KVM GPU Passthrough
Jul 16, 2015

Requirements

  • AMD-V/VT-d capable CPU & motherboard
  • Two GPUs (I am using two different NVIDIA GPUs
  • Enough RAM and disk space for a Windows VM

Guide

  1. Install two GPUs into your system.

  2. Locate the GPU you wish to passthrough to the VM. lspci -nn | grep -i nvidia

    Copy the device id’s which looks like this:

    06:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] [10de:1380] (rev a2)
06:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fbc] (rev a1)

  3. Edit the default grub config (/etc/default/grub) to enable iommu and disable the secondary GPU that you are going to passthrough to the VM. GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pci-stub.ids=10de:1380,10de:0fbc"

  4. Update grub. sudo grub-mkconfig -o /boot/grub/grub.cfg

  5. Next, install qemu. sudo pacman -S qemu

  6. Enable KVM modules. /etc/modules-load.d/20-kvm.conf

    pci_stub
vfio
vfio_iommu_type1
vfio_pci
kvm
kvm_intel

    Reboot.

  7. Create a vfio bind script at /usr/bin/vfio-bind. ``` #!/bin/bash

modprobe vfio-pci

for dev in “$@”; do vendor=$(cat /sys/bus/pci/devices/$dev/vendor) device=$(cat /sys/bus/pci/devices/$dev/device) if [ -e /sys/bus/pci/devices/$dev/driver ]; then echo $dev > /sys/bus/pci/devices/$dev/driver/unbind fi echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id done


   `chmod 755 /usr/bin/vfio-bind`

8. Bind the two devices (this needs to be done everytime you reboot).
`vfio-bind 0000:06:00.0 0000:06:00.1`

9. Get the latest virtio drivers for Windows.
[https://fedoraproject.org/wiki/Windows_Virtio_Drivers#Direct_download](https://fedoraproject.org/wiki/Windows_Virtio_Drivers#Direct_download)

10. Create a blank image for Windows.
`qemu-img create -f raw -o size=120G windows8.1.raw`

11. Start the VM (change the device ids and image paths).

#!/bin/bash rm /tmp/windows_vars.fd cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/windows_vars.fd

QEMU_PA_SAMPLES=128 export QEMU_AUDIO_DRV=pa

sudo vfio-bind 0000:06:00.0 0000:06:00.1

sudo qemu-system-x86_64 -enable-kvm -m 8G -cpu host -smp sockets=1,cores=6,threads=1 -machine type=pc,accel=kvm -mem-prealloc
-balloon none
-vga none
-device qxl
-soundhw hda
-drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
-drive if=pflash,format=raw,file=/tmp/windows_vars.fd
-device vfio-pci,host=06:00.0,multifunction=on
-device vfio-pci,host=06:00.1
-drive id=cd0,if=none,format=raw,readonly,file=/home/USER/windows8.1.iso -device driver=ide-cd,bus=ide.0,drive=cd0
-drive id=virtiocd,if=none,format=raw,file=/home/USER/virtio-win-0.1.102.iso -device driver=ide-cd,bus=ide.1,drive=virtiocd
-drive id=disk0,if=none,format=raw,cache=writeback,file=/home/USER/windows8.1.img -device virtio-scsi-pci,id=scsi0 -device scsi-hd,drive=disk0
-boot menu=on ```

The -device qxl argument will allow you to click into the qemu window and control the keyboard/mouse. It is good enough to get Windows installed but too laggy for anything else.

Notes

  • You can bind a USB hub or USB keyboard/mouse to the VM but then it will be completely bound to the VM and you will not be able to use Windows like a second PC on a second monitor. Instead, you can install Synergy and then you’ll be able to move the mouse just like you would if it were a physical machine set up with Synergy.
  • I have noticed some annoying lag spikes when the Windows VM consumes a lot of CPU resources. The only way I’ve found to fix this is to give the VM less cores. You could also try linux-ck which is what I am testing now. The lag spikes I’m getting aren’t specific to running a Windows VM. It also occurs when compiling code or when Bitcoin loads more than a weeks worth of data. The audio will stop for a second or two and the mouse will lag. It’s very annoying.

Tweaks

References

linux Related
Comments