Introduction

Foreword: This is not a how to. The alignment of software, hardware and luck varies from person to person in this process.

It's been at least a year since I started working toward this. The idea is as follows: many game developers refuse to write their applications for Linux, it's getting better but a lot of the AAA titles will only hit Windows, and if we are lucky Mac. Linux games are usually left to the Paradox Interactives of the games industry. Virtualization isn't new to me, I started working with virtualization with VMWare workstation on Windows 7 probably 5-6 years ago. Hardware assisted virtualization will let you run an operating system and virtualized hardware under another piece of hardware hypervisor along side, or on top of another operating system.

Goal

Achieve hardware accelerated graphics on a Windows OS for gaming without giving up Linux on the desktop.

Requirements

• Two GPU's, different models preferred
• IOMMU (VT-d or AMD-Vi) support on both the motherboard and CPU
• Multiple displays, or a way to switch them
• Arch Linux, or another distro that uses a 4.1+ Linux Kernel
• OVMF
• QEMU/KVM (and motherboard/cpu that supports that)
• libvirt
• This guide
• Other supporting hardware, video cables, power supplies, additional human interface devices

Background

Hardware assisted virtualization is rather new. CPU support of these features started in 2006 with both AMD and Intel on board. Before that software assisted virtualization existed. QEMU, a tool in my process, is just a generic emulator and without KVM(Kernel-based Virtual Machine), another tool, it does not perform anywhere close to as well as with KVM hardware assisted virtualization. But is closer to the software assisted virtualization before hardware extensions existed. QEMU/KVM virtualizes the hardware of these machines and lets the instructions run on the bare CPU directly. This means that I/O is done through a software layer.

Enter GPU's, (Graphics Processing Units) are fancy matrix machines that display graphics in a higher bandwidth than could be possible on single, or multi-threaded cpu's. GPU's render directly out to the monitor, which is capable of taking the high bandwidth data and displaying it on your LCD(or LED) display. So at no point does the rendered display stream go through the CPU bus. This means we cannot render virtualized operating systems interfaces using a GPU and display it back through another display, at least efficiently. Operating systems cannot share these devices. So no picture in picture accelerated displays.

Enter IOMMU(Input/output memory management unit) which provides memory protection and segregation for I/O devices, like GPU's! What this lets us do is make IOMMU groups and assign these to our virtual machines, providing isolation between devices and OS's.

Next step is driver binding. Nouveau, the open source driver for Nvidia display adapters(GPU's) is built into the Linux Kernel and will load for each Nvidia based device it finds connected. So we must use the vfio-pci module included in Linux Kernel 4.1+ (which means your limited in selection of Linux distributions) to bind the target GPU to the target virtual machine. Before 4.1 you had to rely on pci-stub which was a tedious process.

Since newer versions of Windows rely on UEFI(Unified Extensible Firmware Interface) as apposed to the legacy bios we need to utilize it for our machines. For us, that means download OVMF, since were using Arch Linux that means we can simply install it from the AUR and utilize it with a very simple config edit for libvirt.

Execution

Next we tie it all together, QEMU/KVM and libvirt for accessibility. libvirt is a wrapper around hypervisors. Namely it wraps QEMU/KVM, Xen, FreeBSD bhyve, VMWare, Virtualbox, Hyper-V to name a few. It also has a handy interface virt-manager which provides a GUI for managing VM resources. This is very convenient since hand editing virt files is tedious and QEMU isn't very user friendly. We build the VM and before creation, swap out BIOS firmware for OVMF passthrough our GPU, and it's sound subsystem, and switch to virtio-scsi instead of IDE virtual block devices(like disks and drives.) Before booting the VM it is best to add the kvm=off option to the domain(libvirt's name for generic virtual machines). You do this by adding

<kvm>

<kvm>
<hidden state='on'>
</kvm>

to features in the domain's .xml file, located in /etc/libvirt. If you don't, the GPU will not start in Windows because Nvidia detects that it is in a VM and doesn't want consumers doing this kinda thing. Quadro cards do not have this issue, or AMD cards.

That's it, if all this works out you should be able to boot Windows install media, install virtio-scsi drivers and the graphics adapter drivers and achieve native speeds.