GPU Passthrough on Ubuntu 22.4: A Comprehensive Guide
This article help you the below items.
- Virtualization of CPU
- GPU Passthrough
- Install KVM and create a VM using virt-manager
- Map GPU to the created Virtual Machine.
Virtualization of CPU: The below steps will be vary based on the Server CPU and GPU. Here this steps have been performed on HPE DL380 Gen10 server with Intel Processor and base OS is Ubuntu 22.4.
Ensure GPU and CPU support the necessary virtualization extensions (such as Intel VT-x or AMD-V) and GPU passthrough capabilities
- Make sure Intel Virtualization Technology (Intel VT) and Intel ® VT-d must be enabled from server BIOS
Special Note: Some documentation talks about IOMMU, it’s also known as VT-d, is a feature of AMD processors that allows the operating system to manage direct access to hardware devices. This can be useful for virtualization, as it allows the hypervisor to isolate guest operating systems from each other and from the host operating system.
Verify GPU is installed and compatible and the driver is installed.
By running the below command, we can verify, model name of the GPU and compatibility.
lspci | grep -EA1 -i "VGA|3D controller"
Check in the installed GPU in the system using the below command
lshw -c video
Check the installed GPU card driver version using the below command.
Check the driver is compatible or not.
- https://www.nvidia.com/Download/index.aspx
- Select the correct model of the GPU card from the list > And Installed the drive
Enable the the PCI NVIDA A100 GPU to passthrough function.
- Edit the following file /etc/default/grub > Add the below in the grub fil
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on vfio-pci.ids=10de:20f1"- We can identify the pci.ids using the below command.
lspci -nn | grep "NVIDIA"- Marked in yellow, will be the pci.ids
- Open the /etc/default/grub file using a text editor with root privileges (e.g., sudo vim /etc/default/grub).
- Find the line starting with GRUB_CMDLINE_LINUX_DEFAULT and GRUB_CMDLINE_LINUX add the following options inside the quotes:
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on vfio-pci.ids=10de:20f1"
GRUB_CMDLINE_LINUX="net.ifnames=0"
- Update grub using the below command
sudo grub-mkconfig -o /boot/grub/grub.cfg
- Create a new file under /etc/modprobe.d/vfio.conf add the below
options vfio-pci ids=10de:20f1
softdep nvidia pre: vifo-pci
- Update the initramfs using the below command and reboot the hots.
sudo update-initramfs -u
- After the reboot of the host, check NVIDA is configure for Pass-through using the below command.
lspci -k
Install the KVM and the Virt-Manager GUI tool
- Install KVM using the below command.
sudo apt install qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils virt-manager ovmf
Create a new virtual machine in Virt-Manager
- Open Virt-Manager > File > Create new virtual machine > Select “Local Install Media (ISO image or CDROM)” > Forward
- Click on brows and locate the OS > Click on Forward.
- Choose Memory and CPU > Forward.
- Name the VM > Click on Customize configuration before install > Finish.
- Click on Add hardware > Select PCI Host Device > Select the detected GPU from the list > Click on Finish
- Open the virt-manager > run the virtual machine.
Verify the GPU is mapped with the Virtual Machine and install the compatible driver
- First verify the mapped GU is listed in the virtual Machine using the below command.
lspci -nn
- Check if the driver is installed using the below command
nvidia-smi
Since the driver is not installed, lets install the compatible drive for the GPU card.
- https://www.nvidia.com/Download/index.aspx
- Select the correct model of the GPU card from the list > And Installed the driver
Once we identified the compatible driver, please use the below method to install the driver.
- Use the below command to list down the nvidia-driver
apt search nvidia-driver
apt-cache search nvidia-driver- Use the below command to filter the required version for the card based on the official NVIDIA driver page
apt-cache search 'nvidia-driver-' | grep '^nvidia-driver-[[525.105.17]]*'
apt-cache search 'nvidia-dkms-' | grep '^nvidia-dkms-[[525.105.17]]*'
- Install the correct driver version using the below command.
sudo apt install nvidia-driver-525 nvidia-dkms-525- Check the version of the NVIDIA Card using the below command in the virtual machine.
nvidia-smi
praveen.mprv@gmail.com
