Installing non official OSs on a KVM Server with multi-VNIC Guests on Oracle Cloud Infrastructure

Oracle’s official BYOH Installing and Configuring KVM on Bare Metal Instances with Multi-VNIC White paper is the supported and recommended way to configure a KVM Server on Oracle Cloud Infrastructure (OCI) instance. Plus, it provides an up-to-date content!

This tutorial provides an example of how easily you can install Debian and other non official OSs on a Bare Metal KVM instance after you configure your KVM (Kernel-based Virtual Machine) hypervisor on Oracle Cloud Infrastructure (OCI).

This process allows you to run any operating systems you need on top of a BMCS KVM instance as a guest VM on Oracle Cloud. For more information about BMCS official images, see Oracle-Provided Images

Getting Started

KVM, Kernel Based Virtual Machine, is a virtualization software which provides ability to run a multiple guest operating systems with the help of hardware virtualization extensions. It supports a wide variety of guest operating system’s such as Linux, Windows, Solaris, Haiku, REACT OS and much more. KVM can be managed using command line or available graphical tools. Virt-Manager (Virtual Machine Manager) is the most widely used application for managing KVM based virtual machines, it supports creating, editing, starting, and stopping KVM-based virtual machines, as well as live or cold migration of guest machines between hosts.

Requirements

  • A BMCS instance (Standard1.36, HighIO1.36 or DenseIO1.36)
  • Application licenses may require License Mobility through Software Assurance when running on Windows Server Instances. Questions about your licensing rights should be directed to Microsoft or your Microsoft reseller
  • Note that when bringing your license to OCI you are solely liable for meeting your licensing obligations with any 3rd party OS/application you are using on your Bare Metal KVM Server Instance
  • Your KVM VM OS ISO needs to be uploaded into your Bare Metal (BM) KVM Server instance
  • Additional block-storage volume attached to your BM KVM Server instance to hold the KVM VM(s) qcow2 disk image(s)
  • KVM will work only if the CPU has a support of hardware virtualization, either Intel VT or AMD-V. To find whether your cpu supports VT features, run the following command.
$ sudo egrep ‘(vmx|svm)’ /proc/cpuinfo

If the above commands returns with any output showing vmx or svm then your hardware supports VT else it does not. Bare Metal instances have support for “vmx” hardware virtualization.

Installing KVM

Issue the following command to install latest qemu package and also virt-manager which provides graphical interface to manage virtual machines.

$ sudo yum install qemu-kvm qemu-img virt-manager libvirt libvirt-python libvirt-client virt-install virt-viewer

Installing VNC

Run the following command to install VNC packages

$ sudo yum groupinstall “Server with GUI”
$ sudo yum install “@X Window System” xorg-x11-xauth xorg-x11-fonts-* xorg-x11-utils tigervnc-server -y

Now, copy vncserver@.service to vncserver@:1.service.

$ cd /lib/systemd/system
$ sudo cp vncserver@.service vncserver@:1.service
$ sudo vi vncserver\@\:1.service

Replace <USER> to VNC username in vncserver@1.service. Use “opc” username as shown below.

ExecStart=/sbin/runuser -l opc -c “/usr/bin/vncserver %i -geometry 1280x1024”
PIDFile=/home/opc/.vnc/%H%i.pid

Set the VNC password for the opc user defined in vncserver@.service

# su - opc
# vncpasswd
Password:
Verify:
# exit

Configure the firewall to allow VNC connection

Configure VNC to auto start

$ sudo systemctl daemon-reload
$ sudo systemctl enable vncserver@:1.service
$ sudo ln -s ’/usr/lib/systemd/system/vncserver@:1.service’ ’/etc/systemd/system/multi-user.target.wants/vncserver@:1.service’
$ sudo systemctl start vncserver@:1.service

Preparing your KVM Server for Multi-vNIC

On your Oracle Linux 7.x KVM Server Instance do the following:

$ sudo vi /etc/default/grub
# append the following parameters in GRUB_CMDLINE_LINUX line
intel_iommu=on ixgbe.max_vfs=16 rdblacklist=ixgbevf

Below is an example:

$ sudo cat /etc/default/grub |grep CMDLINE
GRUB_CMDLINE_LINUX=”crashkernel=auto LANG=en_US.UTF-8 console=tty0 console=ttyS0,9600 rd.luks=0 rd.lvm=0 rd.md=0 rd.dm=0 ip=dhcp netroot=iscsi:169.254.0.2::::iqn.2015–02.oracle.boot:uefi iscsi_param=node.session.timeo.replacement_timeout=6000 intel_iommu=on ixgbe.max_vfs=16 rdblacklist=ixgbevf

Save and exit vi.

Now, run the following commands to

- blacklist the native Intel VF driver and use the OS native ones

$ sudo echo “blacklist ixgbevf” > /etc/modprobe.d/blacklist-ixgbevf.conf

- enable tuned

$ sudo systemctl enable tuned
$ sudo systemctl start tuned
$ sudo tuned-adm profile virtual-host

- recreate grub to validate all the changes

$ sudo grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

Using your BMCS UI, edit your BM Virtual Cloud Network Security List and open port 5901 if that corresponds to the VNC port you’re using

Source: 0.0.0.0/0
IP Protocol: TCP
Source Port Range: All
Destination Port Range: 5901
Allows: TCP traffic for ports: 5901

Reboot your Bare Metal KVM Server instance to load the KVM kernel modules and to restart gnome. Once the restart finishes, you should be able to access your Bare Metal KVM Server instance through any VNC application like vncviewer

NOTE: you will need to reset opc user password through SSH using passwd command in order to be able to connect into your BM KVM server gnome interface through VNC.

Creating a BMCS vNIC

Next step is creating a vNIC and attaching it into your BM KVM Server instance. Using the BMCS Dashboard, click on the details for your BM KVM instance, select “Attached vNICs” and click on “Create vNIC” option.

Below is an example for Debian using a Virtual Cloud Network (VCN) called “New-BM-172” on Availability Domain1 (AD1)

After you created it, you will see the following:

Pay attention to the “MAC Address” and VLAN Tag info because you will use them later. You are now ready to provision a KVM guest VM and add the above vNIC to it.

Creating a KVM BM instance

As mentioned above, KVM can be managed using command line or available graphical tools. For this tutorial, you will use virt-manager GUI to along with a terminal applcation. Use VNC to connect into your BM KVM server instance, open the gnome-terminal and run

Click on “Create a new Virtual Machine” button and follow the requested options like

Select ISO Image

Point to your ISO location

Choose the amount of memory and number of CPUS.

Select the storage (qcow2 file or device (i.e.; /dev/sdb))

Type a name for your KVM guest VM. You can use the same name you used for your vNIC setup or a different one.

During the guest installation use the same OS hostname you chose in your vNIC setup.

Don’t forget to select “Customize configuration before install” option

Remove the default NIC that was added by default and click on begin installation option but DO NOT start the it yet

Click on “Force Off” option as shown below

Before you start the KVM guest VM installation you need to add your BMCS vNIC to it.

Associating your BMCS vNIC with your KVM VM

Find what is the PCI BUS information related to the vNIC you created that has been associated with your BM KVM Server. After that, you will use the VLAN info you got in the above vNIC step.

Run the following command and replace the VLAN-INFO variable for your VLAN Tag ID to identify the PCI device associated with your BMCS vNIC as shown in the below example

$ sudo virsh nodedev-dumpxml `ethtool -i ens2f0 | grep bus-info | awk ‘{print $2}’ | sed ‘s/^/pci_/’ |sed ‘s/\:/_/g’ |sed ‘s/\./_/g’` | grep “address domain” | sed ‘VLAN-INFOq;d’ | sed ‘s/^ *<address/<address type=\”pci\”/’

Here is what it looks like based on the above vNIC process and note that VLAN-INFO variable was changed to 3 to match the VLAN Tag ID associated with the Debian 88 vNIC.

$ sudo virsh nodedev-dumpxml `ethtool -i ens2f0 | grep bus-info | awk ‘{print $2}’ | sed ‘s/^/pci_/’ |sed ‘s/\:/_/g’ |sed ‘s/\./_/g’` | grep “address domain” | sed ‘3q;d’ | sed ‘s/^ *<address/<address type=\”pci\”/’
<address type=”pci” domain=’0x0000' bus=’0x13' slot=’0x10' function=’0x4'/>

Create a file called kvm-vnic-attach.xml (you can use any name for the XML file) and make sure it matches your vNIC configuration (MAC Address, VLAN Tag ID (BMCS UI Dashboard — Attached vNICS option) and the above virsh nodedev-dumpxml output) as the following one

$ sudo vi /tmp/kvm-vnic-attach.xml
####BMCS vNIC info + virsh nodedev-dumpxml output
<interface type=’hostdev’ managed=’yes’>
<source>
<address type=”pci” domain=’0x0000' bus=’0x13' slot=’0x10' function=’0x4'/>
</source>
<vlan>
<tag id=’3'/>
</vlan>
<mac address=’00:00:17:01:02:1E’/>
</interface>
#### end of /tmp/kvm-vnic-attach.xml

Save and exit.

Attach that new hostdev NIC to your KVM guest running the following command line.

$ sudo virsh attach-device debian8 /tmy/kvm-vnic-attach.xml --config

Device attached successfully

Continue the KVM virt-manager install. Open virt-manager and verify your new vNIC was added into the guest XML configuration

Select Boot Options and set the CD-Rom as the primary one for the booting order

Check if the CD is connected with your local ISO file

and install your KVM guest OS as usual. The only remaining step is setting up a static IP address based on your vNIC/VCN configuration which can be done during the KVM guest install or afterwards.

Below is an example for the vNIC data used on this tutorial.

Static IP: 172.0.0.47

Mask: 255.255.255.0

GW: 172.0.0.1

DNS: 169.254.169.254

Once the install is done, disconnect your CD-ROM ISO, boot your new KVM guest VM and you should be able to connect into it using the public IP address associated with your BMCS vNIC along with the service you have configured.

Debian 8.8.0 Guest on BMCS KVM Server

ssh connection using BMCS vNIC Public IP address

Below are some additional KVM guest OS examples running on a Bare Metal KVM Server Instance

Your BM KVM Server+ multi-vNIC setup on BMCS is concluded!

Additional Security Recommendations:

  • Disable SSH Password Authentication and enable SSH key authentication (add your ssh public key in /home/opc/.ssh/authorized_keys file)
  • Keep the KVM guest firewall enabled and open only the required ports
  • Patch your KVM guest VMs regularly
  • Use VPN whenever possible
  • Install a Bastion Server to login into your BMCS KVM Server and use internal IP address only for the latter
One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.