AWS Firecracker Configure Host & Guest Networking for Testing

Photo by Sander Weeteling on Unsplash

What good is a virtual machine without a network? Not much for most so picking up where the previous post left off, today we will cover how to configure network access for the hello-world micro-vm paving the way to greater projects! If you want to jump into using Libvirt on a custom microvm, see my other article.

First let’s cover some theory, Firecracker works by making use of “tap” interfaces on the host. For internet access each micro-vm requires a dedicated tap. With the tap created will then use iptables to configure NAT. LAN interfaces will be covered later.

Note! I performed these steps on a Fedora 29 host without the jailer as it changes the setup slightly.

Step 1. Configure the host

• First check to make sure that ip_forward is enabled. If the following command has an output of 1, ip_forward is enabled. If the output is 0 use the second command to enable.

#Check if ip_forward = 1
$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
#To enable
$ sysctl -w net.ipv4.ip_forward=1

• Next we need to configure the host’s tap devices

#Create tap named tapvm1
$ sudo ip tuntap add tapvm1 mode tap
#Add an address to tapvm1
$ sudo ip addr add 10.0.0.1/24 dev tapvm1
#start the new tap interface
$ sudo ip link set tapvm1 up

• To handle routing we can make use of IP tables and a few rules

#Change enp1s0f0 to your internet interface
$ sudo iptables -t nat -A POSTROUTING -o enp1s0f0 -j MASQUERADE
$ sudo iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
$ sudo iptables -A FORWARD -i tapvm1 -o enp1s0f0 -j ACCEPT

Step 2. Configure Firecracker

• Now that we have a working tap interface, “tapvm1” we can configure the micro-vm’s network interface prior to boot

#Change the --unix-socket from /tmp/firecracker.socket if needed
$ curl --unix-socket /tmp/firecracker.socket \
    -X PUT 'http://localhost/network-interfaces/eth0' \
    -H 'accept:application/json' \
    -H 'Content-Type:application/json' \
    -d '{
      "iface_id": "eth0",
      "guest_mac": "AA:FC:00:00:00:01",
      "host_dev_name": "tapvm1"
    }'
}'

• Next boot the micro-vm!

Step 3A. Configure the guest via console

There are two methods here for configuring the host. First you can login to the hello-world micro-vm and manually configure the settings, or you can edit the configuration files offline by mounting the filesystem.

• The login way: First boot the microvm and login. Once logged in specify the address, routes and other interface information required for operation.

#The following works well as a startup script
$ ip route flush dev eth0
$ ip addr add 10.0.0.2/24 dev eth0
$ ip route add 10.0.0.0/24 dev eth0
$ ip route add 0.0.0.0/0 via 10.0.0.1

• The micro-vm should now have internet connectivity but no DNS yet. I will update with how to get DNS fully functional once i figure out why resolv.conf isn't respected. Also the hello-world vm has no DHCP client.

Step 3B. Configure the guest via mounting

• This is ideal if you already have configuration files and/or want to do some programmatically setup. The filesystem is just an ext4 image.

$ mount hello-rootfs.ext4 /mnt
$ cd /mnt

I will be updating this as I figure more out!