Python Flask on Unikraft

7 min readSep 7, 2021

Introduction

Recently I came across a technology known as unikernels, which allows for extreme performance and enhanced security through specialization. They are single-address space machine images constructed by library operating systems and a dedicated application, avoiding all unnecessary bloat to create tiny artifacts of high performance.

In this article, I am going to describe the process for how to create a simple WSGI application with Python Flask and then build it into the Unikraft unikernel (a highly maintained unikernel technology currently in development).

Part 1: Python Flask

Python Flask is a tiny WSGI framework for creating simple, but powerful, web applications. It is fully developed in Python with the possibility to add a wide range of extensions as needed. More information about Flask can be found in their official documentation.

The current Python 3 library for Unikraft uses Python version 3.7.4, which means that we will have to download and configure the same version on our local machine. For this task, I am using pyenv, which is a simple Python management tool for handling Python versions. I recommend this article by David Littlefield for further information about how to install and use it on Linux.

Now it’s finally time to get our hands dirty!

First we create a local directory for where we will initialize Flask:

$ mkdir ~/python-flask && cd ~/python-flask

Make sure to install Python 3.7.4 with pyenv (or have it pre-installed locally):

$ pyenv install 3.7.4
$ pyenv local 3.7.4

The second commands makes it so that Python 3.7.4 is used within this newly created directory. We can check that this is the case by running:

$ python version
Python 3.7.4

After Python is successfully setup, we need to install virtualenv as this is how pure Python-based modules are ported to Unikraft (important to note is that this process is different for modules that are not purely developed in Python, see the Python 3 library description for Unikraft for further information). This can be installed by:

$ pip install virtualenv

Once virtualenv is installed, we can create and enable a new virtual environment:

$ virtualenv flask
$ source flask/bin/activate

You can now also notice that a new directory has been created. This directory includes all the environment files needed for running this Python environment, which we also activated with the source flask/bin/activate command. Within this environment, we can install the wanted Python modules — which in this article is Flask:

$ pip install Flask

We can confirm that Flask was installed by opening the Python CLI and importing the Flask library. If no error messages occurs, we know that Flask has been installed successfully into the virtual environment.

$ python
$ from flask import Flask

All that is left to conclude the Python part of the article is to deactivate the virtual environment:

$ deactivate

In the next section, we will look into how we can move this virtual environment into a Unikraft unikernel.

Part 2: Creating a Flask Application in Unikraft

As a prerequisite for this section, we need to install kraft, a CLI tool for defining, building, configuring, and running Unikraft applications. The following guide (or this alternative guide) is recommended for getting it installed locally.

kraft will automatically create a folder structure as follows on your machine:

~/.unikraft
├── apps
├── libs
└── unikraft

We next cd our way into the ~/.unikraft/apps directory, which is where we can initialize and develop all our Unikraft specific applications. For our case, we want to use the existing Python 3 application as a baseline for creating a Python Flask application for Unikraft. Initialize the existing helloworld application (make sure that kraft list update has been executed successfully first):

$ kraft init -t python3@staging app-flask

By using kraft to initialize the existing helloworld application for Python 3 the example application will be downloaded into the new directory ~/.unikraft/apps/app-flask, and all the necessary support libraries can be found in ~/.unikraft/libs.

Listing out the files in ~/.unikraft/apps/app-flask shows following files:

kraft.yaml
Makefile.uk
minrootfs.tgz
README.md

Unfortunately, the current helloworld application for Python 3 in Unikraft is build for an old version of Unikraft (RELEASE-0.4), which requires us (for now) to manually bump the versions for Unikraft and libraries to the latest versions. This can be done by using git to checkout the different versions:

$ cd ~/.unikraft/unikraft && git checkout usoc21
$ cd ~/.unikraft/libs/lwip && git checkout staging
$ cd ~/.unikraft/libs/newlib && git checkout staging
$ cd ~/.unikraft/libs/pthread-embedded && git checkout staging
$ cd ~/.unikraft/libs/python3 && git checkout staging

Next we need to update kraft.yaml to the following:

specification: '0.5'
name: python3
unikraft:
  version: usoc21
  kconfig:
    - CONFIG_LIBUK9P=y
    - CONFIG_LIB9PFS=y
    - CONFIG_LIBDEVFS=y
    - CONFIG_LIBDEVFS_AUTOMOUNT=y
    - CONFIG_LIBVFSCORE_AUTOMOUNT_ROOTFS=y
    - CONFIG_LIBVFSCORE_ROOTFS_9PFS=y
    - CONFIG_LIBUKLIBPARAM=y
targets:
  - architecture: x86_64
    platform: kvm
libraries:
  pthread-embedded: staging
  lwip: staging
  zlib: staging
  libuuid: staging
  newlib: staging
  python3:
    version: staging
    kconfig:
      - CONFIG_LIBPYTHON3=y
      - CONFIG_LIBPYTHON3_EXTENSIONS=y
      - CONFIG_LIBPYTHON3_EXTENSION_EXPAT=y
      - CONFIG_LIBPYTHON3_EXTENSION_LIBMPDEC=y
      - CONFIG_LIBPYTHON3_EXTENSION_UUID=y
      - CONFIG_LIBPYTHON3_EXTENSION_ZLIB=y
      - CONFIG_LIBPYTHON3_MAIN_FUNCTION=y
volumes:
  fs0:
    driver: 9pfs
    source: ./flask-fs.tgz

Thereafter, we remove the minrootfs.tgz file as this only includes the packaged helloworld application and corresponding virtual environment files. Our goal after all is to be able to run Python Flask within Unikraft!

Going back to the Python Flask based virtual environment that we created in the first section, we now move this directory into the ~/.unikraft/apps/app-flask/fs0 directory:

$ mv ~/python-flask/flask ~/.unikraft/apps/app-flask/fs0

The lib-python3 library defines the module root path to be $(PYTHON_ROOTFS)/lib/python3.7, which means that we have to move the module files out of the site-packages and up one level. This can be done by:

$ mv ~/.unikraft/apps/app-flask/fs0/lib/python3.7/site-packages/* ~/.unikraft/apps/app-flask/fs0/lib/python3.7/
$ rm -rf ~/.unikraft/apps/app-flask/fs0/lib/python3.7/site-packages

Before building a Unikraft unikernel, we have to configure it. kraft comes with a handy configuration tool which we can use by running:

$ kraft menuconfig

In the menuconfig, a couple of configurations has to be set:

1. Platform Configuration → KVM
2. Library Configuration
  2.1 Python 3
    2.1.1 Provide main function
    2.1.2 Extensions
  2.2 vfscore: Configuration
    2.2.1 Automatically mount a root filesystem (/)
    2.2.2 9PFS

This process will generates a .config file within the application directory, which describes what the build process will do.

Since Python Flask is a WSGI-based framework, networking has to be configured. Use the following commands to set up a simple network bridge:

$ sudo brctl addbr virbr0
$ sudo ip a a  172.44.0.1/24 dev virbr0
$ sudo ip l set dev virbr0 up

All of our configurations are now in place and we can create a demo application. Create a new file within the ~/.unikraft/apps/app-flask/fs0 directory named app.py. Add the following source code to the file and save it:

from flask import Flaskapp = Flask(__name__)@app.route("/")
def hello_world():
    return "<p>Hello, World!</p>"if __name__ == '__main__':
   app.run(host = "172.44.0.2", port = 5000)

From the application source (~/.unikraft/apps/app-flask) we can now build our Unikraft unikernel by running:

$ kraft build

A build process is now triggered which means that all the external libraries will be pulled and compiled, then the application sources will be packaged into a compact unikernel artifact. Once the build has completed without any error messages, we can launch the unikernel with the following command:

$ sudo qemu-system-x86_64 \
    -netdev bridge,id=en0,br=virbr0 \
    -fsdev local,id=myid,path=$(pwd)/fs0,security_model=none \
    -device virtio-net-pci,netdev=en0 \
    -device virtio-9p-pci,fsdev=myid,mount_tag=rootfs,disable-modern=on,disable-legacy=off \
    -kernel build/app-flask_kvm-x86_64 \
    -append "netdev.ipv4_addr=172.44.0.2 netdev.ipv4_gw_addr=172.44.0.1 netdev.ipv4_subnet_mask=255.255.255.0 -- app.py" \
    -enable-kvm \
    -m 1G \
    -nographic

If everything launches as expected, the terminal window should look something like this:

Booting from ROM..0: Set IPv4 address 172.44.0.2 mask 255.255.255.0 gw 172.44.0.1
en0: Added
en0: Interface is up
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                   Tethys 0.5.0~8ef440e
 * Serving Flask app 'app' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://172.44.0.2:5000/ (Press CTRL+C to quit)

Opening the browser on http://172.44.0.2:5000/ would be return a window with the message Hello, World!.

Part 3: Error-Handling

If you encountered any errors above, please check below for known errors that might occur.

Incorrect Unikraft and/or Library Versions

When trying to run the Unikraft unikernel, you notice this error message:

Booting from ROM..[    0.000000] ERR:  [libuklibparam] param.c @ 276  : Failed to fetch the library
[    0.000000] ERR:  [libuklibparam] param.c @ 555  : Failed to identify the library
[    0.000000] ERR:  [libuklibparam] param.c @ 276  : Failed to fetch the library
[    0.000000] ERR:  [libuklibparam] param.c @ 555  : Failed to identify the library
[    0.000000] ERR:  [libuklibparam] param.c @ 276  : Failed to fetch the library
[    0.000000] ERR:  [libuklibparam] param.c @ 555  : Failed to identify the library

This might occur if the library sources or Unikraft were reverted back to configured versions from the kraft.yaml file. Head to the part describing the git checkout process for the different sources and make sure that the correct versions are set.

Missing Python Source Modules

When trying to run the built unikernel, you notice this (or similar) error message:

Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named ‘encodings’

This comes from missing Python source modules, which can be solved by copying the source modules into the virtual environment. With pyenv, this can be done as follows:

$ cp -r ~/.pyenv/versions/3.7.4/lib/python3.7/* ~/.unikraft/apps/app-flask/fs0/lib/python3.7/

Now, rebuild the unikernel and launch it.