Docker Containers on RISC-V Architecture

Carlos Eduardo
Jun 23 · 7 min read

Containers are part of the vast majority of daily interactions with software and the cloud these days. From building applications in a reproducible way to defining standards in deployment, containers brought ease and agility to IT.

RISC-V is a free and open-source instruction set enabling a new era of processor innovation through open standard collaboration. Born at the University of Berkeley, RISC-V ISA delivers a new level of free, extensible software and hardware freedom on architecture, paving the way for the next 50 years of computing design and innovation.

Together they bring real openness to the future of cloud ecosystem by having a top-to-bottom open solution ranging from the hardware to the end-user software.

Me presenting at the Systems Summit in Switzerland

In this article, first I will show how to have a Risc-V virtual machine, install Golang and Docker into it, then run and build containers in this environment.

Then I will show how you can build Golang, and how to compile and install the entire Docker pre-requisites to have a development environment of your own.

Risc-V Virtual Machine

To start with development, I provide a Risc-V Virtual Machine based on Debian Sid with a complete enviroment where you can start developing and building your applications on the Risc-V architecture.

The VM tarball can be downloaded here. Unpack with tar vxf debian-riscv64–20180608.tar.xz and run with the run_debian.sh script.

Log-in on another terminal window with:ssh -p 22222 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@localhost, the root password is “riscv”.

Soon I will also provide a Fedora development VM, the link will be added here.

Install Golang

Go support on Risc-V architecture is not upstream yet. You can check the progress on this issue. Many of it’s modules have already been upstreamed like x/sys and x/net. Also many libraries and applications already support the Risc-V architecture like VNDR, GitHub’s Hub (git client), Labstack Echo framework and more. Check the tracker on https://github.com/carlosedp/riscv-bringup.

To install, download the tarball from here and install with the commands:

# Copy the tarball to the VM
scp -P 22222 go-1.13dev-riscv.tar.gz root@localhost:
# In the VM, unpack (in root dir for example)
tar vxf go-1.13dev-riscv.tar.gz
# Link the files
rmdir /usr/local/go
ln -sf /root/riscv-go/ /usr/local/go
# Add to your PATH
export PATH="/usr/local/go/bin:$PATH"
# Addto bashrc
echo "export PATH=/usr/local/go/bin:$PATH" >> ~/.bashrc

And you are ready to develop in Golang on Risc-V!

Install Docker

After running your VM, download the Docker pack with the command curl -L -o docker-riscv.tar.gz “https://drive.google.com/uc?export=download&id=1Op8l6yq6H_C_zpZUpvO-zHxwbtcrAGcQ" or download in your host and copy to the VM.

The script contains all pre-reqs already built, systemd scripts to start the services and have you ready.

Unpack the tarball and install using the install.sh script as root. Now you can already do docker info and docker version to check if it’s working (in case docker fails to start, just run systemctl start dockeragain.

Running Containers

As a test, I already pushed a container to DockerHub with a hello-world web application using Echo Framework.

The source for this small application and it’s Dockerfile is in the Risc-V Tracker repository.

Run this container with docker run -d -p 8080:8080 carlosedp/echo_on_riscv and test it with curl http://localhost:8080

Building Containers

To build a container, just follow the default path of building your app, creating your Dockerfile and running docker buildlike the example from the repo. Checkout that tree and use the Makefile for convenience:

Currently, there are no official base images for Debian, Alpine or other OSes yet so scratch is your friend. I’ll soon start building some base images and pushing them into my DockerHub account. Tonis Tiigi already built a Debian image that can be pulled from tonistiigi/debian.


Build instructions

Here are the instructions if you want to build all pre-reqs from source.

Go

First checkout and bootstrap Go Risc-V tree into a host that has Go installed (could be Mac, Linux):

git clone https://github.com/4a6f656c/riscv-go
cd riscv-go/src
GOOS=linux GOARCH=riscv64 ./bootstrap.bash
# Copy the generated boostrap pack to the VM/SBC
scp -P 22222 ../../go-linux-riscv64-bootstrap.tbz root@localhost:

Now on your Risc-V VM/SBC, clone the repository, export the path and bootstrap path you unpacked and build/test:

tar vxf go-linux-riscv64-bootstrap.tbz
git clone https://github.com/4a6f656c/riscv-go
cd riscv-go
export GOROOT_BOOTSTRAP=$HOME/go-linux-riscv64-bootstrap
export PATH="$(pwd)/misc/riscv:$(pwd)/bin:$PATH"
cd src
# Builds go on $HOME/riscv-go/bin that can be added to your path
GOGC=off ./make.bash
# Tests the build (optional)
GOGC=off GO_TEST_TIMEOUT_SCALE=10 ./run.bash

Build Docker and it’s requirements

Create a temporary place for building the packages:

mkdir -p $HOME/riscv-docker
cd $HOME/riscv-docker

libseccomp

Libseccomp builds fine without Kernel support, just applying the PR seccomp/libseccomp/pull/134.

git clone git://github.com/seccomp/libseccomp
pushd libseccomp
git fetch origin pull/134/head:riscv64
git checkout riscv64
./autogen.sh
./configure
make
make install
popd

crun

Crun is the container runtime. Docker uses runc by default but it requires Go CGO that is still not available but crun is built in C and works perfectly.

# Install pre-reqs
sudo apt install pkgconf libtool libsystemd-dev libcap-dev libyajl-dev libselinux1-dev go-md2man libtool
git clone https://github.com/giuseppe/crun
pushd crun
./autogen.sh
./configure
make
sudo cp crun /usr/local/bin
sudo ln -sf /usr/local/bin/crun /usr/local/bin/runc
popd

containerd

Containerd is the container daemon. Risc-V support is already upstreamed.

mkdir -p $GOPATH/src/github.com/containerd/
pushd $GOPATH/src/github.com/containerd/
git clone https://github.com/containerd/containerd
pushd containerd
go build ./cmd/ctr
go build ./cmd/containerd-shim
go build -tags no_btrfs ./cmd/containerd
sudo cp ctr containerd-shim containerd /usr/local/bin/
popd
popd

docker-cli

The Docker client is already upstreamed and support Risc-V.

mkdir -p $GOPATH/src/github.com/docker/
pushd $GOPATH/src/github.com/docker/
git clone https://github.com/docker/cli
pushd cli
./scripts/build/binary
sudo cp ./build/docker-linux-riscv64 /usr/local/bin
sudo ln -sf /usr/local/bin/docker-linux-riscv64 /usr/local/bin/docker
popd
popd

dockerd

The Docker daemon still requires some changes and libraries to be upstreamed. Follow the project tracker for more info. You can build from Tonis tree:

mkdir -p $GOPATH/src/github.com/docker/
pushd $GOPATH/src/github.com/docker/
git clone git://github.com/tonistiigi/docker
pushd docker
git checkout 3de77084d559055e87414c2669b22091a8396990
go build -tags "no_quota_support exclude_graphdriver_devicemapper" ./cmd/dockerd/
#go build -tags "exclude_disk_quota exclude_graphdriver_devicemapper" ./cmd/dockerd/ # On new trees
sudo cp dockerd /usr/local/bin

docker-init

git clone https://github.com/krallin/tini
pushd tini
export CFLAGS="-DPR_SET_CHILD_SUBREAPER=36 -DPR_GET_CHILD_SUBREAPER=37"
cmake . && make
sudo cp tini-static /usr/local/bin/docker-init
popd

docker-proxy

mkdir $GOPATH/src/github.com/docker
pushd docker
git clone https://github.com/docker/libnetwork/
pushd libnetwork
go get github.com/ishidawataru/sctp
go build ./cmd/proxy
sudo cp proxy /usr/local/bin/docker-proxy
popd
popd

Running

# Execute containerd
sudo containerd
# Execute dockerd
sudo dockerd #or with the proxy parameter
# Run docker client
sudo docker version

Or use the systemd scripts below:

[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd
KillMode=process
Delegate=yes
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
[Install]
WantedBy=multi-user.target

Save this at /etc/systemd/system/containerd.service

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/local/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3
# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this option.
TasksMax=infinity
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target

Save this at /etc/systemd/system/docker.service

[Unit]
Description=Docker Socket for the API
PartOf=docker.service
[Socket]
ListenStream=/var/run/docker.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
[Install]
WantedBy=sockets.target

Save this as /etc/systemd/system/docker.socket

And run:

sudo systemctl daemon-reload
sudo systemctl start containerd
sudo systemctl start docker
sudo systemctl enable containerd
sudo systemctl enable docker

Conclusion

Container use on Risc-V architecture is pretty functional. Now the heavy work is to upstream Go, implement CGO support and have base images to build software.

If you have suggestions, want to join and start providing support to some projects, message me on Twitter or open an issue on the tracker repository.

Carlos Eduardo

Written by

Writing everything cloud and all the tech behind it