TL;DR at the end.
My home has an Odroid C1+, a single-board computer the size of a credit card, based on an ARM CPU and made by Korean company Hardkernel. It’s always on and runs a few essentials like DNS, DHCP, and Samba.
I decided to add two services to it, a photo gallery and a password vault. Installing them natively would be a lot of work — prerequisite software has to be installed, web servers and databases need to be configured. Besides, installation instructions assume heavy stuff like Apache HTTPd and MySQL, probably too heavy for this tiny computer.
It would be so much easier if those services came in the shrink-wrapped form, for example as Docker containers. And indeed, dockerized versions do exist!
However, Docker has certain prerequisites that my current Odroid configuration doesn’t provide. This is so because it runs an old Linux kernel, 3.10 to be specific. To upgrade the kernel, I would have to use ARMbian, which I am not too happy to install, since it is not supported by Hardkernel. For now, I stick to the original Ubuntu 16.04 installed on this server.
So far, I have discovered two problems.
Problem 1: Docker detects Apparmor but can’t use it
The first problem comes up right at the beginning, when I try to run the Docker
hello-world container. I get:
docker: Error response from daemon: Could not check if docker-default AppArmor profile was loaded: open /sys/kernel/security/apparmor/profiles: no such file or directory
Yes, it’s true:
/sys/kernel/security/apparmor exists, but it has no subdirectory named
profiles. When I check the status of Apparmor, I get the same complaint:
apparmor module is loaded.
Could not open /sys/kernel/security/apparmor/profiles: No such file or directory
After some searching, my verdict is that Apparmor is probably not fully supported by that kernel. To fix the problem, I deinstall it and disable the kernel module by adding
apparmor=0 to the kernel parameter list. Now, Docker works.
Problem 2: Spotty network access to containers
The next problem is networking. I install two dockerized applications, Lychee and Bitwarden_RS. They launch all right, but I don’t get network access to them. The browser just times out. However, as soon as I launch
tcpdump on the docker bridge,
docker0, access is established. But after a few minutes, again timeouts. Strangely, it seems that whenever I try to analyze network traffic with
tcpdump, the problem goes away. How can a simple probe impact networking?
I install the containers on a virtual machine running the x64_86 version of Ubuntu 20. No network problems there.
Eventually I discover messages in the Odroid’s kernel log:
$ sudo journalctl -k |grep docker0
Nov 26 19:46:03 odroid kernel: device docker0 left promiscuous mode
Nov 26 19:47:49 odroid kernel: device docker0 entered promiscuous mode
Nov 26 20:16:50 odroid kernel: device docker0 left promiscuous mode
Nov 26 20:16:59 odroid kernel: device docker0 entered promiscuous mode
docker0 bridge is promiscuous, I can access the containers. By default,
tcpdump puts NICs in promiscuous mode, but I can also do it manually with
ip link set docker0 promisc on
The precise cause of this behaviour is still not clear to me, but after a few web searches I come to the conclusion it must have something to do with the
br_netfilter kernel module, which doesn’t exist on Linux 3.10. The Ubuntu 20 virtual machine doesn’t require promiscuity for Docker’s networking to work.
To make Docker work on older Linux kernels, consider disabling and/or removing Apparmor and make the
docker0 bridge promiscuous.
Should I run into more problems, I will update this post.