How to run containerized Bluetooth applications with BlueZ

Introduction

For a multitude of reasons, it can be beneficial to containerize applications. These might include security concerns, ease of use or other reasons, such as automated deployment via container platforms like Kubernetes. In the times of huge Cloud-native applications, containers are an integral part of making them flexible, scalable and easy to deploy.

While many applications can be containerized and run rather straight forwardly, this is not the case for ones that require host-specific functionality — as in the case of Bluetooth. In this article, we show how to accomplish the aforementioned task with applications that use BlueZ, the official Linux Bluetooth Stack. We will describe how to build and run containers, which problems we faced along the way — and possible usecases for such a solution.

About Bluetooth

Bluetooth is a Wireless Personal Area Network (WPAN) standard developed by the Bluetooth Special Interest Group (SIG) with version 1.0 being released in 1999. The standard has seen several updates up to the current specification of Bluetooth 5.2. It is currently one of the most widespread wireless technologies, being supported by many laptops, smartphones, single-board computers and IoT-devices.

Bluetooth has many different predefined profiles that expand the core specification and define how certain applications shall communicate with each other. These include streaming of audio, data transfer, printing, remote controls and many other use cases. Bluetooth Profiles allow vendor-agnostic interoperability between devices of different companies, which is a big contributor to Bluetooth’s prevalence.

Bluetooth has two different radio versions, Bluetooth Classic and Bluetooth Low Energy (LE), the latter one being added with core specification 4.0. Bluetooth LE aims at maximum energy efficiency, making it a good fit for low power wireless applications. While adapters with Bluetooth 4.0 or higher support Bluetooth LE, it has separate profiles, such as Generic Attribute Profiles (GATT), which are not compatible to Bluetooth Classic.

About BlueZ

BlueZ is at the heart of many applications that utilize Bluetooth connectivity. It is the official Bluetooth stack for Linux and is therefore pre-installed on many popular distributions, such as the latest versions of Ubuntu and Raspberry Pi OS. However, is compatible to many other distributions too, while being almost entirely platform independent. Bluez supports widely used Classic Bluetooth profiles such as A2DP or AVRCP out of the box, as well as Generic Attribute Profiles of Bluetooth LE.

BlueZ runs in a process called bluetoothd. It mounts the Bluetooth adapter(s) using the Host Controller Interface (HCI), in most cases the adapter hci0, which is the default one of a Bluetooth-capable device with a single adapter. Applications can communicate with bluetoothdusing the interprocess message bus D-Bus, more precisely, the system D-Bus:

Fig 1: Structure of an arbitrary application with BlueZ

About our Application

Our application is a nodejs app, written in TypeScript. It utilizes Bluetooth LE to exchange json encoded messages via the UART GATT with other instances running on different devices.

While the D-Bus calls necessary to communicate with the Bluetooth service can be implemented manually, this would require a lot of work. To make the implementation easier, a number of BlueZ api-wrappers exist for different platforms, such as the one that we are using.

In theory, the runtime environment should not matter, since all necessary inter-process communication is done via the D-Bus, as long as the aforementioned is supported. Consequently, this article can be applied to other applications that use BlueZ as well.

Our goal is to containerize our nodejs app and include all necessary services like BlueZ and D-Bus in the container. Essentially, we want to enclose as much Bluetooth-related software as possible, to minimize dependencies on host devices, as we intended to deploy such a container in a Kubernetes cluster.

About our testing

We have built containers for multiple architectures, including arm/v6, arm/v7, arm64 and amd64. The containers were run on multiple versions of the Raspberry Pi (Zero W, Pi 3B, Pi 4) and a Thinkpad T470s.

Before we start

This article focuses on how to build and run containers that use BlueZ. We assume that the host has all prerequesites so that BlueZ is able to run locally in the first place. That includes a compliant Bluetooth adapter and all necessary kernel modules. While we will not cover setting up the host in this article, if your device is able to run your Bluetooth application un-containerized, it should be able to run it in a container as well.

Basic Solution

Now that we know about BlueZ and the basics of our use case, we can proceed to the realization of the container.

Containerization

With the information we have, we can set a number of requirements that our container has to fulfill:

• nodejs support, in our case Version 14
• BlueZ support
• D-Bus support

From these requirements, we can construct the Dockerfile:

We have chosen node:14.15.5-buster as base image since it combines a suitable version of nodejs with Debians current stable version 10. While we have not tested different images, other Debian-based distributions are likely to work as well. We have not yet tested images based on other distributions like Alpine.

The entry point of the docker container is a script called entrypoint.sh. Since the container does not automatically start the necessary services like dbus and bluetooth, they need to be started manually before launching the application. The basic script consequently looks like this:

Now, we can build the image, for example with docker:

docker build -t our-container .

Running the container

With the container built, it can now be run. However, a few things come to mind:

• the container needs to be able to directly communicate with the Bluetooth adapter, which is handled as a network interface
• the container needs to be able control the adapter administratively

In the case of running the container via docker, these requirements lead to the start options that we need to set when launching:

docker run --net=host --privileged -t our-container

The first flag, --net=host, allows us to handle network adapters natively. The second one, --privileged, gives the containers application the same capabilities as if they were native as well. While the first one is mandatory, there are better solutions for the latter one — but more on that later.

One thing we need to make sure is that no application owning the Bluetooth adapter we intend to use is running on the host. For example, if the bluetoothd service is active, the adapter will not be visible to the container, as it is already owned by a process. If that is kept in mind, this solution should run just fine. However…

Problems with this solution

…while this works, it has a few issues that we need to address:

• Running a container with privileged rights is a security liability, especially if the container executes as root user, which it does by default
• The container startup can fail obfuscatedly

Fixing our problems

With the previously established knowledge, let’s reiterate and fix the problems of our first, basic solution.

In-container rights management

By default, containers run with the root user. This is a problem, especially since the container is running with privileged rights. However, it is not directly possible to just change the user to a non-elevated one. Starting the dbus service requires super-user rights, as does communication to the bluetoothd process, which utilizes the system D-Bus. As our application needs to communicate with BlueZ, this means it would have to be started by a super-user as well —we want to avoid this if possible.

Luckily, at least the latter (running the app as non-root user) can be managed rather easily. The D-Bus allows us to configure the rights that each individual user has by placing a *.conf file in /etc/dbus-1/system.d/. We can set which processes a user is allowed to communicate with, as well as which interfaces can be owned or sent to:

The list of send-interfaces that need to be permitted depends on the user application. This particular configuration allows the user bluezuser to use GATT via BlueZ. Other applications might require additional permissions.

After saving the above file, e.g. as bluezuser.conf, we have to add it to the container via the Dockerfile:

Now, while we can run the app without elevated rights, we still need to address starting the dbus service. One possibility is to allow bluezuser to start applications via sudo. This requires us to install the sudo package, as well as set up the bluezuser so that it can headlessly start processes with root-rights. We can do this by appending the following section to the Dockerfile:

Since now the container is run as bluezuser, we need to prefix the service start commands in entrypoint.sh with sudo:

Making the start-up reliable

While experimenting, we’ve run into seemingly random failures on start-up. One possible cause was easily identified — the adapter could get stuck if the container was run previously and did not exit correctly, not allowing the newly started one to own it. This could be fixed by just restarting the adapter once before launching the app in entrypoint.sh:

However, still some container launches resulted in failures. On some Raspberry Pis, part of a Kubernetes-Cluster, launching did not seem to work at all — even though it worked fine when launching the container locally on the same devices. After a lot of debugging and even more headscratches, the breakthrough came when we observed that launching the user application directly after starting the container did not work, while starting it after waiting for a few minutes did.

The culprit was the bluez service, which in some cases took unreasonably long to start. While we could not yet identify why the start took so long on the Kubernetes nodes, it is clear that the user application cannot be run before the service it depends on is fully started. So to make the start-up process reliable, we need to ensure that all services launched correctly first. This again can be done in entrypoint.sh:

Unintuitively, just checking the services status does not work, since the bluetooth service shows as running right away, even though it is not yet fully started. Instead, we check once per second if the start-stop-daemon, which starts and stops services, is still active. Once the start-stop-daemon is gone from the list of active processes, all services have launched successfully. In our case, applying this change fixed all our issues and the start-up became fully reliable. However, the validity of this solution might depend on your base image, as it will only work the images distribution uses the start-stop-daemon.

Running the container without privileged rights

While we’ve already managed to start the container itself as non-root user, executing with privileged rights is still problematic. In the case of docker however, it is possible to more granularly set its permissions.

The Bluetooth adapter is a network adapter. If we look at the list of privileges a container can receive, we can see that the NET_ADMIN capability allows it to “Perform various network-related operations.” — just what we need.

We can add this permission by replacing the --privileged flag with the --cap-add=NET_ADMIN start option:

docker run --net=host --cap-add=NET_ADMIN -t our-container

And lo and behold, everything worked as intended (and our Sysadmin is happy :) ).

Advanced Solution

Applying the fixes to our previous, basic solution, we have managed to build and run a container that utilizes BlueZ, while not requiring to be run as privileged. In the following, let’s see the new solution we have ended up with:

Dockerfile

Entrypoint.sh

bluezuser.conf

Running the container

docker run --net=host --cap-add=NET_ADMIN -t our-container

Applications

Now that we’ve successfully built a reliable container for Bluetooth applications — what possible usecases exist for such a solution?

While it can be useful to containerize simple monolithic applications such as a Bluetooth sensor running on a Raspberry Pi, a task like this can be rather easily — and in many cases more efficiently — done natively as well. However the containerization allows us many more possibilities…

Bluetooth container as Bluetooth service

This article focuses on an arbitrary, monolithic nodejs-based application. However, the developed procedure can also be used to implement a Bluetooth-capable service, for instance by exposing the BlueZ-api via a REST api that other applications or services could communicate with to access Bluetooth connectivity.

Fig 3: Example structure of a Bluetooth container as service

This structure is advantageous for multiple reasons:

• The Bluetooth container requires more rights than by default, so limiting its functionality to just being a proxy for other applications with regular permissions benefits security
• BlueZ can only be owned by a single process. Deploying multiple applications that include BlueZ on the same device would not work. However, if the Bluetooth service balances the load instead, multiple applications can use it simultaneously
• The Bluetooth service and other services can be individually updated
• Bluetooth can be accessible from other devices as well, allowing applications on devices that are not capable of Bluetooth to utilize it

While this could all still be done natively, it already becomes apparent that containerization is helpful. However, it becomes even more interesting when paired with a container platform like Kubernetes.

Bluetooth as microservice

Cloud-native systems embrace microservices, a popular architectural style for constructing modern applications.

One of the paradigms of Cloud-native is to split applications into microservices that can be individually deployed on-demand. This allows to dynamically deploy capabilities if necessary, while not requiring them to share their code base.

Imagine a Kubernetes cluster with Bluetooth capable devices such as Raspberry Pis, which are part of a big Cloud-native application. Wouldn’t it be cool to be able to dynamically use some of them as sensors or Bluetooth beacons? That is something we can do now.

Fig 4: Cloud with Bluetooth capable devices

Possible usecases for a Bluetooth microservice sensing applications, for example to measure the density of a crowd. It also makes it possible to contact devices that have no Internet-connectivity, but can use Bluetooth. Many other applications are imaginable, such as energy efficient close range monitoring that does not use-up network bandwidth, or fallback mechanisms if for instance the WiFi connection fails.

Finally, we’ve reached a point where native just does not cut it anymore.