Building a keyboard with Elixir
At the amazing ElixirConf in Prague last year I’ve heard firsthand how Nerves is being used and how well it works with Raspberry Pis. The talks I’ve attended enticed even more my curiosity, so it was just a matter of time until I bought a Raspberry Pi Zero and started thinking about a cool idea to try out the Nerves project.
The Nerves project provides an easy way to develop and deploy embedded software written in Elixir. You can rely on the Elixir friendliness and on the battle-tested BEAM to run your device, instead of going with the “traditional” approach, where you code in C and run your firmware on bare metal.
Embedded 101
In this post I’ll try to describe Nerves in a nutshell, but please check the project thorough and well-crafted documentation.
Imagine that you have a “normal” (i.e. not embedded) Elixir application ready to be deployed. You’d probably create an Erlang/OTP release (check Distillery, or the recent mix release introduced by Elixir v1.9 ⚗) that contains everything needed to run your application and deploy it somewhere. You don’t even need to have Elixir (nor Erlang) installed to run your release. An Erlang/OTP release is your application with batteries included 🔋.
The previous description overlooks an important detail: we’re assuming the release will run on an equivalent environment. If you want to run your application on a different architecture, say ARM64, you’d need to build your release targeting this particular architecture. If you’re using a x86 machine, you need to cross-compile your application so it can later run on an ARM CPU. To do this you’ll need a set of development tools (called a toolchain) that produces ARM64 binary code.
Correctly configuring the build environment is possible but not trivial, you need to install and configure the right combination of dependencies. That’s when Buildroot enters the scene and saves the day. It allows you to easily build Linux systems for other architectures, from the comfort of your local development machine.
At this stage you may be asking “Wait, what? Building Linux? With Buildroot? Where is Nerves 😕?”. All the above complexity is handled for us by Nerves 😃.
Nerves
Nerves provides us with a streamlined Buildroot-based Linux for different target architectures (platform), a library of Elixir modules that lets you perform common tasks (framework) and a set of CLI tools to build and update your firmware. This way you can focus on the development of your Elixir application, knowing that everything else is handled by Nerves for you 💪.
Let’s understand this by creating a simple Nerves project targeting the Raspberry Pi Zero. After installing Nerves, create your first Nerves project with the nerves.new generator:
Insert the card in the Pi Zero, export the MIX_TARGET=rpi0 environment variable, get the specific Pi Zero dependencies with mix deps.get, and then run mix firmware and mix firmware.burn, to build and write your firmware to the MicroSD, respectively.
Then, insert the MicroSD card on the Pi Zero and use a micro USB cable to connect your laptop to it. We’re now able to access an IEx shell running on the Pi Zero through SSH. How cool is that 😎?
At first sight, it seems simple, but Nerves did a lot for us. It got the dependencies, fetched the specific streamlined Linux image for Pi Zero, built a release with our minefield OTP application and wrote it to the memory card! And everything done with mix tasks, no need to use a myriad of different tools.
USB Gadget
Have you wondered how the SSH connection worked in this scenario? We were able to connect to the Pi Zero via SSH because the default Nerves Pi Zero Linux image (given by the rpi0 target) configures a virtual Ethernet port over USB. When the Pi Zero boots, it configures a usb0 network interface and starts the SSH daemon.
We can provide different functionalities over USB: a Serial or Ethernet adapter; a HID device, like a keyboard, mouse or joystick; and even a mass storage device, like a pen. We were able to SSH to the Pi Zero because the default Linux image provided by Nerves that we’re using already has configured the g_cdc kernel module. This module provides the Ethernet adapter over USB that we’ve used for the SSH connection.
Remember that we want to build a keyboard? Having the g_cdc module loaded for us is a problem, because it configures the Ethernet adapter and doesn’t allow us to configure any other gadget over USB. What we want is a way to easily configure a custom HID gadget, in this case a keyboard 🔳. And this is exactly what the USB Gadget ConfigFS provides: it lets us create and configure arbitrary USB composite devices from user space.
Configure the custom Nerves image 🔨
Now that we know that we need to customize the Pi Zero Nerves image, let’s create a custom Linux image where we remove the g_cdc Kernel module and add the USB Gadget ConfigFS one. We’ll follow the excellent Nerves documentation on customizing an existing Nerves system (in our case, we’ll start from the “vanilla” Pi Zero Nerves system).
After making sure our custom Pi Zero Nerves system builds without any significant changes from the official Pi Zero Nerves system (mix compile on the custom Nerves system folder works as expected 🎉), we’ll now remove the g_cdc module and add the USB Gadget ConfigFS support.
The linux-menuconfig command will present us a menu that allow us to configure how the Buildroot Linux system will be built. Navigating to Device Drivers->USB Support->USB Gadget Support we can see the following:
We’ll now disable the CDC Composite Device entry and select the HID function inside the USB Gadget functions configurable through configfs:
After exiting from the linux-menuconfig, we persist the configuration change we just made with make linux-update-defconfig:
We’re done with the nerves.system.shell. The changes we made are reflected on the modifiedlinux-4.19.defconfig file:
Note: You can check this repository with the custom Pi Zero Nerves system already configured and ready to be used.
Nerves network 🔌
Before we jump to our keyboard Proof of Concept, we need to configure the nerves_network library (remember to add it to your mix.exs dependencies first) and remove the nerves_init_gadget configuration, since the former will allow us to connect to the Pi Zero via wi-fi and the latter configuration isn’t needed anymore (because we just removed the Ethernet via CDC composite device).
At this stage, we have the custom Nerves system that lets us configure the HID USB gadget and we’ll be able to SSH into our Pi Zero via wi-fi, instead of having to connect to it through the USB virtual Ethernet port provided by the :nerves_init_gadget library.
Using ConfigFS to configure our keyboard 🎹
ConfigFS provides a virtual filesystem for us to configure new USB gadgets. Because we are now loading the USB Gadget ConfigFS module, when the firmware runs on the Pi Zero there will be a /sys/kernel/config/usb_gadget folder that we can use to configure our keyboard.
Now it’s just a matter of creating specific folders and files inside the usb_gadget folder to configure the keyboard. You can check this approach on the ManualConfig module here, but I’ve ended up using the usb_gadget library that is currently under development. This library belongs to the Nerves project and simplifies a lot the configuration of the HID gadget (even if behind the scenes the library is interacting with the virtual filesystem in the exact same way):
We now have everything we need to start sending USB key presses from the Pi Zero to the host!
Sending key presses with Nerves 👉 🔳
Before we actually send key presses, let’s quickly understand how USB Human Interface Devices (HID) work. Every device (like our keyboard) needs to send a report whenever it wants to communicate that a key was pressed. The report structure for keyboards has 8 bytes, where the first byte is used to communicate the modifier states, and the third to eighth byte are used to communicate a given key being pressed (we don’t use the second byte since it’s reserved).
So if we want to send an a character, we need to send a report with the value 0x04 on the third byte, and zeros on all the other report bytes:
00 00 04 00 00 00 00 00You can find a list of all scan codes below:
Armed with this knowledge and after configuring the Pi Zero as a keyboard device using ConfigFS, we can send key presses to the host by simply writing to the device using File.write/1:
keypress_a = "\0\0\x04\0\0\0\0\0"File.write("/dev/hidg0", keypress_a)
If we want to send a capital A, we would send a shift in the modifier (first) byte alongside the a on the third byte:
keypress_a = "\x02\0\x04\0\0\0\0\0"File.write("/dev/hidg0", keypress_a)
Each bit of the first byte corresponds to a modifier, ie., the first bit corresponds to the left Ctrl, the second bit to the left Shift, etc. This is why we send 10 as the first byte value in the last example.
You can find the raw_write/2 and raw_write_and_release/3 functions here. Be mindful that these functions are using the File.write/2, but to write as fast as possible to the device (like we should aim for a keyboard), we should instead use File.open/2 and the IO.write/2 combo 👊.
Hello, world! 👋
Armed with the above knowledge, and combining this with the Raspberry Pi Zero GPIO (general-purpose input-output) pins, we can send USB key presses with the click of a button 🔘. In the video below, we used the GPIO pins to detect that a switch was closed, hence we turned on the red LED and sent the a key press to the host:
In the next article we will see how to configure the GPIO pins and scan a keyboard matrix to detect key presses in a regular keyboard.
Hopefully by now I’ve piqued your curiosity, so please check out Nerves and see for yourself how enjoyable is developing embedded software with it.
In this article we’ve laid down the foundation of an important part of any keyboard firmware: sending key presses. On the next post we will continue the work towards an Elixir-powered keyboard. Stay tuned ⌨️
