When I created a robot 15 years ago out of PVC tubes, acrylic plates, a very heavy laptop and my mother’s yoga mat, that was tasked with collecting brightly coloured balls but was more interested in plain white walls, I could have only dreamt of the Computer Vision possibilities crammed into affordable credit card sized boards available nowadays. Google’s Coral board is one of these, equipped with a Tensor Processing Unit (TPU) that is optimised to run modern deep neural networks very efficiently.
In my previous post I described how we use the Odroid-XU4 as the main compute of our humanoid robots in our RoboCup team. This is another quite capable little board, able to run some deep(ish) neural networks at a reasonable frame rate and resolution (as we showed in our 2018 paper), but the complexity of the models it can handle is still very limited compared to the current state of the art in computer vision. That is why we have started experimenting with the Coral board, to give the perception system of our robots a big boost by using its Edge TPU.
The first step towards this was to get our middleware of choice running on the board: ROS 2. The second Long Term Support version of ROS 2, codenamed Foxy Fitzroy, has recently been released with a lot of improvements, so we decided to get that installed. Whereas with previous versions we were able to build the system directly on the board itself (although it needed a full night to complete), Foxy has become too big and the board runs out of memory when compiling the ROS 2 C++ client libraries. This meant we had to cross-compile the whole thing instead, for which luckily some tools are available, but this method does require some special steps. I will give a run through of those here so others can enjoy the combination of these platforms as well.
Have your board ready
I assume you have gone through the official getting-started guide and successfully flashed the board, have it connected to your local network and reachable from the machine you will do the cross-compilation on, and have initial access set up with the mendel development tool (
mdt), being able to log into your board with
mdt shell. Later on we want to use SSH without
mdt, which should be possible by using the key that it created on first log in; try it out by running:
$ ssh -i ~/.config/mdt/keys/mdt.key firstname.lastname@example.org
My board called itself
deft-llama, replace that name with the name of yours throughout this article.
We will use the ROS cross-compile tool to save us from having to set up a whole cross-compilation toolchain ourselves. It works by running the cross-compilation inside of a Docker container on top of the QEMU emulator. Not the most straightforward way… and not really the fastest either… but it makes the process relatively easy for us. Follow their instructions to install Docker (don’t miss the bit about the
docker group!), and then install QEMU and the cross-compilation and other tools we will be using:
$ sudo apt install qemu-user-static python3-pip curl git rsync
$ pip3 install ros_cross_compile vcstool
I’m assuming you use Ubuntu, if not then you should be familiar with finding and installing these packages on your flavour of Linux. Finally, make sure you have a few gigabytes of free disk space available to cater to the docker environment and compilation output that will be created.
Next we have to fetch ROS 2’s source code, and prepare some other files that will be used to control and configure the cross-compilation process. Start with creating some main directories:
$ mkdir -p ros2-crosscompile/ws/src
$ cd ros2-crosscompile
$ mkdir custom-data
ROS 2’s codebase is distributed over a whole bunch of Git repositories. We have to fetch a file with a list of them and their tags to use, and then use
vcstool to clone them all in one go:
$ curl -O https://raw.githubusercontent.com/ros2/ros2/foxy/ros2.repos
$ vcs import ws/src < ros2.repos
Unfortunately this pulls in some stuff that has dependencies that are not available and that we don’t want to use right now anyway. When running the whole process later on
rosdep is used to install all dependencies and we have to tell it to skip the missing ones to not break. Create a dependencies file that replaces these with empty lists: create the file
custom-data/skip.yaml and edit it to have the following contents:
Next, create the file
rosdep-script.sh (directly inside the
ros2-crosscompile directory) and fill it with the following commands that will be run to put
skip.yaml into the right place and ensure it gets picked up:
mkdir -p /etc/ros/rosdep/custom-rules
cp ./custom-data/skip.yaml /etc/ros/rosdep/custom-rules/skip.yaml
echo "yaml file:/etc/ros/rosdep/custom-rules/skip.yaml" > /etc/ros/rosdep/sources.list.d/10-skip.list
On top of that we will also disable building some packages to reduce the build time a bit, including the Rviz2 and RQT visualisation tools. This is not necessary and you can skip this, but you would probably want to run those tools on your host machine instead of directly on the Coral board. The following commands create empty marker files that tell the Colcon build tool to skip them:
$ touch ws/src/ros2/rviz/COLCON_IGNORE
$ touch ws/src/ros-visualization/COLCON_IGNORE
$ touch ws/src/ros2/system_tests/COLCON_IGNORE
The final file to prepare is
ws/defaults.yaml, which will be used by Colcon to create a compact and optimised release build. Edit it to read:
Phew, that’s the hard bit done!
Do the actual cross-compilation
Now you are ready to kick off the cross-compilation process:
$ ros_cross_compile $(pwd)/ws \
--arch aarch64 \
--os debian \
--rosdistro foxy \
--custom-rosdep-script $(pwd)/rosdep-script.sh \
If all went well this will compile the whole of ROS 2 Foxy Fitzroy for the Coral dev board — or actually for any 64-bit ARM platform. This will take a good amount of time, so this is as good a moment as any for grabbing some dinner, replacing all the screws on your robots, or read your favourite web comic.
Copy to the board & test
When the compilation is finally done, the results of all this work is available in the
ws/install_aarch64 directory. The complete ROS 2 system is contained in there, so we can copy the whole thing over to the board now. We will use
rsync, ensuring the right SSH key is used:
$ rsync -arzP \
-e 'ssh -i ~/.config/mdt/keys/mdt.key' \
Now SSH into the board and ensure that all system dependencies are properly installed on the board:
$ ROS_OS_OVERRIDE=debian:buster rosdep --os debian:buster update
$ ROS_OS_OVERRIDE=debian:buster rosdep --os debian:buster install \
--from-paths install_aarch64/share \
--rosdistro foxy -y \
--skip-keys "console_bridge fastcdr fastrtps osrf_testing_tools_cpp poco_vendor rmw_connext_cpp rosidl_typesupport_connext_c rosidl_typesupport_connext_cpp rti-connext-dds-5.3.1 tinyxml_vendor tinyxml2_vendor urdfdom urdfdom_headers tango_icons_vendor cyclonedds"
Now you are (finally!) ready to try out the standard talker/listener demo on your Coral board:
$ source install_aarch64/setup.bash
$ ros2 launch demo_nodes_cpp talker_listener.launch.py
[INFO] [launch]: All log files can be found below /home/mendel/.ros/log/2020-07-07-15-53-05-727808-deft-llama-12179
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [talker-1]: process started with pid 
[INFO] [listener-2]: process started with pid 
[talker-1] [INFO] [1594137187.273531702] [talker]: Publishing: 'Hello World: 1'
[listener-2] [INFO] [1594137187.279673904] [listener]: I heard: [Hello World: 1]
[talker-1] [INFO] [1594137188.272416191] [talker]: Publishing: 'Hello World: 2'
[listener-2] [INFO] [1594137188.275080792] [listener]: I heard: [Hello World: 2]
If the output looks anything like that, you’re done!
Where to go from here
Now you naturally want to do something much more advanced with the board than printing some text… If you develop your own ROS 2 packages for the Coral board you can choose to cross-compile them similarly to what we did here for ROS 2 itself. However, unless you make things too complex, a workflow where you just compile your packages on the board itself can work fine as well. Because the OS is basically just Debian you can install all the build and development tools straight onto the board.
But of course you want to use the TPU! We have been building a ROS 2 package to work with Tensorflow Lite, including its tools and operators to take full advantage of the Edge TPU. It currently supports image classification, object detection and semantic segmentation. You can check out our repository to give it a try (also without a TPU):
Watch this space for more How-Tos and updates about this setup in the future, and we’d love to see how other people use these tools, so do drop a comment with how you’re getting along with it when you give it a try!