Running ROS workspace tests with GitHub Actions in 3 Steps

Docker, GitHub Actions and ROS logos

In this article I will explain how you can:

1. Setup GitHub Actions
2. Build a ROS workspace in a Docker image
3. Run all ROS tests and get a necessary exit code

So without further ado, let’s begin.

Step 1

Good code has good tests. Ideally tests which are run automatically and the failure of which prevents merge. Enter GitHub Actions. With just 12 lines saved in .github/workflows/run_ros_tests.yml we can achieve this.

name: ROS test runner
on: [push]
jobs:
  run_ros_tests:
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v2
      - name: Build the docker-compose stack
        run: cd ros_ws && docker build .

So what does this do? It tells GitHub Actions that every time we push, we want it to create an ubuntu-20.04 environment, then run the command:

cd ros_ws && docker build .

Which simply moves into our relevant directory, builds a Dockerfile.

It’s that simple. But for this to work we will obviously need a Dockerfile, so let’s have a look at that.

Step 2

Docker is an incredibly powerful tool for developing on and sharing between different systems regardless of environment. Here we’re going to leverage it for both local testing and in GitHub Actions. The Dockerfile for this repo looks like this.

FROM ros:noetic
RUN apt-get update
RUN apt-get install python3 python3-pip -y
RUN apt-get install ffmpeg libsm6 libxext6  -y
RUN apt-get install ros-noetic-cv-bridge  -y
COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
COPY . /ros_ws
WORKDIR /ros_ws
RUN bash bash_scripts/run_and_check_tests.sh

Those of you familiar with Docker won’t see anything here to surprise you, but let’s break it down a few lines at a time.

FROM ros:noetic

All docker images need a base image from which they build, as this is a ROS project, we are going to use the ros:noetic base image.

RUN apt-get update
RUN apt-get install python3 python3-pip -y
RUN apt-get install ffmpeg libsm6 libxext6  -y
RUN apt-get install ros-noetic-cv-bridge  -y

Although the ROS image saves us a lot of issues with setting up the ROS environment, it doesn’t come with many of the programs we are used to developing with and need for the system to run, so we can download them here with apt-get .

COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip install -r requirements.txt

We copy the file requirements.txt , which contains a list of the python packages we need, and we copy it to inside the docker image. Then we simply upgrade pip and install the packages listed.

COPY . /ros_ws
WORKDIR /ros_ws

We copy the whole directory we are in to a directory we create inside the docker container called ros_ws and set that directory as our working directory.

RUN bash bash_scripts/run_and_check_tests.sh

Last but not least, we run a bash (or actually shell) script which will be the subject of our next section.

That’s it. We’ve Created a ROS image, downloaded the packages and programs we need for our workspace to run, copied in our workspace and run our tests from a bash script. Let’s look into that bash script.

Step 3

Running ROS tests can be done pretty easy with catkin_make run_tests provided that a few things are taken into account.

• Tests to be run with rostest are presented in the CMakeLists.txt with add_rostest(path/to/test.launch)
• Tests to be run with pytest, unittest or your test runnner of choice are presented in the CMakeList.txt with catkin_add_nodetests(path/to/test.py) . It is also worth noting that each of these files needs their own cakin_add_noetests , if they are grouped, only the first will be run
• We can get a summary of the test results with catkin_test_results
• Finally it should be noted that unlike other test runners, catkin_make run_tests will always exit with a 0 exit code, even if tests fail

So with these things taken care of or in mind, lets look at run_and_check_tests.sh

#!/bin/bash
declare file="test_results"
declare regex=" tests, 0 errors, 0 failures, 0 skipped"
. /opt/ros/noetic/setup.bash
catkin_make
catkin_make run_tests
catkin_test_results > "${file}"
declare file_content=$( cat "${file}" )
if [[ " $file_content " =~ $regex ]]
  then
    echo "All Tests Passed"
    exit 0
  else
    echo "Not All Tests Passed"
    exit 1
fi

Once again let’s break down what we’re doing here step by step.

declare file="test_results"
declare regex=" tests, 0 errors, 0 failures, 0 skipped"

We declare a couple of strings for later. regex fits the pattern ofcatkin_test_results when all tests have run and passed.

. /opt/ros/noetic/setup.bash
catkin_make
catkin_make run_tests
catkin_test_results > "${file}"

We then source the ROS setup file, build the ROS workspace, run all the tests presented via the various CMakeList.txt files, then finally save the results in a file.

declare file_content=$( cat "${file}" )
if [[ " $file_content " =~ $regex ]]
  then
    echo "All Tests Passed"
    exit 0
  else
    echo "Not All Tests Passed"
    exit 1
fi

We get the content of the results file we just created as a variable in our script, and check if it contains our regex value. If it does our tests have passed and so we exit with code 0, otherwise at least one of the tests failed, had an error or wasn’t run and so we exit with code 1.

That’s all folks. We setup GitHub Actions to build a Docker image, which downloads the necessary programs and packages, builds our ROS workspace, runs all of the tests and exits with a 0 if they all pass or a 1 if they don’t. Thanks for reading.