Sega Mega Drive vs CI Pipelines

When you think about Continuous Integration (CI), what usually comes to mind are web applications and automated tests. In fact, if you are a web developer and do not write any tests, it is high time you started working on them!

A CI pipeline allows you to check the health of your code and prevent obvious mistakes from entering your code repository. It also helps reviewers save time, as they don’t need to point things out that are found by automated tests.

However, not many people see the point of using CI pipelines beyond web applications — but the truth is, they can be used for anything. So how about we use them for… Sega Mega Drive / Genesis homebrew development?

Here at Good Praxis at the beginning of the pandemic, we decided to create a Mega Drive game, in collaboration with YRS TRULY, which would work on real hardware. This meant not writing a game that looks retro, but that actually is retro — however this does not mean we can’t use modern tools, like GitLab’s CI pipeline.

The game we created is called ASAP PLZ, and is a WarioWare-esque take on what it’s like to work at an office. You are given different small tasks that you need to complete as soon as possible.

Title screen from ASAP PLZ

You may be wondering, in terms of Continuous Integration, where do you even begin when it comes to software written for a 32+ year old gaming console? Well, you start by listing your goals; we want to make sure that:

• the software/game builds successfully
• the code commit contains all necessary files
• the software/game can boot
• there are no major visual glitches after booting

To build our game, we used SGDK, a software development kit (SDK) created by Stephane Dallongeville which allows you to use C to write software, instead of resorting to low-level assembly code. This, arguably, allows the code to be more readable, which makes reviewing a lot easier. The SDK contains both the library which abstracts a lot of the console’s functionality, and tools that allow us to compress and prepare assets to be stored in the ROM.

Since we need to compile our code, we obviously have a compilation step in our pipeline. This step can detect syntax errors, issue warnings and check if all referenced files exist. It may sound trivial, but every now and again we do forget to commit a file, which can cause confusion. If our code can compile, this means that, at least on the surface, everything is in order.

The asset tools, which are part of the kit, will also check if our asset definitions are correct, and the files provided have appropriate properties.

All of this means that we can fulfil our first two goals, which is making sure that the game builds and that all required files are there. But how do we actually achieve this?

To work with SGDK, we need the kit compiled and ready. You can download a precompiled version for Windows, but since we want to run our pipeline on a Linux system, we need to compile the code ourselves. But surely we don’t want to build the SDK every time we want to build and test our own code?

Here’s where a custom Docker image comes handy. We wrote a Dockerfile (and a Vagrant file) which installs all the prerequisites and then builds the SDK for us. It looks like this:

FROM ubuntu:18.04
RUN apt-get update
RUN apt-get install -y git build-essential texinfo curl wget unzip openjdk-8-jdk
RUN git clone https://github.com/kubilus1/gendev.git && cd gendev && make && make install
ENV GENDEV="/opt/gendev"

Having the ability to use a pre-built image saves us a huge amount of time, as compiling the kit takes quite long.

We can now reference our image in our gitlab-ci.yml file:

stages:
- build
build:
  stage: build
  image: maciekbaron/sgdk:latest
  script:
  - make -f $GENDEV/sgdk/mkfiles/makefile.gen clean all
  artifacts:
    paths:
    - out/
    only:
      refs:
      - master
      - merge_requests

The above definition uses our prebuilt image (maciekbaron/sgdk:latest) and builds our ROM. It is then saved as an artifact, meaning that once we run our pipeline, we have a built ROM ready to download and test in an emulator!

Our pipeline’s output after successfully building a ROM

So far so good. We can automatically compile and build our game. People reviewing the code can both look at the code and run the game to see if everything is in order. But what if we want to automatically check if the game boots, and see how it looks like? Surely that sounds impossible, the server running our pipeline doesn’t have a screen!

Have you ever heard of “headless browsers”? They are often used in automated tests, simulating a web browser environment to perform functional tests on web applications. There are solutions out there like Cypress which allows you to take snapshots of your site, and even do things like visual regression tests to see if there are major visual differences between builds.

We can create our own solution that does just that: grab the ROM, open it in an emulator, and then take a screenshot, all on a GitLab runner. For this purpose, we created another Docker image which has Xvfb and RetroArch preinstalled.

Xvfb is an in-memory display server which allows you to run software without having a physical display. What is great about it is that we can also take screenshots of applications running, meaning we can see our game in action.

RetroArch is an open-source cross platform frontend for emulators which has been ported to many different platforms. Since it is well supported, it is a perfect candidate for us to run our game.

Having all of the pieces of the puzzle together, we can write a test stage like so:

test:
  stage: test
  image: maciekbaron/retroarch:latest
  artifacts:
    paths:
    # Make sure we store the screenshot we are about to take
    - screenshot.png
  script:
  # Initialise Xvfb
  - Xvfb :1 -screen 0 1024x768x24 </dev/null &
  - export DISPLAY=":1"
  # Sleep 3 seconds to make sure Xvfb is running
  - sleep 3
  # Run retroarch using the Genesis Plus GX core on out/rom.bin
  - /usr/bin/retroarch -L /usr/lib/x86_64-linux-gnu/libretro/genesis_plus_gx_libretro.so “out/rom.bin” &
  - sleep 10
  # Take a screenshot after 10 seconds
  - import -display $DISPLAY -window root screenshot.png
  only:
    refs:
    - master
    - merge_requests

Let’s discuss what’s happening here.

We first initialise Xvfb and assign it to a display. We also set our preferred resolution and color depth. We then export the display identifier for later use. Since we couldn’t find a reliable way of checking whether Xvfb, we sleep for 3 seconds to make sure everything is up and running.

Now that we have our virtual screen, we are ready to run an emulator. We call RetroArch and pass our output ROM to it. We wait 10 seconds, which should be enough time for the game to load and reach the title screen, and we finally take a screenshot. Notice that in the artifacts section we indicate that we want to keep the generated screenshot.

Our screenshot is available as an artifact.

Screenshot generated by our pipeline

And there you have it! Our Sega Mega Drive / Genesis game gets automatically built from source, tested and run in an emulator to generate a screenshot that we can then review.

But this is just the beginning, you can also simulate keypresses using xdotool and generate multiple screenshots showing different states of the game.

No more excuses for not using Continuous Integration!