Using GitHub actions for integration testing on a REST API
In this post, I’m going to show you how to configure a GitHub action as a continuous integration pipeline for a REST API.
Motivation
Andrés, a friend of mine, gave a talk about testing REST APIs, using Postman’s test scripts. For his talk, He decided to include a demo which uses an aside project I have. This project is small and it’s for personal use only (I still need to improve a lot of things), but receiving help for adding some integration tests was something I could not let pass. So I joined efforts with Andrés, and while he worked on writing the tests I worked on getting a basic CI pipeline for showcasing them.
The Project
AHM (Application for hypertension monitoring) is a small project I have been working on for a couple of months now. It aims to provide an easy tracker interface for my blood pressure measurements (blood pressure readings: systolic and diastolic). I started this project with serverless and Golang in mind, but due to an invitation I got from the EDTeam (for which I had to record a Python Course), I then switched it to Python.
AHM service is far for being production-ready, but regardless, you can get familiar with its architecture and implementation checking the repository’s README. For this post, I rather want your attention to be only on a couple of endpoints that the API offers:
The first and third endpoints from the image above, are the ones that the integration tests will cover to a limited extent. Before moving onto the tests themselves, it is also important to be aware of the stack on which the API will run:
- MongoDB: It’s in MongoDB’s documents where the data (measurements) will be.
- Python3 with Flask-RESTful: This great Flask based framework helps me in guaranteeing that the application is REST compliant.
The Tests
Let’s start this section by defining what integration tests are. Software testing fundamentals localize integration testing as the next layer of testing upon the unit ones. They define them as:
A level of software testing where individual units are combined and tested as a group. The purpose of this level of testing is to expose faults in the interaction between integrated units.
The units part I want to combine and test are basically the data layer (the database) and the API layer. Therefore, by having integration tests in place, what I want to validate is that data is being retrieved from the database to the API Layer, and that data is being sent from the API Layer to be stored in the database.
For the scope of Andres’s talk, there were only two endpoints covered:
- GET /v1/measurement: This endpoint lists the latest 10 readings recorded by a user.
- POST /v1/measurement: This endpoint allows us to create a reading for a user.
The tests were implemented using Postman’s Scripts, which allows you to define tests based on the response object provided by the JavaScript’s API that Postman offers. And together with newman, the tests can be run via CLI, meaning you don’t need to have Postman installed for running the collections written with Postman tests scripts.
As it can be seen in the two previous images, the tests written were rather simple and demonstratives. For production-ready applications, more exhausts and edge cases covering tests will for sure, be required. Once you get enough tests you can export the Postman collection together with the Postman environment file and run them via CLI. For instance, if you name the exported files as ahm_api_calls.json and ahm_local.json, with Newman already installed, you can run a command similar to:
newman run ahm_api_calls.json -e ahm_local.jsonYou can check the exported collections I used (I took them from Andrés talk) for the pipeline inside the postman folder.
The Pipeline
With the tests in place, and a mechanism for running them via CLI, all it was left to work on, was the actual Pipeline. I could have selected a fresh installation of Jenkins or a SaaS CI version like Travis, but I decided to go with GitHub actions because I wanted to learn how it works and I wanted to learn for which use cases it can be used (As CI for instance). Besides, the source code was already hosted on GitHub, so when you read something like:
GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.
It immediately clicks in your brain (at least in mine) that you can have the benefits of automation right there, in the same place where your code is hosted.
The targeted CI pipeline will be a Github Action Workflow, which can be defined as a YAML file that needs to live inside your GitHub’s repository root folder, in the directory .github/workflows. Pretty much like everything that automates infrastructure nowadays, you’d need to specify inside that YAML, which steps need to be executed for your pipeline to be considered as done. Another important feature is that within a Github workflow, any step may use Docker, so virtually, you’d would able to run any technology that runs already on Docker. Moreover, there is already a Marketplace with tons of actions (steps) that are ready for being used. In fact, the pipeline I built, uses a few of them.
Let me now share the final pipeline I came up with, which allows to run Postman Scripts on a Python REST API:
“On” section
Controls when the action will run. Triggers the workflow on push or pull request events but only for the master branch.
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]This section is kind of self-explained; the whole pipeline will be triggered only when a push or pull request to the master branch happens.
“Build” section
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x]
mongodb-version: [4.2]
python-version: [3.7]A workflow run is made up of one or more jobs that can run sequentially or in parallel.
Here I’m defining on which operating system the workflow will be run, as well as the matrix version of all the subsystems that the integration tests will use. What is amazing about this, is that with the same pipeline, I can test how my application will behave with different versions of MongoDB and different versions of Python. For this example, I’m using only one version of each subsystem, but really nothing stops me from extending the matrix to include other versions.
“Steps” section
Steps represent a sequence of tasks that will be executed as part of the job.
Within the steps section is where the fun happens. It’s here, in where each specific unit of execution needs to be defined.
System dependencies:
- name: Git checkout
uses: actions/checkout@v2- name: Install Node JS ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}- name: Start MongoDB ${{ matrix.mongodb-version }}
uses: supercharge/mongodb-github-action@1.3.0
with:
mongodb-version: ${{ matrix.mongodb-version }}- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
These four initial steps take care of installing all the subsystems dependencies required for running the integration tests. In other words, at the end of these steps, I will have an Ubuntu machine ready for being used with the version of Node JS (for running Newman), MongoDB (for the data), and Python (for the API) specified in the matrix version. Something to notice is that none of those steps were implemented by myself, I did borrow them (check the uses command) from the Github Marketplace.
Package dependencies:
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
if [ -f api/requirements.txt ]; then pip install -r api/requirements.txt; fi- name: Install Newman dependency
run: npm install --prefix tests/postman/
The next two steps defined in the workflow, take care of the package dependencies installation, for both the API written in Python, and for the Newman package that lives on the NPM registry. As I’m not using any third party steps but my owns, there is not uses section but instead, there is the run section, in which one can put bash instructions for defining the step itself.
Executing the tests:
- name: Run the API and Postman's tests
run: |
cd api && flask run &
sleep 2
cd tests/postman/ && ./node_modules/newman/bin/newman.js run ahm_api_calls.postman_collection.json -e ahm-local.postman_environment.json
kill -9 `lsof -i:5000 -t`
env:
FLASK_ENV: development
API_HOST: 0.0.0.0
FLASK_APP: main.py
DB_NAME: ahm
DB_HOST: localhost
CI: trueThis was the hardest step to implement, and I have to admit, the strategy I used for it, is hacky (but if you can think in something better, let me know). Why is that? Well, all the steps in my pipeline need to be run sequentially, one needs to finish before the other one can be started. That brought me to the question: How can I then run the API and run the tests on independent steps? I could not find an elegant solution for it, so I decided them to run both the API, and the tests within one step, and to use my old friends, the sleep and kill commands for managing a smoothly step termination. The idea behind this step was:
- I run the API in the background, and wait for 2 seconds, giving it enough time to boost and run.
- I then run the collections via Newman. If they fail, the step will be considered failed, and the error will show up on the Github UI.
- If the tests pass, I now need to kill the API so the step can be considered as successfully ended.
Not the most elegant solution of course, but at least, a solution that can be explain it with enough simplicity. Something additional to notice, is that a step can also inject variables in the environment, so all the ones that the API needs, were passed statically via the env section.
The pipeline in action:
If you want to check one of the logs coming from the implemented pipeline you can revise the actions page for the repository:
In there, you can notice all of the steps that were executed and what the results were for each of them. There are also plenty of failed runs that you can revise and learn from them; ones because of security (I don’t want to reach the free quotas) and others that were run while I got the final implementation.
Conclusions
GitHub Actions can be used for building CI and CD pipelines upon Github repositories. It is even possible to include integration testing, which implies having all subsystem dependencies in place (installed and running).
As a TL, if your CI pipeline is “simple enough”, you don’t really need to go and implement it outside of Github, all the software life cycle can be now implemented within the same space with Github actions, which I think is great, because your focus and energy could be localized on one tool only.
I also believe this can give some level of freedom to the QA automation crews, which more often than not, find themselves begging to their TLs for getting some sort of sandbox that allows them to run integration or end to end testing.
With the amount of actions already implemented and available on the marketplace, most of the regular use cases should be already covered, so try finding an action first, before implementing any on your own.
Finally, GitHub actions might be a nice way of getting started with Open Source. I, for instance, filed a feature request for one of the actions I used in my pipeline: https://github.com/supercharge/mongodb-github-action/issues/12.
