Automate Testing With Gitlab Pipelines

How to use Gitlab pipelines to automate testing and display test coverage on a badge.

Photo by Rodion Kutsaev on Unsplash

Introduction to Gitlab pipelines

Gitlab pipelines are a great way to automate certain repetitive tasks during development cycles. Gitlab offers a very flexible environment to define and execute chained scripts in the form of pipelines. Besides, developers can choose under which circumstances pipelines get triggered. For instance, you can execute pipelines after committing to a certain branch.

By the way, do you have certain variables that have to stay out of the repository for security reasons? No problem. You can store those variables in Gitlab and call them during the pipeline execution.

Sounds great, isn’t it? Let’s see an example. This post will show how to create a simple Gitlab pipeline that runs the repository tests, saves the coverage, and displays the results in a badge. Let’s get started!

The use case

We will work with the following use case to illustrate how to create the pipelines:

• We have a Python repository in Gitlab and have developed a series of unit tests with pytest that we want to run every time we commit and push our code.
• We are also using coverage to measure our tests’ coverage of the codebase. We will report the coverage in a badge.
• Our code depends on some variables that we must keep out of the repository for security reasons. Those variables are defined in a file called config.ini .

Agenda

A summary of what we need to do:

1. Create the jobs we need to run under a Gitlab pipeline.
2. Securely save the environment variables and files our codebase depends on.
3. Store the test coverage after the pipeline execution.
4. Add a Gitlab badge to the repository to display the test coverage.

1. Create a Gitlab Pipeline

To define a Gitlab pipeline, we need to add a YAML file called .gitlab-ci.yml to our code repository's root folder.

The default section of the file defines the Docker image to use. Gitlab will pull and build that image and will execute the pipeline on it. The next sections outline the set of jobs to execute. Bellow an initial version of the file we will be using:

default:
    image: python:3.8
    before_script:
       - apt-get update
       - apt-get install -y python3-pip
       - pip install -r requirements.txt
stages:
    - test
    - dummy
test:
    script:
        - coverage run -m pytest
        - coverage report
dummy:
    script:
        - echo "Pipeline finished"

• image : Defines the base Docker image to pull—a python image in this example.
• before_script : Set of commands to execute before the jobs get triggered. It specifies a common configuration for all subsequent jobs.
• stages : Determines the job execution order in the pipeline.
• test : Name of the first job of the pipeline.
• script : A job’s set of commands. In this case, we are running our unit tests and reporting the coverage.
• dummy : A final job was added to illustrate how to list multiple jobs in the stages section.

By default, the pipeline will be executed every time a commit in any branch is pushed to the repository. You can check the executed and running pipelines in the CI/CD section of your project.

View of jobs in a Gitlab project

2. Add environment files and variables.

In this case, our tests and code depend on variables defined in a file that is not part of the repository for security reasons. To make that file available during the pipeline execution, we have two options:

1. To store the file itself as an environment variable of type “file” in Gitlab and place it in its correct location during the pipeline execution. Our preferred option in this case.
2. To store the file content as an environment variable in Gitlab and dump its content into a file during the pipeline execution. Also valid, although I find it easier to move a file (option 1) than using obscure bash commands to find and replace variables in configuration files.

To store environment variables or files, go to your project Settings > CI/CD > Variables:

Navigate to CI/CD settings to add variables

Click on the “Add Variable” button and add the configuration file's content in the field “Value.” Select “File” as a variable type, and use the field “Key” to name your variable.

These variables will be available in the Gitlab runner that executes our testing pipeline as environment variables. Those of type files will point to a temporary path where Gitlab stores the file. We simply need to copy or move that file to the location we need the file to be, which we can be done in the .gitlab-ci.yml .

default:
    image: python:3.8
    before_script:
       - apt-get update
       - apt-get install -y python3-pip
       - pip install -r requirements.txt
       - cp "$CONFIGFILE" "$(pwd)/config.ini"
test
    script:
        - coverage run -m pytest
        - coverage report

Note the bold line added to copy the file, stored as an environment variable, to the location from which our code will read it.

Note: protected variables will be only available for protected branches and tags. If you push code to a non protected branch, Gitlab’s protected environment variables will not be available during the pipeline execution.

3. Save the test coverage.

As you probably have noticed, the pipeline is reporting the test coverage in the execution logs because of the job’s command coverage report .

Execution logs include the test coverage.

We can tell Gitlab to look for the coverage results in the execution logs and save them. Indeed, navigate to your project Settings > CI/CD > General pipelines. Look for the section Test coverage parsing, and copy the regular expression that applies to your testing library. In my case, I will select that available for pytest:

Add the coverage parsing regular expression.

After saving, next time the pipeline succeeds, the coverage results will be stored in Gitlab.

4. Adding Gitlab badges

Finally, we can add a badge to the repository to display the test coverage. To add badges to a Gitlab repository, navigate to Settings > General > Badges. Fill in the link and badge image URLs and hit “Save changes.”

Add a badge in your repository settings.

After saving, the badge will automatically appear on the repository main page:

The badge appears now on the main page of the repository.

Conclusions

With that, we close the use case. We added a pipeline with several jobs to a Gitlab repository. We made use of Gitlab variables to keep sensitive data out of the code repository. We configured Gitlab to save the coverage results, and we created a badge to display the results on the main page of the repository.

I hope you find this post useful and will include Gitlab pipelines in your next projects. To know more, the Gitlab documentation is a great place to explore further options and features of Gitlab CI/CD tools.

Thank you for reading.

References

1. GitLab CI/CD documentation