Build your first Automated Test Integration with pytest, Jenkins and Docker

In this introductory article, we will develop a simple calculator in Python, write tests for it using pytest framework and use a containerized Jenkins to fetch the repository from GitHub and run the tests inside a new Docker container spun by Jenkins. The whole repository is available here.

Requirements

1. A Windows machine
2. Docker Desktop for Windows, having switched to Linux containers
3. Python 3.x
4. PyCharm, or any suitable IDE for Python
5. Git
6. GitHub account

Part 1: Our pytest Project

Let’s create our project directory python-test-calculator and set it up as your Git repository with git init. Create a virtual environment inside this directory (PyCharm will take care of this when you create a new project). The venv folder can be ignored by specifying it in .gitignore. Install pytest, our test framework, using pip install pytest. Create calculator.py file which consists of a set of mathematical functions that are to be tested. This file is placed inside a src folder along with __init__.py (an empty file which enables us to use this folder as a package).

# calculator.py
def add(a, b):
    checkInputs(a, b)
    return a + b
def subtract(a, b):
    checkInputs(a, b)
    return a - b
def multiply(a, b):
    checkInputs(a, b)
    return a * b
def divide(a, b):
    checkInputs(a, b)
    return a / b
def checkInputs(a, b):
    if not isinstance(a, (int, float)) or not isinstance(b, (int, float)):
        raise TypeError("Inputs must be either int or float!")

The tests are written inside a tests folder which are a set of files testing each mathematical function. By default, pytest only identifies the filenames starting with “test_” or ending with “_test” as the test files. Also, pytest requires the test method names to start with “test”. I have written ten simple tests for our calculator. For example, in test_addition.py, we import the add function defined in our calculator.py file, pass the input parameters and assert check for the expected output. The empty __init__.py file in tests folder enables us to run pytest command directly in our root directory.

# test_addition.py
from src.calculator import add
import pytest
def test_add():
    result = add(3, 4)
    assert result == 7
def test_add_string():
    with pytest.raises(TypeError):
        add("string", 4)

Run pip freeze > requirements.txt to save all the packages that have been installed in your virtual environment. This will help you to install dependencies inside your test container by running pip install -r requirements.txt.

My pytest.ini file consists of a single line “junit_family=xunit1” which helps me ignore the warning that is displayed when --junitxml option is used with pytest command.

Push this project to GitHub.

Part 2: Dockerfile for test

So, where does Docker come into play here? Remember, our idea was to have two containers: one for running the tests and the other for running Jenkins (we’ll see this later). Make sure that you have switched to Linux containers, OS/Arch: under Server: Docker Engine - Community in docker version should say linux/amd64. Let us create a Dockerfile which consists of the following lines:

FROM python:3.6-slim
MAINTAINER varunkumar032@gmail.com
COPY . /python-test-calculator
WORKDIR /python-test-calculator
RUN pip install --no-cache-dir -r requirements.txt
RUN ["pytest", "-v", "--junitxml=reports/result.xml"]
CMD tail -f /dev/null

Here, we are using a python image for running the tests as this comes with our required version of python pre-installed. COPY copies the code from our workspace to a new directory python-test-calculator and WORKDIR defines this as the working directory on the container. RUN lets you execute the installation and test commands. ‘-v’ is the option for verbose mode and result.xml is our test report file. CMD tail -f /dev/null keeps the container running even after test completion so that we can copy the result.xml file from the test container to our workspace in Jenkins for it to publish the test report.

Part 3: Dockerized Jenkins

As we all know, Jenkins is an open source automation tool for CI/CD purposes. We can install it by downloading and running the Jenkins WAR file, but this requires Java support. So, let’s have Jenkins running inside a container, with all the dependencies installed, that can build new docker images and save ourselves from complexities.

Create JenkinsDockerfile, which is basically another renamed Dockerfile so as to differentiate it from the one existing, having the following lines:

FROM jenkins/jenkins:lts
USER root
RUN apt-get update -qq \
    && apt-get install -qqy apt-transport-https ca-certificates \
    curl gnupg2 software-properties-common 
RUN curl -fsSL https://download.docker.com/linux/debian/gpg \
    | apt-key add -
RUN add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/debian \
    $(lsb_release -cs) \
    stable"
RUN apt-get update -qq \
    && apt-get install docker-ce=17.12.1~ce-0~debian -y

jenkins/jenkins:lts will be our image on top which we install Docker. Build the dockerized Jenkins image using the below command:

docker build -t jenkins-docker-image -f JenkinsDockerfile .

jenkins-docker-image is our image name. Run the container with the docker daemon mounted, thus enabling us to run docker commands from inside the Jenkins container, using the below command:

docker run -d -p 8080:8080 --name jenkins-docker-container -v /var/run/docker.sock:/var/run/docker.sock jenkins-docker

Verify whether jenkins-docker-container is running with docker ps -a command.

Open http://localhost:8080/ in your browser to see the Unlock Jenkins page. Copy the password from docker exec -it jenkins-docker-container cat var/jenkins_home/secrets/initialAdminPassword and install suggested plugins. Create an admin user by filling in the details and keep the default URL. ‘Save and Finish’ and ‘Start Using Jenkins’. This is the default Jenkins page:

Click on create new jobs to create a new freestyle project of any name. Under the General tab, you can mention the description for your project and specify the GitHub Project URL.

In Source Code Management tab, click on Git and add the repository URL. Enter the credentials by selecting Add->Jenkins and specifying the username and password of your Git account. This step will copy the repository python-test-calculator from the specified GitHub URL into /var/jenkins_home/workspace thereby creating the workspace inside our python-test-calculator directory.

Under Build tab, click on Add build step->Execute shell and copy the following:

IMAGE_NAME="test-image"
CONTAINER_NAME="test-container"
echo "Check current working directory"
pwd
echo "Build docker image and run container"
docker build -t $IMAGE_NAME .
docker run -d --name $CONTAINER_NAME $IMAGE_NAME
echo "Copy result.xml into Jenkins container"
rm -rf reports; mkdir reports
docker cp $CONTAINER_NAME:/python-test-calculator/reports/result.xml reports/
echo "Cleanup"
docker stop $CONTAINER_NAME
docker rm $CONTAINER_NAME
docker rmi $IMAGE_NAME

In the shell script, we instruct Jenkins to build the image as test-image using Dockerfile. The build step will execute our pytest command as mentioned in the Dockerfile. We run a container from this image as test-container and keep it running unless stopped explicitly as the command we chose to run on startup was tail -f /dev/null. We then copy the result.xml from our container into a newly created reports folder in our workspace. Cleanup ensures that the created container and image are stopped and deleted.

In Post-Build Actions tab, we specify the location in workspace of the copied result.xml for publishing. Ensure that JUnit plugin is installed on Jenkins. Save the project configuration.

Click on Build Now in your project to see the magic.

Congratulations!! You just built your first automation setup wherein you tested an application inside a dedicated container using Jenkins to fetch the code from a remote repository and generate the reports.

You can extend this workflow to test any kind of application within an isolated environment in an automated way. All you need to do is to bridge the gaps.

Happy Automating!!