Build your first Automated Test Integration with pytest, Jenkins and Docker

Varun Kumar G
Mar 22, 2020 · 7 min read

In this introductory article, we will develop a simple calculator in Python, write tests for it using pytest framework and use a containerized Jenkins to fetch the repository from GitHub and run the tests inside a new Docker container spun by Jenkins. The whole repository is available here.


  1. A Windows machine
  2. Docker Desktop for Windows, having switched to Linux containers
  3. Python 3.x
  4. PyCharm, or any suitable IDE for Python
  5. Git
  6. GitHub account

Part 1: Our pytest Project

Let’s create our project directory python-test-calculator and set it up as your Git repository with . Create a virtual environment inside this directory (PyCharm will take care of this when you create a new project). The venv folder can be ignored by specifying it in .gitignore. Install pytest, our test framework, using . Create file which consists of a set of mathematical functions that are to be tested. This file is placed inside a src folder along with (an empty file which enables us to use this folder as a package).

# calculator.pydef add(a, b):
checkInputs(a, b)
return a + b
def subtract(a, b):
checkInputs(a, b)
return a - b
def multiply(a, b):
checkInputs(a, b)
return a * b
def divide(a, b):
checkInputs(a, b)
return a / b
def checkInputs(a, b):
if not isinstance(a, (int, float)) or not isinstance(b, (int, float)):
raise TypeError("Inputs must be either int or float!")

The tests are written inside a tests folder which are a set of files testing each mathematical function. By default, pytest only identifies the filenames starting with “test_” or ending with “_test” as the test files. Also, pytest requires the test method names to start with “test”. I have written ten simple tests for our calculator. For example, in, we import the add function defined in our file, pass the input parameters and assert check for the expected output. The empty file in tests folder enables us to run pytest command directly in our root directory.

# test_addition.pyfrom src.calculator import add
import pytest
def test_add():
result = add(3, 4)
assert result == 7
def test_add_string():
with pytest.raises(TypeError):
add("string", 4)

Run to save all the packages that have been installed in your virtual environment. This will help you to install dependencies inside your test container by running .

My pytest.ini file consists of a single line “junit_family=xunit1” which helps me ignore the warning that is displayed when --junitxml option is used with pytest command.

Push this project to GitHub.

Part 2: Dockerfile for test

So, where does Docker come into play here? Remember, our idea was to have two containers: one for running the tests and the other for running Jenkins (we’ll see this later). Make sure that you have switched to Linux containers, OS/Arch: under Server: Docker Engine - Community in should say linux/amd64. Let us create a Dockerfile which consists of the following lines:

FROM python:3.6-slimMAINTAINER varunkumar032@gmail.comCOPY . /python-test-calculatorWORKDIR /python-test-calculatorRUN pip install --no-cache-dir -r requirements.txtRUN ["pytest", "-v", "--junitxml=reports/result.xml"]CMD tail -f /dev/null

Here, we are using a python image for running the tests as this comes with our required version of python pre-installed. COPY copies the code from our workspace to a new directory python-test-calculator and WORKDIR defines this as the working directory on the container. RUN lets you execute the installation and test commands. ‘-v’ is the option for verbose mode and result.xml is our test report file. CMD tail -f /dev/null keeps the container running even after test completion so that we can copy the result.xml file from the test container to our workspace in Jenkins for it to publish the test report.

Part 3: Dockerized Jenkins

As we all know, Jenkins is an open source automation tool for CI/CD purposes. We can install it by downloading and running the Jenkins WAR file, but this requires Java support. So, let’s have Jenkins running inside a container, with all the dependencies installed, that can build new docker images and save ourselves from complexities.

Create JenkinsDockerfile, which is basically another renamed Dockerfile so as to differentiate it from the one existing, having the following lines:

FROM jenkins/jenkins:ltsUSER rootRUN apt-get update -qq \
&& apt-get install -qqy apt-transport-https ca-certificates \
curl gnupg2 software-properties-common
RUN curl -fsSL \
| apt-key add -
RUN add-apt-repository \
"deb [arch=amd64] \
$(lsb_release -cs) \
RUN apt-get update -qq \
&& apt-get install docker-ce=17.12.1~ce-0~debian -y

jenkins/jenkins:lts will be our image on top which we install Docker. Build the dockerized Jenkins image using the below command:

docker build -t jenkins-docker-image -f JenkinsDockerfile .

jenkins-docker-image is our image name. Run the container with the docker daemon mounted, thus enabling us to run docker commands from inside the Jenkins container, using the below command:

docker run -d -p 8080:8080 --name jenkins-docker-container -v /var/run/docker.sock:/var/run/docker.sock jenkins-docker

Verify whether jenkins-docker-container is running with command.

Open http://localhost:8080/ in your browser to see the Unlock Jenkins page. Copy the password from and install suggested plugins. Create an admin user by filling in the details and keep the default URL. ‘Save and Finish’ and ‘Start Using Jenkins’. This is the default Jenkins page:

Click on create new jobs to create a new freestyle project of any name. Under the General tab, you can mention the description for your project and specify the GitHub Project URL.

In Source Code Management tab, click on Git and add the repository URL. Enter the credentials by selecting Add->Jenkins and specifying the username and password of your Git account. This step will copy the repository python-test-calculator from the specified GitHub URL into /var/jenkins_home/workspace thereby creating the workspace inside our python-test-calculator directory.

Under Build tab, click on Add build step->Execute shell and copy the following:

echo "Check current working directory"
echo "Build docker image and run container"
docker build -t $IMAGE_NAME .
docker run -d --name $CONTAINER_NAME $IMAGE_NAME
echo "Copy result.xml into Jenkins container"
rm -rf reports; mkdir reports
docker cp $CONTAINER_NAME:/python-test-calculator/reports/result.xml reports/
echo "Cleanup"
docker stop $CONTAINER_NAME
docker rmi $IMAGE_NAME

In the shell script, we instruct Jenkins to build the image as test-image using Dockerfile. The build step will execute our pytest command as mentioned in the Dockerfile. We run a container from this image as test-container and keep it running unless stopped explicitly as the command we chose to run on startup was tail -f /dev/null. We then copy the result.xml from our container into a newly created reports folder in our workspace. Cleanup ensures that the created container and image are stopped and deleted.

In Post-Build Actions tab, we specify the location in workspace of the copied result.xml for publishing. Ensure that JUnit plugin is installed on Jenkins. Save the project configuration.

Click on Build Now in your project to see the magic.

Congratulations!! You just built your first automation setup wherein you tested an application inside a dedicated container using Jenkins to fetch the code from a remote repository and generate the reports.

You can extend this workflow to test any kind of application within an isolated environment in an automated way. All you need to do is to bridge the gaps.

Happy Automating!!

The Startup

Get smarter at building your thing. Join The Startup’s +794K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Varun Kumar G

Written by

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +794K followers.

Varun Kumar G

Written by

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +794K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store