Code coverage with pytest

Code coverage with pytest-cov plugin

Great software products emerge with a great code base and great code base comes as a result of great testing efforts. It is always a best practice to write tests for the code that you write(even before starting the code implementation whenever possible).

Coming to the topic, suppose we write a test script for a programme, we always want to know how effective the test we wrote is and code coverage can be used for this purpose.

Code coverage measures how effectively we have written our tests scripts. In this article we will be using pytest-cov to measure the code coverage for the examples that follow. Before starting, the install requirement is given below.

pip install pytest==7.1.2
pip install pytest-cov==3.0.0

Lets consider the following simple function. Create a python file called add.py and write the below function inside it.

def find_sign(a: int, b: int) -> str:
    summation = a + b
    if summation > 0:
        ans = "positive"
    else:
        ans = "negative"
    return ans

The above simple function takes in two integers and returns the string “positive” and “negative” depending on the sign.

Now lets write a test function for this. Import the add script with the defined function into this test script. I would strongly suggest to have a look at pytest parametrization for a clearer understanding(Simply put its a way to give multiple (input, output) cases to a test without having to write multiple test functions)

import add
import pytest


@pytest.mark.parametrize("inp1, inp2, exp", [[2, 3, "positive"]])
def test_return_value(inp1, inp2, exp):
    assert add.find_sign(inp1, inp2) == exp

Now lets generate the coverage report for this test script and do some initial analysis.

In the terminal with the test script and add.py in the current working directory, lets type the following command. The first command runs pytest using coverage run and the second command generates the coverage report.

coverage run -m pytest
coverage report -m

The following report is generated.

add.py is the script we are interested in while test_coverage.py is the script where we wrote the tests.

Name               Stmts   Miss  Cover   Missing
------------------------------------------------
add.py                 6      1    83%   6
test_coverage.py       5      0   100%
------------------------------------------------
TOTAL                 11      1    91%

You can observe that out of 7 lines in add.py only 6 lines have been covered by our test script, the number under the Missing column gives the line that has not been tested.

Thats the power and benefit of using coverage when you write test scripts. You are pointed to the line for which you have to add a test case.

If you observe the find_sign function again you notice that that you tested the function for positive output, but not for negative outputs, therefore you can go ahead and update the test script to include that case.

import add
import pytest


@pytest.mark.parametrize("inp1, inp2, exp", [[2, 3, "positive"],
                                             [-3, -4, "negative"]])
def test_return_value(inp1, inp2, exp):
    assert add.find_sign(inp1, inp2) == exp

After adding the inputs that generate a negative output, you can generate the coverage report again.

Name               Stmts   Miss  Cover   Missing
------------------------------------------------
add.py                 6      0   100%
test_coverage.py       5      0   100%
------------------------------------------------
TOTAL                 11      0   100%

Now you can observe that our test script covers the entire add.py script.

The coverage that we discussed so far is known as line coverage or statement coverage(how many lines of the code base have been executed while performing the tests), its one of the basic measures of the effectiveness of your test scripts, there are far more superior measures that you can adapt to. Since the intention of this article is to give you a practical introduction to code coverage, I will cover the other aspects of code coverage in another article.

In the mean time a great youtube video to understand some advanced coverage measures in addition to the line coverage that I have covered in this article. I must acknowledge the following youtube video for a great discussion about code coverage in general

https://www.youtube.com/watch?v=n313mojz9hI&t=1485s

This concludes the introduction to understanding code coverage in testing python scripts. Please comment any questions you might have..