The Startup
Published in

The Startup

A Simple Introduction to Automating Unit Tests in Python

In 1999 a navigation error saw NASA’s Mars Climate Orbiter crashing into the red planet’s atmosphere, and with it burnt years of painstaking work and millions of dollars spent building it. What was the cause? The commands sent to the spacecraft used English units instead of metric units. Mathematical and technical errors like these could result in catastrophic failures, some even ending in serious casualties. Hence the case for testing being an indispensable part of software development.

Unit Testing” is when you test one unit of your software independent of any other units. Typically in Object Oriented Development (OOD) this unit is a class. But, what is testing? In simple terms, testing means checking the output of your code against a set of pre-defined, correct expected output/results. You need to have test data — both input and expected output/results — before you start testing. I said “output/results” because sometimes your code doesn’t output data, rather it modifies some data in your software.

Now, manually testing your code i.e. running it yourself, entering input via a user interface and visually examining the results is gruellingly boring and no doubt cumbersome. Plus, this way of testing isn’t easily scalable or repeatable. Instead you could write a tester class that simply calls all your functions and compares the results with a set of pre-defined ones. However, the problem with this method is that the functions you are testing could call other functions which may run some code elsewhere and so on. This could really mess up the working of your software or even corrupt any valuable data you might have on your database. Remember Unit Testing is all about testing one thing at a time. The solution to this is writing isolated, automated tests with the help of tools like pytest, unittest and nose2.

Since the goal of this tutorial is not to delve deep into the intricacies of software testing, let’s head straight into writing some automated tests in Python. We will be using the unittest Python library to write tests and nose2 to run them.

Prerequisites: Make sure you have Python 3 installed on your PC. You will also need to install the following libraries: unittest, mock, tinydb and nose2. You can install these using pip.

WHAT ARE WE TESTING?

We have here a Python class called “Customer”. The class has some basic functions that allow us to add and retrieve customer data. We are using a very basic python database called tinydb to store the data.

tinydb stores data in a JSON file in the same directory as the python code. You can instantiate the Customer class by providing a customer ID. But when you add a customer you have to provide the customer name, email and type and it will assume the customer ID given during instantiation. A customer is only added if they don’t already exist in the database and only loaded if they do exist.

HAVING A TEST PLAN

It is very handy to have a test plan before you start writing tests. You should think of:

1- What your code does.

This includes (but is not limited to):

  • an overview of what can be achieved from the program
  • what functions the program has
  • what input, if any, do the functions take
  • what results can be expected from the functions.

2- Good and bad case scenarios

When you think of the expected results you need to take into account both successful execution of your functionalities and anomalies that could occur.

Eg: Adding an already existing customer should result in a warning.

3- What does not require testing

Your program may include code where you invoke functions already tested. You don’t want to test them again. There could also be functions from external packages that you don’t have to bother about.

Eg: The tinydb insert function.

4- Of course, the tests themselves

Here we will need to write tests for the following functionalities:

  • Adding a new customer
  • Attempting to add an already existing customer
  • Loading an existing customer
  • Attempting to load a non-existent customer
  • Checking if customer exists or not

WRITING OUR FIRST TEST

As a general rule test files are stored in a directory called “tests” inside the project directory. Now, let’s create a python file called “test_customer.py”. When naming your test files, start with “test_”. Let’s begin by importing all the external packages we need. Also import the Customer class we are testing.

If you find the code snippets hard to read, you can follow tests from here.

from unittest import TestCase 
from mock import patch
from customer import Customer

Now let’s declare our class:

I will use “…” to signify that there is code above or below the code we are writing.

...class TestClass(TestCase):    

def setUp(self):
#Declare test data here
self.test_customer_id = '000'
self.test_customer_name = 'Bean'
self.test_email = 'bean@beanlovesteddy.com'
self.test_customer_type = 'Premium'

Our test class inherits from the TestCase class. Hence why we imported it earlier. A test class can be seen as a collection of tests, each test being a function inside it.

The “setUp” function is where we declare test data and anything else that we need to build our tests (Eg: a fake database, fake HTTP requests). It runs before any of the tests in the class. Also the “setUp” function is called every time a test is run. You can see we have declared some customer attributes in ours.

Okay, let’s write our first test:

...def test_if_function_returns_true_when_customer_exists():
test_customer1 = customer.Customer(self.test_customer_id)
result = test_customer1.is_myshop_customer()
assert result==True

Your function (or test) names should be very descriptive and clear about what exactly you are testing. As a convention they should start with “test_”. In this test, you can see we have assigned the actual output of the function we are testing to a variable. We then check if it matches the expect output which is True. The assert function raises an error if the proceeding statement is false. This makes sure that the test fails with an error if the code is not working as expected.

But this presents a problem that we touched on earlier. When the “Customer” class is instantiated or when the “is_myshop_customer” function is called we are invoking tinydb resources. Our tests should not run code external to the class we are testing. The solution to this is a feature called Mocking.

MOCKING

Mocking allows us to make calls to external resources without actually invoking them. You can call functions, classes and objects and determine what values, if any, those resources return. You can also cause certain actions to occur as a result of those calls. These actions are called side-effects. Mocking is arguably the trickiest bit of unit testing. Read in detail here.

So let’s rewrite our first test with mocking included:

@patch('customer.TinyDB')
@patch('customer.Query')
def test_if_function_returns_true_when_customer_exists(self, mock_query, mock_tdb):
mock_tdb.return_value.search.return_value = [{'customer_id':self.test_customer_id}]

test_customer1 = customer.Customer(self.test_customer_id)
result = test_customer1.is_myshop_customer()
assert result==True

Patch is a mechanism used to tell Python which external resources are to be mocked in our tests. There are a few ways to use patch. Here we have used it in the form of decorators above our function declaration. For each patch decorator, you need to add an input parameter to your test. See the “mock_query” and “mock_tdb” parameters? They need to go in the order opposite to that of your decorators i.e. the bottom decorator should have its corresponding parameter given first.

### For example, patch('customer.Query') corresponds to "mock_query"

In the first line of our test, we assign the “search” function inside an instance of the TinyDB class a return value. If you look at the “is_myshop_customer” function, this return value will be assigned to the “query_result” variable in place of the actual result of our tinydb query.

The return value, here, is a list with one dictionary that has a key that matches our test customer ID. This will cause the “is_myshop_customer” function to return True, passing our test, and ensure that our tinydb database remain untouched. Very cool, right? The rest of the test stay unchanged from its earlier version.

Now, let’s write another test for the same function:

@patch('customer.TinyDB')
@patch('customer.Query')
def test_func_returns_false_when_customer_does_not_exist(self, mock_query, mock_tdb):
mock_tdb.return_value.search.return_value = []

test_customer1 = customer.Customer(self.test_customer_id)
result = test_customer1.is_myshop_customer()
assert result==False

This function tests the converse case, where the tinydb query may return a negative query result causing the “is_myshop_customer” to return False. This shows how a single function in our code may require multiple tests when we take into account all possible scenarios that could occur when our code is run.

Next up is the “add_customer” function:

@patch('customer.TinyDB')
@patch('customer.Customer.is_myshop_customer')
def test_new_customer_can_be_added(self, mock_cust_check, mock_tdb):
mock_insert = mock_tdb.return_value.insert
mock_cust_check.return_value = False
test_customer1 = customer.Customer(self.test_customer_id)
result = test_customer1.add_customer(self.test_customer_name, self.test_email, self.test_customer_type)

mock_insert.assert_called()
self.assertEquals(result, "** Customer Added **")

In this test we have mocked both the tinydb insert function and our own “is_myshop_customer” function. The reason why we mocked the latter is because we have already tested it once and we only test what needs testing.

We then call the “add_customer” function with our test data as input. The line second to the last checks if the (mocked) tinydb insert function has been called. Note how we just check the invocation rather than the underlying working of the insert function. This is because whether or not the insert function adds data to the database is not our concern, in terms of testing. I am sure the function has been well tested inside the tinydb package.

The test for the converse case would look like this:

@patch('customer.TinyDB')
@patch('customer.Customer.is_myshop_customer')
def test_existing_customer_not_added(self, mock_cust_check, mock_tdb):
mock_insert = mock_tdb.return_value.insert
mock_customer_check.return_value = True
test_customer1 = customer.Customer(self.test_customer_id)
result = test_customer1.add_customer(self.test_customer_name, self.test_email, self.test_customer_type)
mock_insert.assert_not_called()
self.assertEquals(result, f"Customer with ID {self.test_customer_id} already exists!")

At the top of every test you can see that there is a patch decorator above every test. To avoid this code duplication we can transfer it to the top of our test class. We still need to keep the corresponding parameter (right at the end).

...@patch('customer.TinyDB')
class TestClass(TestCase):
def setUp(self):
#Declare test data here

self.test_customer_id = '000'
self.test_customer_name = 'Bean'
self.test_email = 'bean@beanlovesteddy.com'
self.test_customer_type = 'Premium'
...

Now, try testing the “load_customer_data” function in a similar fashion.

You can find the full test file here.

RUNNING THE TESTS

You have now reached the simplest part of the process! All you have to do to run your tests is typing in nose2 on your CLI from your project directory.

Running nose2 --coverage term-missing --with-coverage will give you a detailed report of how much of your code has been covered by your tests.

Hope this guide helped you learn something new!

CREDITS

Cover Photo:

Python image by notKlaatu on OpenClipart (Modified)

Background Image by Tayeb MEZAHDIA from Pixabay

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store