Django Test Fixture: setUp, setUpClass and setUpTestData

It is necessary to write tests, and especially while dealing with dynamic type languages like Python. When I started working at medici.tv, I discovered both Python and Django at the same time (I did some small stuffs with Python before, but nothing serious). Soon enough I messed up with the test fixtures until I understood thoroughly about its methods: setUp, setUpClass and setUpTestData.

I will summarise in this article the most important things I got about these methods, and I will focus on their behaviour regarding writing operations to the database.

For a fast check, you can look at the following recap table (assuming you use django.test.TestCase). Of course I encourage you to read on if you want further understanding ;)

First thing first: The TestCase class

If you are having questions about these aforementioned fixture methods, chances are you are working inside a TestCase class. TestCase is originally a class from unittest, a mature, complete and standard testing framework of Python. You can import this class (import unittest.TestCase) and use it separately from what you do in Django.

The most crucial thing to know about TestCase is that a TestCase object is the smallest unit of testing, and each TestCase object can only execute a single test as you can see in the following code.

from unittest import TestCase
class DummyTestCase(TestCase):
    def test_bigger(self):
        self.assertTrue(2>1)
    
    def test_smaller(TestCase):
        self.assertTrue(1<2)
bigger_test_case = DummyTestCase('test_bigger')
smaller_test_case = DummyTestCase('test_smaller')

The TestCase objects (smaller_test_case, bigger_test_case) are instantiated with one method each. This method is the only test that will be executed by the TestCase object.

The TestCase objects are then added and executed inside a TestSuite, but if you are working with Django test framework, you will not have to handle TestSuite yourself.

unittest.TestCase also offers some assert methods such as assertTrue, assertFalse, assertEqual, assertAlmostEqual.

But, my TestCase class is from Django...

Yes, you are using django.test.TestCase that is a subclass of unittest.TestCase. You can look at the following schema for the whole class hierarchy. I took it directly from the Django documentation.

The most important thing to know about django.test.TestCase is its manner to handle changes to the database: the TestCase instance encloses all operations into a database transaction that it rolls back at the end. This is how it does to keep the database clean for the following tests. And this is also one of the main reasons to choose django.test.TestCase over unittest.TestCase when you write tests involving models in Django.

setUp and tearDown: the most basic fixture methods

The methods setUp and tearDown come from the base class unittest.TestCase. These 2 methods are called at the beginning and at the end of the execution of a TestCase instance (and it only executes one single test method, remember?). Implementing setUp and tearDown is equivalent to repeat the same code at the beginning and at the end of each test methods. Hence, you can consider them as a tool to avoid repetitive code in your tests.

from django.test import TestCase
from django.contrib.auth.models import User
class DummyTestCase(TestCase):
    def setUp(self):
        User.objects.create(username='dummy')
    
    def test_a(self):
        self.assertTrue(1<2)
is equivalent to:
from django.test import TestCase
from django.contrib.auth.models import User
class DummyTestCase(TestCase):
    def test_a(self):
        User.objects.create(username='dummy')
        self.assertTrue(1<2)

In Django, the database operations in the setUp and tearDown methods are also enclosed in the one transaction of the TestCase instance; the changes you made in the setUp method will be rolled back at the end of the TestCase execution. That means If you write some models in the setUp method, you do not need to clean them up in the tearDown methods: the changes are to be rolled back anyway.

So, setUp and tearDown are convenient fixture tools to use since they spare you from cleaning up the changes you made to the database. Their downside is that they are called once per test. If these methods are time-consuming, they can slow down significantly your tests. As you encounter this problem, it is may be time to consider setUpClass and tearDownClass.

setUpClass and tearDownClass: a bit of magic

setUpClass and tearDownClass are class methods of the base class unittest.TestCase. They are called once for all TestCase instance of a given class. How can that be done? Actually, the TestSuite that I mentioned above "remember" the classes of the TestCase instances it encountered; when it sees a new TestCase for the first time, it executes tearDownClass of the previous TestCase class, then setUpClass of the following class. The TestSuite can do that because the tests of the same TestCase class are executed consecutively. So far so good.

If you want to override setUpClass and tearDownClass, mind that these methods are also overriden in django.test.TestCase, so don't forget to call super().

from django.test import TestCase
class DummyTestCase(TestCase):
    @classmethod
    def setUpClass(cls):
        super(DummyTestCase, cls).setUpClass()
        # some code

About the database operations, you have to know that setUpClass and tearDownClass don't give the same convenience as setUp and tearDown: operations inside setUp and tearDown will not be rolled back, since their code are executed outside the scope of a TestCase instance. The changes you made in setUpClass, you have to clean them later in tearDownClass. This can be complicated, but moreover, you might ask: "Why can't someone do the clean up for me?". You are right, and that's why setUpTestData got into the spotlight.

setUpTestData: a specific solution for a specific problem

Unlike the previous methods, setUpTestData does not belong to unittest but to the Django framework. It is introduced in Django 1.8 as a class method of django.test.TestCase. It acts pretty much like setUpClass, except that the operations you do in setUpTestData will be rolled back after all tests of the same TestCase class are executed. Actually, setUpTestData is implemented in Django using setUpClass and tearDownClass.

So we can now say that setUpTestData fills your need if what you want if something that:

• executes only once for all tests inside a TestCase class, so that your tests can be faster;
• has all database operations rolled back after all tests of the given TestCase class are done.

I found a blog entry from Makina-Corpus that illustrates quite well how you can use setUpTestData to speedup your tests. The method refresh_from_db mentioned in the entry can be useful as well.

Conclusion

As there are more and more frameworks in the wild, it happens regularly that we start a language with a chosen framework. It can be the case for Python/Django and especially for Javascript. Go through the abstraction and see what lies behind the framework can be a very good thing to do.

Read the doc. This article for example can bring some clarifications and a summary so that you can better understand the issue, but it does not replace the doc and neither do other articles on the net. Here are the 2 links to the doc of unittest and Django testing tools.

I will end this article with a quote from one of the best engineers that I've had the chance to work with ;)

You can not spare yourself from understanding what you do.