To accomplish good quality software today, programmers write tests, because manually testing is too cumbersome. One of the downside of testing is that it takes long time. One could argue that the lost time in writing tests is balanced by having a less buggy code base. And it prevents programmers from pushing bugs to each other, and it could substantially save time, especially if the code base is very large. One of the really time consuming things programmers need to do, today, is that they have to write isolations for tests. In this article I will try to explain that developers should not isolate their tests and they should instead use state restoring. Because test isolations consumes a lot of time and it drains cognitive energy from your brain, while you can achieve something better in less time and less cognitive load with state restoring.
Isolation
Before jumping into anything we need to describe test isolation and why it’s useful? We use an example of a web server and a database.
Test isolation is being used so that we never have to pollute the state of any dependencies. Because when we pollute the state of one test, it could affect the next test. A good example of it is: if we create one user with an id equal to 1 (in a database that have serial id:s), then in the next test the next user would have id equal to 2. The second test is dependent on the first test. The tests is not isolated from each other. The second test is relying on the first test.
In below, the `User` object creates two users and since we communicate directly with the database during our test session, the id is expected to be 2 on the second test case.
And this inter dependency create problems for us: for example, changing the first test would affect the second test. We can solve this by isolating the tests with a method called mocking. In mocking you create a fake object that resembles the actual object as good as possible.
Now, let us do the same example with a mocked object:
We mocked the `User` object and intercepted the create call. Given a certain argument, the create function will return a certain value that is decided by us (though it should resemble the actual object as good as possible). This example of mocking was very straight forward. Though, there is other very complex mocking examples. Like, mocking multiple objects at once and knowing how to mock third party libraries. And one bad thing is that those complex mocking examples are very common. So mocking gets very ugly very quickly. One could also argue if mocking test objects is really testing our code or testing the mocked object? Since in our simple example, we intercepted most of our logic behind the `create` function.
State Restoring
In state restoring, you don’t need to isolate your tests. What you do instead— is that you restore the state of your dependencies. For example, if you have a database that is running a test. Before it runs the next test it will restore the state of your dependencies.
In our example, our database was our dependency. In our first test we created a user. Somehow, we need to undo our mutation of state in our database. One really simple and yet very effective solution is to reset the database after each test.
In the below tests, we are reseting the database after each test:
In this way, we don’t need to figure out some complex mocking. And writing tests is just straightforward. The tests are also a lot safer, because it is testing a real database, instead of a fake one. And there is also no interception of logic, so we can be sure that our test execution is the same during a test session as it would be in a real session and that means our test is safer. We don’t need to rely on that our mocked object resembles a real one as much as possible either, because we are testing the real object.
One of the downside with state restoring is that it takes longer time to execute and it is more prone to failure such as network failure. So it is good practice to run the tests in parallell and also have mechanism that reruns the tests in case of unexpected failures, such as network failures.
Summary
So in summary, don’t mock your tests because it takes long time to figure out where to intercept code. The interception also might intercept a lot of logic, which makes your tests less safe. Instead, restore the state of your dependencies before or after each test, because it is much more straight forward to write your tests. It is also much safer, because it tests an actual real dependency(database in our example) instead of a fake one.
Next Article
My next article will be about a rather innovative view testing strategy that I intend to implement in Flanity (https://flanity.com). You can follow me in Twitter @tingan87 or in Flanity https://flanity.com/users/1.
I have also written about Baseline Testing if you are interested in more testing strategies: https://medium.com/@tinganho/baseline-acceptance-driven-development-f39f7010a04#.2n49bb84d
