Dig Deeper, Test Faster — Subcutaneous Testing with Django Test Client

Selenium is currently one of the most popular automation tools among QAs. Its ability to drive the browser like a user, enable testers to guarantee somehow that the user will be able to use the system without problems. Although that seems to be the perfect world, it comes with some downsides, being one of them the time it takes to test a single scenario.

Let’s face it, a really fast selenium testcase won’t take less than 4 or 5 seconds to execute, while unit and integration tests can test lots of scenarios in less than one second.

But how can we cover scenarios from the UI without taking that long? Well, the answer may be subcutaneous automated testing. The main idea here is to trigger the events the UI would start by calling the underlying layer. You directly call your controllers (called views in Django) sending exactly the same parameters like the UI would do.

Let’s think of a simple example:

Even without implementing any logic, the fastest test took almost 3 seconds to run, and the full suite took more than 15 seconds. If we think a bit about the scenarios, invalid inputs are supposed to be checked in the backend, and the frontend validation may not compensate the costs of running and maintaining such tests in selenium.

What can we do in order to be more efficient? Well, we can split our scenarios in two layers, delegating some of them to a lower layer, like the example below:

By implementing pretty much the same amount of logic, I have achieved a total time of 419 ms for the full suite. The slowest test took me about 361 ms, and the fastest 14ms. Thinking about the business value an invalid login assertion would bring if made using the front-end or the back-end. Which of these is more important? How to balance costs and benefits? Well, that’s up to you to measure and decide along with your business areas, depending on how you measure costs and benefits.

Since at Meus Pedidos we were trying to reduce the time spent building and testing our tools, the decision was to keep Selenium to a minimum and migrate the alternative and exception scenarios to the controller layer.

Our new selenium class would be like this:

The total suite execution time drops from 17s to less than 7s (5s 968s for selenium, 704 ms for the controller, compared to 16s 540 ms for selenium before removing the extra load)

By doing that, you can cover a greater number of scenarios without increasing the test execution time, optimizing your build time.