Testable and modular code

I’ve worked on multiple projects, small and large, personal and professional with teams of different sizes over the past few years and realized the importance of writing testable and modular code. Everybody wants tests, but people shirk away from writing tests, because they take time, are harder to write than the production code itself. During personal projects or when starting to work on a new idea, people(including me) tend to get too focussed on just putting something together, *that works*, which is great for a proof of concept or getting something out of the door asap. Even during those times, if one has testability and modularity in the back of their mind, it can lead to design decisions, that ultimately lead to modular and better maintainable code. One can add tests later on. It greatly reduces the developer time during the lifespan of the project, making everybody more productive.

I get really scared to check in my changes when there are no unit tests. How do I make sure my small innocent, one-line change doesn’t break stuff 10 steps down the stack(heard of the butterfly effect)? How do I make sure, another person who’s working on a change in parallel works well with mine? How can I be confident future development will be backward compatible? I think these questions become more and more important as the size of the project increases, as the number of dependencies increase. You can either be smart and write tests proactively or spend endless time later to debug, when shit hits the fan and people are unhappy. I have been fortunate to have such experiences and learn from my peers over the years.

When I start to implement a new functionality, I like to break it down in to modules or building blocks or interfaces and define contracts among them those building blocks. It’s almost like a puzzle then, arrange those building blocks in a way that solves my problem.

Let me use an example. Consider an over simplified grocery management system, that finds out the number of items sold on a day, does replacement orders and writes the open orders in another table. DB and OrderPlacer are externally available modules/interfaces.

Simple grocery management system

The above code will only work in an ideal world. In a real production scenario, you’d want to handle db connection issues, db read issues, db write issues, failures in replacement ordering apis, etc. All that error handling is going to make this complicated. Assume, we have another function run_with_retry(), that runs a function with given number of retries.

Grocery management with retries

Now imagine adding a few more configs like timeouts for connect(), read(), write(). All this complexity is going hurt the readability and testability of this simple code really hard.

• How to test db connection/read/write failures?
• How to test the behavior after replacement ordering api fails?
• There’s no way to test writes and reads, ordering apis the failure flows individually. Testing each of them has become quite hard.
• What if, some one comes and changes the number of connect retries to really large or just zero. How to guard against such changes?

To reign that complexity, this logic should be split in to multiple modules, so each can be tested individually.

Config

Modules to run queries and processing

More readable, close to the first code sample

• The code above looks so much similar to the simpler first code sample, is much more readable and handles all the failures with DB and OrderPlacer api.
• It also improves the DB api that had only two methods of run() and connect() and has 3 methods now connect(), run(), write(), with each one having its own error handling.
• The OrderPlacer api’s special error handling of sending out emails is encapsulated in a separate module. The main function process() is so much simpler to follow.
• The difference is now we inject the dependencies in to the process() method instead of process() instantiating its dependencies. This is a very powerful construct, sometimes referred to as dependency injection(DI).

You can easily add tests for each of the modules separately, test process() by injecting mocks of DB and OrderPlacer. I’ve some pseudo code examples here, in the interest of time.

Test connect failure

Test order placing failure

There are a large number of mocking/testing libraries available in different languages, example EasyMock(java), Mockito(java), gtest(C++), MagicMock(python) etc. Some examples of dependency injection frameworks are Guice(java), Boost.DI(C++), Fruit(C++).

I’ve highly used factory pattern, which is a very useful design pattern with DI, which I’ll talk about in another post. Please leave a comment, if you’d like me to write about other topics.