Goals of Unit Testing
👋🏼 Hello! After a long hiatus writing about unit testing, we’re exploring the most fundamental testing concept in software engineering, particularly looking to answer these questions about unit testing (at least from our perspectives):
A Unit Test (or Component Test) is a low-level automation test used to verify the behaviour of a small function or component in the application source code. It falls under the static testing type, where a running server isn’t required. Unit tests can run very fast and test small pieces of code in isolation, limiting the need for all system modules to be ready. Traditionally, unit tests were often written by the developers themselves as they’d have better insights into the design and implementation of the source code that the unit tests are testing.
Unit tests can be written across all the popular programming languages and frameworks. Nowadays, most development frameworks provide added unit testing support out-of-the box. With the proper setup, unit tests help to provide fast feedback on the health of the source code, and require relatively low maintenance.
Unit tests — what to write and not?
It is important to write unit tests and equally important to write good unit tests. When project teams don’t write (or have any) unit tests, it usually ends up with lots of regression or stagnation with every new release. Having said that, let’s look at the kinds of unit tests we should focus on!
1. Focus on the component/function behaviour, rather than the UI aspect
With a unit test, the general recommendation is to verify the behaviour of the small piece of code you’re testing. With frontend libraries like React.js and Next.js, unit tests can also test the UI aspect of the frontend component. However, the general recommendation is to verify the behaviour of the component, rather than how it “looks”. This approach will give you the best ROI in terms of testing and maintenance.
2. Cover all possible input combinations in a single unit test
When testing a logic-oriented function/component that accepts a range of input parameters, it is generally recommended to cover all possible combinations.
This will verify the function is designed to gracefully handle most input parameters and return an expected response. Just to reiterate — the range of unit tests provides the most value only if the function/component is part of the core module(s) in your system!
3. Use mocks (with prudence)
When testing a function or piece of code, it could have some dependencies on other pieces of code. If the dependency isn’t ready or if it increases complexity due to further dependencies, mocks are often used to isolate the code under test by “mimicking” its dependencies. Mocks also provide the flexibility to mimic different behaviours with the dependency.
However, over-mocking can make it difficult to test how your code interacts with real-world dependencies. You may miss bugs that arise from those interactions and end up making your unit tests brittle. Hence, use mocks with prudence and follow a pragmatic approach when using mocks.
4. Always include assertions
Assertions are the core statements in the unit test that actually does the verification of expected behaviour. Without assertions, unit tests are essentially incomplete and sometimes useless. Making sure assertions are properly utilised in your unit tests requires code/peer review, as these omissions cannot be captured in the code coverage report (discussed further in final section).
Now, let’s look at the kinds of unit testing you shouldn’t write:
1. Don’t write tests to cover your entire source code
A good unit test suite helps to maintain the development pace over time, and prevents stagnation. Each test has a cost-and-benefit component, and engineers need to consider both in order to develop a test suite that provides a net positive value to the project. Hence, avoid writing unit tests across all modules in your source code and do not test each nook and cranny.
Remember: Each test you wrote is a liability and something that your team will need to maintain through the project.
2. Don’t write tests to verify third-party functions (including infrastructure and database)
External dependencies aren’t owned by your project team, thus there’s no need to write unit tests for them. They need to be managed by the respective code owners and tested within their own codebase. But what if an external dependency breaks? Well, it comes down to the team’s prudence when choosing this dependency. If the third-party code breaks, there should be a channel to triage about the breakage, and allow the project team to reassess the dependency’s inclusion in the project based on the third party’s contracted services and response.
3. Don’t cover cross-module tests in Unit Tests
When tests are needed to verify behaviour that involves multiple modules, we can move on to integration tests to verify such behaviour of an integration. These tests might go beyond the core modules of your system, but that’s fine. However, take note that integration tests still need to minimally cover your core modules as part of its verifications.
How useful are code coverage metrics in Unit Testing?
Code coverage is a common indicator of unit test coverage. One of the popular javascript tools used to measure code coverage is Istanbul. With popular libraries like React.js, these tools come pre-packaged and available immediately for use.
In code coverage metrics, the branch coverage metric is a useful indicator of test coverage at the unit level. It indicates the percentage of branches or decision points that are exercised by the unit tests. The remaining metrics (like line coverage, statement coverage & function coverage) are good-to-have, but not useful indicators of unit test coverage.
Do note that using code coverage as the sole indicator of unit test coverage is a risky bet, as this metric has some downsides:
- High code coverage provides a false sense of security as it is not a measure of code quality
- Unit tests without assertions can still indicate a 100% code coverage
Furthermore, code coverage is a quantitative metric that provides data on the percentage of source code exercised by unit tests, but never helps to understand if the qualitative aspect is also fulfilled.
In this regard, we’d like to introduce a concept that encapsulates the qualitative aspect of unit testing — Use case coverage. With peer review, if the QE/Dev can derive all the valid use cases of the function/component, a metric to understand how many of those use cases are covered by the unit tests can be derived. Measuring use case coverage involves manual effort initially, and can be automated with proper test management setup.
Goals of Unit Testing
The primary purpose of unit testing is to test the underlying source code, so that bugs are caught early and regression is prevented. In achieving this purpose, we need some goals in unit testing that the project team can aspire towards. Let’s look at them below:
Ensure unit testing is integrated into SDLC
All unit tests written should be executed as part of continuous integration, and maybe even with every git push command to verify that recent changes have not caused regression. Unit tests are fast and easily executable, and there’s no reason why you cannot constantly use them.
Focus on unit testing the core modules
Targeting the most important module(s) in your system is a good strategy for unit testing. The important modules may be user-facing frontend or a backend module that contains the business logic. Writing unit tests that test these business logic provides the best ROI for the project team and the product as a whole.
Unit tests can still be written for other parts of the system source code that might contain important modules. In general, the project team should minimally cover the business logic first with unit tests before moving on to the remaining modules in the system.
Target 70% code coverage and >90% use case coverage
In an organisation, mandating a target of 100% code coverage is not a good idea. Without consistent and pain-staking reviews, such targets usually end up with a mad race to the top, where engineers look for ways to achieve 100% coverage through all means necessary — both good and bad.
However, setting an above-average target of 60–70% has been documented to benefit the project from maintaining a decent coverage level across time. On top of this, use case coverage can be aimed higher as it is a qualitative indicator of the number of use cases in the code that are exercised by the unit tests.
Unit testing is a uniquely contextual topic and we hope this overall guideline is useful in your testing journey. If you have questions or more tips to share with us, do make use of the comments section below. See you again soon!
🧙🏼♀ Team Merlin 💛
Application security is not any individual’s problem but a shared responsibility.
