Integration Unit Testing
Integration unit testing is an API testing strategy that combines the core benefits from traditional unit testing and integration testing. It is for API and integration test automation, and for replacing traditional unit testing in the API ecosystem.
A little background
Software backend systems in modern enterprises are often seen to be distributed, like applications, databases, SCADA services, etc. Each backend system, small or huge, specializes in some business functionalities.
To make all the backend systems, plus frontend user interfaces, work well together to support integral enterprise business, distributed architectures or techniques like Microservices, SOA, ESB, Messaging, etc. are often used, resulting in the proliferation of APIs.
Each API represents a small piece of business or integration functionality in a backend system. APIs are exposed through various protocols like HTTP, JMS, FTP, etc. to enable integration with each other.
Here is a sample API:
This sample API exposes an HTTP endpoint, and, when invoked, interacts with three dependencies: another HTTP API, a database and a queue. In the real world, a normal API probably won’t have too many dependencies, but here we just make the sample API look more illustrative.
To test enterprise software systems, there was traditional test pyramid
Traditional unit testing was invented when most backend systems were siloed and monolithic applications. It feels not so fitting into the API ecosystem.
Some characteristics of traditional unit testing when applied on APIs are
- Single-process testing. Everything runs in the same process, including test case, the API implementation under test and its dependencies. To achieve test isolation, the API implementation’s dependencies are mocked. When the API’s input protocol relies on a middleware that can’t be embedded in the same process, like JMS (Solace), normally the input protocol is also bypassed during the testing. So the API implementation’s interactions with other processes are not tested.
- Whitebox testing. It does not test against API definition/specification. It tests API implementation in a whitebox/structural manner.
These characteristics are so as to make the unit tests easy and fast to run. In fact, people have been so enthusiastic on the ‘run’ aspect, that the critical issues coming along have been largely ignored.
Some critical issues with traditional unit testing in the API ecosystem are:
- Duplicate testing. When developer finishes traditional unit testing, he or she faces one question: is my code now good enough to be handed over to tester for testing? Normally the answer is no, because the API implementation code has not been tested with real dependencies such as a real database. By mocking out a dependency database, the developer is unsure whether the SQL statements in the API implementation will work when a real database is used. The developer risks receiving defects soon from tester. With API definition/specification being a well communicated deck for all people in the team, testing against API definition/specification seems unavoidable. If the developer chooses to also integration testing the API, it will be largely duplicate with the traditional unit testing he or she has already done, especially for testing with various API input data.
- Difficult to do. Writing API implementation code is not easy. Writing traditional unit testing code for API implementation code is even more brain power draining and time consuming.
- Some times not practical. APIs implemented with some technologies, like IIB, are just too hard to unit test in traditional way, if even possible. Refer to this post.
- Brittle tests. When doing traditional unit testing on API implementations, the test code is tightly coupled with the API implementation code. A small change to the API implementation code could break many unit tests.
- Knowledge barrier. There is a lot of business knowledge in the test cases which is also valuable to testers, BAs and architects. But unfortunately only developers are able to read the test cases.
To address the issues, this article promotes integration unit testing as a replacement of traditional unit testing in the API ecosystem.
What is integration unit testing?
‘Unit testing’ here means API level test isolation. To test an API, use stubs as its dependencies like HTTP stubs, stub database, stub queues, etc. Stubs here are dedicated systems, like a dedicated database, for integration unit testing purpose. They are replacements of real systems that the API will interact with in production environment. They can be fully controlled by integration unit test cases. Use test case and stubs to control the behavior of the API under test. Treat the entire API under test (instead of one inner piece of it) as a unit.
‘Integration’ here means multi-process. Test case, the API under test and its dependencies (stubs) run in different processes. The API implementation still interacts with its dependencies (stubs) via real cross-process protocols like HTTP, ODBC, JMS, etc.
When doing integration unit testing on an API, stub its dependencies, call its input endpoint (like an HTTP endpoint), and then check the stubs status and data to verify the API’s behavior is as expected.
As can be seen, the behavior tested here is much closer to the behavior of the API in production environment, comparing to traditional unit testing.
Characteristics of API integration unit testing
- Multi-process testing. As described above.
- Blackbox testing. Test API in a blackbox/behavioral manner, so that API’s definition/specification and cross-process interactions with its dependencies are tested. The testing does not care about API implementation details (like using what technologies, having what programming styles, etc.). The testing only cares about the API’s input protocol/data and its dependency stubs’ status/data.
Let’s recap how the issues with traditional unit testing are addressed by integration unit testing, in the API ecosystem.
- No duplicate testing. Traditional unit testing is replaced with integration unit testing, hence no more question from developer as to un-tested API definition/specification or cross-process interactions between the API and its dependencies.
- Easier to do. Integration unit testing does not require programming skills or any knowledge about the API implementation code. With a good testing tool, writing API test cases will be much easier than traditional unit testing.
- Can test all APIs. No matter what technologies are used to implement the APIs.
- Tests are not brittle. Tests are not bound to API implementation code. Due to blackbox nature, tests are data oriented. Refactoring of API implementation code (without changing its logic) won’t need to modify any integration unit test cases.
- Remove knowledge barrier. With a good tool, integration unit test cases can be shared and read by testers, BAs and architects, because no programming skills are required.
In fact, it is very intuitive to do integration unit testing. Many teams have API or integration design documents. These documents describe how the API is invoked, how it interacts with its external dependencies (HTTP APIs, queues, databases, etc.), and the data/message transformations inside the APIs. Developers are naturally inclined to test against the design documents, so as to talk the same language as the entire team (team is straightforwardly aligned around design documents).
So this article promotes a new test pyramid for the API ecosystem.
The good tool
API Test Base (ATB) is a tool designed for Integration Unit Testing (automated) and System Integration Testing (manual, semi-automated or automated) in the API ecosystem.
Integration unit testing examples using ATB:
