Integration Tests with Testcontainers at Ninja Van
It’s not a new insight that writing tests for your applications is a good idea. It will give you a safety net for future changes, guide your design and can help others to get up to speed on your code base. But sometimes it’s not easy to decide what kind of tests are best. Mocking frameworks like Mockito in Java are widely adopted and sometimes it might seem as if they are the only solution for all tests. But for some areas of your applications you might want to reach for other tools. In this article I will show a bit how we use Testcontainers at Ninja Van to test integrations with different data stores.
Why Testcontainers
Like many other users of the JVM we are mostly using Hibernate in combination with JPA to write queries for our relational database, MySQL in our case. Hibernate and JPA provide classes and interfaces that you can use to express your queries and responses. An obvious idea is to use Mockito to test this interaction. That’s not the best idea though. If you mock parts of this API you need to be sure that you know how it behaves. As soon as your assumptions are wrong you might write a test that passes but in reality the implementation is wrong. This problem is also well explained in the classic Growing Object-Oriented Software Guided by Tests which contains the advice to “Only Mock Types That You Own”.
Another very common approach in the Java world is to use some kind of in memory database like H2 or HSQLDB. While this is a far better approach because it tests the framework interacting with a real database it still leaves room for improvement. The main issue is that you are writing tests against a different system compared to what you use in production. The behaviour of those in memory databases can be surprisingly different to a database like MySQL, e.g. in the way how transactions are handled, or they can miss features in the SQL implementation.
Testcontainers implements a different approach. It controls Docker containers from JUnit tests and provides dedicated implementations for many data stores. Those containers can then be used to provide a database to tests that behaves exactly how a production database would do. You can setup the database by inserting data in it, execute your tests and then verify the data in the database.
The following is a very simple example of the MySQLContainer being used as a JUnit Rule to query a DAO using the jdbi library.
Even though this is a very simple example it already makes sure that your application connects to the database correctly and that the SQL query can be executed.
The full code for this example can be found on GitHub.
Integration in our stack
To ease the usage in our systems we have implemented a thin layer on top of the Testcontainers MySQL container. This implementation provides easy access to JPAApi, the interface used for any JPA interaction in Play, the framework we dominantly use for our JVM based services.
Another benefit of our integration is that it can automatically apply our Flyway migrations which not only allows us to provide the tests with the full schema required but we also test the migration files themselves. Flyway provides an easy to use API to execute the migrations in your application (where MYSQL is an instance of MySQLContainer):
Because we want to start each test with empty tables we truncate all of the tables minus the the Flyway schema table after each test run.
Even with all of those changes in place to minimize runtime the integration tests are still a lot slower than unit tests. To allow both types to be executed separately we add a separate config for integration tests in our sbt build.
This allows the unit tests to be executed using sbt test and the integration tests using sbt it:test. And with the JacocoItPlugin you even get code coverage metrics for the integration tests as well.
Combining Containers
I have mentioned before why I think writing service level integration tests can provide a lot of value. And we started doing the same for some of our applications. One example we recently worked on combines containers for MySQL with Kafka and elasticsearch.
The application under test is rather simple conceptually. It listens to a Kafka topic that has change data capture events coming in (we are using Maxwell for this). Those events are parsed, stored in a relational database and indexed in elasticsearch. Testcontainers allows us to have high level tests that cover the basic scenarios of this application.
When it comes to asynchronous processing, as is the case with Kafka, tools like Awaitility can help with writing efficient, readable tests. This is an actual example what one of those tests looks like.
One thing you definitively want to watch out for when implementing those kind of tests is the test runtime. We are using a separate sbt config to run the integration tests but still we want to execute those tests on each PR, so execution speed matters. Unit tests should still be the weapon of choice to test variations in your code flows. But testing your application using real components like database, message queue and search engine at least for some flows can help a lot making sure that your application actually does what it’s supposed to do.
