Perform integration tests anywhere using Testcontainers

And how to apply it easily in a SpringBoot application

Nowadays, it's common for applications to share responsibilities with other applications, which is good for individual development and scaling. On the other hand, it makes it more difficult to execute integration tests properly.

Imagine that you are coding an application using a database or message broker and want to test its behavior locally or even in a CI pipeline. What would be the best option?

First, if you are unfamiliar with integration tests, I would highly recommend reading this nice article about Test Pyramids and Microservices by my colleague Andrews Azevedo dos Reis.

For better analysis, let's create an example using a SpringBoot + JPA application from scratch, which must be tested against a PostgreSQL instance.

The context of this simple application will be inserting and retrieving a Person in the database. I'll write all the steps, but you can find the final source code here.

Creating the project

First of all, we should go to spring initializr and create a project. To simplify, let's import all dependencies needed in only one go. In this case, the project will use Java 17 and Apache Maven.

After importing it on your favorite IDE, the POM dependencies should look something like this by default:

<dependencies>
   <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-jpa</artifactId>
   </dependency>
   <dependency>
      <groupId>com.h2database</groupId>
      <artifactId>h2</artifactId>
      <scope>runtime</scope>
   </dependency>
   <dependency>
      <groupId>org.postgresql</groupId>
      <artifactId>postgresql</artifactId>
      <scope>runtime</scope>
   </dependency>
   <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-test</artifactId>
      <scope>test</scope>
   </dependency>
   <dependency>
      <groupId>org.testcontainers</groupId>
      <artifactId>junit-jupiter</artifactId>
      <scope>test</scope>
   </dependency>
   <dependency>
      <groupId>org.testcontainers</groupId>
      <artifactId>postgresql</artifactId>
      <scope>test</scope>
   </dependency>
</dependencies>

To make our assertions simpler in the future, let's add one more dependency in the POM.

<dependency>
   <groupId>com.github.npathai</groupId>
   <artifactId>hamcrest-optional</artifactId>
   <version>2.0.0</version>
   <scope>test</scope>
</dependency>

Setting up the code

In this project, there will be a Person and a PersonRepository.

//Person.java
@Entity
public class Person {

    @Id
    private int id;
    private String name;
    private int age;
    //getters and setters are omitted
}

For my tests, I would like to insert a Person in my Database and then retrieve it by name and age. Therefore…

//PersonRepository.java
public interface PersonRepository extends JpaRepository<Person, Integer> {

    Optional<Person> findTop1ByNameAndAge(String name, int age);

}

Very nice! Now, it's possible to create the first test, which basically consists in inserting a person and executing findTop1ByNameAndAge successfully.

Unit Test

Of course, before the actual integration tests, let’s create a unit test.

Stubbing everything. No database was used.

Local Database

All right, time for integration tests! If you are unaware of Docker or containers, the good old intuition is to install an actual database and configure its tables, ports, credentials, and everything else in a way that matches the current configuration used by the application.

By doing that and ensuring that the database is in the correct state, with no wrong data caused by another test that was executed (or interrupted) earlier, we could get our tests running correctly.

spring:
  datasource:
    url: jdbc:mysql://localhost:3306/db
    username: sa
    password: password
  jpa:
    hibernate:
      ddl-auto: update

A simple test to check if an entry was inserted into DB.

But what would happen if this same application is built and the tests run on another machine? If PostgreSQL is not installed, it will probably break due to a connection timeout error. Since the test depends on something it cannot control, even correctly written tests could fail.

In Memory Database

SpringBoot supports the H2 in-memory database, so even if you don't have PostgreSQL installed on your machine, it could be used instead. The problem is that H2 is not the database that you should be testing against, but a PostgreSQL, preferentially with the same software version.

It could be a good option for small applications with only JDBC integrations. But remember that using an in-memory database for integration tests has some known drawbacks and should be avoided when possible.

Its implementation is simple, and because @DataJpaTest annotation already configures the H2 database connection, it really looks like the previously shown Unit Test. Kudos for Spring.

H2 Database integration test.

Docker Containers

Instead of installing the whole software, another possibility is to start a Docker container. It's easier and requires less work than installing the software manually. But the challenges with configuration in multiple places (Dockerfile, application.properties, etc.) and ensuring that the environment is clean for the tests are still a thing.

Fortunately, containers are ephemeral, and to get a brand-new and clean database during our tests. It's only needed to destroy and start a new container! And that's the difficult part, right?

Docker containers give a fully configurable environment that can run anywhere. On the other hand, it's challenging to write scripts to control its lifecycle by hand. It seems a lot of effort, especially for testing purposes.

Of course, there are some workarounds and strategies to do this using Maven, but wouldn't it be perfect if it was simpler to configure and launch it in sync with the desired integration test executions?

Testcontainers

What if the containers' lifecycles and configurations could be controlled directly on our test codebase? With Testcontainers, the problem with configuration and clean environments for tests are gone in a very fancy way.

Using some nice annotations, it’s possible to configure a specific Docker Container to be run within our tests and shut down as soon as our tests are done. Being a perfect solution for executing integration tests both locally and in CI/CD pipelines (As long as they have Docker installed, of course).

Integration test using SpringBoot + JUnit5 + PostgreSQL Testcontainers module.

And beyond PostgreSQL, Testcontainers has plenty of other modules, such as MySQL, Azure CosmosDB, AWS DynamoDB, Kafka, Selenium, and many others. And if the desired module does not exist, it's possible to create your own generic container and use your own image to perform integration tests with whatever you want in a controlled environment.

Example using Generic Container

In fact, Testcontainers goes beyond and lets you import your own Dockerfile or even create a "Dockerfile" via DSL inside your integration test 🤯.

new GenericContainer(
         new ImageFromDockerfile()
                   .withDockerfileFromBuilder(builder ->
                         builder
                                 .from("alpine:3.16")
                                 .run("apk add --update nginx")
                                 .cmd("nginx", "-g", "daemon off;")
                                 .build()))
                 .withExposedPorts(80);

Bonus 1: Beyond testing

Testcontainers also provide the possibility of using its APIs for other purposes beyond testing! So, if you want to mess around creating containers using Java or any other supported language, you can go ahead! There are even some experimental usages of Testcontainers being used as a local development environment fully integrated with Spring Dev Tools!

Bonus 2: Testcontainers Cloud!

What if you cannot / don't want to have Docker installed on my computer (Or I have a MacBook with an M1 processor that won’t run most of the images)?

Fortunately, AtomicJar has a solution to this problem. Testcontainers Cloud lets the user run tests pointing to containers in the cloud. It is currently in private beta testing but will soon be available. :)

I have applied for the private beta, but unfortunately, I still could not get access. For more information, check the announcement.

Do you think you have what it takes to be one of us?

At WAES, we are always looking for the best developers and data engineers to help Dutch companies succeed. If you are interested in becoming a part of our team and moving to The Netherlands, look at our open positions here.

WAES publication

Our content creators constantly create new articles about software development, lifestyle, and WAES. So make sure to follow us on Medium to learn more.

Also, make sure to follow us on our social media:
LinkedIn — Instagram — Twitter — YouTube