Golang Test Containers
In the realm of software development, integration testing plays a crucial role in ensuring that the various components of a service or application interact correctly with each other before the entire assembly is put into a live environment. While each part must work in isolation, the reality is that constant changes to the code can generate unexpected interactions, causing the system not to behave as it should once the elements are assembled. These tests not only affirm that the connections between the different modules are solid but also allow developers and testers to catch anomalies early and minimize risks.
With the introduction of Test Containers, developers now have a robust tool to create isolated and consistent testing environments. These lightweight, throwaway instances of real runtime environments allow us to simulate various external components, ranging from databases to third-party services, without the risks inherent in testing against production systems. But what exactly are Test Containers, and how do they revolutionize how we approach integration testing?
This article will delve into the world of Test Containers, providing a comprehensive demonstration of their application within the Golang ecosystem. We will explore the advantages they offer for setting up predictable and isolated test environments and discuss the potential drawbacks and considerations developers must keep in mind. By weighing the pros and cons, we aim to provide a balanced view that helps you assess the suitability of Test Containers for your integration testing needs. Finally, we will conclude with a summary of how Test Containers contribute to delivering high-quality, robust software ready to face the challenges of the real world. Stay tuned for an in-depth exploration of this powerful testing strategy.
What are Test Containers?
Test Containers is a testing solution that provides ephemeral and isolated environments using Docker containers. It allows developers to run databases, queuing systems, web servers, and other services that their applications require for integration testing. This tool ensures that each test runs in a clean and controlled environment, mitigating inconsistencies between different development environments and simplifying the configuration and deployment process by using container definitions that can be versioned and shared among the team, facilitating collaboration and increasing test reliability.
Containers are lightweight, portable software units that encapsulate the code of an application along with all its dependencies (libraries, binaries, configurations) so that the application can run quickly and reliably from one computing environment to another.
It should be noted that these containers will only be deployed when the tests are necessary, i.e., during the Continuous Integration phase and, if necessary, during Continuous Deployment. These containers, being volatile, stop working the moment that phase is finished and allow the pipeline to proceed with the next configured step. This provides a significant advantage: the ability to recreate the same tests without the need to configure specific elements for our environments.
In the world of Go development, the idea of Test Containers has gained much popularity, and it is not difficult to understand why. Imagine putting together a piece of furniture: you may have all the pieces in perfect condition, but you won’t know if they fit together until you try to assemble everything. The same thing happens with software components; test containers allow us to test how those “pieces” of the software interact without the need to assemble everything in our production environment. Let’s say you’re working on a Go service that needs to interact with a PostgreSQL database. You could set up a local database for your integration tests, but that would force you to manage a lot of details like making sure all developers have the same version of PostgreSQL, that the initial state of the database is the right one for each test, among other cumbersome issues.
This is where Test Containers come into play. They are like temporary furnished apartments for your components while you test them. With them, you can automatically raise a PostgreSQL instance in a Docker container every time you run your integration tests. This ensures that they are in a cool and controlled environment without affecting your system or that of your teammates. Test Containers offer you this convenience thanks to libraries that facilitate their use. In Go, packages such as testcontainers-go allow you to integrate Docker containers into your test flow. Imagine the following simple example:
package main
import (
"context"
"database/sql"
"log"
"testing"
_ "github.com/lib/pq"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/wait"
)
// TestIntegrationWithPostgres test used just for this article purpose
func TestIntegrationWithPostgres(t *testing.T) {
ctx := context.Background()
req := testcontainers.ContainerRequest{
Image: "postgres:13",
ExposedPorts: []string{"5432/tcp"},
Env: map[string]string{"GLOBANT_POSTGRES_PASSWORD": "secret"},
WaitingFor: wait.ForLog("database system is up and ready"),
}
postgresContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
t.Fatalf("Could not start container: %v", err)
}
defer postgresContainer.Terminate(ctx)
ip, err := postgresContainer.Host(ctx)
if err != nil {
t.Fatalf("Could not get container IP: %v", err)
}
port, err := postgresContainer.MappedPort(ctx, "5432")
if err != nil {
t.Fatalf("Could not get container port: %v", err)
}
dsn := fmt.Sprintf("host=%s port=%s user=postgres password=secret dbname=postgres sslmode=disable", ip, port.Port())
db, err := sql.Open("postgres", dsn)
if err != nil {
t.Fatalf("Could not connect to database: %v", err)
}
defer db.Close()
}In this code snippet, we are pulling up PostgreSQL in a Docker container every time we run our tests. The library testcontainers-go takes care of all the configuration necessary to make our PostgreSQL instance ready for testing, including starting and stopping the database automatically to ensure a clean environment. This strategy is so powerful because it eliminates ENV variables that could affect the outcome of our tests. With Test Containers, you can focus on what’s important: making sure the code works as expected in an environment that closely simulates production without the complications of handling external services directly.
Demo
Before starting, we must have preliminarily configured these artifacts, which will allow us to use cache (Redis), create containers (Docker), and deploy them in a virtual machine (Vagrant).
Initially, we will be configuring our repository in the following relatively simple way:
- build: it will contain base scripts to download dependencies and run our unit tests; along with this, we will contain a vagrant file within which you can run the various unit tests using the docker containers to be deployed in this.
- config: as its name suggests, this will contain configuration files, which initially will be, for this post, the configuration of cache under the Redis implementation.
- domain: it will work as a package where we will place our structs of the business logic of this component, as a test, and as the scope of this component, we will only have a struct.
- logic: it is contemplated as a package where all the business logic will be as its name indicates; in this package, the users are obtained and stored in cache as specified in the use case to be tested.
Focusing on the relevant points for this post, in the following file, we can identify simply how we obtain a list of users, and these are stored in cache (Redis), a fairly simple use case that will serve to demonstrate the benefits of Test Containers.
package logic
import (
"context"
"fmt"
"local/test-container-demo/config/cache"
"local/test-container-demo/domain"
"strconv"
)
// GenerateValues retrieves a list of users, selects three
// random users from that list, stores them in a cache,
// and then retrieves and prints the cached data for each of
// those users. It returns the full list of users and the
// identifiers of the three users that were cached.
func GenerateValues(addr string) (domain.Users, []string) {
ctx := context.Background()
rCache := cache.SetUp(addr)
var tracker []string
users := domain.GetUsers()
if len(users) > 0 {
for i := 0; i < 3; i++ {
randomUser := users.GetRandomUser()
userID := strconv.Itoa(randomUser.ID)
err := rCache.Set(ctx, userID, randomUser.ToString(), 0).Err()
if err != nil {
panic(err)
}
tracker = append(tracker, userID)
}
}
for _, index := range tracker {
val, err := rCache.Get(ctx, index).Result()
if err != nil {
panic(err)
}
fmt.Printf("cache value: %s\n", val)
}
return users, tracker
}Up to the moment of this implementation everything is seen in a quite simple way, the problems or headaches begin to arise at the moment we want to test these functionalities, in a certain way there are alternatives as “mock” to this type of calls but what this does is merely to assume a response that a supposed service is supposed to answer (intentional redundancy).
The same developer who performs the test contemplates this assumption, leaving aside some degree of credibility that this may work. Considering the above, without implementing a mock call to this service (Redis), we would get an error message of the following type because it fails to find it in its context.
Probably you will not have the same error, probably a different one, but most likely, you will get an exception because if we do not have in our environment an instance of Redis running, most likely we will get this type of exception, however, that does not mean that placing the IP address of an instance of Redis own environment would be ideal as this would compromise in some way the information that may be stored in that instance and even modifying another that should not be compromised. It is here where we test and explain a little more about the implementation of the Test Containers’ and we can validate by executing the following script in our base project.
cd build && vagrant up && vagrant provision && vagrant ssh && cd /app && ./build/run_tests.shAfter executing this script, we will see the output shown in the image below, and Test Containers deploy base containers with a specified image, like Redis, which our code details. Then we use a temporary instance that ends with the test, automatically closing the connection and removing the container thanks to a defer statement. It is how, in this way, fast and simple, we can interact with an actual instance of a service that we can have in test time and only at the time of executing our integration tests.
Just with this example, we can imagine a lot of other functionalities or possibilities within a more professional work environment; that is, let’s imagine the hypothetical case of having our private image repository (for example, a private version of Dockerhub) in which we can deploy instances at the time of integration testing of another service, for example a calculation component or a database for integration testing, this allows us to generate an added value to our tests instead of just assuming certain types of responses.
package logic
import (
"context"
"local/test-container-demo/domain"
"testing"
"github.com/stretchr/testify/assert"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/wait"
)
// TestGenerateValues creates a temporary Redis container to test
// the GenerateValues function. It ensures that users and their
// associated cache entries are generated and not empty.
func TestGenerateValues(t *testing.T) {
ctx := context.Background()
req := testcontainers.ContainerRequest{
Image: "redis:latest",
ExposedPorts: []string{"6379/tcp"},
WaitingFor: wait.ForLog("Ready to accept connections"),
}
redisC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
panic(err)
}
defer func() {
if err := redisC.Terminate(ctx); err != nil {
panic(err)
}
}()
testCases := []struct {
name string
users domain.Users
}{
{
name: "Test 1",
},
{
name: "Test 1",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
addr, _ := redisC.ContainerIP(ctx)
users, tracker := GenerateValues(addr)
assert.NotEmptyf(t, users, "users can not be empty")
assert.NotEmptyf(t, tracker, "tracker can not be empty")
})
}
}TestGenerateValues function checks if the GenerateValues one works correctly with a "Test Redis Container", this one confirms the ability to connect to the instance, produce values, and verify that the resulting user list and tracking data are populated, the container exposes the default Redis port and waits until it’s ready to accept connections. Once the tests are complete, the Redis container is stopped and removed.
Pros
When considering the integration of Test Containers into the testing workflow, several advantages become apparent; these benefits would contribute to a more reliable and efficient development process:
- Isolation and Consistency: Test Containers ensure that each test runs in an isolated environment created and destroyed for every test. This reduces the risk of tests conflicting with each other, and you can be confident that tests are not reliant on specific states on a developer’s machine or a build server, lending consistency across different environments.
- Replicable Environments: By leveraging Docker containers, Testcontainers makes it easy to replicate complex application stacks, including databases, message queues, web servers, or any other services that can run in a container. This helps in accurately testing code against the same software stack used in production.
- Ease of Use: Even though managing Docker containers can sometimes be complex, Test Containers abstracts a lot of the boilerplate setup and tear down logic into a simple API. This makes it more approachable for developers to integrate complex dependencies into their tests without extensive knowledge of Docker commands.
Cons
After delving into the potential downsides of Test Containers, it’s important to consider the following limitations that could impact the testing process and overall development workflow:
- Docker Dependency: Test Containers require Docker to be installed and running on any machine that runs the tests. This can be a limiting factor, especially in environments where Docker is unavailable or not preferred due to security or resource constraints.
- Performance Overheads: Spinning up Docker containers can add significant startup time to tests compared to tests that mock out dependencies. This can slow down the feedback loop during development and increase the total build time in continuous integration pipelines.
- Complexity and Resource Usage: Using real instances of dependencies might expose tests to additional complexity and flakiness since you are now dealing with real processes that need time to startupstart up and might have their own issues. Also, running many containers might consume significant system resources, which could be a limiting factor on developers’ machines or CI servers with limited memory and CPU.
Conclusion
In summary, ‘Test Containers" in the Golang ecosystem provide an innovative solution to the challenges of writing integration tests, promising consistent, replicable, and isolated testing environments. While developers must consider the balance between their efficiencies and associated complexities, ‘Test Containers” offer a powerful means to increase confidence in software reliability in the pre-production stage.
The key takeaway here is the realization that ‘Test Containers” can substantially fortify the testing phase, provided that teams are conscious of the resources and infrastructure at their disposal. They form part of a broader toolset, ensuring that software is not just developed but delivered with quality and robustness and ready for the rigors of real-world operation.
