Acceptance Tests On CI Pipeline
In this post, I will try to explain the story of our testing process. The story has begun with refactoring one of our legacy applications. As a scrum team, we develop features and test them in sprints. If testing is successful then we ship them to our production environments. However, this delivery process sometimes takes more time than expected because of environment based problems. So these problems increase our Lead Time.
Lead Time: It is one of the 4 key metrics. Basically, it is an average time between starting development and deploying to production. High performer teams have low Lead Time metrics. For more information, I highly recommend you read the Accelerate book if you haven’t read.
I will briefly tell about the testing strategies and what we do about with them to decrease our Lead Time in further of this article.
Test Pyramid
The test pyramid describes the relative proportions of each type of test that you need to write.
- Unit tests — test a small part of a service, such as a class.
- Integration tests — verify that a service can interact with infrastructure services such as databases and other application services.
- Component tests — Acceptance tests for an individual service.
- End-to-end tests — Acceptance tests for the entire application.
Component Tests
Components tests are acceptance tests for a service. They verify your system behavior in isolation. In this testing strategy, you stub your dependencies and run your tests through them. The major difference between unit tests and component tests is that unit tests verify a single piece of part of your code, though component tests verify behaviors of all the units for each service.
Notice that all service dependencies are stub. Tests run against these stub dependencies so that testing operations are going to happen in an isolated environment and won’t affect the business code.
All the information we’ve talked about above, we decided that our API(service) needs to be tested independent of its dependencies, and also this operation should be automated. To create an isolated environment, we’ve used Docker-Compose. You can create containers for external dependencies and database and run them with docker-compose.
The below diagram shows the system architecture of our application. As you can see there is 2 external dependency. In Trendyol, we do care our services should be loosely coupled as much as possible. More external dependency means you have more failing scenarios to handle with. However, it may not possible in some scenarios. Contract testing also is considered in such scenarios.
Dockerize Everything
First, we need to build a custom couchbase docker image for the persistence operations of our application. Why custom couchbase docker image? Its because buckets and indexes have to be created before the tests are started to run.
Couchbase API endpoints are a good choice to create buckets, indexes, etc.
Couchbase DockerFile
As you remember there were 2 external API in the diagram above. To create an isolated environment also these 2 external API should be mocked. Wiremock is a good alternative to mock APIs and easy to use. As you can see Config folder copied from /home/wiremock path in the dockerfile.
The config folder contains the parameter patterns and responses of the application’s requests.
Wiremock Dockerfile
Run Dockerized Images
So far, all dependencies of the application are dockerized. Also, our application had already dockerized due to deployment. Now, the next step is to run all prepared docker images and see that tests are running.
It is important to specify the Links and Depends_On sections. Depends_On expresses the start order of containers. Our API has to be started after its dependencies start. And the Links make541 containers reachable by other containers by creating a network.
After the docker-compose file has prepared, now it is time to run in ci pipeline. We added a new stage(Test) in our GitLab ci pipeline.
When new codes are pushed to the repository, our pipeline will be triggered. The test stage runs the “docker-compose up — build …” script and docker containers(service, database, external dependencies, tests) start working. When all the containers are up then tests hit the service. If all the tests are passed successfully, the pipeline can be continued. Otherwise, pipeline fails!
To sum up, I tried to tell how we do the testing process with the help of docker-compose and ci pipeline. Also, there are other approaches to do that. For example, Trendyol Search Team deploys their feature-based containers to Kubernetes. You can read this post written by Gökhan Yılmaz Gökün for better understanding.
