In-Cluster Testing Framework for Docker containers
Make Continuous Delivery with Quality REAL
Microservice architecture has been adopted by software teams as a way to deliver business value faster. Container technology enables delivery of microservices into any environment. Docker has accelerated this by providing an easy to use toolset for development teams to build, ship, and run distributed applications. These applications can be composed of hundreds of microservices packaged in Docker containers. These containers are deployed and running on AWS or any other IaaS (GCP, Azure, OpenStack)
In a recent NGINX survey [Finding #7], the “biggest challenge holding back developers” is the trade-off between quality and speed. As Martin Fowler indicates, testing strategies in microservices architecture can be very complex. To address this complexity, we need a framework that developers working in the Docker ecosystem can easily adopt without a behavior change.
As Testing becomes increasingly complex, how do we achieve continuous testing without sacrificing the speed of deployment while making it simpler for developers?
Introducing “Tugbot” — A framework for in-cluster testing
Just like a tugboat brings container ships safely and securely into port, Tugbot brings quality Docker containers into production on infrastructures like AWS. Tugbot makes Continuous Testing REAL by providing a framework for testing microservices in containers. Any kind of test (e.g integration, component, contract tests, security, chaos, etc) can be run with five-six lines in a “Test Container” (standard Docker container).
Leveraging Docker LABEL and integrating into Docker events via the Docker Remote API, Tugbot simplifies testing of microservices in Docker containers running in clusters. Tugbot standardizes results collected for analytics to continuously improve the quality of software. These “test containers” can be shared. Just like Github has enabled social-coding, this provides collaboration for testing — “social-testing”.
If a team has a few services, testing can be simple. Testing as part of the CI build process, primarily unit, works for services in isolation. However, the complexities of running services in an environment dev/test/stage or even production can cause significant challenges. As we scale to 10, 15, 20+ services, where services and environment are changing by events caused outside the CI process, testing becomes complex. A CI system is not dealing with a real environment with real data or security and governance issues in production environments. In microservices architecture, the CI flow is “per service”. Unlike a CI for a monolith, managing user acceptance tests, integration, component, exploratory tests as part of a running system is very challenging. This is where running tests on services in a REAL enviroment is needed. A framework to simplify and standardize the way we run our tests in clusters (Swarm, Kubernetes, or Mesos) as part of the CD (Continuous Delivery) process can address some of these challenges. These tests can run periodically, continuously, or triggered based on any event in ANY environment. This framework will enable developers to speed up development and not have to trade off quality for speed. The framework introduces a test container, event driven testing, standardized collection of results, and sharing (ie. “social testing”).
The framework addresses the following use cases:
- Simplify and standardize running ANY test (integration, component, performance, chaos, security, etc) using a standard Dockerfile — A “test container”
- Trigger test on specific events — Event-driven testing
- Docker events: image update, new container, etc (supported)
- Timer events: CRON — time interval, etc
- Host events: kernel updates, host restart, package update, config update
- External event: User driven, etc
3. Standardize collection of test results from all machines. Aggregating and analyzing test results over time enables tracking quality of the services running in containers deployed in clusters with the context of the environment and the tests
4. Share “test containers” — “social testing”. By creating test containers, development teams can share these containers with their peers to streamline the test creating and enable standardized collection of results
Tugbot is available as open source and currently supports Docker swarm. We plan to add support for Kubernetes next.
Tugbot — Open Source Framework
- Tugbot repo
- Example app repo with Swarm support — This is the same application that was used at DockerCon 2016. You can clone it to see how tugbot works in action. We have also integrated “pumba” with network emulation our open source chaos injection tool inspired by Netflix simian army.
We gladly accept pull requests.