Kafka Streams: Testing

Writing comprehensive tests for a Kafka Streams application is essential, and there are multiple types of tests that should be considered by the developer before the application even reaches QA. These tests, covering unit, local integration, and component, are explored and demonstrated in this article.

The accompanying Kafka Streams Spring Boot application source code is available here.

This is one of a series of articles covering different aspects of Kafka Streams. Jump to the ‘More On Kafka Streams…’ section below to explore.

Test Goals

Tests should always be shifted left where possible, such that they live and run close to the code, following the principles of the Test Pyramid. The largest number of tests should be unit tests that are fine grained and written before (via Test Driven Development) or during the development of application classes. Next are local integration tests that are fewer in number and coarser grained, but ensure that end to end flows run as expected. Component tests then run the application up in a container, with fewer coarse grained tests that treat it as a black box, verifying the configuration and deployment locally, on top of the end to end flows.

As well as the ability to run these tests locally on the developer’s machine, these tests are automated in the pipeline…