Automated Integration Testing

Zihao Liu
Nov 26, 2018 · 7 min read
Photo by Ilija Boshkov on Unsplash
  1. save time taken by manual testing.
  2. increase design and implementation flexibility by having a specification expressed through test cases.

Integration Testing Overview

Integration testing is a way of testing software by grouping software components together. In a complex software system, there are numerous interconnections applicable to a particular feature, which makes writing integration tests difficult. This can be illustrated in Figure 1. In this workflow, we have test input data that is provided to a system component A, which then interacts with components B and C. The result of the output is then verified with the expected output.

Figure 1: an example integration test workflow

Catalog System Overview

Jet’s product catalog primarily focuses on the task of matching product offer sources and then representing them holistically so that other teams such as search, discoverability, and front-end can utilize this information. The system processes around 750,000 messages for product data updates and snapshot indexing per minute throughout the entire system on a typical day (peak traffic above 1 million messages per minute during holiday’s); it consists of a collection of micro-services, which interact through a message bus. Some services will perform a simple transformation on their inputs with relevant product and business information; others will enrich inputs with relevant updates to the product data snapshot (e.g. product data information for Nike shoes), or projections of the product data to databases for business analysis in response to these inputs.

Integration Testing in Catalog System

With this background information, we can write a simple test case to verify if multiple offer sources can create a SKU. The pseudo-code looks like:

Send pricing, product and image data for offer source "a"
Expect SkuUpdate event with offer source "a"
Figure 2: a specific integration test workflow

Domain Specific Language

Ideally, we would like the test case to be concise, extensible and reusable. F#’s computation workflow feature allows just that. It gives developers the flexibility to write code in a simplified and understandable format close to plain English. The logic in the above test case can be rewritten in the following DSL:

Integration Testing for Catalog APIs

Prior to adding integration tests, testing for catalog APIs needed to be completed manually with coordination between product owners and engineers. This manual testing requires the use of tools such as Postman to verify that API endpoints produce a desired output. This is time-consuming and quite inefficient, as each feature release means additional coordination between product and engineering team. Integration testing automates this process by systematically checking for inconsistencies in the API output. In addition, since most of our APIs serve as front gates to the system, it also checks that a component functions correctly through the API response. With the help of integration testing, several major features in the catalog system can be safely released with each deployment, including functionality related to product matching, taxonomy, and grouping.

Conclusion

Overall, integration testing is a good way to guard against bugs that might appear in a production system. It also saves development and debugging time since any issue can be fixed after the test case for that feature fails. Hopefully, the information presented in this blog post would be informative for testing your production system. Happy coding and testing!

Acknowledgments

Designing and engineering a significant system is hardly an individual endeavor. The author would like to thank the Jet’s catalog team who have contributed to this work. For the post editing, the author is thankful for feedback from Ivan Glushchenko, Noah RobisonCox, James Novino, and Lev Gorodinski for advice on the structure and content of the post.

Jet Tech

Sharing our engineering org’s learnings & stories as we build the world’s best experience to shop curated brands and city essentials in one place.

315

315 claps
Zihao Liu

Written by

Zihao Liu

Jet Tech

Jet Tech

Sharing our engineering org’s learnings & stories as we build the world’s best experience to shop curated brands and city essentials in one place.