Life at Chime
Published in

Life at Chime

How we made our React Native tests run 5x faster

By Arthur Lee, a Chime Software Engineer

Every day, millions of members use the Chime mobile app to achieve financial peace of mind. At Chime, we use React Native to build our mobile app. We cut app releases from our main branch every week so that we can deliver the best experiences to our members as quickly as possible.

When moving at such a high velocity, we also have to make sure we aren’t causing any regressions along the way—which can negatively impact our members, who rely on Chime for necessities like groceries, gas, and paying bills. Despite having a QA process that validates every release, we want to catch issues early so they can be addressed without disrupting the release cycle. We have a Continuous Integration (CI) suite with automated tests that run for every mobile Pull Request (PR). A majority of these tests are end-to-end (E2E) tests written using Detox, which tests our app’s features in an iOS simulator against a mocked backend.

Problem

As Chime has grown, so has our mobile engineering organization. In the space of 12 months, a once ten-person team quickly grew to over 30 developers across multiple teams. As we wrote more code and shipped more features, our CI times ballooned.

Last year, running the basic test suite took approximately an hour, and running the full suite took as long as two hours! This led to a myriad of issues: Developers were afraid to add more tests because they worried about “slowing everyone down,” resulting in lower test coverage, and developers couldn’t reliably predict how long it would take for them to get their PRs merged—causing frustration for product managers. Plus, last-minute fixes and improvements didn’t ever make the release cut. Developers were constantly context switching, lowering productivity.

From one hour to 35 minutes

At the time, our CI tasks had been set up to run serially, which made sense given the team’s former, smaller size, but it soon became clear that we would need to run tests in parallel. However, this wasn’t as straightforward as it seemed. Unlike unit tests, which are lightweight to bootstrap, our Detox E2E tests take over 20 minutes to bootstrap (because Detox needs to build a custom version of our iOS app before tests can begin).

While we could theoretically build the app N times across N workers for every PR, that would be a waste of resources, especially given the high costs of macOS workers. Instead, we were able to leverage CircleCI’s workspaces feature to build the app once and share the built app with multiple test-running workers with minimal overhead. As a bonus, we were able to use lower-cost linux workers for our lint, type-checking, and unit tests.

With these techniques, we lowered the runtime of the full test suite to 35 minutes — a 40% reduction in runtime.

From 35 minutes to 15 minutes

While our initial speed increase was significant, we weren’t totally satisfied. No matter how many parallel workers we added or how few tests we ran, the suite still took longer than 20 minutes because of the iOS build step. Ideally, developers shouldn’t have to wait longer than 20 minutes to get feedback on their PR.

To make matters more interesting, a bulk of the 20-minute build time was spent with Xcode building the native portions of our app. But over 95% of our PRs only change React Native TypeScript code. Our CI system was building the same native code over and over again!

It turned out we can skip the iOS build completely for a majority of our PRs. Using the git hash of our ios/ directory as a cache key, we were able to create a cache of our iOS Detox builds. If a cached build was available, we would use a powerful (16 CPU) linux worker to build our React Native JS bundle using Metro — only taking about a minute. We would then copy the bundle into the built app before it is loaded onto the simulator. With these improvements, a 20-minute build process has been reduced to one minute in most cases.

Results: Happy developers

A process that used to take over an hour– as long as a flight from San Francisco to Los Angeles– now takes 10–15 minutes. In the time it takes to grab a coffee, developers get feedback about their PR, enabling them to ship more code safely and with less context switching. More importantly, these improvements have helped prepare our infrastructure for the years ahead — with the ability to scale up to hundreds of workers, our test suite can grow without any adverse effect on CI times. This lets us keep our codebase safe and our teams efficient and focused on delivering the best experience to our members.

--

--

--

Learn what it’s like to work at Chime

Recommended from Medium

Full-Stack To-Do List application with Node, NestJS, Angular 2+, Postgres, Liquibase and Docker…

Making Music With Machine Learning: Introduction to Magenta Music.js

If you happen to work in one of these areas, it’s worth giving TypeScript a shot. And if you

Customizable Node-red alternative

Introduction to Nightwatch JS

How to Build Tailwind CSS Plugins and Optimize for Production

What are Web Frameworks? And Why Should We Move Towards Them?

An Alternative Way to Use Angular Templates

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Talent at Chime

Talent at Chime

More from Medium

Introducing Fabric to react-native-screens

A better design pattern for instant feedback in React Native

Chain React Conf vs COVID — 2022

What makes Hermes engine (React Native) fast?