Migrating from Mocha to Jest

Running our test suite with Mocha took 12+ minutes. In CI with our beefy build machines we’re now able to run the entire Jest suite in 4 minutes 30 seconds.

Gary Borton
Jun 15, 2017 · 6 min read
Come check out our new digs at 999 Brannan, our second office in San Francisco!

We’d been using Mocha at Airbnb since September 2013, but due to increasing growing pains, we’ve recently migrated from Mocha to Jest. The migration actually turned out to require minimal changes to our tests and infrastructure, and provided a myriad of benefits.

Comparison of Mocha vs Jest, testing that 1 + 2 === 3

Minimal Changes to Tests

Most teams will not have to change the contents of their test files. In fact the below snippet was enough at Airbnb to paper over the differences between our usage of Mocha and Jest.

This is possible because the Jest API and the Mocha API are similar with only slightly different function names. Which functions you need to control for (or change in your tests) depends greatly on which functions you’re using. We also had a few test calls, which were 1:1 replaceable with it which is what we’ve decided as standard. Below is a quick example of what our tests look like before and after our migration to Jest.

Simplified Testing Architecture

To solve this problem we wrote some custom logic to batch our tests into chunks, run them in separate processes, collect coverage on each, and then merge the coverage into a single report at the end.

Coverage was collected (and enforced) in a different CI job than our test suite. The test suite itself was later parallelized by dispatching chunks of the test suite to different worker machines and aggregating the results at the end.

Jest automatically handles splitting tests across processes, collecting and aggregating coverage. This was a major perk for us. By utilizing this fact we were able to remove our custom logic in both jobs, and rely solely on Jest to handle this intelligently for us.

Improved Performance

  1. Parallelization. This is probably what Jest gets the most attention for, and for good reason. If you aren’t already parallelizing your CPU-bound work, you can expect to see a large performance gain by doing so.
  2. As stated above we were already parallelizing our tests at Airbnb, but we were doing so by getting a list of all our test files and dividing them equally among our workers. This left opportunity for a worker to get either an abnormally fast or abnormally slow queue of tests, and led to wasted CPU cycles. Jest instead takes a round-robin approach, and runs your slowest tests first, helping you to squeeze the most out of your processing power.
  3. It has a built-in babel transform cache. Applying transforms to code is very CPU intensive. By utilizing a cache that is shared across processes, you can dedicate your CPU to running your code and cut a lot of time off your runs.

With Mocha our suite took ~45 minutes to run locally, and sometimes it wouldn’t complete at all due to the memory pressure of running our full suite in a single thread. With Jest it’s down to 14.5 minutes locally. We saw a similar improvement on our build server with Mocha clocking it at 12+ minutes (after our work to parallelize across machines) and Jest finishing in 4.5 minutes.

Reducing Flakiness

Being run in isolation means that it is impossible for a test to fail due to side effects of other test files in your suite. This is especially helpful for errors that are thrown in setTimeout calls that happen after a test has completed. Now it’s much easier for us to investigate a flaky test by checking the test and source file for any asynchronous code.

After migrating to Jest and fixing the tests that failed in isolation we were able to reduce our flake rate to ~1%. This saves our developers hours of time per work day as they no longer have to wait for a build to fail, and repeatedly rerun the test suite until it passes. Additionally on the rare occasions that flake does happen, we’re able to more accurately identify where it is coming from. It’s easier to identify, because of file runs in its own process, so it is guaranteed that flakiness is coming from within that file. With Mocha, a bad timer in file x, could cause a test to fail in file y.

If you’re dedicated to reducing flakiness, you can take this a step further and reduce flake in your tests even more by killing timers that are set in your tests:

Taking Performance a Step Further

This turns out to be really problematic in Jest. Because each test file is run in a clean virtual machine, Jest reruns the spec_helper.js file once for each test file. In the case of the above example, importing chai-enzyme starts a chain that imports all of enzyme, which then imports all of React and ReactDOM. This takes 480ms even for tests that do not include React. In our case 480ms * several thousand files meant that we were spending over a minute just setting up this library. With Mocha, we didn’t feel the pain from this because it isn’t parallelized, and only runs the spec_helper.js file one time.

To get around this, we got a little bit creative with Jest’s mocking capabilities. By utilizing the callback on jest.mock() we were able to intercept enzyme imports, and load chai-enzyme only for tests that need it.

Developer Sentiment

Just wanted to come in here and give a big ++ to anyone who worked on the Jest implementation. The amount of dev time it saves is uncanny. Like, what used to take 3 hours to write now takes like 30 minutes. No joke.

We are always looking for talented, curious people to join the team. Or, if you just want to talk shop, hit me up on twitter any time @garyborton.

Airbnb Engineering & Data Science

Creative engineers and data scientists building a world where you can belong anywhere. http://airbnb.io

Thanks to Joe Lencioni and Adam Neary

Gary Borton

Written by

Software Engineer @airbnb.

Airbnb Engineering & Data Science

Creative engineers and data scientists building a world where you can belong anywhere. http://airbnb.io

More From Medium

More from Airbnb Engineering & Data Science

More from Airbnb Engineering & Data Science

React Native at Airbnb: The Technology

More from Airbnb Engineering & Data Science

More from Airbnb Engineering & Data Science

Sunsetting React Native

More from Airbnb Engineering & Data Science

More from Airbnb Engineering & Data Science

What’s Next for Mobile at Airbnb

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade