Use Easy-Fix to Run Integration Tests Like Unit Tests

Dan Rathbone
Sep 7, 2016 · 5 min read

Software is complex, so testing it is difficult. To make tests comprehensible, engineers divide up responsibilities between different types of tests. Unit tests are comparatively simple — specific and granular, they narrowly target methods and functions in isolation. Integration tests focus on the interaction between software components. For web projects, this includes web browsers interacting with web services, though in this article we’ll simply focus on web services.

Compared with unit tests, integration tests are broader in scope and slower in execution. They may, for example, run “live” (in a test environment) where they interact with remote systems or mutate a database. Such complexities have a cost: the tests require more time and effort to set up, to run, and to understand when they fail. It seems tempting to avoid this complexity, by “mocking” the integration tests. Integration tests can be designed to run with curated “mock” data, loaded from local storage, which mimics the interactions with other components.

It is not immediately clear if testing with mock data is a good idea. Tests with mock data run quickly and consistently, like unit tests, which is appealing. However, mocked integration tests don’t actually interact with remote systems — the mock data limits the scope of the tests. If the remote system changes, the mock data can become stale, invalidating the test results and potentially providing false confidence in the components under test. Live integration testing takes time, but when it succeeds, it provides real confidence that the target software components actually work together.

Don’t compromise between mock and live integration tests

Easy-fix works like this:

  • Write your integration tests such that they run live, allowing network requests, database mutations, etc;
  • Wrap the side-affecting asynchronous tasks with a method from easy-fix.
  • Run those tests, which have a new option:
  • In ‘live’ mode, your tests run just as you wrote them, side effects allowed.
  • In ‘capture’ mode, your tests run like ‘live’ mode, but the arguments and response for each wrapped task are serialized and saved.
  • In ‘replay’ mode, the wrapped tasks are not called. Instead, they respond with the saved (mock) data.

In summary, easy-fix provides an easy way to capture test data, and feed this data back into the tests.

Usage Example

before( function () {
easyFix.wrapAsyncMethod(helpers, 'authenticate');
easyFix.wrapAsyncMethod(isd.IroRequest.prototype, 'getProductOffers');
easyFix.wrapAsyncMethod(isd.TaxRateRequest.prototype, 'getTaxRates');
easyFix.wrapAsyncMethod(restrictions, 'getRestrictonsForUpcs');
easyFix.wrapAsyncMethod(productFees, 'getFeesForUpcs');
});

The actual test code doesn’t need to be modified for easy-fix. A test of a walmart.com checkout might look like this:

it('checkout one item', function (done) {
attemptCheckout(
testUser,
checkoutDetails,
function (err, resp, body) {
var bodyJson;
expect(err).to.be.null;
expect(resp.statusCode).to.equal(200);
bodyJson = typeof (body) === 'string' ? JSON.parse(body) : body;
expect(bodyJson.data).to.exist;
expect(bodyJson.error).to.not.exist;
done();
}
);
});

Note that the test code doesn’t show any awareness of easy-fix. When we execute the test, though, the $TEST_MODE environment variable will determine if the tests interact with remote systems. In “live” and “capture” modes, the checkout service may make it’s related network requests and record the checkout in the database. In “replay” mode, the test does none of this, running only with fake mock data.

➜  export TEST_MODE=live
➜ mocha test/int/endpoints-with-auth.js -t 20000 -g 'checkout one item'
✓ checkout one item - UPCE (12704ms) 1 passing (16s)➜ export TEST_MODE=capture
➜ mocha test/int/endpoints-with-auth.js -t 20000 -g 'checkout one item'
✓ checkout one item - UPCE (14311ms) 1 passing (17s)➜ export TEST_MODE=replay
➜ mocha test/int/capture.js test/int/endpoints-with-auth.js -g 'checkout one item'
✓ checkout one item - UPCE (71ms) 1 passing (224ms)

Here at Walmart Labs, on the Store Services team, we started writing our integration tests to run live. An example test suite with a few dozen integration tests took around two minutes to run. That’s not the end of the world, but it’s slow enough that engineers would typically run them only when they considered committing new code. After integrating easy-fix, these tests run (in ‘replay’ mode) in 4 seconds. The tremendous speed-up allows the engineers to run the tests (in replay mode) much more frequently — such as every file save.

Fast but not magical

Tests in ‘replay’ mode, using mock data, typically test only one side of the target interaction. In practice, tests in replay mode still catch many (but not all) errors. The quick execution allows them to be run more frequently, finding many errors earlier in the programming process.

Conceptually, easy-fix is another record-playback system, which is nothing new. There are several examples of other projects that do this, such as

All these examples, though, focus exclusively on mocking HTTP calls. Strangely, we couldn’t find any existing record-playback systems that worked for other asynchronous tasks, which is why we created easy-fix.

To be clear, the crucial difference is that easy-fix will wrap any asynchronous nodejs method. Easy-fix can mock HTTP calls, database access, or anything else where you can serialize the requests and responses.

Software testing is still complex, but easy-fix provides one single, useful simplification. With easy-fix, you don’t need to compromise between mock or live integration tests.

The easy-fix source code is available at: https://github.com/walmartlabs/easy-fix

Walmart Global Tech Blog

We’re powering the next great retail disruption.

Thanks to Dave Cadwallader

Dan Rathbone

Written by

An independent software engineer living and working in Seattle, WA (http://danrathbone.com/)

Walmart Global Tech Blog

We’re powering the next great retail disruption. Learn more about us — https://www.linkedin.com/company/walmartglobaltech/

Dan Rathbone

Written by

An independent software engineer living and working in Seattle, WA (http://danrathbone.com/)

Walmart Global Tech Blog

We’re powering the next great retail disruption. Learn more about us — https://www.linkedin.com/company/walmartglobaltech/

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store