HTTP API Testing - A Slightly Different Way To Assert

By George Lutz

At Trimble MAPS, we support hundreds of web APIs, some used directly by users, others only used internally within our products. We test these APIs with tools like Postman and JMeter - both are incredibly valuable.

Here, we’ll focus on Postman but the idea is generic to any HTTP test client.

Most API tests inspect the response body, typically a JSON object, for some expected result. If the response fails to match the expected response then the test fails. Which parts of the response should be inspected though? Take this JSON response for example:

[{"id":0,"name":"George","timestamp":”3898394839023984"},{"id":1,"name":"Bob","timestamp":"3838291839013981"}]

The timestamp is variable, therefore we can avoid testing it. Tests like this will work well:

var data = JSON.parse(responseBody);
tests["check name 0"] = data[0].name === "George";
tests["check id 0"] = data[0].id === 0;
tests["check name 1"] = data[1].name === "Bob";
tests["check id 1"] = data[1].id === 1;

Yet we can do better by performing a deep compare on the expected response against the actual response:

var data = JSON.parse(response.body);
 
// data_expected is a raw JSON object not a string. 
var data_expected = [{"id":0,"name":"George","timestamp":"3898394839023984"},{"id":1,"name”:"Bob","timestamp":"3838291839013981"}]
 
// use UnderscoreJS isEqual function to deep compare the objects.
tests["match full response body"] = _.isEqual(data, data_expected);

Advantages:

1. Less code. We’re checking every single field for free, without lots of repeated, error prone code checking individual fields.
2. Scales well. Imagine a much larger object. The code to diff the entire object would be exactly the same. This code is reusable by swapping in a different expected value.
3. Comprehensive coverage. Not only does this check flag a difference in values but it flag any change at all. For example, a new field added to the response. We want to know when anything changes.

But we missed one thing. The aforementioned evil timestamp field changes constantly, breaking the diff, while the rest of the response remains constant. Luckily, there is a way to omit checking of dynamic key/value pairs. Let’s try again:

var data = JSON.parse(response.body);
 
// data_expected is a json object not a string. 
// timestamp is removed from the expected response.
var data_expected = [{"id":0,”name":"George"},{"id":1,"name":"Bob"}]
// use underscore js to omit the dynamic item from each element in the array.
var _data = _.map(data, function(o) { return _.omit(o, "timestamp"); });
 
// as in the initial example, the isEqual deep compare is now workable.
assert.ok(_.isEqual(_data, data_expected), "match full response body");

Unless much of a response object is dynamic, we prefer to use the deep compare.

Interested in joining our team at Trimble MAPS? Click here!