Unit Testing Express Middleware Behavior in ECMAScript 2015

Please note: I’ll walk through this pattern in more depth at MobileTea Boston on 9 March 2016. Sign up here: http://www.meetup.com/mobiletea/events/228891215/


As anyone who knows me professionally would attest, I am a very strong advocate of test-driven development.

In fact, unless I am experimenting with a framework or library I do not yet know, I almost always use tests to drive the development of software I write.

Most other engineers I encounter need little selling when it comes to the virtues of TDD, but many experience challenges when they actually try to write tests against a project they are developing.

This is understandable.

There really is not a silver bullet when it comes to setting up tests. There are a lot of great tools out there, but putting them together can be challenging, as can meeting the idiosyncratic needs of specific libraries.

Two years ago, I set out to solve those challenges, specifically with respect to testing Node.js webservers that use the Express framework. After working with a few dozen engineers in my department on this issue, I was able to synthesize this system for testing Express middleware.

The reception has been almost entirely positive. I and many developers I know use some form of the pattern I developed.

The pattern has undergone some improvements in recent months.

For starters, when originally written, the pattern used ECMAScript 5.1 and ran on Node 0.10. As such, it now can benefit from some revision to use such things as native promises, improved asynchronous flow control, and the ES’15 module loader.

Additionally, the more I used the pattern, the more I found myself agreeing with some who felt that I should strengthen separation of concerns. So, I have done so here.

Finally, the old pattern brought in some libraries that are no longer necessary and seem to have fallen out of some favor (e.g., Chai). I have replaced those with native functionality (e.g., Assert) where appropriate.

In light of these points, I would like to take a moment to describe a revised pattern for Express middleware.

Next week, I will return to do a second post describing an implementation of the pattern using ES’15 generators offer with Koa.

The Challenges of Testing Express

There are three challenges with testing Express middleware that, in my experience, cause people to avoid doing it:

  1. They assume that they are doing unit testing, when, in actuality, they are doing integration testing.
  2. They cannot sufficiently separate concerns to test middleware in isolation.
  3. They cannot exercise adequate granular control over asynchronous behavior to facilitate testing.

Let’s look at each of these challenges in light of a very stripped-down example Express application:

Confusing Unit and Integration Testing

import express from "express";  
export var app = express();
app.use(function (req, res, next) {  
res.send('hello world');
next();
});
app.listen(3000);

When one writes a webserver like this, it is virtually impossible to test at the unit level. Without delving into the internal workings of Express, there is no way to extract middleware that is declared inline and test it. Resultantly, most developers end up testing by writing something like this:

import app from 'app.js';  
import * as request from 'supertest';
it('sends "hello world" on the response body', (done) => {  
request(app)
.get('/')
.expect(200, 'hello world', done);
});
});

This is not an altogether unreasonable approach . . . to acceptance testing. In other words, if your contract with a front-end developer is that your API provide the response “hello world”, you can run the test any time you want to ensure that you met your contract, and you could sleep well at night.

The problem is that this is not unit testing.

Why does this matter? Look at the following example:

import express from "express";  
export var app = express();
const middleware = [  
function (req, res, next) {
req.responseString = 'HELLO WORLD';
next();
},
function (req, res, next) {
res.send(req.responseString.toLowerCase());
next();
}
];
app.use(...middleware);
app.listen(3000);

Now you are going to start to have problems unit testing middleware in isolation. You can still run an acceptance test for your contract, in which case this “refactor” would still pass the test. However, given that middleware are independent behavior, testing the contract will not be sufficient.

What you really want to know is whether the first middleware will add “hello world” to the responseString attribute of req, and whether the second middleware responds with the responseString value on req, no matter what that value may be.

Separation of Concerns

You will have to separate concerns to accomplish this, as follows:

// app.js
import express from "express";  
import {behavior1, behavior2} from "behavior.js";
export var app = express();
const middleware = [  
function (req, res, next) {
req.responseString = behavior1();
next();
},
function (req, res, next) {
const response = behavior2(req.responseString);
res.send(response);
next();
}
];
app.use(...middleware);
app.listen(3000);
// behavior.js
export function behavior1 () {  
return 'HELLO WORLD';
}
export function behavior2 (responseString) {  
return responseString.toLowerCase();
}

What we have now is one “behavior” that returns the string “HELLO WORLD”, and a second that converts it to lowercase. The first middleware puts the result of the first behavior on the responseString attribute of the req object, and the second sends the lowercased version of the string to the client.

Now, the ability to write unit tests becomes a reality. You can keep your acceptance test — it should still work — but you can also test the behavior in units:

import Assert from "assert";  
import {behavior1, behavior2} from "./behavior.js";
describe('behavior', () => {  
describe('behavior1', () => {
it('returns "HELLO WORLD"', () => {
return Assert.equal(behavior1(), 'HELLO WORLD');
});
});
describe('behavior2', () => {
it('returns "foo" when passed "FOO"', () => {
return Assert.equal(behavior2('FOO'), ‘foo');
});
it('returns "bar" when passed "BAR"', () => {
return Assert.equal(behavior2('BAR'), ‘bar');
});
it('returns "hello world" when passed "HELLO WORLD"', () => {
return Assert.equal(behavior2('HELLO WORLD'), 'hello world');
});
});
});

In reality, you probably don’t need so many tests for behavior2, but I put them in here to generate a point. Your second behavior is actually a lowercasing behavior in the abstract. It really has nothing to do with “hello world,” except that the endpoint called it with that string. You can now reuse behavior2 in other parts of your application where you need that behavior and you can be confident that it works.

Testing Async Behavior

Things get a little hairier when testing asynchronous behavior, but not insurmountably so. Fundamentally, what is needed is a way to control the flow of a test of asynchronous behavior.

Let’s take the following asynchronous behavior, which, though it may seem rather silly, will suffice for this post:

// async-request.js
export function asyncRequest () {
return new Promise((resolve, reject) => {
process.nextTick(() => {
resolve('HELLO WORLD');
});
});
};

This asyncRequest method uses process.nextTick to defer the execution of the function declared within it until the next tick of the event loop. As such, asyncRequest returns to its caller without having resolved the string “HELLO WORLD”. Instead, it returns a promise, which resolves in the next tick.

So, now imagine that behavior1 leverages asyncRequest to return “HELLO WORLD” in asynchronous fashion:

import {asyncRequest} from "./async-request.js"
export function behavior1 () {  
return asyncRequest();
}

Then, the endpoint can also be rewritten:

//app.js
const middleware = [  
function (req, res, next) {
behavior1().then(function (response) {
req.responseString = response;
next();
}
},
function (req, res, next) {
const response = behavior2(req.responseString);
res.send(response);
next();
}
];
 app.use(...middleware);

The endpoint now uses the promise returned from behavior1 to control asynchronous flow and only move onto the next middleware in the stack once asyncRequest resolves “HELLO WORLD”.

How do we test this?

it('returns "HELLO WORLD"', () => {  
return behavior1().then((result) => {
return Assert.equal(behavior1(), 'HELLO WORLD');
});
});

There are some libraries that will clean up the tests a little, such as assert-promise:

import {assertPromise} from "assert-promise";
// . . . .
it('returns "HELLO WORLD"', () => {  
return assertPromise.equal(behavior1(), 'HELLO WORLD');
});

When employing these strategies for testing ahead of actual coding, in a test-driven development fashion, the result is fully tested behavior-focused test-driven development of the behavior that is composed into Express middleware.

Show your support

Clapping shows how much you appreciated Morris Singer’s story.