Testing Express js with node-vm

TL;DR: Check out the code below for the example and the demo repo for runnable demo.

Recently we started work on splitting our app into two:

• the marketing pages: landing pages, funnels, promotions, etc.
• the webapp: plays music and interacts heavily with our APIs

We wanted to do this mainly to give the marketing team autonomy and not tie them to our release cycle.

Requirements

We wanted to have the speed of a statically generated site but the flexibility a of a fully fledged node server so we started with react-static and built our own server on top of Express.

This server would receive all the incoming requests to www.idagio.com and do the following:

• select the user locale based on geolocation
• serve statically generated A/B variants of the landing pages
• redirect requests to the webapp domain for legacy URLs

Testing

We wanted to test our server and looked at various articles and recipes on testing Express routes and mocking dependencies. Most suggestions we could find didn’t fit our use case.

One of the issues we couldn’t easily resolve, for example, was stubbing response.sendFile. Generating static site with build-specific configuration just for testing was clearly a bad approach as well.

We have spent some time with supertest and tried stubbing require calls but we weren’t happy with any of the approaches. The issues were speed and the amount of code you had to change just to get the tests to run.

So our goals were:

• stub the Express API
• stub request/response that Express handlers interact with, including response.sendFile
• stub require calls to isolate our test

Node Virtual Machine

Previously, when testing logic specific to Service Workers, we relied on node-vm’s functionality to recreate browser specific Service Worker API in a test.

The VM API has this fantastic quality, where everything can be defined in your own code, including process and require. It is also smashingly fast!

[…] Inside such scripts, the sandbox object will be the global object, retaining all of its existing properties but also having the built-in objects and functions any standard global object has. Outside of scripts run by the vm module, global variables will remain unchanged.

So, we realised that we could easily stub all the require calls that our handlers relied upon, while also stubbing express API.

⚡️ So fast! ⚡️

The following code demonstrates this technique…

The code

Here’s a simple server that just returns a static HTML file:

import express from 'express';
const app = express();
app.get('/a', (req, res, next) => {
  res.sendFile('dist/a.html');
});
app.listen(process.env.PORT, () => {
    console.log(`Server: listening on port  ${process.env.PORT}`);
});

and the test which asserts you are sending the right file (using mocha):

import vm from 'vm';
import sinon from 'sinon';
import path from 'path';
import assert from 'assert';
import {transformFileSync} from 'babel-core';
const transpiledCode = transformFileSync(
  path.join(__dirname, '../server.js')
).code;
function getServerInContext() {
  const getHandlers = {};
const listeners = {
    get: (request) => {
      const fn = getHandlers[request.path];
      const response = {
        sendFile: sinon.spy((filePath) => filePath),
      };
      const next = sinon.spy(() => ({}));
      fn(request, response, next);
return { response, next };
    },
    listen: sinon.spy(() => ({})),
  };
const express = () => ({
    get: (getPath, fn) => {
      getHandlers[getPath] = fn;
    },
    listen: listeners.listen,
  });
const sandbox = {
    express,
    require: (target) => {
      const packages = {
        express,
      };
      return packages[target];
    },
    process: {
      env: {
        PORT: 3000,
      },
    },
  }
vm.runInContext(transpiledCode, vm.createContext(sandbox));
return { listeners };
}
describe('Testing server', () => {
  describe('Setup', () => {
    it('Should listen on env port', () => {
      const { listeners } = getServerInContext();
      sinon.assert.calledWith(listeners.listen, 3000);
    });
it('should serve correct file for the path', () => {
      const { listeners } = getServerInContext();
      const request = { path: '/a' };
      const { response } = listeners.get(request);
const sendFileCall = response.sendFile.getCall(0);
      const sendFilePath = sendFileCall.args[0];
assert(sendFilePath, 'dist/a.html');
    });
  });
});

We are fairly happy with this approach, mainly because the only things we actually need to be there are stubbed and all this stubbing code is right before our eyes in the same file.

Bye console.log 👋

Another interesting advantage that we couldn’t easily find with other testing approaches was ability to easily silence/remove console.log calls from within tests to not pollute mocha test output:

const sandbox = {
  // simple noop to omit all logs, but could be a function
  // that collects all logs for later inspection
  console: () => {},
  express,
  ...
}

There is one slight disadvantage where due to transpilation step our error trace doesn’t always match line numbers. This could be addressed with adding source maps to the mix, but we haven’t yet explored this path.

We put the code above in an example repo here:

IDAGIO/node-vm-testing

Thanks to Misha Reyzlin for all his support and input.