Express.js under the hood

Digging into the mysteries of Express.js

Express.js is a minimalist, unopinionated Node.js framework sitting on top of node http.Server. There are plenty of tutorials out there for beginners, but we want to help existing users really get stuck in and learn Express inside out. This series of posts will analyse the logic of Express — line by line. Over the course of six posts, you will gain in-depth knowledge of Express; you will learn how to write any app; you will even be able to contribute to the project on Github. Hold onto your hats: the mysteries of Express are about to be revealed!

Chapter 1: Initialisation

App handler

The first thing you’ll do when you begin writing any app is to create an Express instance using the following snippet:

import {createServer} from ‘http’;
import {express} from ‘express’;
const app = express();

Let’s take a look at what’s actually happening here.

When you first call express() the following method is called:

//lib/express.js
module.exports = function createApplication() {
    var app = function(req, res, next) {
        app.handle(req, res, next);
    };
    mixin(app, EventEmitter.prototype, false);
    mixin(app, proto, false);
    app.request = { __proto__: req, app: app };
    app.response = { __proto__: res, app: app };
    app.init();
    return app;
}

It’s a factory method that creates a new Express app each time you use it. The created app can be described as a request handler, with additional methods attached to it (by calling mixin that uses Object.defineProperty)

You probably already noticed that the signature (the number and type of parameters) of the handler is similar to what we pass to http.createServer. However, there’s one additional parameter, called next, that’s not used by node http.Server. To find out the purpose of next, let’s check out the app.handle source code.

// lib/application.js
app.handle = function(req, res, done) {
    var router = this._router;
    done = done || finalhandler(req, res, {
        env: this.get(‘env’),
        onerror: logerror.bind(this)
    });
    if (!this._router) {
        done();
    }
    router.handle(req, res, done);
};

When next is defined, the Express app calls it once all middlewares and route handlers have been called (and there was no response). It’s used to pass errors up the chain so it’s possible to have a global middleware for all failures. When next is not defined, the Express app uses the finalhandler library to end the middleware chain.

Next will be defined only when you use a sub-app pattern to modularize your code. Take a look at the following snippet:

import express from ‘express’;
import api from ‘./api’; // let’s assume it’s an express app
const app = express();
app.use(‘/v1’, api);

Because we pass the the Express app (api) to app.use, it will land on the middleware stack and will be entered when there’s a matching path in the incoming request. We’ll talk about this more in the next post, when we discuss routers (and why I think they are the most important part of the codebase).

Going back to the initialisation example — once we define a handler, we attach request and response to Express linking their prototypes with the node.js built-in http.IncomingMessage (often referred to as req) and http.ServerResponse (often referred to as res). We will discuss their purpose later on. In a nutshell, this pattern will be used to decorate req and res on a per request basis as patching built-in Node objects can cause bugs and is ill-advised.

Listening

There are two ways you can start listening to incoming requests. As already described, Express is a tiny layer on top of http.Server. So it needs to start the server in a way you would’ve done manually:

var server = http.createServer(function(req, res) {});
server.listen(process.env.PORT || 3000);

In Express, you use app.listen to start the server so let’s take a look at its implementation to see how this problem has been addressed:

app.listen = function(){
    var server = http.createServer(this);
    return server.listen.apply(server, arguments);
};

As you can see, the method proxies the call to the underlying http.Server which is returned after initialisation. It passes this to createServer because the Express instance is actually a request handler with a similar signature (see above). After successful initialisation, the server starts listening and the newly created instance is returned. If you need to operate with a pure http.Server instance, you should assign a returned reference and use it according to your needs.

The common approach in the MEAN stack world is to use Socket.io with Express. Socket.io expects the http.Server instance as its argument, which becomes tricky with Express. From the previous snippet, we’ve seen that http.Server is returned after the Express app is successfully mounted and the server is listening for incoming requests. It means that any further operations on the http.Server will be done live, which may cause a production lag before Socket.io starts. Therefore, you might want to create a Socket.io server before starting to listen to real incoming requests so both Express and Socket.io are all started at the same time, which renders the above app.listen method ultimately useless.

Let’s take a look at the alternative initialisation featured in Socket.io docs:

var app = require(‘express’)();
var server = require(‘http’).createServer(app);
var io = require(‘socket.io’)(server);
server.listen(80);

The first line creates an Express app. Easy. In the second line, we createServer manually and we pass the Express app as the request handler. As we haven’t started listening yet, we can safely plug in Socket.io without worrying about a request that may come in the middle. After all the dependencies are created, we can safely start listening. Note — we start listening directly on the http.Server as there’s no longer a need to use an Express proxy to approach this.

In this post, we’ve learned how the Express initialisation works under the hood and why there are two ways we can start the server listening to incoming requests. It’s the beauty of the Node.js world that every module is highly extensible. We will see this in further posts as well, especially when looking at router implementation which, thanks to a few clever tricks, can be nested in an unlimited number of ways.

See you in the next post when we will investigate `Router` class and discover what the mythical `lazyRouter` in Express app stands for!