Hacking Node Core HTTP Module

One of the modules I wrote to support NodeJS at @WalmartLabs is something I had to do out of necessity. It hacks and monkey patches the http module in NodeJS core. In this blog I will go over the why and how of it.

Uppercase vs. Lowercase HTTP headers

We have an SOA micro service architecture implemented in Java. It supports registration and auto discovery, client authentication, and a service proxy. The implementation adds a few custom HTTP headers.

One of the things our NodeJS app platform needed to provide was a standard way to connect to our internal micro services. I quickly ran into an issue; the services I was testing against all returned HTTP 500 with errors indicating that my requests were missing the custom headers.

After some sanity checks and trying with hand crafted cURL requests, I realized that the Java SOA libraries were only allowing the custom HTTP headers in uppercases. Unfortunately Node core automatically converts all HTTP headers to lowercase, which is fine because HTTP headers are case insensitive per RFC 2616.

The first thing I tried was to find the contacts for the team owning SOA. After some time it was clear a fix was not trivial because:

1. SOA v1.0 was deployed to all services and updating all of them would take some time.
2. The SOA team was working on ver 2.0 and have no available cycle to fix the issue.

That left me in a very hard place. Was our NodeJS endeavor blocked?

Monkey patch to the rescue

The dynamic nature of JavaScript enabled a phenomenon known as monkey patching. It allows you to replace or alter code that was not meant to be public. Generally it’s bad, but at times, it ends up being the only way to move pass a critical blocker.

Given that the lowercasing of the HTTP headers is hardcoded in Node core, it was clear monkey patch was my only option. So I started looking into the core http modules.

At the time, NodeJS was getting ready to release 4.2.4. Without much frills, I easily found the location where outgoing HTTP headers are converted to lowercase.

Monkey patching that code was also straightforward. I just copied the method setHeader, patched it, and replaced the original version:

var http = require("http");
http.OutgoingMessage.prototype.setHeader = function(name, value) {
  // original code etc
  var key;
  if (!name.startsWith("FOO_")) {
    key = name.toLowerCase();
  } else {
    key = name;
  }
  // original code that updates this._headers
};

And that did the trick. Upper case for any HTTP headers that starts with FOO_ from NodeJS. Of course our actual internal headers use another prefix signature.

Testing

Unit testing this was slightly tricky. Node core also converts all incoming HTTP headers to lowercase. So the test similarly monkey patches http IncomingMessage to capture and verify HTTP headers in their original form.

The actual test uses mitm to fake a HTTP server, and then sends out a request to it with some custom HTTP headers in uppercases. The server’s request handler then verifies that they didn’t get lowercased.

Node 8.3.0

The release of NodeJS 8.3.0 came with the brand new v8 engine TurboFan with much better optimization. I immediately tested our app’s React renderToString performance. The result was consistently a 60% reduction in rendering time for one of our production apps. That’s a huge gain. So we immediately set out to run regression with Node 8.3.0.

However, Node 8 has some major changes to the http module. One of it was that the OutgoingMessage object now hides the _headers field. Instead of this._headers, it does this[outHeadersKey], where outHeadersKey is a Symbol created here. What’s more, that’s an internal module the user code has no access to. Since a symbol is always unique and there’s no way for me to gain reference to the internal outHeadersKeysymbol, I can’t modify the outgoing headers with monkey patch anymore.

I spent some time looking at the pull request that made those changes to understand the rationale behind them. I also looked at the officially added methods for user code to access HTTP headers. There didn’t seem to be anything I can do about altering the hardcoded lowercasing behavior.

I threw my hands up for a moment upon realizing that. Is this the end to my monkey patch? Our SOA v2.0 has been released, but I am not sure about its adoption status. Does this mean using Node 8 was not an option for a while? It’d be a huge bummer if we can’t take advantage of all that performance gain.

The Symbol API

With no clear option forward, I started studying the new Symbol API document here.

Initially nothing stood out except the Symbol.for method. I thought that’s my key to gaining reference to the core’s outHeadersKey symbol, so I didn’t even read the doc and immediately tried it out. Well, after 5 minutes there was no luck, and I went back to actually read the doc. And I found the reason:

In contrast to Symbol(), the Symbol.for() function creates a symbol available in a global symbol registry list.

Ah, it’s the global symbol registry list, right! At this point I thought it was game over, but I continued to browse the document on Symbol.

Then I noticed the Object.getOwnPropertySymbols API. Not sure if this would work, I went to try it out.

Long story short, it worked, and here is how I managed to gain access to the internal outHeadersKey symbol.

const http = require("http");
const assert = require("assert");
function hackFinity() {
  const x = new http.OutgoingMessage();
  const s = Object.getOwnPropertySymbols(x);
  assert(typeof s[0] === "symbol" &&
    s[0].toString() === "Symbol(outHeadersKey)");
  return s[0];
}
const outHeadersKey = hackFinity();

Whew, monkey patch back in business and uppercase HTTP headers out from NodeJS.

mitm

Since mitm hacks Node at the core level, it stops working in Node 8. Not surprisingly, the unit tests stop working also because they depend on mitm. So I had to change the tests to start a real HTTP server on the localhost listening on a random port, and then send requests to it with the custom HTTP headers for verification.

Conclusion

Monkey patch is not a good idea, but it’s one of the blessings I found with using JS that got me out of hard places on a few occasions. It should only be used minimally when absolutely necessary. The little scare with Node 8 was a wake up call for me and I realized that I should’ve follow up with the SOA team to make sure there’s plan to fix the issue. For the time being, we are still moving pass this blocker with monkey patch. So far I haven’t notice any negative effects, but it’s something to look into further.