So I still can’t run my Node.js 8 code on AWS Lambda, or can I?

It was an absolutely ordinary weekday at Zenitech when we decided to create an absolutely ordinary NPM lib to wrap some duplicated code we had. Let me explain: 5 different Git repositories contained code for 5 different microservices which we called GameUp. The lib supposed to abstract one of the features spanning in all 5 of those microservices. And I would not be writing about it now if we hadn’t noticed that one of the microservices was deployed on AWS Lambda. You see, all microservices except the one running on Lambda were deployed on Node.js 8 and we definitely took chance to use async/await JS feature (only available on Node.js 8 and onward). And we really liked that — to be honest we did not want to go back to callbacks and Promises! However, since Lambda still does not have Node.js 8 runtime as one of its options (and looks like its yet to come), we had a problem which we needed to solve somehow.

Isn’t Babel the answer?

At first it looked like a problem that should be quite easy to solve with Babel: we write some code for Node.js 8 and then transpile it to something poor current Lambda can understand. And that’s what we did! On another absolutely ordinary weekday we were at the point when there was something for Babel to crack. And suddenly we realized that its not that easy.

Here is a piece of code we used async/await in:

const someFunction = async (users) => {
  const start = new Date();
  const results = await promisify(map)(users, (async user => {
    const promisedUser = await Promise.resolve(user);
    ...
    return something;
  }));
  return new Date() - start;
}

After transpiling the start variable was put to async call function scope and was not reachable for new Date() — start expression. There were around 5 other issues we faced with transpiled code — very similar to the one discussed above, all related to async/await alone. Babel has at least 2 ways to transpile async/await — none of those worked for us.

Also, in my opinion, having to use Babel for back-end apps sounds like an unnecessary overhead. Though, I do find it useful for front-end apps.

In other words, Babel let us down and it looked like we had to reintroduce Promises / callbacks to end up with a mixture of 3 kinds of asynchronous code handling.

Then one weekday, that looked as ordinary as ones I mentioned before, idea comes to my mind. The idea is based on my previous attempts to try to spawn a child process in Lambda. For example, I was able to execute Headless Chromium in a child process in Lambda. Of course, the problem with such approach is that Chromium binary is more than 50 MB in size (does not fit to Lambda deployment package), so as soon as Lambda Function starts it has to download the binary from S3, and that means the whole thing becomes slow. Not to mention that Chromium process consumes hundreds of MBs of Memory. The good thing is that Node.js is not Chromium and it looks like it might be different enough to be run efficiently in a child process.

Not that childish Node.js child process

Node.js is small. Its compressed size is less than 10 MB. In Lambda terms there is more than 40 MB of space left for you code (Lambda package limit is 50 MB).

Node.js is quick to start. I could not find any well written source to support this statement, but it should be obvious from the small number of core modules and widely used third-party modules such as nodemon. In Lambda terms: my JS code is executed as soon as new process starts.

Node.js only consumes as much Memory as it is required to run application. If application is small — Memory consumption is small. I ran node command (that opens the Node.js shell) on my Linux machine and it consumed 7.5 MB. In Lambda terms: 7.5 MB is not that much compared to minimum amount your Function could be allocated, which is 128 MB of Memory.

Now let’s fit peaces together:

1. Node.js of your desire (for example Node.js 8) is bundled together with your JS code into a Lambda deployment package.
2. Upon initial Lambda Function execution a child process is spawned that starts the bundled Node.js runtime.
3. Node.js child process executes JS code.

I am sure that quite a few Lambda fans thought of this idea already, however, I could not find it in any form on the web (NPM to be exact). Therefore, I am proud to introduce you to a new-born open-source NPM module called lambda-node-runtime .

lambda-node-runtime

Feel free to find all the specific details in README.md or in source code (really not that much of it) to understand suitability for you Lambda Functions. There are some things you need to be aware of so Benchmarks section talks about some implications of using the module in your Lambda Function.

Is there a future for it?

Node.js 8 will eventually come to Lambda and so please all of those waiting. The paradox is that JS language evolves fast and so Node.js keeps releasing new versions. It appears Lambda fans will have to queue at Lambda “door” for the latest Node.js release again and again. Until things settle down. While waiting, let’s see if Lambda users will find lambda-node-runtime solution useful.