Advanced Node.js Lambda Function Concepts
In this three-part-series we’ll look at getting a Node.js application which performs image resizing running on AWS Lambda. We’ll look at how Node.js applications work, then we’ll configure our AWS account and get the AWS CLI utility working. We’ll Create a Lambda Function within AWS and configure an Application Gateway which is required to route HTTP requests to our Lambda Function. All the while we’ll be looking at features of Lambda, both basic and advanced, while also taking into consideration security implications.
In this third and final post of the series we’re going to write the code for a more advanced Lambda Node.js application. This app will perform image resizing by accepting a URL for an image, then downloading and resizing the image. We’ll look at how to perform deploys of complex Functions from the command line. We’ll also look at some miscellaneous Lambda concepts which are very useful to know. If you’re feeling a little lost, please give the previous post “Basic Node.js Lambda Function Concepts” a read.
Sample Image Resizer Application
Now that we’re familiar with the basics of Lambda Functions let’s go ahead and build one! This Lambda Function will perform the “Hello World” equivalent in the serverless world, which is image resizing. The way we’re going to resize an image is to initiate the process by making an HTTP request through an API Gateway, have that gateway invoke our Lambda Function, and have that function download an image from our servers using HTTP. It’ll then perform the work to resize the image for us.
Initial Setup
Create a new empty directory, and issue the following commands within it to setup the project:
npm init
npm install --save jimp
touch resizer.js # Our lambda function
touch invoke.js # Convenient for testing
chmod +x invoke.jsCreate the Image Resizer
Now we’re ready to create our resizer application. Edit the resizer.js file using your favorite editor and paste in the following content:
const jimp = require('jimp');
const http = require('http');
const fs = require('fs');const DOWNLOAD_FILE = '/tmp/rawfile';
const RESIZE_FILE = '/tmp/resizedfile.png';
const DEFAULT_SIZE = 32;module.exports.downloadAndResize = (event, context, lambdaCallback) => {
if (!event.url) return lambdaCallback(new Error('missing event.url'));
const start_orange = Date.now();
console.log('Request (orange box) starts now');
const size = event.size || DEFAULT_SIZE;
downloadFile(event.url, DOWNLOAD_FILE, (error) => {
if (error) return lambdaCallback(error);
fileSize(DOWNLOAD_FILE, (error, dlFileSize) => {
if (error) return lambdaCallback(error);
resizeImage(DOWNLOAD_FILE, RESIZE_FILE, size, (error) => {
if (error) return lambdaCallback(error);
fileSize(RESIZE_FILE, (error, rsFileSize) => {
lambdaCallback(null, {
ok: true,
dlFileSize,
rsFileSize
});
console.log(`Request (orange box) ends now, operation`
+ ` took ${Date.now() - start_orange}ms`);
const start_purple = Date.now();
console.log('Etc Background Work (purple box) starts`
+ ` now');
removeFile(DOWNLOAD_FILE, (error) => {
// Cannot call lambdaCallback
if (error) return console.error(error);
removeFile(RESIZE_FILE, (error) => {
// Cannot call lambdaCallback
if (error) return console.error(error);
console.log(`Etc Background Work (purple box) `
+ `ends now, operation took `
+ `${Date.now() - start_purple}ms`);
});
});
});
});
});
});
};function downloadFile(url, destination, callback) {
const file = fs.createWriteStream(destination);
const request = http.get(url, (res) => {
res.pipe(file);
});request.once('error', (error) => callback(error));
file.once('finish', () => callback());
}function resizeImage(filename, newFilename, size, callback) {
jimp.read(filename, (error, image) => {
if (error) return callback(error); image
.resize(size, jimp.AUTO)
.write(newFilename, callback);
});
}function removeFile(filename, callback) {
fs.unlink(filename, callback);
}function fileSize(filename, callback) {
fs.stat(filename, (error, data) => {
if (error) return callback(error);
callback(null, data.size);
});
}
The application isn’t too complex but I’ll attempt to explain it anyway. What it does is first download an image from a remote source, read the size of the downloaded image, resize the image and save it to a new location, check the size of the new image, and then responds with the size of the image. After the response has been sent the application then removes the downloaded image and the resized image from disk. (Of course, a real application would keep the resized image. We’re just replying with the metadata for simplicity).
Notice how this application is performing some work after the response has been sent, namely, deleting the two temporary files. With the default behavior of Lambda, the request cycle won’t be considered complete until we no longer have work queued up on the event loop. So our application will actually wait for the deletions to occur before ending. However, had we set context.callbackWaitsForEmptyEventLoop to false, we would then introduce the risk of a race condition, where the two files might not be deleted until after the next request starts.
Another thing we’re doing is something which would normally be considered an anti-pattern in Node.js. Specifically we’re reusing the same temp files between different requests. With a normal Node.js application, there could be multiple requests happening in parallel, and by reading and writing to the same file, we could actually respond to one persons request with another person’s images! However, with Lambda, we’ve got this guarantee that only a single Node.js instance will run on a machine, and that only a single request will happen at a time, so this usual anti-pattern isn’t that bad.
Create a Simple Invoker
Now what we’re going to do is create a simple program to help us invoke our function. This will also help illustrate that Lambda is ultimately requiring our handler and executing the function we export. Edit the invoke.js file using your favorite editor and paste in the following contents:
#!/usr/bin/env nodeconst handler = require('./resizer.js');// Usage: ./invoke.js http://example.org/image.jpg <size=32>
const url = process.argv[2];
const size = process.argv[3]
? Number(process.argv[3])
: undefined;handler.downloadAndResize({ url, size }, {}, (error, data) => {
if (error) return console.error('FAILURE', error.message); console.log('SUCCESS', data);
});
We can now invoke our handler function in a manner similar to what Lambda does. Execute the following command to run your code locally:
$ ./invoke.js \
http://node.green/logo.png \
100You should see output like this:
Request (orange box) starts now
SUCCESS { ok: true, dlFileSize: 9022, rsFileSize: 12683 }
Request (orange box) ends now, operation took 225ms
Etc Background Work (purple box) starts now
Etc Background Work (purple box) ends now, operation took 3msDeploying the Function via CLI
Deployments for Lambda work by uploading ZIP files to AWS using the CLI utility. Assuming you’re in the directory which contains your Lambda function you can execute the following two commands. The first one will recursively zip the contents of your application into a file called function.zip in the parent directory. The second command will perform the deployment of your ZIP file. Of course you'll want to set FUNCTION_NAME to be the name of your function.
$ zip -q -r ../function.zip .
$ aws lambda update-function-code \
--function-name ${FUNCTION_NAME} \
--zip-file fileb://../function.zip
$ aws lambda update-function-configuration \
--function-name Resizer \
--handler resizer.downloadAndResizeOnce the deployment is complete you should see a small summary containing metadata about the deployment. Your changed code will also be immediately available, any handlers serving requests with the old code will finish and any new requests will be sent to freshly deployed instances.
Now that our new Function has been deployed, let’s invoke it. Of course, if we make the POST request that we made previously, we will get an error that the URL isn’t present. This is because when we modified the code we set it up to throw that error. So what we’ll need to do is provide a JSON POST body which provides a url attribute.
$ curl -X POST https://d34db33f99.execute-api.us-west-1.amazonaws.com/Production \
-H 'Content-Type: application/json' \
-d '{"url": "http://node.green/logo.png"}'And if all goes to plan you will see the following response:
{
"ok": true,
"dlFileSize": 9022,
"rsFileSize": 2046
}Enabling CORS via API Gateway
If you plan on allowing web-based clients such as web browsers to communicate directly with your Function via API Gateway it’ll be important to enable CORS (Cross Origin Resource Sharing). CORS is how modern browsers determine how a browser on one domain can interact with resources on another domain. If a page is loaded at one domain, and would like to make a POST request to another domain, the browser will first send an OPTIONS request to the other domain. The response will contain headers which describe which actions can be taken and who can take those actions.
To enable CORS, visit your API Gateway, select the endpoint you would like to enable CORS on, click the “Actions” dropdown, and select Enable CORS.
On the screen that appears you can then configure how CORS will work with this API Gateway endpoint. For example, if you only wanted the domain intrinsic.com to be able to POST to this endpoint, you could set the Access-Control-Allow-Origin field to 'intrinsic.com'.
Dynamically Reconfiguring the Handler Name
Earlier we saw that the syntax for naming our handler functions looks like filename.funcname. For example, if we export a function named handler inside of a file named index.js, then our handler name would be index.handler. What this means is we can actually ship different files within our Function and also export different methods. Another useful feature is that we can dynamically tell AWS which of these filename / export name permutations to choose from, on the fly, without the need to redeploy code.
The command to dynamically reconfigure a Lambda handler looks like this:
$ aws lambda update-function-configuration \
--function-name ${FUNCTION_NAME} \
--handler ${FILE_NAME}.${METHOD_NAME}When your application is reconfigured in this manner, AWS will begin creating new instances of your application with the appropriate environment variables set. AWS will also ensure all the currently running requests finish. New requests will then begin going to the new instances. The old instances will then all be destroyed once they have no more requests to handle. None of the old instances will be reused, so any state within the processes will be lost.
The Importance of Quick Startup Times
One thing to keep in mind when running Node.js applications on Lambda is that the amount of time it takes to bootstrap your application is much more important. For example, when running a typical Node.js application outside of Lambda, you’re probably creating several Node.js instances per deployment and not deploying too incredibly often, perhaps a few times a day. If your application takes a few seconds to be ready it’s probably not a big deal (assuming you wait until the new application instances are ready to handle requests before switching traffic from old to new). However, remember that with Lambda, AWS will dynamically create more instances all the time as traffic increases. This means that every time a new instance is created to serve a request, we’re now delaying that request by our startup time. These can really start to add up.
This article was written by me, Thomas Hunter II. I work at a company called Intrinsic where we specialize in writing software for securing Node.js applications. We currently have a product called Intrinsic for Lambda which follows the Least Privilege model for securing applications. Our product is very powerful and is easy to implement. If you are looking for a way to secure your Node.js applications, give me a shout at tom@intrinsic.com.
