A Gulp Workflow for Amazon Lambda
Environment Variables, NPM, and Uploads
In this earlier article I talked about how excited I am about Amazon’s new Lambda service and where it fits into a modern realtime stack. But any nontrivial development of Lambda functions will require a simple, automated build/deploy process that also fills a couple of Lambda’s gaps such as the use of node modules and environment variables.
Gulp, Not Grunt
First, the build process itself. We all agree there are technical debates out there that can distract from what we should be working on, and Gulp vs. Grunt (or Make for that matter) is no exception.
I prefer coding over configuration, so for me Gulp is an easy choice. I also agree with their guidelines, which suggest that we should not build a plugin for something that can be done easily with an existing node module. This is why I am going to show you a sample gulpfile making use of the AWS node module rather than release a plugin:
Let’s tackle the key pieces.
Using Node Modules in a Lambda Function
Most non-trivial functions are going to lean on NPM modules. Lambda facilitates this by allowing you to upload a zip file containing the modules you need.
It is a miserable slog to be manually installing dependencies or copying folders around, so I prefer to lean on gulp-install to take care of this part. The non-obvious thing is to be careful about splitting development dependencies from production dependencies within package.json:
You will know you got it right if the lines under “dependencies” map one-to-one with require statements at the top of index.js.
Using Environment Variables to Configure Lambda Functions
As I had hoped, the dotenv module gets us there. The goal is to avoid committing into source code any sensitive information such as password or tokens. Heroku and Elastic Beanstalk allow you to set environment variables directly, a feature I anticipate seeing in Lambda shortly.
But for the time being, the “env” task above copies a local git-ignored file over to your dist directory where dotenv will be looking for it. To make this feature easy to work with, I committed “config.env.sample” as a reminder of where to put the file. A user with deploy rights can rename this file to “config.env.production,” specify sensitive variables, and deploy. Problem solved.
Uploading Using the AWS Node Module
From here, the rest is mechanical. We are going to clear out a dist folder and then copy over everything we need. We can use Run Sequence until gulp/orchestrator can natively handle non-dependent ordered tasks. Then we push it all into a zip file. Boilerplate stuff.
If there’s anything tricky about using the AWS node module, it is the configuration. Rather than pass around deploy configuration within gulp or stash it all away in package.json, we lean on ~/.aws/credentials just like the AWS CLI.
Armed with a gulpfile like the one above, you get everything compiled and uploaded like so: