I am psyched about Amazon’s new Lambda service for asynchronous task processing, but the ideal development and testing cycle is really left to the engineer. While Amazon provides a web-based console, I prefer an approach that uses Mocha. Below you will find the gritty details using Kinesis events as a sample input.
Amazon’s Web-Based Testing Console
Certainly better than nothing, the web-based testing console that Amazon provides gives you an interface to see execution results for Lambda functions given sample events.
Of course, this makes for a laborious workflow during the development cycle. Each time you need to test out the execution results of the function, you have to precompile, package, zip, and upload before manually invoking it with a sample event you have copied in.
If there are multiple data scenarios to test, you have to copy those in one by one and review the output manually. Obviously the console is not intended for this.
Local Development and Testing
I found this approach useful for local development and automated testing:
- Create development and test environments
- Capture real events from Kinesis to use as testing inputs
- Simulate Kinesis event creation and Lambda execution
- Call the handler function and test outcomes using Mocha
- Automate testing with Circle CI
Create Development and Test Environments
If your function requires environment variables, you will need to create these locally. I use dotenv and a local .env file. If the local environment variables are nonsensitive (e.g. they do not include AWS Access Keys or other third party services credentials), you can check this file into source control. Otherwise, add to .gitignore.
In this case of this example, we use Firebase and have different URLs for development, test, staging, and production. These are reflected in our environment files.
Capture Real Events from Kinesis
If you have just started getting your hands dirty with Kinesis it can be useful to capture the specific output that Kinesis events generate. Piping this output to stdout can give you a bit of trouble when you copy from the console, so I also pipe directly to a flat file. Here’s a simple example:
If your Kinesis stream is live and generating a high volume of events, you might want to throttle the pipe. In any case, after capturing a few events, you will have an idea of precisely what Lambda will be seeing.
Simulate Kinesis Event Creation and Lambda Execution
Having viewed the live events, it is easy enough to write a test function that reliably encodes a raw payload the way your upstream code does, like so:
The events you capture from the above method will be strings, but when they are sent to Lambda they will be base64 encoded and packaged within an object containing an array of records. For testing, this must be replicated. In addition, if your code is asynchronous, Lambda expects you to call context.done() so that it can exit its own process.
Thankfully, this results in two boilerplate test functions that can be used across projects. Since the context object is called after the Lambda function concludes, it can be useful to create for each test a custom context including the test’s assertions.
In the same vein, we have found it useful to expose a boilerplate handler function within index.js itself:
Call the Handler Function and Test Outcomes
With all the Lambda, Kinesis, and Firebase sugar sorted out, tests with Mocha are nothing unique. You can use whichever assertion library you prefer, and each test can call upon different raw payloads of data to ensure that all data scenarios are covered.
Once initial local development is complete, it is still a good idea to manually invoke your function against your sample events using Amazon’s web-based console, but provided you are getting the same results at critical milestones, you are probably safe to lean on Mocha for ongoing development.
Automate Testing with Circle CI
Since you have your development and production environments abstracted to environment files (if needed), it is a short hop to creating a test environment and automating the testing process.
We use Circle, while some use Travis or Jenkins. The point is to have a place where your tests will be executed each time code is pushed.