AWS Lambda : Missing piece of AWS Orchestration

Last winter I was working on a small internal tool; one that should pull some files from an S3 folder sanitize and aggregate them and drop them in another S3 folder, no more than 10 lines of code. Running an EC2 instance, even a tiny one, to handle couple of calls per day, each running less than 5 seconds, was just wasteful. Spending a single minute every day to check it is health would’ve taken longer than the actual job at hand. Setting up a Data Pipeline with a ShellCommandActivity was almost equally wasteful and not responsive enough. Setting up a job queue and some webhooks would have taken unnecessarily long development time for such a simple job. AWS was missing its trigger.

I was so happy when I read about then newly released Lambda. I didn’t know node.js, console and S3 bucket notifications had a couple of bugs and develop-test cycle was cumbersome, but it didn’t matter. Lambda was the most elegant solution; it was there when I needed and gone when it is done.

Since then Lambda came a long way; it got Python and Java in addition to node, has a much better console and some templates to start with, it is more reliable and some limitations are relaxed. It can be triggered on a schedule or by mobile devices now. And of course, it became best friends with AWS API Gateway to build serverless backends. Nowadays people build frameworks on top of it to create serverless applications.

Lambda is changing the notion of traditional backend in such a radical way that its other benefit; connecting AWS resources to each other in the most simplest way is just overshadowed. By making it possible to trigger Lambda from other services, Amazon simply increased value of all services that can interact with Lambda. It became truly “on demand”. I am using Lambda to make decisions as I pass data between services, start/stop services and sanitize some calls from mobile devices to our backend.

Next time you design a backend system or an internal tool, consider Lambda to be the glue that keeps your bigger blocks together.