Scalability is getting easy

A quick jump into the future of cloud computing

AWS Lambda

For those who don’t yet know, AWS Lambda is a very promising service that Amazon announced at their latest re:Invent conference. The service allows you to run custom code in response to events. The typical example they use to showcase the service is a thumbnail creator running every time there is a new image dropped into a S3 bucket. This is brilliant! Think about it for a second. You don’t need to manage infrastructure anymore or worry about software updates and security patches. What about scalability and availability? Amazon handles it all for you, running your code on as many machines as required.

When auto-scaling isn’t enough

Today’s auto-scaling technology is a good progress toward responding to increasing user requests, yet they are still insufficient. Remember the 2013 Super Bowl massive website outage? Most auto-scaling technologies have a common shortcoming: in order to boot new instances, you first have to collect analytics and reach a threshold of unacceptable response time. Then, your saviour instances will start booting. Few minutes later (and probably too late), those new instances will be ready to respond to requests. In other words, you must either predict the requests spike or suffer a few minutes down time.

What does it have to do with AWS Lambda? Well, imagine if your web application faced a similar problem. You might have one or many servers running your app and some clients using it, but you can’t predict when they’ll actually log on and run calculations. And when they do, you need to respond now. You don’t have the luxury of time to spin up a new instance; your SLA wouldn’t be respected nor will your users be happy. On the other hand, you can’t simply have many permanent instances, as it would cost too much to operate. Wouldn’t it be better if you could just run those heavy computations on the cloud instead? Wouldn’t it be paradise if you had a single server handling all the basic stuff like authentication and logging while executing all the CPU-intensive tasks in AWS Lambda? Of course it would be.

This isn’t easy to achieve with Lambda alone. Why? Because in order to run your code on Lambda, you need to manually zip your code, upload it to AWS, invoke the function through their SDK, store the result somewhere like on a S3 bucket and poll AWS CloudWatch to know when the function finished executing so your server can get the result back.

Introducing Lambdaws

This is why my friends and I wrote this library, which runs on top of AWS Lambda. With a single line of code you can cloudify any stateless asynchronous function and make it run on Lambda. In fact, one line is a lie— it’s more like a single character:

var cloudedFunction = λ(yourAsyncFunction);

I do not pretend this solves all the scalability problems modern applications face, but I do think this is where the future of cloud computing is heading. Not having to worry about underlying infrastructure is the dream of many developers, including myself.

Sooner or later, platform-as-a-service will make it trivial to build highly-scalable, high-availability applications. I think giving developers the ability to scale their functions directly is a great step toward this dream.