Oct 3, 2017 · 2 min read

Originally published at

The biggest usage of serverless technologies today is around event-driven workflows in the backend of a system. Examples include receiving an event from an S3 bucket when a video is uploaded, then using a serverless function to spin up a Docker container to transcode the video. However, serverless technologies can provide even bigger wins in when they are used for API services.

One of the biggest challenges faced when hosting an API service is the volatility of traffic. Here’s an example of traffic patterns for a typical API service:

Notice how the traffic goes up and down every day in relatively large amounts. Before serverless there were two technologies to help handle increased traffic loads: request queuing and auto-scaling of compute resources.

Request Queuing

A request queue is a layer that sits between your load balancer and your application. When many requests come to the server in parallel faster than the application can handle them, the queue will hold some of the requests until the application can handle them. But this adds latency to requests:


If your request load varies too much, it may require auto-scaling of compute resources. While this works, there are two more challenges in using this approach. First, you need to hire or contract with expensive engineers to convert existing services to be auto-scalable. Second, if you have very high traffic volatility, where large spikes occur at random times, auto-scaling often does not occur quickly enough to handle all the traffic.


The solution to these problems is the horizontal scalability of serverless technologies:

  • Using an API gateway offloads the complexity of request queuing management to the cloud service provider, which also can horizontally provision the bandwidth at this layer to reduce latency.
  • Auto-scaling latency is a non-issue because serverless compute resources can also be scaled horizontally with high utilization efficiency.
  • General developers can write serverless functions, which offloads to the infrastructure service provider the expensive operational task of enabling auto-scaling of compute resources.

Give serverless technologies a try the next time you have scalability concerns around your API services!

Stacks on Stacks

The Serverless Ecosystem Blog by Stackery.


Written by


Run serverless in production with Stackery's serverless operations console. (

Stacks on Stacks

The Serverless Ecosystem Blog by Stackery.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade