Lightweight Serverless API Using AWS Lambda and ALB

Published in

Driven by Code

7 min readJun 19, 2019

Introduction

Over the last few years of getting better and better at writing Lambda code and introducing more and more functionality without the use of Elastic Compute Cloud (EC2) instances or EC2 Container Services (ECS) containers (that is, becoming less and less serverful), I had started to notice a new serverless pattern:

I need to do a small task X
I write a single-function Lambda that does X
I want to share X as a simple Application Programming Interface (API)
I just want a simple, lightweight interface for my Lambda via a HyperText Transport Protocol, Secure (HTTPS) endpoint

Amazon Web Services (AWS) recently announced support for triggering a Lambda function from an Application Load Balancer (ALB). This feature started to make me think there was a way to service HTTPS API requests via ALB rather than API Gateway. A few features (you could call them drawbacks) always prevented me from immediately grabbing API Gateway to trigger my Lambdas. If any of these resonate with you, you might consider using ALB in certain cases:

I want to keep the API inside my Virtual Private Cloud (VPC) network.
I don’t need or want the API to be public (you could make the ALB public if you wanted — not that I recommend that; you could even then put a Cloudfront distribution in front of your ALB if you wanted to do that too — although at that point you should probably migrate to using Gateway API).
I don’t need to specify the routing for API requests and I don’t need a lot of request validations because I’m not writing a full API contract, just an internal source of some outputs and perhaps supporting a light administrative User Interface (UI).
I don’t need to generate a Software Development Kit (SDK) for clients.
I don’t need the full set of authentication options available with Cloudfront (although, as I said, you could add Cloudfront or migrate to API Gateway later).

I found a really good article on the differences (and similarities) between API Gateway and an ALB Lambda implementation from the Serveless folks. I would recommend you read their article in full to find the subtle difference between the two solutions.

A drawing showing the way requests would flow, triggering a Lambda and returning a response.

Python ALB Hello World

My first step in deciding if I could write a Lambda API and serve it via ALB was to determine the effort and time required to start a “Hello World” application. The first result from a Google search shows an existing Python implementation that was so ugly, I almost wanted to stop researching a solution. I wanted a simple, fast, lightweight implementation I could use and recommend internally that fit the following criteria:

I should be able to take an existing Lambda function that does X and serve it via an API. Ideally, I’d like to install any dependencies in a single file to keep the Lambda footprint and deployment small.
I don’t want to handle HTTPS overhead like headers, response codes, and so forth (unless I have to in specific use cases, like returning an error).
I don’t want to have to write a bunch of boilerplate code or library for marshalling objects, converting hashes (or dictionaries), routing locations details, and so forth.
I don’t want to use a heavy application framework like Flask or Django, say, nor do I necessarily want to have all that abstraction and weight hidden in another layer of medium- to heavy-weight wrappers like Zappa. Incidentally, Zappa already deploys an API Gateway solution for you if you want, and I was already shying away from that solution when I started thinking about this project.

I pressed on to find two key ingredients I knew had to exist: first, a lightweight (one file) implementation of the Python Web Server Gateway Interface (WSGI) to marshal AWS events to my application; and second, a similarly lightweight WSGI framework that I could use to offload the HTTP-specific headers, response codes, content-types, and so forth for me.

APIG-WSGI

For the first part, I found the apig-wsgi project on GitHub which takes in the event context from an API Gateway (or an ALB) event and moves around the dictionary of objects into a WSGI object. It even creates the Lambda handler for you. Best of all, it installs in a single file! This satisfied my instinct to keep things small and lightweight. Installing dependencies in a single file is good because it means that you don’t have to bundle a lot of files together with a build script or pipeline, and it also means that you don’t have to support a lot of libraries and dependencies that aren’t even used directly in your code.

Bottle

For the second part, I was already aware of the Python Bottle framework, which is similar in design to Flask, without a lot of dependencies or libraries. In fact, Bottle installs in a single file and even includes a web server (which we won’t need). The only issue was gluing everything together, which turned out to be far simpler and more rewarding than I thought.

I was cheating a little bit if you are paying attention to counting my dependencies. Instead of having “no dependencies” or even “a single dependency”, I already have two dependencies (three, if you start to include Boto3 for my Lambda AWS actions), and at least three files (my code, and two dependency libraries).

Implementation

The first step is to install the two dependencies via `pip` in my local Lambda project directory. Then I needed to import the two libraries. Then I needed to instantiate a Bottle app object. I also needed an entry point for the ALB event and context. To prove my point, I needed to print “Hello World!” in a Bottle route. And lastly, I needed a test case for running the code locally before I deployed it. Here is the result:

The largest part of this code is the example JSON event used for local development! If you run this in your Python interpreter, you’ll see the following glorious output:

{'statusCode': 200, 'headers': {'Content-Length': '12', 'Content-Type': 'text/html; charset=UTF-8'}, 'body': 'Hello World!'}

This was an amazing accomplishment in only a few lines of code, and already the project was looking a hundred times better than the AWS ALB Lambda Hello World example.

Making a RESTful-ish API

I was now well on my way to transforming a simple Lambda into a REpresentational State Transfer (REST) interface. Only two obstacles stood in my way to confirming this would work: first, I needed some way to pass parameters into my application; and second, I needed to actually point an ALB endpoint at my Lambda.

For the first point, I know that Bottle allows you to parameterize the request path and automatically transform the values into parameters to your function call. There is another possibility you can use that inspects either the query parameters from a GET method or body of the POST parameters. I’ll go through both examples for completeness.

Path and Query Parameters

You can create parameters in Bottle by using angle brackets to specify a parameter name. These can get very fancy if you want to find out the full details. In this example, I decided to make a parameter for the health check using an `/internal/` path. I’ll actually use this health check later on, when I configure the target group from the ALB to the Lambda. I also added a `create_post` function that will do something with DynamoDB (just as an example) and then return the result.

Other than handling the GET and/or POST parameters, I found this solution to be smooth, elegant, readable, and lightweight. Best of all, in engineering terms, “it works.”

ALB Integration

The only further icky part is making an ALB target group pointed at the Lambda. I’ve written a Terraform module to do so, but it’s ugly and lengthy and not illuminating to the project at hand. Suffice it to say, you’ll need to create an ALB, a Lambda function and body, a Lambda permission for ALB to trigger the Lambda, a role, a role policy and an execution policy for your Lambda, and all the supporting ALB pieces: security group, target group, target group attachment, listener rule, and certificate. Phew! In my defense, an API Gateway Terraform would be just as big a mouthful to digest and swallow. Here are a few screenshots from the console that will be more illustrative:

Summary

I had succeeded in developing a pattern for a lightweight conversion from pure Lambda implementation to an HTTPS REST API. The pattern does introduce two new single-file dependencies and a few lines of import and decorator boilerplate. The “Hello World!” example is, humbly, a hundred times better than the one proposed by the AWS GitHub example, and could even be improved with Jinja2 templates or a library like Dominate.

Configuring the ALB permissions and supporting glue was messy and tedious at first, but it was abstracted out in a Terraform module that, unfortunately, isn’t very useful or general (the main issue is policy documents and AWS infrastructure specifically tied to what the Lambda accesses).

This pattern easily extends as needed to scale up to an API Gateway pattern by simply tying the Lambda to an API Gateway function instead of ALB. This might be the case if you require more enterprise features than Cloudfront would offer. Following all the examples I’ve set so far, no code changes are necessary to trigger this Lambda code from either the ALB or API Gateway mechanisms.

We are hiring! If you love solving problems please reach out, we would love to have you join us!