The Serverless API

A recipe for fully dynamic browser apps

Blunt Jackson
Jan 3 · 10 min read

AWS Nomads #4: How to provide dynamic content and functionality to your web app. Everything here can be done from a mobile browser, and will cost less than a cup of coffee per month.

Serverless doesn’t mean no servers. It means more servers than you can imagine.

So, you are sitting down to build an API.

If you are an old-timer like me, you might once have done something crazy like write your own dynamic template processor as a C-language Apache module. Less insanely, many developers will have dipped into the waters of LAMP (Linux + Apache + MySQL + PHP), or some Rails framework or another, or perhaps a Nginx + Node.js, setup, or if you have been in one of those shops, perhaps JSP. Don’t talk to me about Microsoft variations. I don’t want to know.

All of these follow a common principle: first, you have your server.

If you are beyond the prototyping stage, probably several. If you are successful, perhaps dozens, or maybe hundreds. Perhaps thousands.

You point your DNS to your load-balancer, which distributes traffic to your servers, which handle the requests, which activate your business logic code, which accesses your persistent data storage.

In many ways, “serverless” is not that much different: the architecture diagram is not going to be dramatically different: the load-balancing and http serving accesses your business logic which interacts with persistent storage. The names are changed to protect the innocent. What’s different we are trading whole departments of networking, system administration, and operations for some specialized expertise in the arcane science of AWS configuration.

Let’s roll up our sleeves.

The “Lambda” is the heart of true serverless. Although, of course, the code will run on a server, somewhere, we don’t know where is is, or what it is, and we have no access to it. Trust me: we don’t want to.

The idea is, rather than existing as a traditional application, our business logic is segmented into discrete and independent functions. In formal computer science, a ‘lambda’ is a small, independent, anonymous function. With thoughtful design, we can organize our business logic into one or several lambdas.

This article isn’t going to provide step-by-step instructions; Amazon’s recipe sequences are perfectly adequate. But there is some context that will help you navigate the world of the lambda.

(A) Pricing: The lambda service charges for both CPU time and number of executions, after a fairly generous monthly free-access grant. Initial development and low volume websites are easily free. (See more about pricing in FAQ#1).

(B) Initial Access: Lambdas cycle out of memory if not used. Loading and running a lambda for the first time if there is no recent activity has an extra performance cost. This is typically a fraction of a second. For very low traffic applications, this is great, because we save a ton of money by not paying for a server that we’re not using. But suffice it to say that small functions are better than big ones! This should drive design: if you have disparate portions of your API, they should probably be written as different resources, rather than a single function that handles all api traffic, or (possibly worse) handles all api traffic and dispatches to secondary lambdas. The beauty of it is there is a perfect alignment here between peak performance and healthy software design.

(C) Lambdas have no access to system resources. In particular, disk. This is not like writing an application. Everything a lambda might need should either be internal to the code of the function itself, or available in another AWS service.

(D) You have your choice of runtime: Node, Python, Go, Java, Ruby, or (God help you), .NET. Personally, I find it feels as if it was designed for Node. But pick your poison.

Here is the code for a node.js “Hello World” standalone lambda:

exports.handler = async (event, context) => {
const response = {
statusCode: 200,
body: JSON.stringify('Hello, friends!'),
};
return response;
};

Note: the full response object is itself written as the body of the HTTP response, not just the body element.

Here are instructions for creating and testing an appropriate lambda.

The lambda is your business logic; API Gateway maps to your http(s) server.

Amazon has a comprehensive tutorial for this, but there are a number of tricky bits. Because these services and systems are designed in a very generalized way, the flexibility of each service can be an impediment to clarity. The API Gateway is not the worst, but it can be challenging. I recommend going through the tutorial before reading the following concepts, as the gotchas here make more sense once you have some hands-on experience.

Tricky concepts:

A. Resource, Method, Stage, and Base Path Mappings — The resource represents the path or path hierarchy (/my-resource) for an HTTP request to be routed to a specific lambda. You can additionally route based on the HTTP method of the request (GET, POST, etc.). Or, within the console, you can select method ‘ANY’ which will route all methods to your lambda. I discuss stages in more detail below, but note that the stage forms part of the url on the API gateway’s published url. (/prod/my-resource). But many times you will want to use a custom domain (api.mysite.com), and you can use base path mappings to route requests to a resource/method/stage combination. (Which is easy to forget about because it lives in the Custom Domain Name section of the API Gateway console; you will want to remember this when you try to delete a resource, and AWS will not let you because there is a base path mapping referencing it.)

B. Lambda Proxy Integration — This alters the way the request is passed to the lambda and received from the lambda. Using a proxy integration takes on a convention of how headers and parameters are passed to your function, and how your response should be formed. Although it is generally a simplification, it is not the default option. You may want to go with it for simplicity and consistency across lambdas.

C. Testing your Setup — The API Gateway console allows you to test your integration with the Lambda separately from issuing a request to the gateway itself; the in-console testing facility is available when you select the “Method” for your resource. But to truly end-to-end test your api, you will need to deploy it…

D. Deployments & Stages —You can’t deploy your API without having at least one stage; but you can only create the stage as part of the deployment process.

The theory is: you can set up different stages for different in-progress versions of your app. The different stages can be used for development, for testing, for integration, and (of course) for production. But, as of this writing, you can only create the first stage as part of deploying the API for the first time. Going to the stages menu and trying to create your first stage prior to deploying it doesn’t seem to work. So, create a stage as part of your initial deployment.

E. The URL Endpoint — Once you have configured, tested, and deployed your API, you will have an endpoint that looks like this: https://h3pae271qr.execute-api.us-east-2.amazonaws.com/prod

You can totally use this, but you might not want to. In the first place, it is on the domain amazonaws.com, so if your app is in a different domain, you can run into cross-domain challenges. Secondly, it can expose more than you might want to people sniffing your network traffic.

Also note, this represents a regional implementation, and for higher volume and higher performance sites (at higher cost) you may want to use an edge implementation, although that is beyond the scope of this article.

To change this endpoint to something on your domain (api.mysite.com), you will need to go through the setup for a Custom Domain Name. You will need to obtain a secure server certificate (free, from Amazon), configure your DNS entries (through Route53 or your existing nameservers), and establish your base path mappings.

Here are the full AWS instructions for setting up your custom domain. One thing to be very clear on is that the Custom Domain Name for API Gateway is using Cloudfront under the hood, and this effectively repackages your URL — but the URL it offers will not work if directly referenced. It only works when requested via the sub-domain you CNAME to it. And make sure you create the CNAME to the domain published by the Custom Domain Name panel, and not the one published by the API Gateway resource configuration.

Finally, as with anything involving DNS or CloudFormation, changes take time to propagate. Any time you make a change to this any of this, you can expect at least an hour of wallclock time, or more depending on your DNS provider. (I use Google, which seems to update within 15–20 minutes, regardless of your TTYL settings.)

If everything has gone smoothly, you will now have your API, accessible under your own domain, vending your Hello message to the world. But without persistent storage, this is unlikely to be useful.

You don’t need to use DynamoDB. AWS offers a variety of storage and database options. But DynamoDB is a great choice for many applications.

There are three steps: creating the table, granting permissions, and accessing the table from our Lambda.

A. Creating the DynamoDB is extremely simple. Load up the DynamoDB service in the console, and create a table. Give it a primary key and you can hand-edit in some content. (Amazon instructions for creating a table.)

B. Granting Permissions is not so simple. In this case, Amazon’s instructions are both head-spinning and overkill. (Check it out.)

Here is a simple(r) procedure to grant permissions:

i) Create a Role — the Role menu is currently third down, after Users and Groups.

ii) Select ‘Lambda’ as the service that will use this role, and move on to permissions.

iii) This role will use the ‘AWSLambdaBasicExecutionRole’ permissions policy. Start typing that in the filter window, and select it. Skip tags or set some as you like, and move on to the final Create Role step”

iv) Name your role. The name is arbitrary, so something like MySiteLambda. Proceed, you have your role, but now you need to give it access to DynamoDB, so:

v) Select the role from the role list.

vi) “Add Inline Policy” — this is a link off to the right at the same level as the “Attach Policies” Button.

vii) Select DynamoDB.

viii) Open the Actions subsection. You can set very fine grained controls on what methods the Lambda can access, and in general more restrictive is better. GetItem and PutItem are a good starting place.

ix) Almost done: select the resources submenu, and select “Add ARN to Restrict Access”.

x) Use the ARN provided by the DynamoDB service.

xi) Select Review Policy and give this inline policy a name.

xii) Configure your Lambda to use this role (See the “Execution Role” block in the lambda configuration page), and you’re good!

C) Recode your Lambda.

This being node, there are several design patterns you can use to access your table. Following is the simplest, in which we will get from dynamo and return via our api a message: “Hello Friend.” This assumes that the table created is named: TestAPI and the primary key field is msg-keyand the key entered into DynamoDB for this field is hello-messagethe message field is display-text:

const AWS = require('aws-sdk');
const ddb = new AWS.DynamoDB.DocumentClient();
exports.handler = (event, context, callback) => {

var params = {
TableName: "TestAPI",
Key: { msg-key: "hello-message" }
};

ddb.get(params).promise()
.then((data) => {
callback(null, {
statusCode: 200,
body: { title: data.Item.display-text }
});
})
.catch((err) => {
callback(null, {
statusCode: 500,
body: { error: err.errorMessage }
});
})

}

Assuming I entered a record in DynamoDB:

msg-key: "hello-message"
display-text: "Hello Friend!"

That “Hello Friend!” is exactly what I should get back.

Woah! You now have a robust, versatile, scalable, secure application server farm without any operations, security patches, system administration, database administration, network staff required. If you are small, this costs somewhere between nothing and pennies per day.

F.A.Q.

This setup is affordable for low to mid volume websites. The different components have different pricing structures, but the whole setup looks like this, as of January 2020 for a very basic setup. Because AWS services have many options, and each option is priced separately, this only describes a very basic setup, and prices are subject to change!

The setup:

  • API Gateway, no frills
  • Lambda, pay-as-you go, no frills (Note you pay for both requests and for the number of seconds the lambda runs in a month.)
  • Dynamo DB, pay-as-you go, no frills, no backups
  • Secure Server Certificates: free.

All pricing is per month.

I don’t know why this table has so much empty space after it!?

Conclusion: for a low volume API (< 1M req/mo), you can bring just about everything in on the free tier in the first year, and only pay for the gateway after that. But we all want to be wildly successful, and if you are there are some options that can save money as you grow, such as provisioning dedicated hardware. And you will probably want backups on your DynamoDB!

Add this to your payment on static assets on S3, and you can get very close to delivering a high quality low-volume website for under $2 per month, even out of the free tier.

This is distressingly common. I have tried to share some of the core concepts in this article that minimize the confusion around what happens where, but the extensive flexibility and multi-purpose design in AWS means there are a lot of obscure options that can confuse.

The good news is, you are not alone. Google your error messages, and you will likely find an abundance of different solutions, one of which will probably work. Or at least this has been my experience. And believe me, there was a lot of googling.

You can do whatever you want! I don’t know what the risk levels are in a lambda being somehow compromised by hackers, but assuming it is non-zero, the more restrictive permissions are a protection against all kinds of chicanery. But because dynamo tables are associated with the account, not with any api or domain, I would take special care to ensure that an API can never access a table for another app or site.

Ok nomads, I recommend the bar for this one. When you miss a configuration step and are getting access denied errors throughout your stack, you are going to want the beer. And you are going to want to be in a place where a certain amount of cursing is acceptable.

The Startup

Medium's largest active publication, followed by +567K people. Follow to join our community.

Blunt Jackson

Written by

Building web applications since 1992. Crikey, that’s a long time.

The Startup

Medium's largest active publication, followed by +567K people. Follow to join our community.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade