Building a Serverless REST API in Go

Serverless, or FaaS (Functions as a Service), is becoming an important trend in the software architecture world. At AWS re:Invent 2016, there were more than forty sessions scheduled to talk about serverless architecture, compared with less than ten sessions a year earlier. The obvious reasons why people are quickly adopting serverless architecture — as Mike Roberts summarizes — include:

• Reduced development and operational cost
• Automatic scaling that can accommodate various traffic patterns
• Easier operational management

So what is serverless architecture anyway? According to ThoughtWorks,“Serverless architecture replaces long-running virtual machines with ephemeral compute power that comes into existence on request and disappears immediately after use.” Based on this definition, REST APIs are perfect for utilizing serverless architecture since any code being run needs to be stateless and the execution is so quick.

In this post, we will describe how to develop a serverless REST API in Go (Golang). More specifically, we will show how to develop a REST API in Go, deploy it as an AWS Lambda function, and finally publish it via Amazon API Gateway.

Because AWS Lambda does not support Go natively, we will use a framework called Apex to support our choice of language (you can use the framework of your choice or even no framework in this instance). Apex does so by building a Go binary for Amazon’s server architecture (GOOS=linux GOARCH=amd64) and running it in a child process of a Node.js Lambda function. Apex also provides a variety of workflow related tooling for building, deploying, and testing Lambda functions, rolling back deploys, viewing metrics, tailing logs, and more. We will show how Apex can simplify most of the Lambda related work and let us focus on developing the Go code.

Tip: While Apex supports using Go to develop Lambda functions, it also supports those native languages, including Node.js, Python, and Java. When using native languages to develop Lambda functions, another popular framework that you may consider is the Serverless Framework.

Getting Started

To get started, install the apex tool and the corresponding Go package as follows:

Then follow the instructions in the Apex documentation to set up the AWS credentials needed for deploying Lambda functions using Apex.

Creating the Lambda Function

Tip: For those who are impatient, you may check out the complete source code described in this post from my golang-serverless-restapi open source project.

Create a Go project folder. Under the project folder, create a folder structure like the following, as required by Apex. You may also use the command apex init to do this initialization work.

An Apex project may have one or multiple Lambda functions, each of which is organized as a sub-folder of the functionsfolder. For our demo project, we only need a single Lambda function named apis. The function is configured through a project-level configuration file named project.json and a function-specific configuration file named function.json. In these files, we specify parameters, such as memory limit, timeout, which Apex uses when deploying the Lambda function to AWS.

The project.json file should contain the following content:

And the function.json file may contain:

The above configuration files read that the project is named as rest, and the Lambda function named apis should reserve 128MB memory and take no more than 10 seconds when running.

The .apexignore file contains a single line as follows, which notifies Apex that none of the Go source files should be included in the deployment distribution:

And finally the main.go file is the Go source file containing our REST API implementation and the wiring with the Apex framework.

In the above code, we first create an HTTP handler which implements thehttp.Handler interface, as we usually do while developing a REST API. This handler is responsible for taking various HTTP requests and processing them accordingly. While in this example we use the standard http.ServerMux handler, if you’re using a third party HTTP routing framework such as Echo or Gin, you should still be good to go as they should all provide a handler.

We call apex.HandleFunc to register a function that is called each time the Lambda is invoked. The function takes two parameters:

• The first parameter event is a byte array in JSON format which represents the input parameters used when the Lambda function is invoked
• The second parameter ctx contains contextual information (e.g. request ID, function name) for the function invocation.

The data returned by the function will be marshaled by Apex in JSON format and set as the output of the Lambda. An error may be returned indicating a failure of the Lambda invocation.

Within the function registered by apex.HandleFunc, we first parse the Lambda event into a standard http.Request. We then call the HTTP handler’s ServeHTTP method to handle the request. Note that we use ahttptest.ResponseRecorder to record the response of the handler. And finally we convert the response into the format expected by the Lambda.

When a Lambda function is hooked with Amazon API Gateway (to be described shortly), the format of its input and output can be described as the following structs:

So in the main function described earlier on, the ParseRequest function should convert LambdaInput into http.Request, while FormatResponse convert httptest.ResponseRecorder into LambdaOutput, and FormatErrorbuild a LambdaOutput from an error.

This is the primary adaption work you should do if you already have a REST API that you want to convert to serverless.

Since the data conversion work is very straightforward, we will not show the implementation here. Please find the complete source in my golang-serverless-restapi open source project if you want to learn more.

Deploying the Lambda Function

With the help of Apex, deploying a Lambda function is as easy as running a single command as follows:

The above command will build each Lambda function under the functions folder and deploy them to the AWS region us-east-1. We can run the same command multiple times. If Apex detects any change to the binaries, it will create new versions and alias them with latest.

We can test the deployed Lambda using Apex or via AWS console. For example:

Configuring Amazon API Gateway

Although we have deployed the Lambda function to AWS, it is not currently accessible by Internet users. We need to use Amazon API Gateway to delegate the HTTP requests from the Internet users to the Lambda function. To achieve this goal, we will make use of the proxy feature of Amazon API Gateway.

Take the following steps to configure Amazon API Gateway:

1. Log into the AWS console and switch to the API Gateway page. On that page, choose “Create new API” and enter the API name as “hello” (or any other name you prefer).
2. In the “Resources” tab of the “hello” API, click on the “Actions” dropdown button and select “Create Resource”. In the “New Child Resource” page, select “Configure as proxy resource”, and then click on the “Create Resource” button. See the screenshot below as a reference.

1. In the “/{proxy+} — ANY — Setup” page, choose “Lambda Region” as us-east-1 (or the actual AWS region that you used to deploy the Lambda function) and enter the “Lambda Function” name as rest_apis. Click the “Save” button to complete the proxy resource setup. See the screenshot below as a reference.

1. Click on the “Actions” dropdown button and select “Deploy API”. In the popup window, choose [New Stage] in the “Deployment stage” dropdown list, and enter prod in the “Stage name” input field. Click on the “Deploy” button to complete the deployment.
2. At this point, you should be redirected to a page showing the deployment information about the “hello” API. You should see an “Invoke URL” on the page which looks like https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod. This is the root URL that everyone can use to hit our REST API.

If you prefer using AWS CLI, you may also run the following commands to achieve the above steps:

Trying it Out

Run the following commands to verify the REST API is accessible and working as expected:

In the AWS console, locate the Lambda function named rest_apis and check its monitoring result to verify that the Lambda was invoked indeed.

Lastly, remember the default root URL of our REST API is not user friendly. Amazon API Gateway lets us set up a custom domain name (e.g. api.example.com) so the URL looks simpler and more intuitive. Follow the AWS documentation for more on how to achieve this goal.

Final Notes

In the above, we have shown how to develop a serverless REST API from scratch. However, the same procedure can also be taken to convert an existing REST API to serverless. To do this, the only change needed is to modify the NewHTTPHandler function to use your existing HTTP routing code.

The proxy feature of Amazon API Gateway is very easy to use. However, using it also means that we cannot leverage many other advanced features offered by the Gateway; including authentication, error handling, rate limiting, etc. This is probably not an issue for an existing REST API since these features were very likely already implemented at the API layer. If you are developing a REST API, you may want to consider more advanced usage of the Gateway. Be warned, though, the configuration of the Gateway is not a trivial task, which is especially true when the API is under active development with frequent changes.

Originally published at developer.capitalone.com.

For more on APIs, open source, community events, and developer culture at Capital One, visit DevExchange, our one-stop developer portal. https://developer.capitalone.com/