Quickly Deploy a Golang GeoIP Microservice
With the help of Aegis for AWS Lambda
Recently Amazon announced native support for Go in Lambda. They also cut someone in half on stage. Cool. Honestly, the announcements made at re:Invent were exciting and soon after, Go was available to use with Lambda in early 2018.
I’d even say deploy a serverless geoip microservice service in seconds if it weren’t for the size of the data set. However, for about 30 minutes of coding and less than 100 lines, we really have a nice service here.
The Results & Performance
Let’s cut to the chase. You know how a simple geoip lookup works. We’ll go over how the code works, but most folks are interested in how it actually works. Is putting a geoip lookup microservice in AWS Lambda something you’d want to do? Sure!
The end result is an API that you can call to get a JSON response with geolocation data based on your IP address as well as an RPC to use with other Aegis based functions (or anything that makes AWS SDK calls to invoke a Lambda).
The API is pretty quick too. On average, it takes about 1 second to return a response.
There’s a few things you can do to tweak performance as well with Lambda, such as increasing the amount of memory the Lambda uses and keeping the container “warm.”
The actual time spent loading the geoip data set (using Maxmind’s free GeoLite2 City database…thank you Maxmind!) is about 1–1.5 seconds. However, subsequent Lambda invocations can re-use the same container if executed soon after the initial invocation. So pinging the API every few minutes would be what’s known as keeping it warm. That way, the burden of the initial load of that data set is avoided/minimized.
After that data set is loaded, handling the request is extremely fast. Using 1,204MB of RAM with the Lambda, along with Go, produces some great results. Most of the time is really spent in the actual invocation of the Lambda, API Gateway, some measurable time with DNS as well. The Aegis framework will wrap all handlers with XRay tracing by default (this strategy can be changed) and so we can easily see that it took 1ms for the Lambda to handle the request and 0.1ms for the actual function that did the look up.
Again, this service not only works with API Gateway, but is a microservice in its own right. That is to say, other Lambdas can invoke it. It’s still acceptably fast (an average of 60ms or so isn’t bad, you’re still billed for 100ms anyway). Here’s an example XRay trace:
Show me the Code
You can take a look at the go-lambda-geoip repository here. It’s pretty small. Just a main.go file of under 100 lines and a config file for Aegis.
You’ll need a few prerequisites in order to deploy it into AWS yourself. The readme has instructions. It should only be a few commands and a few minutes before you’re in business. Given you are a Go developer with AWS credentials all setup on your machine already, it is likely the fastest way to deploy a geoip service for you, period.
The way Aegis (and Lambda) works with Go is through a blocking process that listens for incoming messages. AWS uses TCP under the covers for this, so it’s fair to think of it like a web server to a degree.
Side Note: In the past there have been several shim approaches for getting Lambda to work with Go. Some involved piping data through stdin while others used HTTP servers. Looks like Amazon went with a similar TCP solution. Many of their structs were nearly identical to Aegis, so migrating it from a Node.js shim to native Go was a breeze!
Aegis adds some convenience in by allowing you to route the events to various handlers along with providing default/fall through handlers. It does this for various types of events; requests from API Gateway, CloudWatch event rules, or Lambda invocations via the SDK. Aegis will continue to add more triggers/routers/handlers over time.
In this case, we registered a handler for the root path to the API endpoint as well as an RPC handler which will handle any invocation of Lambda with a conventional JSON message format.
{
_rpcName: "lookup",
ipAddress: "127.0.0.1"
}Since there is no way to know if it was invoked via the SDK (unlike the API Gateway which carries a very specific signature), the convention is an ”_rpcName” key with a value that references the handler. Like all “routers” in Aegis, there could also be a catch all.
The intent of “RPC” handlers is to allow this Lambda to be called from other Lambdas (or anything using the AWS SDK really).
So that’s basically what you’re seeing in the code. The bulk of it being the loading of the data set and registering the handlers. This all then “starts” with a simple, blocking, call:
listeners.Handle()Note that we could load the data set in each handler, but that’s wasteful. That means every single invocation would go through the process of loading that large-ish file each time. It takes about 1–1.5 seconds for this. So you’re going to have a slow function on your hands.
To mitigate this, the code loads that data set in the main() function before the handlers listen for events. Now on the initial invocation, it loads the data set, but on subsequent invocations the Lambda container is likely to be re-used and so your responses will be much quicker.
The APIGatewayProxyResponse struct provides us with the client IP address if using the API Gateway. Otherwise, you’ll need to pass it in the RPC event JSON. Then the lookup() function is very simple here. It just uses another Go package that reads from the data set. It returns city level information.
That’s all there is to it. For illustrative purposes, here’s the code:
Aegis is a deploy tool as well (actually that was its first goal). It will build, zip, and upload the Lambda code along with setting up the API. It will create an execution role in IAM for you too. You can adjust configuration if you like in the aegis.yaml file.
The snippet above is likely to become dated, so if you want to actually deploy this service, use the repository. Note that it also does not include the XRay timings around the geoip data set loading that are seen in the screenshots above.
The best part of all this, it’s serverless. It runs fast enough to stay within the minimum billing per request (100ms) and if you don’t use it, you won’t be charged. In general, geolocation services (more the geocoding aspects) can get rather expensive, but using Lambda really helps reduce that cost.
In future articles (or perhaps even videos), I’ll go through how to use RPCs with Aegis and cover more architectural decisions and options available to you with Aegis.
