Here we Go — (not just) an AWS Lambda performance test
By Sonja Pfeifer, Full Stack Developer
Always start with why
Searching for AWS Lambda performance on medium there are already plenty of interesting articles, so why bother writing one for Marc O’Polo? As you’ll discover this article is not just about the results of the performance test but how we overcome the issue integrating the new programming language into our existing CI / CD pipeline.
Our current setup
We started building micro services on AWS a couple of years ago, with our preferred programming language JavaScript. The more services we moved to the cloud, the more mature our CI / CD pipeline became. Starting off with dedicated stacks defined in CloudFormation templates with yaml, we later moved on to building theses stacks with CDK. This also became a starting point to migrate JavaScript ECMAScript 6 Lambdas to TypeScript. First we started building new functions in TypeScript and whenever we needed to adapt, migrating existing functions as well.
As stated above there are many interesting articles about the AWS Lambda performance with different runtimes, thats how Go came to our attention. Next to overall good performance and low memory consumption, a fast cold start is one of the key benefits from using Go. That sounded good to us and we wanted to see if those promises can be kept.
Which lambda to pick?
There were still some JavaScript Lambdas left to migrate so why not migrate it to Go, if it wins the test. We checked which service could improve through the key benefits of Go, especially the cold start time. Apart from that we wanted to see how the language is interacting with AWS services and at the same time not relying one of the custom packages we build. A simple lambda got our attention:
- called irregularly, so a cold start is quite frequent
- called through api gateway
- selecting and returning data from RDS MySQL
That was the perfect playground for our test, as we could see the Go integration with two frequently used AWS services. By the time starting our test, for AWS Lambdas we used three different runtimes: Node.js (ECMAScript, TypeScript) and Python. Within other areas we also develop in .NET Core. So after a quick discussion we decided to compare the existing Node.js lambda with a .NET Core 3.1 and a Go version. Since it was the first implementation with Go we checked the language documentation and starting getting familiar within the various playground exercises.
On how to build a lambda in Go, AWS provides useful documentation and helpful examples to start coding. As the first draft was ready to be tested we locally build and zipped the code and manually uploaded it into the AWS console. The stats of the first execution were quite good and we could see why Go ‘is becoming more and more popular’. In the next implementation iterations we cached the DB connection within the init() function and moved the struct declarations to a separate file. Now the code was ready to be challenged.
We did a couple of rounds calling each lambda and then checked the stats in the CloudWatch console.
Performance results
The JavaScript version of the lambda wasn’t too bad, the average cold start time and variance in execution time would work for the service. The second language, .NET Core struggled with the cold start. Go delivered what was promised:
Checking the execution times (without cold start) the situation changes. The .netCore version performed better than the JavaScript lambda:
Summarising the results, Go impressed us with:
- the lowest memory consumption — more than half compared to C#
- short cold start time — around the average execution time for C#
- overall low execution times, mean of 10.4 ms with a standard deviation of 4 ms
We were thrilled with the outcome!
Bringing it alive — integrated
Our current setup can easily be deployed through individual stacks from bitbucket using an alpine docker base image. For each resource type there is a dedicated builder class which provides all necessary options and associated functions. We found this interesting article from AWS on how to build Go lambdas with CDK and used it as a starting point. Within the lambda builder, we extended the createLambda function to set the build options if lambda runtime is set to Go.
Running the bitbucket pipeline we got following error:
Bundling did not produce any output. Check that content is written to /opt/atlassian/pipelines/agent/build/infrastructure/cdk.out/asset.Subprocess exited with error 1
Ok that didn’t work as hoped. As the message stated no output was produced but why?
For debugging reasons we locally build a docker container, copied all resources and ran ‘cdk synth’. Awesome — we got the same error message so we could start debugging it now. The logged path variables showed that the current directories were correct and Go was correctly installed.
Just like with Docker bundling, you must make sure that you place the Go executable in the
outputDir.
Ok thats what we’ll do now. Within the local attribute we add:
exec(`cp -r ${path.join(outputDir, ‘main’)}/. ${outputDir}`)to manually copy the resources to the output directory. Running ‘cdk synth’ again the output directory now contains the source files and the build worked. Now we copied the compiled Go module from the docker container and uploaded it to AWS. Unfortunately, running the lambda produced following output:
{
"errorMessage": "fork/exec /var/task/main: no such file or directory",
"errorType": "PathError"
}Even though containing the same files on build — the locally build module was around have the size of the module build inside the container. Something was off.
Searching for any hints this github issue held our answer. We switched to statically linking the libraries, using ldflags.
GOARCH=amd64 GOOS=linux go build -ldflags '-linkmode external -w
-extldflags "-static"' -o main main.goAgain, we ran ‘cdk synth’, copied the module and uploaded it to AWS — now the lambda was running successfully!
We extended lambda builder, pushed the changes and ran the bitbucket pipeline. Bundling produced output, the lambda was updated and could successfully be executed.
Conclusion
Testing new technologies and (from time to time) programming languages is one of the small adventures we love at Marc O’Polo. Getting started with Go was pretty straight forward using first steps and tutorials provided at the language documentation. On a first glance the integration into the existing CI/CD pipeline seemed to be easy as well. While overcoming the issue with bundling the module, we dived deeper into the Go world than expected at the start.
You want to shape the digital future of the fashion industry? Start creating with us. The Digital Intelligence & Tech Teams at Marc O’Polo are always looking for talented and driven software engineers, data engineers, data scientists, data analysts, ML Engineers or project managers to join our team. Learn about life at Marc O’Polo and visit our career site.
