Feb 3
How We Built a Serverless Backend Using GraalVM, AWS Lambda and Astra DB (Part 2)
Authors: Frank Rosner, Raffael Stein
In this article, we continue the series of building a serverless backend using GraalVM, AWS Lambda, and Astra DB by completing the last goal of setting up a Lambda function to use the GraalVM native image runtime.
Introduction
In this two-part blog series, we’re building a serverless order processing API using Astra DB and AWS. As you might recall from the first part, we set ourselves three goals:
- Access Astra DB from within AWS Lambda.
- Write automatic tests for our Astra DB client.
- Set up the Lambda function to use the GraalVM native image runtime.
Let us recall the application architecture and review what’s still missing. So far, we’ve accomplished the first two goals and successfully implemented and tested an Astra Client that we can use inside our AWS Lambda function.
We can now focus on the third goal: implementing the Lambda function and running it inside a GraalVM native image runtime. We’ll also hook it up to API Gateway, so that the end user can call it via an HTTP API. The complete source code is available on GitHub.
Implementation
Lambda handler
In order to process incoming API Gateway requests via AWS Lambda, we need to implement a RequestHandler<APIGatewayV2HTTPEvent, LambdaResponse> which creates and uses our AstraClient.
APIGatewayV2HTTPEvent is implemented in the aws-lambda-java-events dependency and contains a JSON representation of an HTTP request. LambdaResponse is a simple data class we wrote that captures HTTP response fields we want to use: body and status code. You can find more information about the event and response format in the official documentation.
The following code shows a very simple handler implementation that decodes the incoming payload into an Order object and stores it in Astra DB. If the operation was successful, we return the respective order ID. Otherwise, we return an error.
Now that we successfully implemented our Lambda handler, we will look into building and packaging it so it can be executed inside a GraalVM native runtime.
Build steps
The goal of our build pipeline will be to generate a Lambda runtime package since we have to provide the entire runtime to AWS Lambda if we want to use GraalVM native code.
We’re going to use Maven as our build tool, and we’ll combine the following Maven plugins and dependencies to generate the runtime package:
lambda-runtime-graalvm(dependency) to generate the custom Lambda runtime.native-image-maven-plugin(plugin) to generate the native GraalVM image.maven-assembly-plugin(plugin) to package everything into a zip file.git-commit-id-plugin(plugin) to tag the zip file with the commit hash.
Let’s dive into each component in detail.
Custom Lambda runtime
By including the dependency lambda-runtime-graalvm, we’re adding the class com.formkiq.lambda.runtime.graalvm.LambdaRuntime, which is the main class for the native image.
It does everything a custom runtime needs to do, including the invocation of our Lambda handler function with the event payload. Please consult the Custom AWS Lambda runtimes documentation for more information on how to build custom runtimes from scratch.
Building the GraalVM native image
Next, we configure the native-image-maven-plugin. The following listing contains the plugin definition. We’ll go over it in more detail in the upcoming paragraphs.
In the executions section, we include the native-image goal as part of the package phase, which will invoke via ./mvnw package. In terms of plugin configuration, we’ll define an image name and the main class, which we pulled in via lambda-runtime-graalvm in the previous section.
Once you build a native image, it only includes code that is reachable from the configured main class, which will break some dynamic features offered by the JVM, such as reflection and URL protocols. To make sure our application works nevertheless, we need to pass a couple of build arguments:
— no-fallbackmakes sure we get a standalone image, or the build fails.— enable-url-protocols=httpenables HTTP support.-H:ReflectionConfigurationFiles=../src/main/resources/reflect.jsonspecifies the configuration file which contains all reflective accesses our application might perform.— no-servertells the builder not to start a dedicated build server but instead build the image in the builder process.
For more information on native-image build arguments, please consult the GraalVM Native Image Options documentation. The next listing contains the contents of reflect.json, which contains a bunch of data classes that we need to serialize and deserialize with Gson, as well as our Lambda handler class which the runtime needs to instantiate based on the qualified class name passed to the Lambda function.
Now that we can build our Lambda runtime as a GraalVM native image, we only need to package it into an archive which we can upload to AWS Lambda.
Zipping it up
Packaging the native image in a zip file will be done with the maven-assembly-plugin. Analogous to the native-image plugin, we will execute the plugin goal as part of the package phase.
The file assembly.xml contains the output file format (zip) and defines what we’re zipping.
The zip file name will have the git commit appended to make it easier to distinguish different packages when we update our Lambda function. The variable ${git.commit.id.abbrev} is defined by the git-commit-id-plugin.
Now running ./mvnw package generates the Lambda zip file. The next section covers creating the Lambda function and API Gateway resources to deploy and wire everything together.
Infrastructure
In order to create our infrastructure, we are using Terraform. Terraform is also used to create our Astra DB instance using the DataStax Astra Provider, but we’ll not discuss it in this post. Please check out the blog post, Let’s Get Started With Terraform for Astra DB, for more information on the DataStax Astra Provider.
Lambda
The Lambda function resource and surrounding resources are created through the terraform-aws-modules/lambda/aws module. The following snippet contains the lambda_function module definition.
The important properties of the module are:
- Our native runtime image zip file.
- The handler function.
- Environment variables holding the credentials.
It’s good practice to manage the credentials in a secret store, such as AWS Secrets Manager. For simplicity’s sake, they are passed as Terraform variables here.
API Gateway
With the Lambda function in place, let’s create the API Gateway resources to call our function via HTTP. This requires an API (aws_apigatewayv2_api), a stage (aws_apigatewayv2_stage), an integration (aws_apigatewayv2_integration), as well as two routes (aws_apigatewayv2_route).
The API defines the protocol type HTTP:
For this pet project, we will only need one stage:
The integration allows our routes to invoke our Lambda function:
We require two routes, one for order retrieval (api-gateway-get) and one for order persistence (api-gateway-post):
And that’s it! Now, after applying the Terraform plan, we can use our serverless order API.
Demo
We use the terraform apply command to create the entire infrastructure, including the API gateway, the Lambda function, and the Astra database.
Terraform plans the necessary changes and — due to the --autoapprove flag — immediately takes action. After completing the Terraform command, all our components are ready to be used. The Outputs section shows us the URL of our API gateway endpoint. We can invoke our API by sending JSON queries via the curl command.
Our /order endpoint accepts a JSON document containing order details. It persists the order in Astra DB and returns the persisted object, which now contains an orderId. Now we can use curl to retrieve the order again.
Conclusion
This concludes our two-part series of developing a backend using serverless technologies. In Part 1 we successfully implemented and tested an Astra Client that we can use inside our AWS Lambda function. Part 2 shows how you can implement the Lambda function and run it inside a GraalVM native image runtime.
You can look at the entire project over on Github, and if you have any questions or want to know more about this project, feel free to reach out to us at @FRosnerd and @raffael on Twitter.
Follow the DataStax Tech Blog for more developer stories. Check out our YouTube channel for tutorials, and follow DataStax Developers on Twitter for the latest news about our developer community.
