Java in AWS Lambdas: how to play nice with other services

Beyond HelloWorld
See our Twitch session on this blog post!

As discussed in our architecture post, we’re migrating our on-prem services onto the cloud, and we’re betting on serverless with Java Lambdas.

So what does this mean for our API call performance?
How are they deployed through the CDK?
How do we code to make this work with other AWS services?
…. and 3rd party libraries?

Disclaimer
I Love My Local Farmer is a fictional company inspired by customer interactions with AWS Solutions Architects. Any stories told in this blog are not related to a specific customer. Similarities with any real companies, people, or situations are purely coincidental. Stories in this blog represent the views of the authors and are not endorsed by AWS.

First things first, how cold starts affects us

While researching our options, we quickly ran into a bunch of articles declaring that Java was slow to run on Lambdas, compared to other languages. The issue seems to be caused by “cold starts”, i.e. when a Lambda is invoked the first time, it needs to spin up its environment (e.g. load libraries) and that takes time. Once it has its cold start, the lambda stays “warm” and can handle subsequent requests very fast until it is eventually killed by lack of requests over a period of time.

Quite frankly, we were a bit surprised on the pushback we saw on the internet regarding using Java in a serverless way. However, we are a Java shop… and switching on the fly to another language while the covid-lockdown clock is doing a countdown will probably set us up for guaranteed failure. There’s just no way we can train all our software developers in one fell swoop, and going from being Java experts to novices in some other language is frightening for both management and for the developers themselves. How can we even ensure we’re following best practices in a brand new language, when no one knows it? How do we know we’re not creating security holes? How much will training dozens of people cost us,.. not only in money but also in time? How many of our best programmers would we lose because of the switch?

It seems that depending on what the Lambda does, the “cold start” can take a few seconds. As on-prem developers, we don’t bat an eye when a server takes 30s to start up, so why do a few seconds matter for a Lambda? The difference here is that once an app server has started, it holds on to a minimum number of db connections which are shared across the entire application.

Lambdas on the other hand, are just microservices.. they just do one thing. So a Lambda will hold just 1 db connection and can handle just 1 request at a time. If a 2nd request comes in while the Lambda is busy with the 1st request, a new copy of the Lambda will be spawn up and will have to connect all on its own. So now we have 2 users suffering through the “cold start” instead of just 1. If we’re lucky enough, request #3 will be handled by the 1st lambda if it was already done processing, and that user will have an already warmed up lambda with a previously established db connection. In order to mitigate the waiting, we need to make sure we have a sufficiently low cold start.

So the question remains.. can we make Java work in a serverless world?

So let’s find out. We set up a HelloWorld Lambda and executed it several times. Since the cold start only happens the first time the Lambda is executed every XX minutes (30? 40?), we tricked the Lambda into reinitializing itself by changing the memory settings (from 2048MB to 1024MB and then back up to 2048MB). The HelloWorld Lambda took about 0.5s to initialize, so we decided to proceed with the real test… connecting to a database.

Serenity Now!! Establishing a DB connection

Establishing a connection to the database has been and will probably remain the biggest bottleneck in any database-powered application (hence the creation of connection pooling!!). We went ahead and tried to test that using our Java Lambda, and came up with 7s. After some initial performance tuning, we brought this down to 2.5s.

This means that the first user to invoke a Lambda will have this much latency, as well as the first user for every new Lambda spun up in order to take care of an increase in requests.

So what about the subsequent runs while the Lambda is ‘warm’? Those were under 100ms, which was great.
So how long does the Lambda stay ‘warm’? There’s no formal promise given by AWS, but the common ballpark on the internet seems to be around 20–40min (but beware, there are official disclaimers that the environment can be killed at any time). Around that timeframe, if no requests come in the Lambda will be killed and a brand new will replace it on the next request. However, as long as requests keep coming in, the Lambda could stay warm forever. All in all, having an occasional 2.5s latency offset a regular sub-100ms latency on a retail app is acceptable for our purposes.

Note: If you’re interested in how we did performance tuning, please check back in the next weeks, we will write an article about that too!

Lambda Tips

With that doubt out of the way, we decided to proceed and put our serverless eggs in the Java Lambda bucket. After setting our lambdas up, we have a couple of things to note:

• The event argument is the info accompanying the request (e.g. query params)
• The context argument is the info about the invocation (e.g. requestId, lambda memory configuration)
• You can return an object or void from the function. If it’s an object, it will be serialized into its text representation (e.g. object into json, primitives into strings)
• Besides the event argument, you can also set Environment variables on a Lambda, e.g. region, log levels, endpoints for your Lambda to use.
• You can’t see the Java code inside the Lambda in the console, like you sometimes can with other languages 😠. You can however, download the code by using the Export menu to inspect what got uploaded.

Writing Handler arguments that other services call

Now there’s a bit of a twist on the handler arguments… the types of the arguments depends on what is calling the Lambda. For example, a handler will typically be declared as

public class Handler implements 
   RequestHandler<Map<String,String>, String>

… but if it’s called from ApiGateway, it’ll be a different story:

public class CreateSlots implements     
   RequestHandler<APIGatewayProxyRequestEvent,     
                  APIGatewayProxyResponseEvent>

This is because ApiGateway passes in a ton of information on the request, and the Request argument needs to accommodate for it.

If you don’t adapt your handler arguments appropriately, you’ll probably wind up with an obscure error: “Cannot deserialize instance of java.lang.String out of START_OBJECT token”, as the Lambda can’t marry what it’s getting vs what it’s expecting. Quite frankly, figuring out what to pass in wasn’t a walk in the park either, and I wound up relying on George Mao’s blogpost to figure it out.

And so, if you’re calling your Lambda from another service, check out what the corresponding class is for the event and use that in your handler arguments.

A word on including 3rd party libraries

One biggggg thing to take into account when using Java Lambdas, which is a huge difference from on-prem development is the amount of 3rd party libraries to include in your Lambda. You cannot, I repeat, you cannot load up your Java Lambda functions carelessly with libraries like you did on your app servers, since Lambdas run in a constrained environment.

Adding a library might take a penalty on the cold start, depending on what it does. Also, a library might contain transitive dependencies, i.e. rely itself on other libraries which ultimately adds to the zip file. One thing to watch out for particularly, is whether the dependency uses reflection, since this heavy operation can negatively impact cold starts.

So how did this affect our development?

• we opted out of using MyBatis for importing and executing SQL within a file (reduced cold start by 150ms) and instead manually read in the file line by line
• we took out using SecretsManager SDK (which takes 1.2s!!) for reading in db credentials, and changed to use IAM authentication instead for customer-facing Lambda functions (it was a best practice anyway!)
• we even had heated discussions about whether or not to include log4j (adds 700ms to the cold start!)…. I lost that battle 😕
• we added Lombok using Gradle’s compileOnly dependency feature, which uses the library during compile-time to generate the getters/setters, but doesn’t include it in the final zip file

Note: do NOT load the entire Java SDK. Cherry pick what you really need instead!

For those libraries that you cannot do without, there are ways of optimizing and dwindling down what gets added to your Lambdas. We’ll cover that in subsequent posts, but for now just remember that a lean Lambda is a fast Lambda.

Deploying Java Lambdas with CDK

In the previous post, we explained how to write CDK in Java. Now we’ll talk about the bundling process of the Lambdas by the CDK.

Since we’re including Java Lambdas with external libraries with our infrastructure, we need to make sure the *.java files are compiled and bundled into a Zip file together with the libraries, which will be later uploaded into the AWS account.

We could just manually run ./gradlew build and let the CDK pick up the generated lambda.zip, but we’d rather incorporate this step into our CDK deployment.

To do this, we pass in the bundling instructions when defining a Lambda. First, we define the commands needed to build & bundle our Lambdas in our local environments (our laptops):

ProcessBuilder pb = new ProcessBuilder(
    "bash",
    "-c",
    "cd ../ApiHandlers && ./gradlew build "
        + "&& cp build/distributions/lambda.zip " + outputPath);

This is basically what we would’ve done had we manually built and bundled our app onto a zip file thru the command line, and it’s a simpler version of the default bundling functionality, which uses Docker:

List<String> apiHandlersPackagingInstructions =
    Arrays.asList(
        "/bin/sh", 
        "-c",
        "./gradlew build "
        + "&& ls /asset-output/"
        + "&& cp build/distributions/lambda.zip /asset-output/");

The Docker version is slightly different, since it uses the asset-output as the target directory for the zip file. Anything in that directory will be uploaded to the Lambda.

So which one gets executed? Well, first it tries the local environment version and only if it fails, it falls back to using the default Docker version.

You can see the complete code in the declaration of the DefaultLambdaRdsProxy within the ApiStack. If you’re interested in a more thorough explanation (yes, I know this was hard to digest!) of what’s going on during CDK bundling, try this one (go to Local Bundling section).

I hope you’re able to make good use of this info, and as usual, if you have any feedback don’t be shy and drop me a line ;-)

Over the next weeks, we will go deeper into how to performance tune your Java Lambdas, so don’t forget to circle back!

Useful Links

Java Lambdas for this blogpost
CDK script bundling the Java Lambdas
Lambda Handler Api Gateway
Classes to import for Lambda Handlers

Writing your CDK in Java

A Serverless Java Solution for Deliveries