Dec 9, 2018

6 min read

AWS custom runtime for lambda really works: How I developed a lambda in Perl

At re:Invent 2018, AWS announced a new lambda feature called custom runtime for lambda and what exactly it means is you can develop lambda functions in not only the out-of-box supported runtimes like Java, C#, Python etc but also using any other programming language.

So I decided to give it a try and guess what its worked as promised. I am able to develop and deploy a lambda function using Perl (no kidding!!).

Lets quickly review the key points for lambda custom runtimes and then we will go through some real code to try it out.

The runtime is responsible to execute the lambda function’s code, it reads the handler info (file name and the method name) from the pre-defined environment variable _HANDLER
The runtime passes the event data to the function handler, and posts the response from the handler back to Lambda.
The custom runtime runs on the standard lambda execution environment, so it will be your responsibility to bring along any library or language support you need for the custom runtime.
Make sure the lambda Runtime is as below on AWS console or “Provided” in the create-function lambda CLI.

The runtime can be included in your function’s deployment package, or in a layer.
A custom runtime’s entry point is an executable file named as “bootstrap”, it should be in the lambda deployment package or in a referenced lambda layer

Key environment variables that you should be using in the bootstrap file to initialize and executethe lambda task:

_HANDLER: The format is [file-name].[method], the method in this file will be executed
LAMBDA_TASK_ROOT: This is where on lambda execution environment the deployment package files will be available. The value is “/var/task” and its a read-only location

I have mentioned a few of the key steps in bootstrap executable file below, for additional details refer here:

Initialize the function — Load the handler file.
Handle errors — If an error occurs, call the initialization error API and exit immediately.
Get an event — Call the next invocation API to get the next event. The response body contains the event data. Response headers contain the request ID and other information.
Create a context object — Create an object with context information from environment variables and headers in the API response.
Invoke the function handler — Pass the event and context object to the handler.
Handle the response — Call the invocation response API to post the response from the handler.

Lets deep-dive on how we can develop a lambda using perl runtime:

A lambda function code in perl needs the perl interpreter and the dependant libraries.

I downloaded the ActivePerl community edition for linux (x86_64) from here
I created a lambda layer using the download file, you cannot upload it as-is due to size limitation, lambda layer only supports a zip file upto 50MB so to keep the size within limit I had to delete the man and html directories, the final zip file only has below directories:

The lambda layer looks as below:

Let's take a quick look into the bootstrap file:

Its a shell script

#!/bin/shset -euo pipefail# Initialization - load function handler
# The function handler is of the format [handler-file-name-withon-extn].[handler-method]# The handler file/script name
lambda_script_name="$(echo $_HANDLER | cut -d. -f1)"# The handler method/function name
handler_name="$(echo $_HANDLER | cut -d. -f2)"# Processing
while true
do
  HEADERS="$(mktemp)"  # Get an event
  EVENT_DATA=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
  REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)cd /tmp# The directory location $LAMBDA_TASK_ROOT which is /var/task is readonly,
  # So copy the perl_runtime.pl.tpl file from layer location /opt/ to /tmp
  cp /opt/perl_runtime.pl.tpl perl_runtime.pl# Copy the perl lambda script from LAMBDA_TASK_ROOT
  cp $LAMBDA_TASK_ROOT/$lambda_script_name.pl $lambda_script_name.pl# Update the perl runtime file with the actual script name and
  sed -i "s/PLACEHOLDER_SCRIPT_NAME/$lambda_script_name/g" perl_runtime.pl
  sed -i "s/PLACEHOLDER_HANDLER_NAME/$handler_name/g" perl_runtime.pl# Execute the handler function from the script
  export PERL5LIB=/opt/perl/lib
  RESPONSE=$(/opt/perl/bin/perl perl_runtime.pl "$EVENT_DATA")# Send the response
  curl -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response"  -d "$RESPONSE"
done

Extract the lambda handler file name and the handler method from the environment variable _HANDLER
Run an infinite loop to handle incoming requests
Get the EVENT_DATA from the lambda runtime API: http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next
Get the REQUEST_ID from the HEADERS
I have a template perl script perl_runtime.pl.tpl that acts as a glue code between the bootstrap and the actual lambda handler file, I have kept it so that I can create a perl script with the actual handler file name and the function name.

#!/usr/bin/perluse strict;my $lambda_script_name = 'PLACEHOLDER_SCRIPT_NAME.pl';require "$lambda_script_name";
my $handler_return_value=PLACEHOLDER_HANDLER_NAME(@ARGV);
print $handler_return_value

The bootstrap executable file and the perl_runtime.pl.tpl file is in a separate lambda layer named as perl-runtime:

The bootstrap file and perl_runtime.pl.tpl are already in PATH and available under “/opt/”
The bootstrap executable file copy over the perl_runtime.pl.tpl file to /tmp and replaces the placeholder variable with actual values and create the actual perl_runtime.pl
The bootstrap executable also copy over the actual lambda handler file to /tmp so that it can be invoked by the perl_runtime.pl
As we already have perl interpreter as runtime and the lambda will refer it, so the interpreter and the libraries are already available under “/opt/perl” and the bootstrap executable using them as below to involve the perl_runtime.pl script:

export PERL5LIB=/opt/perl/lib
RESPONSE=$(/opt/perl/bin/perl perl_runtime.pl "$EVENT_DATA")

Lambda Handler Script:

The lambda handler script file “lambda_script.pl” just read the incoming json event and add a reply field and return back the same.

use strict;
use warnings;
use JSON;sub handler {
  my $event = $_[0];
  my $reponse = decode_json($event);
  $reponse->{'reply'} = "Thanks AWS custom lambda runtime!! Now even perl can be a runtime!!";return encode_json($reponse)
}1;

The actual execution flow:

Lambda function deployment:

Create perl interpreter lambda layer:

This is the perl interpreter and the library zip file.

aws lambda publish-layer-version --layer-name perl --zip-file fileb://perl.zip

Create perl-runtime lambda layer:

It has the bootstrap executable shell script and perl_runtime.pl.tpl files

aws lambda publish-layer-version --layer-name perl-runtime --zip-file fileb://perl-runtime.zip

Create the lambda function:

The perl-lambda-poc.zip file contains the lambda_script.pl file, the actual lambda function handler.

aws lambda create-function --function-name perl-lambda-poc \
  --zip-file fileb://perl-lambda-poc.zip \
  --handler lambda_script.handler \
  --runtime provided \
  --role arn:aws:iam::1234554321:role/my-lambda-role \
  --layers perl perl-runtime

You can invoke the lambda function with any json payload, I tested it with:

{
  "request": "Hey perl lambda!!",
}

And the response came back as:

{
  "request": "Hey perl lambda!!",
  "reply": "Thanks AWS custom lambda runtime!! Now even perl can be a runtime!!"
}

AWS custom runtime for lambda really works: How I developed a lambda in Perl

Lets deep-dive on how we can develop a lambda using perl runtime:

Lambda function deployment:

More from Avijit Sarkar

Get the Medium app

Avijit Sarkar