AWS Lambda and DynamoDB with Rust

In a previous post we saw how easy and straightforward it is to create AWS Lambda functions using the Rust runtime.

Let’s take this a step further and demonstrate how can we add the ability to interact with DynamoDB from AWS Lambda functions using Rust.

What we are building

In this post, we are going to create a Rust function validate_serial_unique which takes a string serial number value and checks if it already exists in a DynamoDB table.

The function returns true if the provided value is unique (does not exist) and returns false if the provided value is already part of the collection and thus is not unique.

Creating a DynamoDB table

In order to be able to test our function, lets create a DynamoDB table named assets which is going to host a collection of items uniquely identified by their serial number. The primary key of the table will be a field name serial_number .

Once the table has been created, we need to ensure that we can access records in the table from a AWS Lambda. You may want to check this post on how to configure the IAM security profile so that the lambda function has access to DynamoDB resources.

The Rusoto Crate

In order to interact with the DynamoDB API, we need to adopt the rusoto crate as a dependency of our lambda function. It’s an open source Rust SDK for various AWS services. (and it’s awesome, in my opinion 😍)

By default, rusoto depends on OpenSSL which means we also need to add an additional dependency for OpenSSL as described in the readme of rusoto_core crate. At the time of writing, this creates the complication that OpenSSL will have to be built along side our binary and this can be a tricky procedure depending on your current development environment.

As an alternative, rusoto allows the use of a feature flag to replace OpenSSL with rustls. The rustls crate doesn’t require additional steps to build.

Adapt the dependencies section in your Cargo.toml to include the following two packages:

[dependencies]
...
rusoto_core
= {version = "0.35.0", default_features = false, features=["rustls"]}
rusoto_dynamodb = {version = "0.35.0", default_features = false, features=["rustls"]}

Querying DynamoDB from Rust

Querying DynamoDB from Rust is really quite straightforward thanks to the excellent API provided by the rusoto crate. In order to read an item from the table, we are going to initialize and execute a GetItem operation.

We begin by importing the dependencies into scope:

extern crate rusoto_core;
extern crate rusoto_dynamodb;
use rusoto_core::Region;
use rusoto_dynamodb::{DynamoDb, DynamoDbClient, GetItemInput, AttributeValue};
use std::collections::HashMap;

Next, we are going to create a method which takes a serial number and checks if it already exists in the DynamoDB table which I created earlier.

fn validate_serial_unique(serial_number: &str) -> bool {
let mut query_key: HashMap<String, AttributeValue> = HashMap::new();
query_key.insert(String::from("serial_number"), AttributeValue {
s: Some(serial_number.to_string()),
..Default::default()
});

let query_serials = GetItemInput {
key: query_key,
table_name: String::from("assets"),
..Default::default()
};

let client = DynamoDbClient::new(Region::EuCentral1);

match client.get_item(query_serials).sync() {
Ok(result) => {
match result.item {
Some(_) => false,
None
=> true
}
},
Err(error) => {
panic!("Error: {:?}", error);
},
}
}

Let’s explain what happens here step by step.

To query the DynamoDB table, we need to prepare a GetItemInput instance which holds information about the table and keys we would like to retrieve. This will result in a GetItem operation.

The assets table we created earlier uses a primary key field named serial_number which uniquely identifies all records in the table, so we can initialize the query_key hash map with a single value with key “serial_number” and a string value of the serial number value we want to lookup.

let mut query_key: HashMap<String, AttributeValue> = HashMap::new();
query_key.insert(String::from("serial_number"), AttributeValue {
s: Some(serial_number.to_string()),
..Default::default()
});

let query_serials = GetItemInput {
key: query_key,
table_name: String::from("assets"),
..Default::default()
};

Next, we are ready to connect with the DynamoDB instance. For this purpose we create a client instance for the region in which the DynamoDB instance is hosted. My test database is located in the EuCentral1 region:

let client = DynamoDbClient::new(Region::EuCentral1);

Then, we can execute the query. For simplicity, I am executing the function synchronously, however the rusoto SDK provides full support for futures that you can take advantage of. (Futures are a similar concept to Promises in JavaScript or Tasks in C#).

client.get_item(query_serials).sync()

If successful, the operation returns a Result which contains an optional item matching the query. If there was a matching element in the table, the result item will have some value and this is what we use to determine if the serial number is unique:

match client.get_item(query_serials).sync() {
Ok(result) => {
match result.item {
Some(_) => false, // already exists => not unique
None
=> true // no result found => unique
}
},
Err(error) => {
panic!("Error: {:?}", error);
},
}

AWS Lambda Handler

In order for our code to run in AWS Lambda, we need to adopt two more dependencies:

Open Cargo.toml and add the following under the dependencies section. I am going to include the package versions at the time of writing but feel free to lookup the latest version of the crates.

[dependencies]
serde = "1.0.80"
serde_derive
= "1.0.80"
lambda_runtime
= "0.1.0"

Let’s import the creates at the top of main.rs

#[macro_use]
extern crate lambda_runtime as lambda;
#[macro_use]
extern crate serde_derive;

The AWS Lambda runtime expects to be able to call a function with the following signature:

pub type Handler<E, O> = fn(E, Context) -> Result<O, HandlerError>

The first argument of the handler method is the event object which contains the payload that triggers the execution of the lambda function. We haven’t defined that object yet, so let’s do it now:

struct ValidationEvent {
serial_number: String
}

Since both the event payload and response are going to be serialized and deserialized, we need to annotate them with the Serde macros:

#[derive(Serialize, Deserialize)]
struct ValidationResult {
#[serde(rename = "isValid")]
is_valid: bool,
errors: Vec<String>
}
#[derive(Serialize, Deserialize)]
struct ValidationEvent {
#[serde(rename = "serialNumber")]
serial_number: String
}

Finally, we need to implement the Lambda event handler and register it using the lambda! macros:

fn main() -> Result<(), Box<dyn Error>> {
lambda!(validation_handler);
Ok(())
}
fn validation_handler(event: ValidationEvent, ctx: Context) -> Result<ValidationResult, HandlerError> {
Ok(validate_serial(event.serial_number.as_str()))
}

This last part makes it possible for incoming ValidationEvent requests to be passed to our lambda function handler and result in a ValidationResult responses to be sent back.

The ValidationResult is a custom enumeration type which defines the kinds of validation errors our AWS Lambda function will return:

enum ValidationError {
InvalidFormat,
AlreadyExists
}

Since we are going to serialize the response to JSON, I prefer to also implement a way to associate a string representation for each of the validation errors which are going to be part of the output response of the function:

impl ValidationError {
fn value(&self) -> String {
match *self {
ValidationError::InvalidFormat => String::from("invalid_format"),
ValidationError::AlreadyExists => String::from("already_exists"),
}
}
}

Finally, the response we are going to eventually serialize and return:

struct ValidationResult {
is_valid: bool,
errors: Vec<String>
}
fn validate_serial(serial_number: &str) -> ValidationResult {
let mut result = ValidationResult { is_valid: true, errors: Vec::new() };

// other validators...

if !validate_serial_unique(serial_number) {
result.is_valid = false;
result.errors.push(ValidationError::AlreadyExists.value());
}

return result;
}

Building for the AWS Linux environment

From the AWS Lambda documentation we see that AWS Lambdas are executed x86 64bit Linux environment. In this case, we should make sure that our Rust project is also targeting and successfully building in that environment.

Note: if you are currently on a x86_64 Linux environment, then you can skip this section.

I found an excellent tutorial which guided me through the needed steps in order to be able to build for this architecture. You can read it yourself here.

rustup target add x86_64-unknown-linux-musl

Let’s configure our project to build against this target.

  • In the project root folder, create a subfolder named .cargo
  • In this subfolder, create a file named config with the following contents:
[target.x86_64-unknown-linux-musl]
linker = "x86_64-linux-musl-gcc"

You can read more about the cargo configuration file in the docs.

On macOS we also need to install the corresponding linker tools so that cargo can produce a binary compatible with x86 64-bit Linux. Luckily, that’s not hard using Homebrew.

brew install filosottile/musl-cross/musl-cross

Once installation is complete, it is possible that the new musl-gcc binary is not discovered during the build process. To address this, we can manually create a symbolik link to it as follows:

ln -s /usr/local/bin/x86_64-linux-musl-gcc /usr/local/bin/musl-gcc

Running cargo build --release --target x86_64-unknown-linux-musl at this point should be successful.

Publishing the executable

Before we can create the AWS Lambda, we need to package the executable.

The AWS runtime expects a binary named bootstrap so we need to rename it before publishing. Here is a command line snippet that can take care of renaming and zipping the binary for upload:

cp ./target/x86_64-unknown-linux-musl/release/aws_validate_serial ./bootstrap && zip lambda.zip bootstrap && rm bootstrap

Next, head over to the AWS Lambda dashboard and create a new function.

Our function will use a provided runtime

Next, upload the lambda.zip package

Note: you can leave the value in the Handler field as is, it doesn’t really matter in our case.

Click Save.

Testing the Lambda function

Before we can test our function, let’s add a few records to the assets table in DynamoDB.

Let’s try a test event for the lambda function with the following payload:

{
"serialNumber": "123456b"
}
Our function returns a validation error

Let’s try a valid payload:

{
"serialNumber": "new12345"
}
Our function returns that the input is valid

It works! 🎊🤩

You can find the complete source for this post over at GitHub.