OneByte
Published in

OneByte

Extend Lambda’s functionality using AWS Step Functions

How to use Step Functions to extend Lambda functionality.

Step Functions

Overview

AWS Lambda is a great service to run your code in a serverless environment. But it also has its own shortcomings like state management and limited execution time. Currently, Lambda’s max execution time is 15 mins, which means we cannot use it to run long-running tasks and there is no way to manage the state unless we use external services.

One question arises here that why use lambda for this purpose because it is not built for running long-running tasks and state management?

The reason to use it is due to its strong integration with other AWS services (i.e Kinesis, SQS, etc), the devs can focus on their code without worrying about the infrastructure. So, these benefits make it a strong contender for these types of tasks.

The article is about Step function utilization to overcome lambda’s limitations. I will use AWS RDS Disaster Recovery Plan (migrating manual snapshots to another region) as an example to demonstrate it. Snapshot creation and migration is a time-consuming task, so it will serve as a great example for this purpose.

If you are interested in the AWS RDS Disaster Recovery strategy read this article.

Pre Requisites

  1. RDS instances must have Multi-AZ configuration enabled to avoid IO suspension.
  2. Create an AWS KMS key in the destination region. It will be used during snapshot copy/migration operation.
  3. Create Role and Policies for lambda.

Details

This section provides detailed guidelines on how to develop the whole workflow:

  1. Develop a lambda function that will create and migrate the snapshots. Lambda needs to modify the lambda event object to add state variables and return the event object so that Step Function can decide what to do next. The code given below contains skeleton code for the lambda:
Lambda Skeleton

2. Once the lambda is developed, the next step will be to create a state machine for the lambda to manage its state so that we can run lambda again and again until all the snapshots are migrated to another region. The JSON given below contains the config for the state machine:

State Machine Configuration

The state machine will look like this:

State Machine

3. After lambda and state machine creation create an AWS Cloudwatch Event that will StepFunction trigger the state machine based on your requirements.

Final Thoughts

I hope you enjoyed this story. Please share your feedback about anything that can be improved or I missed. Thank you.

--

--

--

We’ve been shaping the innovation road-map since our inception, one byte perfection at a time.

Recommended from Medium

Extensibility in Elixir Using Protocols

Analyzing trends and patterns in growth of a company with SQL (Part — 2)

5 Best Cross-Platform Desktop Desktop Frameworks

Configuring Kubernetes Role-Based Access Control with Lens Spaces

Parsing ICD Codes With Python

Ease of UI releases- micro frontends with module federation

Don’t worry, be happy: How to build your future tech career in 5 simple steps.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Irtiza

Irtiza

DevSecOps Engineer https://irtizaali.com/

More from Medium

How to integrate ASP.NET Core Web to CloudWatch on AWS Beanstalk?

Adding HTTS And Custom Domains To An API Hosted On AWS Fargate Using CDK

Lambda as a container image sometimes could be tricky for you

Trigger “specific” ECS Tasks with S3 payloads