AWS Serverless Framework
The Serverless Framework has got an open-source CLI and a hosted dashboard and together we get full serverless application lifecycle management. We can develop, deploy, troubleshoot, and secure our applications with radically less overhead and cost by making use of the Serverless Framework.
Serverless Framework with AWS
The Serverless Framework can be used for deploying our code to many cloud providers like AWS, Apache OpenWhisk, Cloudflare Workers, Microsoft Azure, and Google Cloud Platform.
At Fasal we make use of the serverless framework to deploy and manage our AWS services. The Serverless Framework helps in deploying our AWS Lambda functions, along with the AWS infrastructure resources which are required. It’s a CLI that gives us structure, automation, and best practices out-of-the-box, allowing you to focus on building polished, event-driven, serverless architectures, comprised of events that trigger the functions.
The Serverless Framework differs from other app frameworks because:
- It manages both code and infrastructure
- It supports Node.js, Python, Java, and much more languages.
Core Idea
Functions
A Function is an AWS Lambda function. AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. You can trigger Lambda from over 200 AWS services and software-as-a-service (SaaS) applications and only pay for what you use.
Events
Framework as an Event is anything that triggers an AWS Lambda Function to run. Events are infrastructure events on AWS such as:
- An AWS API Gateway HTTP endpoint request (e.g., for a REST API)
- A CloudWatch timer (e.g., run every 5 minutes)
- An AWS S3 bucket upload (e.g., for an image)
- A CloudWatch Alert (e.g., something happened)
- An AWS SNS topic (e.g., a message)
Resources
Resources are AWS infrastructure components that our lambda function needs such as:
- An AWS S3 Bucket.
- An AWS DynamoDB Table.
- An AWS SNS Topic.
- Anything that can be defined in CloudFormation is supported by the Serverless Framework
Services
It's like a project file, and we can have multiple services for a single app. A Service is the Framework’s unit of organization. It’s where we define our Functions, the Events that trigger them, and the Resources. A file named serverless.yml or serverless.json can be used to describe a service in YAML or JSON format, at the root directory of the project.
Serverless Use Cases
Below are some of the use cases for the AWS Serverless Framework:
Image Processing
Workflow Automation
Scheduled Tasks
Web and Mobile Apps and APIs
IoT
Serverless Use Cases at Fasal
Plugins in Serverless Framework
The Serverless Frameworks' new features can be extended using a plugin and the plugin is custom JavaScript code. At Fasal, we use a specific workflow so we install a pre-written plugin or we can also write one to customize the framework as we need.
Automating CI/CD workflow for serverless apps with CircleCI
At Fasal, to minimize redundant communication across teams and manual scrutiny we need to invest in automating CI/CD processes. We make use of CircleCI as a CI/CD tool. Know more about CircleCI.
Automating complex workflows
As an AI-powered company, we are constantly developing complex workflows that include continuous incoming data, processing these data, and analyzing this data in a timely fashion. In this respect, we have made use of the AWS Step Functions to develop and manage such complex workflows. In the next section, I will be discussing more AWS Step Functions.
AWS Step Functions
AWS Step Functions is a service from Amazon Web Services that makes it easier to orchestrate multiple AWS services to fulfill tasks. Step Functions help to create steps in a flow where the output of one step becomes the input for another degree. The best part is all using a visual workflow editor.
Step Functions gives quite a bit of favorable functionality, such as automatic retry handling, triggering, and tracking for each workflow step, and ensures whether the order of steps is correct. Although that list might at first seem unexceptional, it turns out it’s not at all insignificant to ensure all these things happen correctly in workflows that contain dozens of steps and hundreds of parallel executions. Step Functions take on a lot of the tasks that had to have previously happened in the application.
How do AWS Step Functions work?
Behind the scene, Step Functions is a state machine, and its primary abstractions are called states. A Step Functions configuration constitutes a map of all possible steps and the transitions between them.
Amazon States Language is based on JSON is proprietary to Amazon and is used to define the states and their transitions.
Why are AWS Step Functions integral to the Serverless ecosystem?
When building Serverless applications, we have our growing workloads and developers usually want to scale the application according to it, while we also need multiple teams to work simultaneously on different parts of the application at low cost. In the Serverless model, we have one of the best practices to accomplish these goals is by separating the business logic of an application into a set of decoupled services. A large Serverless application can consist of tens or hundreds of Serverless services.
The challenge comes when many services all need access to various parts of a shared state. To manage these services efficiently, the teams must also be able to organize the flow of data through all application services in a single place, and this is exactly what AWS Step Functions handles. Step Functions have become an essential part of the Serverless ecosystem because of all the state and data management needed to keep Serverless systems working effectively at scale.
How do AWS Step Functions integrate with other AWS services?
Step Functions can be used for task execution — with synchronous or asynchronous execution models (wait vs. callback) using the following AWS services:
- Invoking Lambda functions
- Running AWS Batch jobs
- Running a task in Amazon Elastic Container Service (ECS)
Step Functions can also be used to insert or fetch an item from your databases such as Amazon DynamoDB. They can be also used for messages and notifications e.g. by publishing a topic in Amazon SNS or sending a message in Amazon SQS.
How do AWS Step Functions work with the Serverless Framework?
We can do it in two ways:
- First, by using the Serverless Step Functions plugin, you can include Amazon State Language for Step Functions in your serverless.yml file and deploy it using the Serverless Framework.
- Second, we can use the Serverless Framework to create the AWS Lambda Functions-based services that Step Functions will orchestrate.
Benefits of using AWS Step Functions
- Quickly create complex sequences of tasks
- Manage state between executions of various stateless functions
- Decouple application workflow logic from business logic
- More efficient workflows with parallel executions
Fasal’s use case on developing Serverless Framework using AWS Step Functions
At Fasal, we use satellite data for crop health assessment. The satellite data pipeline is a complex workflow and includes the following key processes:
Step 1- Getting polygon coordinates of the field
Step 2 — Pre Processing the shape of the plot using boundary coordinates.
Step 3 — Data Capturing (Sentinel 2A Data for processing)
Step 4 — Data Processing & Generating NDVI, SAVI, SIPI, EVI images
Step 5 — Clipping of shape-file of geotagged fields
Step 6 — Generating Colour coded Image with improved image quality.
To perform this operation we make use of the following methodology.
As mentioned earlier, we can use the Serverless Framework to create the AWS Lambda Functions based services that Step Functions will orchestrate. We then use CircleCI to automate the complete flow.
These are the jobs running in parallel in CircleCI where the three lambdas we need are configured using the Step Functions and they run in parallel. After the lambdas are launched the code of the Serverless Framework for configuring the Step Functions using the Lambda Functions is launched our entire workflow is ready.
I hope this article helped you to see how powerful and seamless deploying with AWS Serverless Framework is. AWS Serverless Framework is evolving very rapidly and its adoption is also growing exponentially. There are many tools and services already present making the adaptation of this technology easy.
