Step Functions Overview: A beginner’s viewpoint

Recently, I had an opportunity to work on AWS Step Functions. Learning Step Functions was quite interesting because of its visual features. It also sparked me to think about how our serverless journey can be enhanced by moving to step functions.

In this article, I would like to discuss the benefits of using Step Functions, my experience and challenges while working on it and how we can benefit by implementing it.

Step Functions:

AWS Step Functions is a serverless orchestrator that developers can use to build distributed applications and business processes by integrating multiple AWS services.

We can configure our Step Functions workflow to call over 200 AWS services and run a series of event-driven workflows that maintain the application state.

The use case I worked on was to create the functionality of a REST endpoint to return an Id for a created Request. The design architecture was to generate a new request Id and transform the data as per the expected response. To achieve this, we would need to interact with a couple of AWS services. Given, that it was not overly complex, it seemed like a good candidate for a pilot step function design. We came up with the architecture, created components and wired them up with step functions. Although, we had struggles finding documentation in AWS documents or other blog posts on the Internet for integrating API Gateway with Step Functions directly.

We spent some time reading, to understand the concepts. The step function took care of the overheads of integrating the components. Once everything was wired up, the transformation of JSON/data flowing through was easy, by using flows like “Map” and “Pass” states.

Here are some of the pros and cons, which I understood while working on the task:

Pros:

1. Visual nature: Step Functions enables us to create complex workflows by simply dragging and dropping AWS services on the AWS console. You can also copy the generated snippet of this workflow and paste it into terraform files which then could be used to deploy in different application environments.

Another interesting feature is the “Workflow Simulator”. In Step functions, individual states receive JSON as input and usually pass JSON as output to the next state.

The simulator helps to understand what information is flowing through the different states, so we can filter or manipulate the data to get the expected result much faster.

2. Service Integration: Using AWS Step Functions Service Integrations, you can configure your Step Functions workflow to call over 200 AWS services. Minimal configuration is required to sort IAM roles and permission needed to access one service from another.

3. In-built Error Handling: State machine automatically handles errors and exceptions with built-in try/catch and retry-logic with exponential backoff.

4. Pay per use: Billing is metered by state transition or by number of executions, and you do not pay for idle time, regardless of how long each state persists

Cons:

1. Choosing between Standard and Express workflow: We can select the type of state machine we want to create. One may choose any depending on their use case. Although Express workflow is much cheaper in comparison to standard and its supported execution start rate is greater than standard, it comes with a visual disadvantage in monitoring and debugging the workflow.

2. Learning Curve: Step functions are fairly new and there are not many blogs on service integration of many AWS services. Also, most of the documentation that you might find on Step Functions is usually done on the AWS console. Hence, if you are trying to configure a workflow via terraform, it could be a bit challenging.

Conclusion:

This is just an overview of my experience working on Step functions. The goal of step functions is to build complex workflows or processes. This helps decouple our workflow logic from business logic and decreases application complexity. The developers can concentrate on building and designing the architecture by putting minimal effort into sorting permissions, etc. However, I would highly recommend having a good understanding of the IAM policies and other configurations associated with the service.

Also, w.r.t step functions, understand if it will benefit your serverless journey in terms of the use case, learning curve, pricing, etc before introducing a new service to your application.

Further reading:

· https://aws.amazon.com/step-functions/?step-functions.sort-by=item.additionalFields.postDateTime&step-functions.sort-order=desc

· https://docs.aws.amazon.com/step-functions/latest/dg/welcome.html

· https://docs.aws.amazon.com/step-functions/latest/dg/troubleshooting.html