Leveraging AWS Step Functions to Call Third-Party APIs Directly with AWS CDK

Serverless architectures have revolutionised the way we build applications, allowing us to focus on writing code without worrying about managing infrastructure. AWS Step Functions, a serverless orchestrator, simplifies workflow management by coordinating multiple AWS services into serverless workflows. Previously, invoking third-party APIs from Step Functions required additional layers of abstraction, typically using AWS Lambda functions. However, with the introduction of the HttpInvoke step in Step Functions, this process has become even more seamless and efficient. In this blog post, we’ll explore how to leverage this new feature using AWS CDK and TypeScript, and discuss the benefits it brings to serverless architectures.

Getting Started with AWS Step Functions and HttpInvoke

AWS Step Functions allow you to build scalable, distributed applications by defining state machines that orchestrate the execution of various AWS services and custom code. Until recently, invoking external APIs involved creating and managing Lambda functions to handle the HTTP requests. However, the HttpInvoke step simplifies this process by allowing Step Functions to directly call third-party APIs without the need for intermediary Lambda functions.

Credits: AWS

Using AWS CDK and TypeScript to Implement HttpInvoke

AWS CDK (Cloud Development Kit) provides a high-level construct library that allows developers to define infrastructure as code using familiar programming languages such as TypeScript. Let’s see how we can leverage CDK to implement the HttpInvoke step in Step Functions.

1. Define the Step Function: First, we define our Step Function state machine using the AWS Step Functions module provided by CDK. This state machine will include the HttpInvoke step, which will invoke the third-party API directly.

import * as stepfunctions from '@aws-cdk/aws-stepfunctions';
import * as tasks from '@aws-cdk/aws-stepfunctions-tasks';

// Eventbridge connection
const connection = new Connection(this, "InvokeTealiumAPIConnection", {
  authorization: Authorization.basic(
    "testingHttpInvoke",
    SecretValue.unsafePlainText(`testingHttpInvoke`)
  ),
});

// HttpInvoke Task
const httpInvokeStep = new tasks.HttpInvoke(this, 'httpInvokeStep', {
  url: 'https://api.example.com', // URL of the third-party API
  method: 'GET', // HTTP method (GET, POST, etc.)
  headers: stepfunctions.TaskInput.fromObject({ "Content-Type": "application/json" }), // Headers 
  requestBody: stepfunctions.TaskInput.fromObject({}), // Optional request body
  connection
});

// Define state machine
const stateMachine = new stepfunctions.StateMachine(this, 'MyStateMachine', {
  definition: httpInvokeStep, // You can customise based on your requirement
  timeout: cdk.Duration.minutes(5), // Optional timeout for the state machine
});

In this example, we create an HttpInvoke task that calls a third-party API using the provided URL and method. We can also specify a request body and headers if needed.

2. Error Handling with Dead Letter Queue (DLQ):

To handle failures during API invocation, we can implement a Dead Letter Queue (DLQ) using AWS CDK. This ensures that any failed executions are captured and can be analysed later for troubleshooting.

const dlq = new sqs.Queue(this, 'MyDLQ', {
  visibilityTimeout: cdk.Duration.minutes(5), // Visibility timeout for messages in the DLQ
});

httpInvokeStep.addCatch(new stepfunctions.SqsSendMessage(dlq));

Here, we create an SQS queue to serve as the DLQ and configure the HttpInvoke step to send failed executions to this queue using the addCatch method.

Benefits of Using HttpInvoke in Step Functions

• Simplified Architecture: By eliminating the need for intermediary Lambda functions, the HttpInvoke step simplifies the architecture of serverless workflows, reducing complexity and cost.
• Improved Performance: Directly invoking third-party APIs from Step Functions reduces latency by eliminating the overhead of additional function invocations.
• Enhanced Error Handling: Integrating DLQs with HttpInvoke steps allows for better error handling and troubleshooting, improving the reliability of serverless workflows.

Conclusion

The introduction of the HttpInvoke step in AWS Step Functions streamlines the process of invoking third-party APIs from serverless workflows. By leveraging AWS CDK and TypeScript, developers can easily implement this feature and benefit from its simplicity, performance, and enhanced error handling capabilities. As serverless architectures continue to evolve, features like HttpInvoke empower developers to build scalable and resilient applications with ease.