Executing Parallel Tasks In Step Functions

In this post, we’ll see how we can execute parallel tasks in Step Function.

Our scenario will be to execute a step function when an instance gets into running state, which will first create AMI of the instance and then as parallel tasks, will attach an IAM role and security group to the instance.

Step 1: Create an IAM role for our Lambda functions with AmazonEC2FullAccess and AWSLambdaBasicExecutionRole permission. Also, add following inline policy

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*"
}
]
}

Step 2: Create 3 functions,EC2_AMI,EC2_SG,EC2_IAM, with Python3.7 runtime and 2 minutes timeout.Use the role which we created in Step 1.

Code for EC2_AMI function:

def lambda_handler(event, context):
import boto3
ec2= boto3.client('ec2')
response=ec2.create_image( InstanceId=event['Input']['detail']['instance-id'],Name=event['Input']['detail']['instance-id'],NoReboot=True)
return(event['Input']['detail']['instance-id'])

Code for EC2_IAM function: Specify Instance profile ARN and name in the code.

def lambda_handler(event, context):
import boto3
client = boto3.client('ec2')
response = client.associate_iam_instance_profile(
IamInstanceProfile={
'Arn': '',
'Name': ''
},
InstanceId=event['Input']['Payload']
)

Code for EC2_SG function: Replace sg-xyz with your security group.

def lambda_handler(event, context):
import boto3
client = boto3.client('ec2')
response = client.modify_instance_attribute(InstanceId=event['Input']['Payload'],Groups=['sg-xyz'])

Step 3: Create a Step Function with following definition. Replace <account_id> with your account id.

{
"StartAt": "Create_AMI",
"States": {
"Create_AMI": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:<account_id>:function:EC2_AMI:$LATEST",
"Payload": {
"Input.$": "$"
}
},
"Next": "Add_IAM_Role_And_SG"
},
"Add_IAM_Role_And_SG": {
"Type": "Parallel",
"Next": "Final State",
"Branches": [{
"StartAt": "Add_IAM_Role",
"States": {
"Add_IAM_Role": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:<account_id>:function:EC2_IAM:$LATEST",
"Payload": {
"Input.$": "$"
}
},
"End": true
}
}
},
{
"StartAt": "Add_SG",
"States": {
"Add_SG": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:<account_id>:function:EC2_SG:$LATEST",
"Payload": {
"Input.$": "$"
}
},
"End": true
}
}
}
]
},
"Final State": {
"Type": "Pass",
"End": true
}
}
}

Step 4: Create a CloudWatch event rule with the following Event Pattern.

{
"source": [
"aws.ec2"
],
"detail-type": [
"EC2 Instance State-change Notification"
],
"detail": {
"state": [
"running"
]
}
}

Specify your Step Function as Target and make sure the IAM role you are using for Step Function has the permission to invoke our 3 lambda functions.

That’s all from a setup point of view.

Now try launching an instance and you’ll see AMI gets created for that instance and designated IAM role and security group is also attached to this instance.

This gives us a fair bit of idea about using the Step function and you can use a similar setup for tasks like the text to speech conversion etc.

👋 Join us today !!

️Follow us on LinkedIn, Twitter, Facebook, and Instagram

https://avmconsulting.net/

If this post was helpful, please click the clap 👏 button below a few times to show your support! ⬇

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Vinayak Pandey

Vinayak Pandey

Experienced Cloud Consultant with a knack of automation. Linkedin profile: https://www.linkedin.com/in/vinayakpandeyit/