SQS as an event source to trigger Lambda

Published in

Slalom Technology

5 min readJun 29, 2018

Putting an end to Polling

AWS Lambda is event-driven compute. What this means is that your Lambda functions only run when triggered and you pay only for the compute time you use. Up until today, the AWS event sources that could trigger Lambda were the following:

Absent from this list was AWS Simple Queue Service (SQS).

At the AWS Summit San Francisco, SQS as a built-in Lambda event source was announced…

..and then one week ago…

As of today, this functionality is live!

Why does this matter?

Prior to the availability of SQS as a built-in Lambda event source, an inefficient work-around solution was to poll until an SQS message was received and ready for processing.

The Polling Hack

CloudWatch Events, an existing event source, can periodically trigger a Lambda function (via a cron schedule) and wake up to check if a new message resides in SQS and act upon it.

How To Stop Polling

Leave the polls to the politicians and cross-country skiers.

As of this writing, you can now perform the following steps to have SQS serve as an event source to trigger Lambda:

Create an IAM Role — you can reuse a managed policy like AWSLambdaSQSQueueExecuionRole.

2. Create a Lambda function (this example uses Node.js) using the aforementioned IAM role. Add an SQS trigger to the function and add a console log message (“SQS Trigger Fired!”) to the function code.

3. Create a new Queue.

4. Configure Trigger for Lambda Function.

5. Configure Incoming Messages to Trigger a Lambda Function.

6. Save and see your configuration under the Lambda Triggers tab.

7. Test your configuration by sending a message to your Queue.

8. Open CloudWatch and navigate to your Lambda function’s log stream.

9. Open the Log Stream and confirm that the console log message (“SQS Trigger Fired!”) has made it into the log.

10. Rinse and repeat with your processes.

This new addition to your AWS Solution Architecture toolkit is much more efficient than polling and is serverless.

Now you know…and knowing is half the battle. The other half is applying this event-driven pattern to your architecture.

January 2023 Update

AWS Simple Queue Service (SQS) can be used as an event source to trigger AWS Lambda, allowing for the creation of a serverless architecture. When a new message is added to an SQS queue, it can trigger a Lambda function to process the message. This allows for the decoupling of different components of an application, as the message producer and message consumer do not need to be aware of each other.

To set up SQS as an event source for a Lambda function, the SQS queue and the Lambda function must be in the same region. The Lambda function also needs the appropriate permissions to access the SQS queue. Once these requirements are met, the SQS queue can be added as an event source for the Lambda function through the AWS Management Console or the AWS CLI.

Using SQS as an event source for Lambda can be useful in a variety of situations. For example, it can be used to process messages from a messaging system, handle the results of a long-running process, or process data from a stream. SQS also allows for the use of a dead-letter queue, which can be useful for handling messages that fail to be processed by the Lambda function. This feature allows for messages that failed to be processed to be sent to a separate queue for further investigation and/or redelivery.

AWS Console Instructions

In the AWS Management Console, navigate to the SQS service.
Create a new SQS queue by clicking on the “Create Queue” button. Give the queue a unique name, and select the “Standard Queue” option.
Navigate to the Lambda service in the AWS Management Console.
Create a new Lambda function or choose an existing one.
In the Lambda function’s configuration page, navigate to the “Triggers” section.
Click on the “Add trigger” button and select “SQS” from the list of available triggers.
In the “Configure triggers” section, select the SQS queue that you created earlier.
In the “Batch size” field, specify the maximum number of SQS messages that should be processed by the Lambda function at a time.
Click on the “Add” button to create the trigger.
In the Lambda function’s configuration page, navigate to the “Permissions” section.
Click on the “Add Permissions” button, and select the “Simple Queue Service” as the service that will invoke the Lambda function.
Select the SQS queue that you created as the source of the event and assign the proper permissions.
Save the Lambda function configuration and test the function by sending a message to the SQS queue and check the Lambda function’s logs to confirm that the message was processed correctly.

Now the SQS queue is set up as an event source for the Lambda function and any new message added to the queue will trigger the Lambda function to process it.