SQS as an event source to trigger Lambda
Putting an end to Polling
AWS Lambda is event-driven compute. What this means is that your Lambda functions only run when triggered and you pay only for the compute time you use. Up until today, the AWS event sources that could trigger Lambda were the following:
- Amazon S3
- Amazon DynamoDB
- Amazon Kinesis Data Streams
- Amazon Simple Notification Service
- Amazon Simple Email Service
- Amazon Cognito
- AWS CloudFormation
- Amazon CloudWatch Logs
- Amazon CloudWatch Events
- AWS CodeCommit
- Scheduled Events (powered by Amazon CloudWatch Events)
- AWS Config
- Amazon Alexa
- Amazon Lex
- Amazon API Gateway
- AWS IoT Button
- Amazon CloudFront
- Amazon Kinesis Data Firehose
- Other Event Sources: Invoking a Lambda Function On Demand
Absent from this list was AWS Simple Queue Service (SQS).
At the AWS Summit San Francisco, SQS as a built-in Lambda event source was announced…
..and then one week ago…
As of today, this functionality is live!
Why does this matter?
Prior to the availability of SQS as a built-in Lambda event source, an inefficient work-around solution was to poll until an SQS message was received and ready for processing.
The Polling Hack
CloudWatch Events, an existing event source, can periodically trigger a Lambda function (via a cron schedule) and wake up to check if a new message resides in SQS and act upon it.
How To Stop Polling
Leave the polls to the politicians and cross-country skiers.
As of this writing, you can now perform the following steps to have SQS serve as an event source to trigger Lambda:
- Create an IAM Role — you can reuse a managed policy like AWSLambdaSQSQueueExecuionRole.
2. Create a Lambda function (this example uses Node.js) using the aforementioned IAM role. Add an SQS trigger to the function and add a console log message (“SQS Trigger Fired!”) to the function code.
3. Create a new Queue.
4. Configure Trigger for Lambda Function.
5. Configure Incoming Messages to Trigger a Lambda Function.
6. Save and see your configuration under the Lambda Triggers tab.
7. Test your configuration by sending a message to your Queue.
8. Open CloudWatch and navigate to your Lambda function’s log stream.
9. Open the Log Stream and confirm that the console log message (“SQS Trigger Fired!”) has made it into the log.
10. Rinse and repeat with your processes.
This new addition to your AWS Solution Architecture toolkit is much more efficient than polling and is serverless.
Now you know…and knowing is half the battle. The other half is applying this event-driven pattern to your architecture.