Subscribe an AWS Lambda function to an SNS Topic in CloudFormation (..and work)
Lots of posts (including mine) on subscribing a Lambda to an SQS that is in turn subscribed to an SNS. But what if you do not need an SQS Queue? Then read this and all your hopes and dreams will come true (~Pedro).
For those that just want to see the CloudFormation template (SAM) here you go. If you want a little explanation then read on.
There are 4 parts to this process:
- Create the SNS Topic. The ExampleTopic above defines the SNS Topic we are going to create. In this topic definition we are also creating the subscription for the Lambda. This is what creates the subscription shown in Figure 3.
- Allow the SNS Topic to call the Lambda Function. ExampleFunctionInvokePermission is the key to allow the SNS Topic to invoke the Lambda Function. If you ever tried this before in a SAM Template or through the console and noticed that you have the subscription configured but your Lambda never gets invoked this is probably why.
- (Kinda sorta optional.) Allow anything in your AWS Account to send an SNS message to this topic. ExampleTopicPolicy allows any resource in your AWS account to send a message to this SNS Topic. This is overly permissive. You should grant publish rights to the resources that need to send messages to this topic. Use this as an example and lock it down by modifying the ARN in line 46.
You can deploy this sample stack to your own AWS account if you wish. In order to do so you need to ensure the following prerequisites are configured:
- Install yarn.
- Install AWS CLI and AWS SAM CLI.
- Configure the AWS CLI with your SECRET_ID and SECRET_KEY.
- In the package.json file modify the sam-package script to include an s3 bucket name for a bucket in your account. You can create a new bucket on the command line with this command: aws s3 mb s3://your-bucket-name
Once all the prerequisites are configured simply run the command:
This will deploy the stack and if you log in to your AWS Console you should eventually see the stack in the CloudFormation console.
Once the stack is deployed you can navigate to the SNS Topic list and you should see your new topic in there.
Clicking on your topic will show the details of your topic.
You can see your subscription was created and it is hooked up to the Lambda we defined in the template.yaml file.
If you click on the Access Policy tab you will see the access policy we defined in our template.yaml file. This policy allows anything in your AWS account to send a message to your SNS Topic. See my notes on this being overly permissive above.
Now let’s test it out. Click on the Publish Message button on the SNS Topic Details page and you will see a form like Figure 5. Type in anything for the subject or leave it blank. In the Message Body section you can also type what you want. When you have crafted your SNS Message body click the Publish Message button on this form.
You sent your first SNS Message to your Lambda Function. Now let’s see the log message.
Navigate back to your CloudFormation console and click on the example-lambda-sns stack. Then click on the Resources Tab. Your screen should look like Figure 1. If you click the link listed under the Physical ID column of the ExampleFunction row it will link directly to the Lambda Function’s configuration screen (Figure 6).
From this screen click the Monitoring tab at the top.
This view shows you some neat metrics about your Lambda Function. If you look at the third graph on the top row you should see 1 execution of your Lambda represented by a green dot. This means the lambda executed and succeeded. Click the View logs in CloudWatch button now and we will link directly over to your CloudWatch log group for this lambda function.
You should have one log group. Click the link (the log stream name) and we will see the actual log message that our lambda output.
If you click the second row in the log it will expand and show you the entire log message. Here you can see that our Lambda was invoked when the SNS Topic received the message.