Automated verification of SNS publishing with ngrok tunnels.

Yesterday I was writing an integration test to verify that an SNS broadcast topic I was leveraging in front of a DynamoDB stream was being pinged every time an item was added to DynamoDB.

“This’ll be easy”, I laughed. I’ll just write something like;

sns.subscribe({topicName: "myTopic"}, subscriptionHandler)

and then verify subscription handler is invoked.

Not so fast!

Alas to my dismay, there is no such way to subscribe to an SNS topic and read messages unlike SQS. Which makes sense once I stopped to think about it. SNS is a pub sub system and does not maintain messages. If there’s no subscribers at the time of the message it’s simply lost.

Which is perfect for my broadcast use case but not so much for my automated testing needs. How to verify this without commissioning infra just for testing 🤔.

What goes in the box?!

As I looked through the list of listener types I began to get dismayed;

email — nope!

sms — hell no!

lambda — nah, I don’t want a lambda just for an integration test, I want my deployed app to mirror prod as much as possible

https — ah maybe we can do something here!

When adding an automated test to our CI/CD pipeline perhaps we can have an http listener process the message and verify. But our pipeline will have a listener on localhost and SNS requires a web address. How can we stitch them together 🤔.

ngrok to the rescue

ngrok is a beautiful service. For free, it provides secure introspectable tunnels to localhost dynamically. It’s as simple as;

Stitching it all together

For the complete Infra BDD test we require

• a local listener to handle SNS forward requests.
• an ngrok tunnel to the listener on a local port
• to subscribe to SNS with the ngrok tunnel URL
• save the SubscriptionARN for later
• add an item to DynamoDB
• verify it broadcasts the message to SNS
• verify the received message is as expected.

Sounds complex but it’s really not too bad, having a local listener store the requests is a few lines of node.

And there we have it. Automated verification of our SNS topic to plug into our CI/CD pipeline.

Why do this?

People who don’t do automated infra as code verification tests often think this is an elaborate waste of time. But we do this for the compounding effect it has. We never need to test this again and we detect a critical vulnerability before it makes it’s way to production.

As part of this test I discovered that my publishing lambda didn’t have permission to publish to SNS. I updated the CDK code to provide, I checked in and the IAM role was updated and test passing.

It’s just a beautiful way to work and saves so much stress of manually testing this stuff.

About Me

An AWS Certified Solutions Architect Professional with a passion for accelerating organisations through Cloud and DevOps best practices.

If you want to work together contact me over on brianfoody.com, on LinkedIn or Twitter for a chat.

And don’t forget to follow the Lambda Lego publication.