Lambda for Asynchronous Message Processing through SQS

Most of the modern application developers are moving towards serverless paradigm which will be one of the key paradigms for enterprise application development in next few years. FaaS is the processing component of the serverless architecture. In this blog post, I’m going to explain how to use FaaS and serverless architecture for a real-world use case. When we are talking about FaaS and serverless there are few leading service providers around these technologies and obviously, AWS is the top of the list. So I’m going to use AWS services for this exercise.

Use case description

ReserveNow.LK hotel booking service provider which manages an application to handle hotel bookings which are purchased by their client hotels. They currently have a requirement to process booking requests asynchronously for further analyze request patterns and failures to enhance their service in the future. Since they don’t want to introduce any delay to booking request handing process due to this requirement, they have decided to push these booking messages to a message queue.

  1. It should detect booking failures and notify failure state to the Ops team via an SMS message

Selected AWS services

  • Since we need to decouple booking handling and asynchronous booking request processing, we need a queueing system in-between. AWS Simple Queue Service (SQS) is the best solution for this. So booking engine will publish messages to the SQS, and our application will poll messages from SQS and do required post-processing for further analysis.
  • Detected anomalies should be persisted to a NoSQL database which obviously forces us to choose DynamoDB for that.
  • For notifying Ops team via an SMS about the booking failures, we can use AWS Simple Notification Service(SNS).

Setting up the resources

Enough of explanations and boring stuff… Let’s go to some fun stuff with the implementation of our application using AWS services.

  • AmazonSQSFullAccess
  • CloudWatchEventsFullAccess
  • AmazonDynamoDBFullAccess
  • AmazonSNSFullAccess

Implementing Lambda Function

Now we have set up all the required resources, the only remaining part is the coding part of the Lambda function which is the most interesting part for a developer. If we go through the logic that we need to implement, it’s as below,

  • Go through each message and check for booking anomalies. Booking anomalies will be detected on two rules, if booking start date is 6 months after the current date or booking period (the difference between booking end date and start date) is greater than 20 days, it will be detected as booking anomaly. See following sample booking request to understand the structure of the message
{
"resellerId": "81023I",
"bookingRef": "SL-booking-NR-1284404",
"bookingReqProcessingState": "Success",
"bookingRequest": {
"startDate": "2018-11-19",
"endDate": "2018-12-02",
"pax": "2",
"city": "PAR",
"grade": "NR"
}
}
  • If there are booking failures (based on the value of the “bookingReqProcessingState” field value), those booking failures should be notified to the users via an SMS message (to be simple, let’s send an SMS message only for one number)
  • Delete all processed messages from the SQS queue (SQS won’t deleted messages automatically once you read the messages. You have to specifically delete messages once you complete the processing for that message)

Create Lambda Deployment Artifact

It’s quite straightforward. Create a folder with name BookingInfoProcessor and go inside that folder and create package.json file with the following content.

{
"name": "BookingInfoProcessor",
"version": "1.0.0",
"dependencies": {}
}

Testing

To test our application, we need to have sample messages in the SQS queue. For that go to SQS services section and find your queue in the queue listing view. From that view, you can publish messages to your queue using Queue Actions drop-down list, by selecting the Send a Message option. From that, you can easily send messages to your queue as you want with required changes to the payload of the above sample message.

Call To Action

  • Clap. Appreciate and let others find this article.
  • Comment. Share your views on this article.
  • Follow me. Manjula Piyumal to receive updates on articles like this.
  • Keep in touch. LinkedIn, Twitter

Software Engineer who loves to work on integration space. I speak Java, Scala, Javascript, English and Sinhala