Serverless queue based notification service (FaaS)
(Touching AWS SQS, Lambda and Python…)
The Monolith application that we were using is creeping with bigger cronjobs at a certain point of each day and it felt like the server is craving for more resources and it’s sitting a lot idle during the rest of the day!
We could scale our whole infrastructure, right? Yes, sure! That’s what I thought at first, why can’t we just level up our resources and call it a day? The problem lies there is, we don’t know when it will cave in again and it feels wrong that a certain part of our Architecture is growing bigger but we will just be trying to fit our whole infrastructure.
By now, we all heard of a buzzword in our industry “Serverless”. I was as always late to the party so when I came to know about it, I was like whaaa…t! Is something really running without Memory and CPU cycles, what, how? Ahh, I was very much wrong in there! According to Mike Roberts Serverless is something like this,
Serverless architectures are application designs that incorporate third-party “Backend as a Service” (BaaS) services, and/or that include custom code run in managed, ephemeral containers on a “Functions as a Service” (FaaS) platform.
I would suggest reading this Blog of Mike Roberts from https://martinfowler.com if you are having the same expression as mine.
Parts that were growing bigger…
There are services that run as cronjobs from our core application, this is sending messages via third-party services to a large number of users of our systems. The user numbers are rising and so sending all the notification to those users via a Celery service from our Django backend seemed as an overgrowing cell in a monolith body!
Let’s learn and grow with Serverless
An (over)simplified architecture of our system is like the image on the top. The Django backend is communicating with the PostgreSQL in RDS and Facebook Messenger. With some Celery workers backend, it processing notifications for users sending those to Messenger API.
Now let’s propose and set a new structure for the cronjobs for sending that notification via a new Serverless Architecture. Let’s see what we need there and for what purposes,
- Functions, to perform the required tasks we need some functions as a service(FaaS). So comes in the AWS Lambdas.
- We need a queue system to store the users we will try to notify so we can use AWS SQS.
- We wanted to store some of the dynamic contents like database hooks and some secure keys to access messenger APIs. We realized that environment-variables & secret-keys stored from the AWS management console on Lambdas get destroyed on each lambda deployments! For now, we chose DynamoDB to store those.
- We also want some of the tasks to automatically trigger during some point in time. For scheduling events from Lambdas, we’ve used Scheduled CloudWatch Events.
Based on these AWS resources let’s propose a barebone architecture for a “Serverless queue based notification service” as the following,
Tools used Implementing proposed architecture
- Programming Language: Python 3.71, supported for the longest period of time in AWS Lambdas.
- Micro-framework: Chalice, it’s simple yet stable and just fits our needs here.
Notes: The deployment, local development+testing and library dependency supports are just awesome with Chalice! If you collaborate don’t forget to add the file .chalice/deployed/dev.json, it exposes the AWS ARNs and APIs on the codebase but without it, other developers won’t be able to deploy on the same stack! Issue: https://github.com/serverless/serverless/issues/4471 - Mostly used libraries: boto3, psycopg2
We are still in the development and R&D phase with this but all this seems to be working on our staging servers for now.
TODO:
- Improve and simplify the serverless architecture
- Check how large concurrent operations fit in our SQS stacks and Lambdas
- More studies and cost estimations required on AWS resources SNS, SQS and CloudWatch Events.
This blog is open for any related discussions and learnings, Kudos everyone!
