Vaccine Slot Notifier using AWS & Python
Learn how to build your own notification system using few simple steps!
India announced that from May 1, 2021, it will start vaccination of people in the age group 18–44 years. Earlier, it was only for all 45+ years old. Also, it is mandatory to be registered and booked for an appointment on the COWIN portal prior to coming for vaccination. India has a population of 594.6 million for the age group 18–44 years and only a single portal to register for vaccination. So definitely, this called for the techies to use their skills and build notification systems to save everyone from the hassle. The tweet below pretty much sums it up 😜
There are a plethora of notification systems out there and they are quite helpful as well. I wanted to create my own version and just thought of documenting it as the same process can be replicated across many other things, not only COVID vaccinations.
Coming straight to the point, the following are the steps I used to come up with my own notification system. Will elaborate on each step later on.
- Locate the source of data.
- Fetch and Manipulate data.
- Create a pipeline that implements step 2 indefinitely and also sends notifications.
STEP 1: LOCATE THE SOURCE OF DATA -
- The best thing is that public APIs are already available so you can straightaway check out this link and fetch the data as per your needs.
- A more generic way is to use developer tools and find the API through the Network tab. Better to filter by XHR requests in case there are a lot of requests.
STEP 2: FETCH AND MANIPULATE DATA -
Now that you have got the source figured out, you just need to create a script that will fetch the data from the source and manipulate it into your preferred format. For scripting purposes, I have used Python and my preferred format is a table owing to readability.
Few points regarding the code:-
- Since I’m using AWS lambda, the code is for the lambda function. It’s just that it is encapsulated inside the function lambda_handler.
- In order to import pandas and requests into my lambda function, I used lambda layers. Follow this article to add non-standard python libraries to lambda.
- Lambda runs the code as per the UTC timezone. I forgot to import pytz and also it isn’t a standard library. So, that's why manually changing the timezone in line 13.
- I have hardcoded the districts in line 14 because I and my friends wanted updates for Delhi NCR only. So, the logic can be implemented as per your needs.
- As I built the service for my friends and myself, I added the recipients as an environment variable. You can read them from your DB or from S3 as per your needs.
- send_email function is in utils.py. I have taken this amazing article as a reference to create this function.
STEP 3: CREATE A PIPELINE THAT IMPLEMENTS STEP 2 INDEFINITELY AND ALSO SENDS NOTIFICATIONS -
Above is the simple AWS architecture used for the vaccine notifier. The schedule expression enables EventBridge to trigger the lambda function in step 2 after every 2 minutes. The frequency totally depends on your needs.
AWS provides 2 notification services: SNS & SES. SNS enables you to send messages or emails in a simple text format. SES enabled you to send formatted emails or text messages using HTML. You can even send emails with attachments. Since Pandas provides a function df.to_html() to convert dataframe into HTML, it motivated me to use SES.
- SES is a pretty easy to use service. You just need to go to SES in your AWS console and add the email/identities there. As soon as you create an identity, a verification mail is sent to the same and on verification, a verified with a green tick is added as the status of the identity.
Please note: SES mails can only be sent to verified mails in sandbox. Also, the sender needs to be verified irrespective of the access(sandbox/production).
- As I was using it personally, I added the emails manually. To automate the whole process you can simply use boto3.
- AWS initially provides sandbox access to SES and there is a cap on the number of emails(200/day) that you can send. But, you can request production access with a valid reason and AWS would be happy to grant one. Also, once you have production access, you can make changes to the verification mail template as well.
