Transmissions, for Django

This is a post about one of the most common components I have had to rewrite for all consumer apps and websites I have worked on. It is very likely that your product calls for users to be notified.

I built Transmissions for the 3rd time when joining MakeSpace. First time was in PHP at Tripl, second time was less clean, in Python at Lifesum to send millions of notifications every month, and finally in a much cleaner Pythonic form this year thanks to my colleagues, especially Zach Smith who is also maintaining the package nowadays.

Sending a user notification

So you need to email, sms, or push a notification to a user. The rule is in your code and there may be a couple of ways you have to reach out to the given user. These rules and codes are everywhere in your code base, and hopefully hitting just a couple of base classes handling emails and other channel. But you sometime realize that the rule is triggered right after a user event and the email is not supposed to be sent before 3 days in the future. If your product has users and doesn’t have this requirement, you are probably not communicating too much with your users.

For example, Send an update to the customer in 24h after one of their friends accepts their invitation to the service.

Triggering notifications for the future

There are two common ways to look at notifications:

• Trigger notifications when the conditions are met
• Regularly check for conditions (with a cron job) and send right away

It is very common to build a system of the second kind with a “Notification sub-system” that knows all the triggers. The problem I have with it is that you disconnect the logic of notifications from your key systems of billing, social, bookings, gamification, etc. and regroup them all in a way that doesn’t make much sense.

Ideally (IMHO), each component triggers notifications ahead of time, or for immediate delivery, and associate with them some rules. Instead of looking every 5min for which invitation confirmation emails to send from 24h ago, trigger that email right away and add a condition in the notification to cancel itself if something changed. This is how Transmissions works.

Channels for Email, SMS, Push, Slack, etc.

In early implementations, I thought this component was all about channels and creating a domain language for SMS and Email to work the same way. Fast-forward a few years now working in Python, and we all know it takes 3 lines of code for any channel to be setup.

Transmission doesn’t really come with any Slack, SMS, Push integrations yet, because that’s the part we can all import or build differently.

Show me some code!

Transmissions relies on Celery for much of the logic but also for the code structure. In celery, you define tasks (methods or classes), and call them (delay) anywhere in your code.

Define your notification

A notification is a class that you’ll trigger in the future (or immediately). So it is up to you to have it extend your favorite channel.

Trigger from anywhere in your code

Anywhere in your code, you can now just schedule to send this notification with a line of code.

List all the user notifications

If you are building a notification menu in your app, it will be useful to keep track of the emails scheduled (for admin users to check), or the ones submitted (for the user herself to see).

Notifications come with a set of properties to make them easy to maintain and faster to load:

• Target user: who should receive or see the notification
• Trigger user: who originated the trigger (user who invited, sent a message, poked, etc.)
• Content: A generic foreign key to any other model in your system to be referenced (a message, a booking, an event, a tweet, …)
• Data: Additional data (list, dictionary) to be associated with the notification. This should be avoided not to get your notification DB to explode when you have millions of entries, so it should not exist anywhere else in you system. Try to use Content as much as you can
• Dates: when it was created, scheduled for and processed at. But also, when it was seen (when the user listed it was part of a notification menu) or consumed (when the user acted on it).

A system you can trust

Notifications as all sent from a single central loop dispatching individual tasks for all notifications. The loop only looks at tasks status and once a minute it checks for all notifications not yet processed.

If your system goes down, the loop will resume notifications when it comes back up.

If a notification fails and throws an exception, the system will mark it as BROKEN to avoid retrying in case it failed to do post-processing but succeeded in sending the email, SMS or other message.

If you have a spike, it’s only up to you to make sure to have enough workers (celery) running, because the loop can trigger thousands of them per second. If you don’t, the system will fill up the queue and the spike should just take a little longer to be processed.

If your channels are slow, it should not try to process the same notification twice in parallel. Each worker will try to acquire a lock on the notification before starting, or give up for later retry. We use Django cache to manage the locking mechanism, which works great on Redis.

If a channel 500s for a few hours, the system will work just as normal and keep retrying. Admittedly, we should add a cache mechanism to slow down retries once we notice many error message.

Rebuilding the Facebook notifications

Admittedly, this may not be Facebook-scale quite yet. Mostly because storing all notifications in one table has its limits, but I honestly wouldn’t change much to the setup.

Definition: Every Facebook action that triggers a notification would have it’s own class with a mixin to select one or multiple channels.

There is no reason not to have a single notification use multiple channels! A channel could also be supporting Daily Summary by recursively triggering a daily notification based on user settings.

Triggers: Any part of the code (Messenger, Pokes, Likes, Comments, …) can trigger the notification right away. Triggers and definitions would most likely live in the same components, but the mixins for channels and detecting user settings would be in a more global base class.

Displaying a badge: The app badge or top bar navigation badge with a counter for unread notifications would be cached with the number of processed notifications for which Seen is null.

Clearing the badge: As soon as a user would open the notification menu, all current notifications (visible or scheduled until now) would be marked as seen (with the current date and time), clearing the list of unseen, and up to the app to clear a cache for that query.

Highlighting notifications: Even though the notification list is cleared, it doesn’t mean that you have clicked on that Clash of the Titans invitation, so it would still be highlighted.

Consuming notification: Two possible ways: When opening a message through any interface, your app can lookup the notification to the current user and with the message set as Content to mark it as Consumed (and Seen). But obviously, you can also mark something as Consumed when a user clicks on the given notification.

Channels: As you can see, most of the logic is independent of the channels. A base model can take care of loading user settings and deciding whether to pass itself to the Push channel, the email channel, etc.

How are you solving this?

I do not remember how I was doing before I was using scheduled tasks (I do actually, it was some ugly cron jobs). But I can really not remember how my code looked like before Transmissions or its previous generations attempted in Python and PHP.

The code is on MakeSpace’s Github and on PyPi, please contribute and comment! And email me if you feel like joining MakeSpace full time since we have yet to open source more components!