Task Scheduling with NodeJS and Redis

Sonny Trujillo Jr.
May 26, 2019 · 5 min read
Image for post
Image for post
Sample push-notification from uSTADIUM app

The Problem

Building an API isn’t that complicated once you understand the basics. We send HTTP requests to the server, it does some work, and then returns the requested data. Simple. But what happens when that request requires work that’s outside of its scope? For example, when I mention a user the system needs to send a push notification to the affected users. Processing the notification during that request’s lifecycle would delay the final response. As our notification system became more complex, it was clear we needed to put more thought into it.

  1. A notification is constructed and inserted into the database.
  2. That notification is mapped to the set of users who will receive it.
  3. We retrieve a list of all devices for the users we need to notify.
  4. We send a push notification to every device they have registered with us.
  5. We update the send status of that notification and remove invalid devices tokens.

The Task Queue

Task Queues manages a list of work that needs to be completed in a separate process. One system adds work to the end of the queue while another pops work items off the top. We need to create a Task object that represents the work described above and then add it to the Task Queue. Before we could begin I needed to ask a few fundamental questions.

1. Where will the Task Queue live?

We were already using Redis as a caching system, so when I began looking for ways to build a queue Redis was the obvious choice. Not only was it well built to handle this pattern, but there are a lot of resources online discussing how it’s built. There are many other options for this, and if you’re using Google App Engine you should look into Google Cloud Task Queue, which offers more built-in features.

2. How do we know when an item has been added to the queue?

I spent a bit of time trying to figure out. I did not want to poll Redis every n milliseconds for new jobs. Two methods showed up on my radar. The first is the Redis Pub/Sub system. For this method, I would have a function that subscribes to a channel and receives messages on it. These messages will alert me to a new Task that is ready to run. The second is to use a simple Redis list as the queue and the blocking list pop primitive, BLPOP, to wait until an item is ready to be removed from a queue.

3. What do we submit to the task queue?

“Well we submit the task, duh…” was what you might be thinking, but queues only support adding strings, so we can’t really push an object. We have to push a key onto the end. I’ll admin this question confused me more than it should, mainly because I wasn’t sure what the “best” method was. Was the key a primary ID into our database or should it be a reference to some object in Redis? Where do we draw the line on the work that has to occur? I decided to send the events primary key ID to the queue and allow the Task to decide how to handle it. For example, if a user upvotes a post, I’ll push the ID of the vote action to the vote_queue, once it’s popped off the queue the service will know how to handle it.

The Set Up

Ok, I’ve set up the problem and answers to some of the questions I had (hopefully those answer some of yours), now let’s take a look at a big picture diagram of how this will work.

Image for post
Image for post
Example setup of the task queue.

The Code Sample

The TaskScheduler.js file is a basic example of how we can add a task to the database and then push it onto the end of a task queue. Once it is pushed onto the queue it will begin processing when the TaskManager starts listening.

Improvements

There are many areas for improvement since the code I’ve given is just a basic example. One question you might need to ask is where will you place your TaskManager. If you add it directly to your server it could overload the system during levels of high usage, but this depends on what type of work your tasks are performing. In our system, we extracted all of this out to a new microservice with a simple API for checking its state.

Overview

The method described isn’t too complex, but it decouples the business logic from the application logic. With this minor change in place, we can begin iterating on the performance of the system and build more robust queues and services. We can also reproduce this method to handle various long-running processes like recommendation systems, text processing, etc.

The Startup

Medium's largest active publication, followed by +683K people. Follow to join our community.

Sign up for Top Stories

By The Startup

A newsletter that delivers The Startup's most popular stories to your inbox once a month. Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Sonny Trujillo Jr.

Written by

Hi, my name is Sonny and I am a Senior Software Engineer at Wayfair.

The Startup

Medium's largest active publication, followed by +683K people. Follow to join our community.

Sonny Trujillo Jr.

Written by

Hi, my name is Sonny and I am a Senior Software Engineer at Wayfair.

The Startup

Medium's largest active publication, followed by +683K people. Follow to join our community.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app