Queueing: Optimizing for critical path

One of the biggest performance killers happen to be the non-essential actions that are being executed on a critical path of your application. Let’s examine a simple registration example:

router.post('/register', function(req, res){
    var user = new User(req.body);
    user.save( function(err) {
        if (!err) {
            user.savePreferences(req.pref);
            user.sendWelcomeEmail();
            res.render(‘first_time_welcome',
              { title: 'Welcome' + user.name, errors: [err] }
            );
        } 
    });
});

The code looks pretty straight forward. User submits the registration form, the code saves used details and preferences, sends out a welcome email and shows the user a “welcome” page.

Optimizing for user experience — you want to display the confirmation page as soon as possible, so the user can start shopping/posting/doing whatever is most beneficial to your business. As such, you want to make sure that the registration process is fast and painless. So let’s time it and make sure that it is. On average, the code above returned the desired “welcome” page in 2.7 seconds, with occasional spikes taking up to 10 seconds. Not exactly within acceptable performance range, especially since the code is pretty slim, doing two quick database inserts (for user.save and user.savePreferences) and is not doing any heavy logical computation.

Adding timing to individual actions, it appears that the function that takes up the most cycles is user.sendWelcomeEmail.

User.sendWelcomeEmail = function(callback){
  if (!do_not_email(user.email) {
    var email = new Email();
    email.subject = "Welcome " + user.firstname;
    email.cta = getDealoftheDay();
    email.secondary = getOffers(user);
    email.generateBody(user, function() {
      email.send();
    });
  }
};

Bingo! user.sendWelcomeEmail retrieves information from the database, does real-time targeting, and invokes to be sent out immediately. There are likely some optimizations that can be done to the code to speed up the process (with caching, pre-processing and other techniques), but the real question that any developer should be asking themselves is — why are we sending a welcome email during the registration process?

Does it have to be done now?

To answer this question you have to understand what the word “now” means to your business. Should the welcome email be sent to a new user upon registration? Yes. Should the welcome email be sent before user is shown a “welcome” page? Probably not.

Anything not pertinent to the immediate business outcome should not be in the critical path

The immediate goal is to take user into the site . Welcome email, while maybe be just as valuable from the business return perspective, is a secondary goal based on timing. Which means user.sendWelcomeEmail function does not belong in the registration process. And this applies to any other functionality that does not assist in accomplishing the primary goal. Anything not pertinent to the immediate business outcome should not be in the critical path. Which means that all the other processes need to be moved outside of the critical path.

Enter queuing

Queueing is also a good way to queue up a number of actions that can be processed at a pace slower than realtime user experience. The application would add an entry to your queue and external job would pick up those entries in the order of appearance (FIFO).

The are a number of ways (and tools) to implement a queue, but most commonly used ones are MQ-specific tools and databases.

Message Queue Tools

There are tools available to use as a basis for your queue. If you’re reliant on Open Source tools, RabbitMQ is probably the most popular and easy to set up tool. If you’re heavily invested into Amazon ecosystem, SQS may be something to look into.

Database

You can also implement your own queue using any database. Traditionally, you would create a queue table and have a cronjob pull the new data every so often (based on your latency requirements) and process it. Additionally, some of the modern databases implement queueing functionality, allowing for jobs to subscribe to your queue table and wait for new entries, instead of traditional periodic pulls.

The right way

Going back to the registration code, it would require a minor change to the flow.

router.post('/register', function(req, res){
    var user = new User(req.body);
    user.save( function(err) {
        if (!err) {
            user.savePreferences(req.pref);
            sendToQueue(‘welcome email’,user);
            res.render(‘first_time_welcome',
              { title: 'Welcome' + user.name, errors: [err] }
            );
        } 
    });
});

Additionally, your cronjob just need to iterate through the new entries in queue and call the same user.sendWelcomeEmail function that was previously in the critical path of our registration process.

while (var user = readFromQueue('welcome email')) {
   user.sendWelcomeEmail();
}

Now, the welcome emails goes out within 1 to 5 minutes of user registration, which is an acceptable latency, given that the time to “welcome” page has been cut down to 300 milliseconds.