Anyone who’s written an appreciable amount of code in JavaScript has encountered callbacks. This is particularly true on the server with Node and IO.js. While callbacks have the benefit of being simple and easy to understand, they unfortunately also lend themselves to a number of issues. In this post I’ll be discussing some solutions to common issues with callbacks: Deeply nested callback stacks, inconsistent callback behavior, callback dependent code fragility, and misbehaving callbacks (ones that return multiple times).
Deeply nested callback stacks
Sometimes you need the value of a callback to call other functions that also take callbacks. This results in “nesting” callbacks. Imagine you need to stat a file to get it’s size before reading it (lets pretend that there’s not a statSync method). In order to do this you might write something like this:
var fs = require(fs);
function statRead (filename, maxRead, dataCallback) {
fs.stat(filename, function (err, stats) {
if (err) {
return dataCallback(err);
}
if (stats.size >= maxRead) {
return dataCallback('file too large');
}
fs.readFile(filename, function (err, data) {
return dataCallback(err, data);
})
});
}Notice here that we’re already 2 “indents” in by the time we’re ready to return data. It’s not hard to imagine what happens if you have lots of dependent operations. This is commonly referred to as “callback hell”.
There are several libraries designed to patch this problem. Most are similar to the very popular async.js. Async allows you to “chain” responses together so that the response from the last function becomes the input to the next function. This flattens the callbacks into an array instead of an ‘arrow’. Lots and lots of blog posts and tutorials speak very thoroughly about async.js so I won’t repeat that here. Instead I’ll solve the problem another way: with a non-callback based framework called FRHTTP.
FRHTTP is a node framework available on npm designed to solve these types of issues using functional programming. In FRHTTP the code looks like this:
route
.when({
name: 'stat file',
params: ['filename'],
produces: ['stats'],
fn: function (producer, input) {
producer.fromNodeCallback(['stats'], -1, fs.stat, null, input.filename);
}
})
.when({
name: 'read file',
params: ['filename', 'maxSize', 'stats'],
produces: ['data'],
fn: function (producer, input) {
if (input.stats.size > input.maxSize) {
producer.error('file too large');
}
producer.fromNodeCallback(['data'], -1, fs.readFile, null, input.filename);
}While this seems initially more verbose, it’s a fair bit of boiler plate and some just for easy debugging. The key line here is:
producer.fromNodeCallback(['stats'], -1, fs.stat, null, input.filename);Once the callback returns, if successful it puts a value in ‘stats’. The system then looks for something that can run with this value. It finds our next ‘when’ block and runs that. In most situations, this won’t get any deeper than 1 level. In some edge cases it might get 1 level deeper, but you won’t see this grow to 7 or 8 deep.
Inconsistent callback behavior
As a best practice, functions that take callbacks should be consistent about how they run the callback function. They should either always do so synchronously or asyncronously. Unfortunately not everyone is aware of or always follows this advice. Lets examine some more code.
function oops() {
var cancelMe = undefined; setTimeout(function () {
if (cancelMe) {
cancelMe();
console.log('timeout error.');
}
}, 5000); cancelMe = misbehaving(function () {
cancelMe = undefined;
});
}
The snippet above sets a timeout to cancel the call to a misbehaving function if it takes more than 5 seconds. Unfortunately, sometimes misbehaving calls the callback asynchronously and our code works, and other times it calls the callback synchronously and our code incorrectly reports that the function timed out. It may even crash because we’re calling cancelMe on a completed operation.
FRHTTP again comes to the rescue here with ‘fromNodeCallback’. Since all fromNodeCallback can do is produce a value which then goes back into our framework, it removes this inconsistency and forces the function to always appear to behave asynchronously.
Callback dependent code fragility
In our first code example we showed a callback dependent on another callback. This is often the case, especially in web services where you may need 4 or 5 bits of interdependent data before you can produce a final result. In a callback system this leads to carefully crafted list of callbacks, some of which need to pass a bit of data they receive from the last callback on to the next without doing anything with it. Unfortunately this leads to very fragile code.
Lets imagine a situation where Alice wrote an Express route which pulled data from several Mongo documents to generate a report for end of month sales. The end product consists of 7 callbacks chained together. 3 months later, Bob, a new hire, is asked to add a few fields to the report. He needs values from callback 2, so he sticks a new callback in between 2 and 3, but forgets to forward a field (which can legitimately be null on occasion) needed by callback 5. The route is now broken but may not be broken in an obvious enough manner to detect immediately.
Resolving this type of problem with any async type library is difficult, but FRHTTP is perfect for this type of situation. In FRHTTP, Alice would have written 7 ‘when’ blocks. When Bob came along he could just add one anywhere in the list. The system will ensure that each function gets all the parameters it needs.
Misbehaving callbacks
Sometimes functions have issues that result in them calling the callback more than once (excluding functions that are designed to call the callback multiple times. In that situation you should really use events, but that’s a topic for another blog post). Again, let’s look at some code.
function boom(req, res) {
mightCallbackTwice(function (err, data) {
if (err) {
res.write(err);
res.end();
}
else {
res.write(data);
res.end();
}
});
}If the callback gets called a second time, res.write will get called on a closed connection, resulting in a crash. This crash will be very difficult to find in production as there won’t be an obvious reason that the connection is closed.
This again is a situation where FRHTTP can be of service. The `fromNodeCallback` function ends the producer after the first response. Any further calls will call the producers value method, but since the producer was ended, no further values are actually produced.
Check it out
You can find FRHTTP on GitHub. There’s pretty detailed documentation on using it by itself as well as integrating it with an Express or other Connect style framework app. I’ve also written a tutorial series on it which you can find here on Medium (part 1, part 2, and part 3) as well as a tutorial on integrating with ExpressJS.
About me
I’m passionate about highly performant, highly scalable software. You can find me on Twitter, GitHub, LinkedIn, or follow me on Medium. I’m always looking for interesting problems to solve, if you have one and need help, feel free to reach out.
