Promises And FRP Streams In Node: Better Together

As developers began using JavaScript and Node for larger and often more complex projects, the issues with callbacks became apparent, and several alternatives were proposed. In this post I’ll be talking about mixing the power of promises and functional reactive programming streams as paradigms for asynchronous programming.

Promise primer

While there are several excellent promise libraries available for Node, I’ll be talking about ES6 promises. The reason I’m preferring these over others is that over the next 18–24 months I believe that they’ll become the preferred promise on Node.

The basic premise of promises is that instead of taking a callback, a function returns a promise. The returned promise can be in exactly one of three states, pending, fulfilled or rejected. Fulfilled and rejected are “exit states”. Once a promise enters one of these states, no further state transitions are permitted.

As a consumer of the promise, you register a function to be executed when the promise is fulfilled and optionally a function to be executed if the promise is rejected.

app.get('/pendingOrders', function (req, res) {
    loadUserOrders(req.user).then(function (orders) {
        res.send(200, JSON.stringify(orders));
    }).catch(function (error) {
         res.send(500, JSON.stringify(error));
    });
});

One entry isn’t all that interesting (in fact it looks a lot like a callback would), but promises can be chained such that you have a list of actions that happen asynchronously but you can treat as a single group. If an error occurs, you can provide one error handler for the whole group.

More than the simple case

In the scenario in the promise primer, we had to perform a couple of actions:

• Load a user’s orders
• Return those orders to the user

Promises are an excellent choice for these types of linear activities; things that in synchronous I/O languages like Go would just be a single function, and where JavaScript adds seemingly unnecessary complexity to the code.

Now let’s make this scenario a little more complex (and potentially a little more real world):

1. Get a user’s orders from our website
2. Get the user’s orders from our internal sales system
3. Determine which orders where shipped via UPS vs USPS
4. Get the latest tracking details for orders shipped via UPS
5. Get the latest tracking details for orders shipped via USPS
6. Merge all this and send that info back to the caller

In this more advanced scenario steps 1 and 2, as well as 4 and 5, are independent of each other. This is the exact type of scenario where promises by themselves don’t really answer the problem. You can make this work with just promises but it involves lots of the same juggling we are trying to escape by not using callbacks (promise.all can be of some use but not without the risk of introducing fragility and dependency). This also happens to be the exact kind of scenario where FRP streams can come to the rescue, and keep you focused on adding value instead of data reference counting, or worrying about where to position functions so they get called correctly.

As an FRP stream framework we’ll be using FRHTTP. The FRHTTP framework is relatively approachable and is built for Node (and in fact the most common job for Node, serving data).

FRHTTP can work with promises or callbacks, but it’s primary value add in this scenario is allowing you to worry about transforming your data, while letting the system manage moving your data through it’s various states and ensuring that the right functions are called at the right times.

Let’s take a look at how the more complex scenario would look in a combination of FRHTTP and promises (fair warning: so we don’t get bogged down, we’ll use lots of imaginary APIs to get data from various systems):

The first thing to note here are the individual ‘when’ functions. Each function takes the following parameters:

• a name (for debugging and error reporting purposes, you can pass null though it’s better to use a reasonable name)
• what inputs your function requires
• what outputs your function is allowed to produce (don’t confuse this with data types, this is a list of the names of the parameters you produce).
• a function that will transform the inputs into the outputs
• any advanced options. takeMany and triggerOn are used in this example. Setting takeMany to true means that any time any of our inputs are produced, call this function again. Setting values for triggerOn filters what data will cause our function to be re-executed.

Notice that each when function performs a single task, and doesn’t concern itself about where data comes from or what happens to data that the function produces. We also didn’t write any “connecting code”. Let’s take a look at what’s going on.

We begin on line 4 by pulling out the userId from the route, and creating an empty shippedOrders array which we’ll be filling in later. The empty array is necessary because no ‘when’ gets called until all the inputs it requires are available so if we didn’t provide an empty starting point, our order aggregation function (merge) wouldn’t get called.

Lines 9- 14 and 15–20 grab orders from our two separate systems. They don’t care about each other, only about the input they require (the userId).

Lines 21–31 are where things get easy with FRHTTP and a bit harder without. This is where we grab the orders and separate them out into UPS and USPS shipped orders. The takeMany option tells the system that any time anyone produces an ‘orders’, call this function to transform it.

Lines 32–39 and 40–47 get order details from UPS and USPS respectively. Again we use takeMany to do the “next step” on any value we take as input. We don’t care how many times either one of these get called, we’re just concerned with the single step. It’s the system’s job to worry about the rest.

Lines 48–51 merge the orders (now with their shipping info) back into a single set. We again use takeMany to get called any time one of our inputs changes, but in this case we also use triggerOn. Because we need the previous array of all orders, and we produce this array, we don’t want to be notified when it changes (since we’re the ones changing it). The triggerOn option allows us to specify that we only care when a new order with shipping info is available, allowing us to ignore the array.

Lines 52–54 return the response to the user.

While your functions are concerned with the business logic, FRHTTP concerns itself with which functions to call and how often to call them. Once no more functions can be called based on what data has been produced, FRHTTP calls the render function allowing you to respond to the caller.

If in the future we needed to inject some new behavior or additional transform into the route, we can simply attach a new ‘when’ function onto the end of the list of ‘when’ functions. There’s no need to find the right place to inject the behavior, the new block will just be called at the appropriate time by the system based on the data it requires.

As you can see, promises and FRHTTP make a powerful pair that allows you to focus on adding value without having to worry about the plumbing of how to move data around.

Learn more

If you’re interested in learning more about ES6 promises you can read about them here.

If you’re interested in learning more about or using FRHTTP in your next or existing project you can learn more about it on GitHub. You can use FRHTTP either in stand alone mode or as part of an ExpressJS application.

About me

I’m Amir Yasin; a polyglot developer deeply interested in high performance, scalability, software architecture and generally solving hard problems. You can follow me on Medium where I blog about software engineering, follow me on Twitter where I occasionally say interesting things, or check out my contributions to the FOSS community on GitHub.