Imagery by Jeremy Booth; special thanks to Eric Baer, Brooklyn Zelenka & Jason Trill for their pervasive helpfulness and knowledge; extra special thanks to John-David Dalton for not only lodash itself but humbly bringing to light some of the inaccuracies herein and contributing tidbits of arcane knowledge about the inner workings of lodash.
Why using `_.chain` is a mistake.
There are two not insignificant problems with any application or library that makes use of _.chain: its most common use promotes importing the entirety of lodash to work, and it’s hard to extend with new methods. It’s worth us understanding why people want to use _.chain in the first place and figuring out how we can use some functional programming techniques to gain all of its benefits less its drawbacks. We’ll be looking at how to get a 2x build-time performance increase and 1.5x bundle size decrease by turning this:
import _ from "lodash";_.chain([1, 2, 3])
.map(x => [x, x*2])
.flatten()
.sort()
.value();
into this:
import map from "lodash/fp/map";
import flatten from "lodash/fp/flatten";
import sortBy from "lodash/fp/sortBy";
import flow from "lodash/fp/flow";flow(
map(x => [x, x*2]),
flatten,
sortBy(x => x)
)([1,2,3]);
🚧 Note: This article makes use of several modern JavaScript language features: imports, fat arrow functions and constant variables. If you haven’t seen them before then check them out for an optimal reading experience!
Prior Art
The lodash library, by John-David Dalton, offers many things JavaScript doesn’t: consistency, performance and a suite of functions for doing common or otherwise error-prone tasks. There are almost 300 functions in lodash, with the package receiving over 20 million downloads per month. It’s a fantastic piece of work.
lodash: A modern JavaScript utility library delivering modularity, performance, & extras.
It turned out to be handy to connect many of these little utility functions together in a way that was more legible (and in some cases performant).
Consider the following operation:
(_.flatten(_.map([1, 2, 3], x => [x, x*2]))).slice().sort();This is arguably more clear as a series of “passes”, visualizing the input object flowing from one function into the next one and out the end:
_.chain([1, 2, 3])
.map(x => [x, x*2])
.flatten()
.sort()
.value();People really, really liked _.chain. And things were good, for a time.
The Problem
But there was trouble in paradise. As time went on, more functions were added to lodash; people realized that maybe they didn’t need all that functionality. People also realized they’d like to add their own functionality: new methods as part of _.chain; but they found that these new methods didn’t always play well with one another.
It Can Get Big
Web applications that include lodash can be big. Today two versions of lodash (v4) exist: the kitchen sink with everything and a more minimal build with less functionality; these packages have a build size (min+gz) of approx. 21 KiB and 4 KiB respectively. Finding the right balance can be hard, and ideally you’d like to just import what you need, since most apps will use only a fraction of the total functionality in lodash. However, because of the way _.chain works, it can be rather ineloquent to import the particular bit you need.
This is because the _.chain function takes, as input, a (typically array-like) object and returns, as output, a chain object whose methods correspond to the current chainable methods in lodash.
_.chain = (array) => wrap(array, _); // Rough concept of chainBecause of this you either have to import lodash wholesale, or extend the global lodash object with the methods you want.
Wholesale import is the most common case, resulting in bringing the entirety of lodash into your build. It’s the most heavy-handed but also the most convenient:
import _ from "lodash"; // Import everything._.chain([1,2,3]).map(x => x+1).value(); // Use the methods.
To avoid the larger build sizes associated with using a global import, you can mix your desired methods into an initially empty global lodash object:
import chain from "lodash/chain";
import value from "lodash/value";
import map from "lodash/map";
import mixin from "lodash/mixin";
import _ from "lodash/wrapperLodash";// Add the methods you want. The object generated by chain() will
// now have these methods.
mixin(_, {map: map, chain: chain, value: value});_.chain([1,2,3]).map(x => x+1).value(); // Use the methods.
This is obviously an extremely convoluted process, having to import the empty wrapper prototype and then bind your methods before calling them. Not to mention, modifying globals is never a good thing. There is a less convoluted mechanism provided by lodash called _.runInContext, but it’s most sadly available only in the monolithic build. So we’re kind of stuck between a rock and a hard place.
It’s Not Nice to Extend
Adding your own processing methods is less than ideal; they must be added or injected into this wrapped chain object. There are a handful of ways that this can be achieved from the downright dirty to the almost acceptable.
Let’s give ourselves a fictitious function we want to use that tests an array of strings and returns those entries that have vowels in them:
import filter from "lodash/filter";const vowels = (array) => filter(array, str => /[aeiou]/i.test(str)
As talked about previously, you can use _.mixin in both monolithic and modular fashions. The monolithic version has a fairly nice implementation going for it:
import _ from "lodash";_.mixin({vowels: vowels});_.chain(['ab','cd','ef']).vowels().value(); // Use the methods.
That wasn’t so bad, but when turning to the modular approach, things start to get particularly messy:
import chain from "lodash/chain";
import value from "lodash/value";
import map from "lodash/map";
import mixin from "lodash/mixin";
import _ from "lodash/wrapperLodash";mixin(_, {
chain: chain,
value: value,
map: map, // Add existing lodash methods you need.
vowels: vowels, // Add your own lodash methods.
});_.chain(['ab','cd','ef']).vowels().value(); // Use the methods.
The cleanest “non-polluting” way to use _.chain is with _.tap or _.thru. While it doesn’t mutate global objects and preserves the flow-like structure, it still requires every function be wrapped in one of these methods. Again, in both monolithic and modular forms:
import _ from "lodash";_.chain(['ab','cd','ef']).thru(vowels).value();
And again the modular version gets fairly obtuse:
import chain from "lodash/chain";
import value from "lodash/value";
import mixin from "lodash/mixin";
import thru from "lodash/thru";
import _ from "lodash/wrapperLodash";// Have one file that only imports core lodash methods you need.
mixin(_, {
chain: chain,
value: value,
thru: thru,
});chain(['ab','cd','ef']).thru(vowels).value();
Neither of the two approaches, (with respect to their modular versions), is particularly attractive. So what can we do about this?
Much Ado About Something
There are some existing solutions to these issues, from the simple “just don’t use chain” to the nuclear “build a whole new library” (trine). Fortunately it’s also possible to solve this problem in a simple, sane manner with the nice functional programming tools that are already available in lodash toolbox.
The Solution
It turns out that _.chain is totally replaceable with another pattern. The two key mechanisms of this approach that lodash provides are: function currying and function composition; both of these tools are relatively commonplace in functional languages, (like Haskell, Elm, OCaml and PureScript), but are incredibly underrepresented in the world of vanilla JavaScript.
Currying
To talk about currying we must first talk about partial application. Partial application means taking a function with n arguments and returning a new function with n-i arguments (given i<n), where the i arguments are bound to the new function. That is saying we are allowed to sequentially “lock-in” every parameter except the last one (otherwise it wouldn’t be partial). In lodash we can do this with _.partial.
const add = (a, b) => a + b;
const add5 = _.partial(add, 5); // now `a` is bound (locked-in) to 5
const add5 = (b) => 5 + b; // this is what it looks like insideadd5(3); // equivalent to invoking add(5, 3);
Function currying is the process of successive partial applications, until the last argument is given at which point the result of the function is returned. A curried version of the add function might be written as:
const add = (a) => (b) => a + b;add(1)(2); // Repeated partial application.
This makes it possible to create new functions that “lock in” values for all arguments except the last one. For example, we could take the curried add function and create a new function that always adds 5:
const add5 = add(5); // Implicit partial application!Composition
Function composition lets you achieve the essential essence of what _.chain does (pass results from one step to the next) with the major caveat that the composition works only on functions. However, this still typically results in significantly greater legibility for long sequences of combined functions. For example, a composed version of:
const add8 = (x) => add5(add3(x));Could be written with more visual clarity using composition:
const add8 = compose(add5, add3);You will see that with these two primitives we can take existing lodash functions and create the same behaviour of _.chain without all of its notable deficiencies.
All Together Now
So remember with currying you partially apply the arguments in order. If we look at the majority of the existing lodash functions they take the array as their first argument:
import "map" from "lodash/map";map([1, 2, 3], (x) => x*2);
This is a problem because we cannot partially apply the mapping function to _.map. We want to be able to make it so that all the functions we generate only take the input array as their only argument:
const double = map((x) => x*2); // We want this!double([1, 2, 3]); // It now operates on just the input array!
Thankfully lodash provides all of their functions in a functionally-compatible, curried, data-last way via the fp package. This package is available with cherry-picking (which we want to keep the build size down) in lodash since v4.1.0.
import map from "lodash/fp/map";map((x) => x*2)([1, 2, 3]); // We have this!
The documentation for this rearg behaviour (and much more) is available as part of the the lodash/fp documentation. So now we have all the tools to remove _.chain from our original:
import _ from "lodash";_.chain([1, 2, 3])
.map(x => [x, x*2])
.flatten()
.sort()
.value();
Combing the individual methods that were previously in the chain above by using compose results in identical functionality:
import map from "lodash/fp/map";
import flatten from "lodash/fp/flatten";
import sortBy from "lodash/fp/sortBy";
import compose from "lodash/fp/compose";compose(
sortBy(x => x),
flatten,
map(x => [x, x*2])
)([1,2,3]);
This essentially allows for an exact 1:1 mapping of all the things you previously used _.chain for. It’s simply just a different syntax.
Caveats
You’ll probably be asking a couple of (perhaps well deserved) questions like: “Why are the arguments reversed?”, “What happened to .value()”? and most importantly “Why is it not working?!”
Argument Ordering
The arguments seem to be reversed because that applies the functions in the computationally correct order. The original code did a map first, then a flatten and then a sort. Remember how compose works and you’ll see that is exactly what’s happening:
compose(
sortBy(x => x),
flatten,
map(x => [x, x*2])
);Is the same as:
sortBy(x => x)(
flatten(
map(x => [x, x*2])
)
)([1, 2, 3]);Where the map is computed first, then passed to flatten and then finally passed to sort. However, because this can be confusing, lodash provides an inverted composition mechanism called flow which allows for ordering in the opposite (arguably more natural) direction:
import map from "lodash/fp/map";
import flatten from "lodash/fp/flatten";
import sortBy from "lodash/fp/sortBy";
import flow from "lodash/fp/flow";flow(
map(x => [x, x*2]),
flatten,
sortBy(x => x)
)([1, 2, 3]);
No Wrapper Object
Since composition works on the actual array, not the lodash chain wrapper object, the .value() call from _.chain is not needed to extract the actual result from the chain object.
As an interesting aside: The lack of monolithic build also means that you lose the performance benefits of lazy evaluation lodash provides. However the lack of _.chain does not.
“In fact our composition with flow/flowRight uses chaining internally (when monolithic) to support shortcut fusion in composed methods”
Given the typically fairly small constants in general front-end lodash usage, the performance difference is likely negligible.
Careful with Currying
Not all things in _.chain will work as a copy-paste job into this new fantasy land of composition. Remember that in order for everything to work, flow or compose must be called with a series functions that pass the correct type expected by each subsequent function (this case is arrays all the way down).
Let’s look at a simple chained sort:
_.chain([1, 6, 2]).sortBy().value();What looks like its corresponding composed version will not work:
flow(sortBy)([1, 6, 2]);Why? Well because sortBy is curried and the type signature looks like:
const sortBy = (sorter = _.identity) => (array) => ...This results in an invocation that returns a function that returns the initial array — hardly the valid sort function sortBy was expecting to get and certainly incompatible with the rest of the functions in the composition chain. It boils down to:
sortBy([1, 6, 2]);But what you really wanted was:
sortBy(/* _.identity */)([1, 6, 2]);Which translates back into the final composed form as:
flow(sortBy())([1, 6, 2]);Since JavaScript is untyped you’ll have to watch out for these kinds of errors yourself. The flow type checker is an appealing option that, with some level of massaging, might be able to help address this particular issue.
The lodash/fp group of functions can behave a little bit differently than their vanilla counterparts. In case you missed it, there is a complete guide to this functional family of lodash available on the wiki page.
Further Exploration
This whole pattern is not new: existing languages have leveraged these same techniques for a long time. JavaScript has also been slowly evolving to start incorporating more and more of its functional brethren’s toolset. There are a lot of great thoughts about this style of programming and other resources to get you started thinking functionally.
For example, a function declaration in Haskell is always curried and can be read as a function taking in one argument and returning a new function that takes the remainder of the arguments:
add :: Num -> Num -> Num
add = x y = x + yadd 2 4
This looks awfully familiar to our curried JavaScript (with the one difference that JavaScript is much more explicit about making these kind of curried function calls):
// Num -> Num -> Num
const add = (a) => (b) => a + b;add(2)(4)
Haskell has, equally, language-integrated support for function composition providing the infix dot operator:
add8 = add5 . add3Which, again, looks awfully familiar but less elegant in JavaScript land:
// add5 . add3
const add8 = compose(add5, add3);Haskell is not the only language to provide first class support for these features. For example, Elixir and bash scripts have pipe operators (the symbols |> and | respectively) as infix equivalents to flow.
Not to be outdone, there is now a language extension proposal to JavaScript to add the |> operator, offering you some very clean looking code indeed:
[1, 2, 3]
|> map(x => [x, x*2])
|> flatten,
|> sortBy(x => x);What does it all mean?
The lodash toolset is not the only framework to suffer from this same kind of anti-pattern either–the library that shall not be named also draws ire from its users over a poor size-to-usage ratio because every method is chained through a global object.
While no panacea, powerful functional tools make solving certain classes of problems elegant and simple once you wrap your head around how all of the dots connect. Composition is a great abstraction in a lot of cases, and certainly offers an extremely clean, compelling alternative to _.chain.
Our simple example in its complete form:
Built with webpack:
The production output bundle size is 1.4x smaller and webpack compilation time is almost 2x faster. Like weight loss, your results will vary depending on framework dietary restrictions. With the advent of tree shaking (unsupported even for lodash-es presently unless you use a babel plugin) and other smarter build tools, real modularization will go a long way to making lean web applications a reality.
Fight the darkness. Make JavaScript great again. Stop using _.chain. 🙏 ⬆️
