Terrible Lodash

Checkout terrible-lodash on github.

A few weeks ago a package was unpublished from npm which caused many large frameworks to suddenly stop working. The package, which was 13 lines long, raised the question “Why? Why is this is own package!?”

Some argue that breaking everything out into small components removes complexity from the programmer — that they have guaranteed behavior no matter what they are trying to accomplish.

Others point out that all this does is push complexity and bloat onto a project’s dependency tree, creating an install process that becomes unpredictable with many possible points of failure.

There is a lot of room for browbeating and comments on the internet, but lets create an example to look at the end result of each philosophy.

Making Lodash Terrible

Lodash, the most depended on javascript library, is a collection of utility functions of varying levels of complexity. Each of these functions are a few lines long and are built to do one thing well. Its a great library that is easy to include on any project.

An interesting feature of lodash is that all of its functions are unbundled into their own npm packages so that a developer can install the individual functions they need.

Lets create a lodash bundle that has every function as a dependency and compare it to vanilla lodash on size and install time. The overhead introduced by the new package, terrible-lodash, will perform cause the package to be larger and install slower, but by how much?

Its easy to build this test case. First we need to install all of the individual functions.

Then re-export all of those packages as functions (309 in total).

export {default as _arrayeach} from 'lodash._arrayeach';
export {default as _arrayfilter} from 'lodash._arrayfilter';
export {default as _arrayincludes} from 'lodash._arrayincludes';
// .. and so on

Build the project with rollup (which is a whopping 2MB file), and then compress with uglify + gzip.

And then push the project to npm.

Now we have an easy way how much overhead this has added to our dependency tree.

Installing vanilla lodash takes ~3 seconds, and takes up 4.4MB on disk:

While terrible-lodash takes ~22 seconds and uses 8.3MB:

By sticking to the idea that each function should be broken down into its smallest component we have created a library that is much larger, takes longer to install, and makes many many more network requests.

Lodash is installed ~1 million times every day — if it was packaged like terrible-lodash then, in a single day, there would be approximately:

• 300 million more requests to npm1
• 231 days of additional install time2
• 3.8 TB more disk space used for bloat3

The art of programming is the art of organizing complexity, of mastering multitude and avoiding its bastard chaos as effectively as possible.

Edsger Dijkstra

This toy example illustrates a core problem that developers deal with — managing complexity. Taking the terrible-lodash approach and breaking each function into its own package does not get rid of or reduce any complexity, but instead moves this complexity onto our tool chain. This would be fine if our tools were perfect, but this is not the case. The internet goes out, NPM goes down, people publish broken code and there is nothing you can do about.

There is no clear cut answer on what the best size of a package should be. While this answer is tied to the robustness of the tools that we use to build, lets focus on building robust systems that work by managing complexity is a reasonable and logical way.