Small modules: it’s not quite that simple

There’s a post currently doing the rounds that articulates the ‘small modules’ philosophy. It’s written by Sindre Sorhus, whose npm profile is enough to make all but the most prolific developer feel wholly inadequate, and so carries with it a degree of authority.

Like a lot of ideas that seem to be self-evident and inarguable, ‘small modules’ contains a lot of truth but does not, I believe, bear close scrutiny. But because it is espoused by so many leading members of the JavaScript community, scrutiny is all too rarely applied.

I think I know why: it’s because the small modules philosophy favours library authors (like Sindre) at the ultimate expense of library users. Because library authors typically have larger megaphones (more Twitter followers, more credibility on GitHub and Hacker News, etc), their voices are heard more often.

But library users are more numerous, and so their needs are more important.

Before I begin

I’m someone who has benefitted a great deal from npm, and from the wealth of code available therein. In particular, I’m a frequent user of some of Sindre’s modules. The web is a better place because of him, and — even though node modules have certain design flaws from a client-side perspective — node and npm did more to introduce the concept of modularity to JavaScript developers than a thousand blog posts could have. Thank you Sindre, and thank you everyone involved in making node and npm.

What follows is not an absolutist claim that small modules are ‘bad’, or that we should explicitly strive to make ‘big modules’, but rather an attempt to bring out some of the nuance that is often glossed over by proponents of the small modules philosophy.

Why npm is the most popular package manager

You’ll often see presentations showing that npm has more modules than any other package manager, and is experiencing faster growth. The natural assumption is that this is because a) it is the best package manager, and b) the JavaScript community is the most energetic and productive.

I offer an additional explanation: that we in the JavaScript world have a higher tolerance for nonsense and dreck. Perhaps my favourite npm package is peter, which, when required, prints the following to the console:

Once upon a day there was a person, named Peter DeMartini
He is awesome…
Have a nice day…

It’s an extreme case, but npm is riddled with packages of similar value — packages with no documentation or README, that never advanced beyond version 0.0.1, and which have a dozen or so downloaders per month of whom the majority are most likely bots. I should know, I’ve written a few.

This happens because npm makes it ridiculously easy for people to release their half-baked experiments into the wild. The only barrier to entry is the difficulty of finding an unused package name. I’m all in favour of enabling creators, but npm lowers the barriers right to the floor, with predictable results. Bear this in mind when encountering npm stats.

The ensuing discoverability problem

Discoverability is often cited as npm’s biggest flaw. Many blog posts — scratch that, entire websites — have been created to try and mitigate the difficulty of finding what you need on npm. Everyone has an idea about how to make it easier to find needles in the haystack, but no-one bothers to ask what all this hay is doing here in the first place.

Eventually, you find a package that seems to suit your needs, more or less. But your problems have only just begun. It’s up to you to evaluate the library: does it have tests? Can you understand the source code? Is it actively maintained? Is the documentation easy to find and consult?

The most vocal small modules proponents don’t have this problem, because they practically live on npm and already know which packages they like — possibly making it harder for them to appreciate how severe the discoverability issues are for the rest of us.

Nostalgia

Back when JavaScript development was almost synonymous with jQuery development, our job was so much easier. jQuery has hundreds of functions, but once you’ve learned a few, you can predict with a high degree of accuracy how most of the others will work: the library is learnable. If you do get stuck, you can head to api.jquery.com and find the answers you need in a few seconds. Hipster millenial front-enders (including yours truly) may prefer other tools nowadays, but jQuery really is a masterpiece of design.

It’s also large enough that a community exists around it — a community that keeps it up-to-date, writes tutorials and blog posts, organises meetups and conferences, and answers your questions on Stack Overflow. You can go a long way as a developer with the jQuery Swiss Army knife in your pocket.

The ‘small modules’ adherent, by contrast, is able to use best-of-breed tools for her niche requirements. But she must learn where they are to be found, and develop an encyclopaedic knowledge of them, and if she gets stuck the chances are she’s on her own.

But what about…?

Yes, small modules are easier to write. Yes, they’re easier to test. Yes, it’s easier to adhere to semver. These are all things that make your life as a library author easier. As we’ve seen, they come at a cost for others.

This pattern exerts itself in another form: small modules allow library authors to become lazy. Why include that six-line helper function when you can do a one-line `require`? Never mind that the required module has its own dependencies, with their own dependencies, until your users suddenly find that `npm install` involves downloading 70Mb of gubbins split into 15,000 files. I exaggerate, but barely.

Back to the future

Recently I’ve been wondering what it would take for the pendulum to swing back the other way, so that we can once again use libraries like jQuery (broad, learnable, battle-tested, well-documented, actively-developed) without being subjected to modularity shaming.

I believe the answer lies in ES6 modules. I’ve been working on a tool called Rollup which presents one possible way that we could enjoy the benefits of a large utility library like jQuery (or D3, or Underscore/Lodash, or even Three.js) without adding unnecessary bulk to our projects, by only including the code you actually use. (It does this by statically analysing your code and its dependencies — no configuration required, and it’s extremely fast. The effects are drastic.)

But I don’t want this to be a ‘you should use Rollup’ post — I’ll write that post when it becomes true (right now it’s a promising experiment, but an experiment nonetheless). Instead, I want us to consider more broadly the trade-offs involved when we talk about small modules, and to remember that the needs of the many outweigh the needs of the few.