The beautiful chaos of the NPM ecosystem
Say what you want about the attitude towards micro-dependencies, npm (and by extension, yarn) get plenty right.
What follows is a bit of a rant because it can be frustrating to witness the mob-mentality playing out in a way that doesn’t seem so destructive on the surface, yet that’s exactly what the popular JS and PHP bashing turns out to be: just calling it all shit because the status quo is to call it shit, no matter what is being achieved through it. Rather than paying attention to what you enjoy doing as a programmer it can often appear to be the case that you’re just waiting for someone to dig up a bit more dirt on the things you love to hate so you can get back into that drama.
JavaScript is a weird language to single out here, because it is not the only language that provides a half-way decent dependency management system. RubyGems is fantastic, and it’s great that the community converged on Bundler as the solution to fetching and managing libraries beyond a simple gem install for the one-off cases. How you build those Gemspecs is up to you — Bundler just wraps the existing RubyGems infrastructure, which sounds familiar when you look at Yarn. Rust’s Cargo is equally fantastic, and the approach Go takes is unconventional but fairly intuitive to a beginner who doesn’t want to think about versioning (what could be simpler than plugging a github URL into your source file and having it just work?).
It’s easier to make the point with JavaScript/Node because it often gets a lot of stick for the attitude the NPM organisation and the surrounding community has towards building a project. A guy called Jon Schlinkert recently got called out for publishing hundreds and hundreds of simplistic packages to NPM, and for having the audacity for big projects like Webpack including some of them as dependencies. I think it’s unfair to call someone out for doing this because the number of downloads often speaks for itself: my completely unfounded speculation is that JS is more accessible to aspiring developers now than PHP and HTML ever were back in the late nineties and just beyond the turn of the millennium. I was cutting my teeth on PHP 4 and an FTP client connected to some shared hosting service way back in 2004 and back then even CSS2 was a distant dream for web devs. But now… we have more inclusive communities with huge numbers of volunteers posting articles and running weekend workshops, never mind the massively open online courses and the boot camps that are now available to those who want them.
The point is that a user trying to google how to figure out whether a number is odd or even, in JS, is likely to come across a github repo or an npm package that does the job for them already, and since the attitude to programming in JS is to stand on the head, shoulders, knees and toes of giants… well, it stands to reason that they’re more likely to run npm install --save is-slightly-smiling-face-emoji or the yarn equivalent than dive into the source code and see how it works. You can’t expect a beginner to do that even if they have the curiosity, because you can’t rely on the fact that these packages have intuitive implementations. The is-odd package might not be exporting parseInt(n, 10) % 2 === 1, but some crazy thing that handles all kinds of edge cases you would never have dreamed of anticipating.
It sounds ridiculous to those of us who’ve spent years or decades programming because we have a year or decade long head-start over those who are coming in fresh. And to some level it is ridiculous, because you don’t see the same kind of extreme micromanagement of functions in any other language. Imagine installing GCC or Clang and setting up a new C++ project, and then realising there’s no standard library included, and you have to sudo apt install libcppstd_iostream-dev to work with standard input and output on a terminal.
This brings me to my main point: the extent to which people extract functions into packages is extreme but optional, and nobody is forcing you to add is-paamayim-nekudotayim to your list of dependencies to see if a string contains ::. Having enjoyed the experience of setting up C and C++ projects, where the solution to using an external library is to either install whatever version of the headers are available through the OS (e.g. with apt on Debian or Ubuntu) or to manually copy the file into your repo and wire it all up with CMake, or Make, or whatever else you use to tell the compiler what to do, or even to copy the code and compile it first so you can link to it… I pretty much spent half a day telling the IDE that the test library I copy/pasted into the project lived in a subdirectory only to get inexplicably broken builds, and CMake looks powerful but seriously complicated. To be fair it’s not unique, you get the same by being introduced to JS through webpack and babel, or Python through pip and virtualenv, or Haskell through Cabal.
With that said, NPM might be chaotic and mismanaged and not-entirely-professional and whatever else you want to call it, but it (along with other package managers) make it so much easier to share code and have it available to your own project that it’s worth accepting the flaws.
That isn’t to say that it couldn’t be vastly improved; the unpleasant experience of starting a C project from scratch doesn’t mean that the JS ecosystem gets a free pass. More that the benefits it brings far outweigh some of those drawbacks (some of which have been dismissed by the effort put into Yarn), and if the option was to give all of that up in order to bring order to the chaos, then it would be easy to see just how much this stuff is taken for granted by those who can only see the faults.
