Gophers: We Don’t Need No Stinking Package Registry
NPM, Python, and Ruby’s Fails Prove It All
The recent massive security failure of NPM happened because of a bunch of just plain bad assumptions by the original architects (some of which even the creator of Node calls out very publicly). The first bad assumption? THAT YOU NEED A STINKING REGISTRY TO BEGIN WITH. 😁
The best part about the Go community is challenging the herd mentality on everything. Guess what? Having a registry adds all kinds of risk, security being just one of them.
What if the thing goes down or becomes unworkably slow (as NPM has done on countless occasions)?
What if the original maintainers just go away leaving a public package with no support and no way to add additional maintainers?
What if you can’t even contact a maintainer because they don’t exist anymore but have been hacked and made an out-of-date package now a primary zero-day vector for hackers?
These are all serious questions that any experienced IT architect would have considered before pulling the trigger on the monumental amount of work behind creating, supporting, and maintaining a registry in the first place.
Unfortunately, NPM bolted ahead like the naive young unicorns they see themselves as without asking these important questions (even making fun of experience people who challenged them on it) and, based on all the events since NPM’s creation as a for-profit enterprise, have proven to have alternative motivations driving decision making. Frankly, it is shameful and embarrassing (and yeah, when the creator of Node agrees with me I don’t have to apologize for that opinion).
What can we learn from the NPM complete and utter fail?
WE DON’T NEED NO STINKING REGISTRY
It’s 2018. Registries are so 1990s.
We have a ton of source management and hosting options now that cover every need that a registry would have provided, AND in a way that allows for forked continuation of abandoned packages or alteration without the additional unnecessary and wildly insecure package management registry.
When people cite the lack of dependency management of any kind I repeat back the very well-informed, experienced, proven-in-the-heat-of-Google-development, conclusion from the Go FAQ:
“Versioning is a source of significant complexity, especially in large code bases, and we are unaware of any approach that works well at scale in a large enough variety of situations to be appropriate to force on all Go users.” — Go FAQ
Perhaps the most dramatic difference between Go community thinking and NPM insanity is that Go promotes less separate dependencies going so far as to suggest downloading a complete copy of a package you depend on and placing it in your
externals or other location directly in your own package source tree. This forces every developer to do their own research about what they are including. It forces responsibility.
NPM, on the other hand, results in literally hundreds, perhaps thousands of cross dependencies none of which are ever personally checked by the developers using those dependent modules. This is insanity. Not only does it produce all the bloat and file system naming limitation bugs (for which
yarn was one solution), but it requires an immense amount of complexity just to lock down the versions in a given application or module that needs all those dependencies. Complexity breeds bugs and security vulnerabilities, as we have seen over and over again.
The most disgusting part is that NPM leadership are encouraging irresponsibility by promoting a culture of small modules invoking a bastardized version of the toolbox pattern made famous by UNIX where one thing has essentially one job that it does well. The ugly result of that is people squatting on package names they never use, which is compounded by the fact that you cannot EVER remove a package under any circumstances. I know. I tried and had to email the NPM support team who refused to hear me say, I AM NO LONGER SUPPORTING IT AND THERE ARE NO OTHER SUPPORTERS. The moronic response was, “well we cannot be sure someone else it not using it, and we can’t take that chance.” They didn’t even look at the thing to realize that would be impossible if not very unlikely.
You know why they keep all that crap? Because in their young like/subscribe mindsets they think having more shitty packages and being able to say “we are the biggest package registry in the world” is actually something to be proud of. At least CPAN has GOOD stuff in it.
By the way, I’m picking on NPM, but Python has been almost as bad. No one can figure out what to do but there is also no good way to share and get modules from GitHub as Go has so astutely built into the system.
Ruby and CPAN suffer from the problem about maintainers. I happen to be very aware of it because I created the Ruby INI parser library years ago and completely forgot about it with life only to later find frantic pleas into the Netverse from people wanting to update it and unable to do so because there was no solid way to fork it. Had that been done with GitHub anyone using the package would likely still have their own built into their dependency local source directories even if the entire project or GitHub repo went away. Now that is brilliant.
Having a registry builds a implicit dependency on the registry itself, a single point of failure and is simply unnecessary. Not having it forces responsible architectural decisions and development decisions.
Nathan Youngman also wonderfully captures this topic in this blog post.