Dealing with a Dependency Hell
Maybe npm was not the best way to deal with this.
How many times have you asked yourself, do I need a dependency for just this? Turns out, multiples times you won’t. Many of us, try to choose a package to solve a very concrete problem that we are having in the moment. But, if you are able to solve it yourself in a util, do yourself a favor and just implement it there.
Reducing the number of dependencies, or simply selecting light ones above heavy ones, can have a drastic impact on initialization time, speed, and vulnerabilities.
It took months before my team and I, years ago, realized that we could drastically reduce the size of our node_modules folder from GigaByte scale to a few dozens of MB.
Not only that –it depends on the nature of your project, but– we had some dependencies back then that required some libraries to be installed in the host OS. That left us with two choices: Package our app with a bigger bundle (not acceptable) or restrict the number of users we could reach.
So we decided to look deeper, see what was strictly necessary and rework those things where we could find compromise.
How did we do it?
The journey to prune out unneeded deps
Visualization.
The picture above was the map of netbeast-cli dependencies when we started. I found a small utility, that uses D3.js to print a hierarchical tree of user dependencies.
But here was the root of the problem, Mosca was too big of a package, so we found an alternative in development that was stable enough. After replacing Mosca in favor of Aedes:
The order of magnitude of our dependencies was reduced by 10. Take aways? Use packages of just what you absolutely need basis. Visualize and learn.
Take this further
This technique may work in other environments and framework, but you should try to look for what tooling is there available.
If you are lucky enough and have webcpack as a packager (i.e. many front-end developers) you can try out https://github.com/chrisbateman/webpack-visualizer
This is a very easy way to see the immediate impact of a dependency, or your own files over your site loading time.
About code entropy
The more entangled is your own code, it can also have important side effects. Implicit relationships are the worst, but even explicit dependency and calls increase the complexity of a project. There are two important highlights that we need to pay special attention to:
- Cyclic dependencies. In all programming languages they may imply initialization problems, or will be strictly forbidden, or introduce unexpected side effects. Possibly interpreted in different ways by different machines. We should try to minimize them.
- Mental overweight. There is a point where we will start losing the thread of our own code; That moment it will start being it’s own thing and we will forget important details. Keeping a lean, clearly directed structure of our codebase will ensure we have things under control longer, making it more maintainable, testable, allowing it to grow naturally as we hire.
In Scope 🔬 the team is researching ways to add this check directly on GitHub (ideally upon Pull Request) by the moment we can enjoy the early feature of Hot Paths 🔥 that allows maintainers to see which paths have the most modifications and try to detect whether a certain structure is causing regressions or not, and therefore, needs a refactor.
Hopefully with your support as users they can find out ways of implementing other information of this kind, for example trying to find ciclic dependencies upon PR and letting reviewers be aware of the side effects.
🙇♂️ Hopefully this is of good use to you. Thanks for reading.
· Jesús Darío · 🐦 Twitter · Author of 🔬 Scope · 🏡 Yeti Smart Home
