Optimising npm dependencies
Learning about transitive dependencies, rational code reuse and dependencies test in your builds.
Do you recognize yourself in one of the following situations:
- you are 🔫 killing🔫 jobs on Jenkins to make yours start sooner
- you are amazed by the number of folders in
- you shared this Medium post
- you shared one of the comics below:
In the same time, I believe that app developers should take responsibility for choosing the good parts:
- check dependencies including transitive dependencies
- rationally reuse code
Sometimes copying file or code snippets is the best way of code reuse
To take a decision, you need to measure it. Analyzing transitive dependencies is not out of the box feature of NPM. But there are tools for this. I will list some of them:
- Webpack Visualizer webpack plugin generating nice diagram from your bundle; there are similar tools for Rollup and Browserify
- cost-of-modules CLI calculating size impact of installed modules
- https://bundlephobia.com web app which estimates cost of adding package into your app
- http://npm.anvaka.com/ web app visualise dependency tree of the package
Why my node_modules are still a mess?
Looks like problem solved, right? Yet, in projects, I’m working on we use npm-cache because the install is taking ⏳too long⌛️, and bundle size counts megabytes.
I don’t believe that the reason for it is a “natural project complexity” and we cannot do better.
I think developers, including me and my pals, tend to skip using tools to look deep into transitive dependencies of the packages they install. I have two hypothesis about it:
- The natural flow of discovering dependency is something like 🔍 Google 👉 npm.org 👉 GitHub 👉 $ npm install; Google, NPM, and GitHub are not telling us about hidden cost of package we install; other tools do not fit into this flow and we keep forgetting to use them before it’s too late;
- Many projects are team efforts; it’s not about me and you checking dependencies but also about our colleagues; my experience is that best practices like code style and test coverage works when enforced by automation/CI tools; nobody cares until build is not failing;
To test ideas I made a simple CLI called npm-consider which you can use instead of
npm install when adding a new dependency and in your automated builds.
Installing packages carefully
If you care, you can type
npm-consider install instead of
npm install and check package size with all transitive dependencies without downloading them:
You can see when a package was updated, size, number of dependencies and licenses. If nothing looks suspicious, choose Install and tool will run
npm install or
yarn add as usual. If you are not sure, print dependency tree and calculate the impact of the package on your project.
Real life example
When making this tool I was about to add license-compatibility-checker package:
Package itself has just two dependencies, but with all transitive dependencies, it will count 77 packages. It will double the number of my dependencies! In the detailed view I saw that update-notifier brings most of the dependencies:
But the only thing I need from it was
licenses.json, a 6Kb file. I simply copied thefile into my project and kept original copyright.
Test npm dependencies in your automation scripts
In my case tool saved ~30% of install time. Nevertheless, you and your colleagues will be forgetting to use it. That’s why you have to use it in your CI scripts along with other tests.
If you specify configuration in your
npm-consider will verify your package against the limits, and exit with
exit=1 if limits are not satisfied:
Now you know when someone added a creepy dependency to your project.
Think about the cost (including carbon footprint) of all those CPUs, which are compiling terabytes of JS code we don’t really need. You can help it if you:
- use tools to check your dependencies, including transitive dependencies;
- rationally reuse code: don’t include 77 packages when you need just one file, simply copy it;
- enforce rules above with dependency size/number tests and include them in your CI scripts
If you support ideas in this article, please clap. If you don’t, write a response and let me know what do you think! Thank you!