Guide to managing npm packages in your package.json
Learning the npm commands of how to install or upgrade packages isn’t hard. What I did find difficult for a long time were the best practices behind it. I would mostly use the packages that others had added to projects as I thought that they would have had good reasons to add them and I didn’t know which to add. Or I would think that all dependencies needed to be updated all the time for your application to keep working.
I’ve learned a few things since then about managing packages in web projects that I would like to share with you. These things have worked well for me, but if you’ve had different experiences, please share them with me in the comments.
Considerations before adding a package to your project
The first thing to consider here is that there is always a risk to adding a package to your project. Every package you add means another dependability on an external source. Since you have no control over what happens to these packages, there’s a chance others made it or are changing it in ways that could harm your project. In 2016 half of the internet broke as someone unpublished their npm packages (it did become harder to delete npm packages afterwards). More recently someone updated a package which broke one of its dependents. Or this alarming story of how someone could be reading along from your installed packages (spoiler alert: in this case it’s fictional, but it is still worrying).
All this said, it’s not my goal to make you so afraid of npm packages, you don’t want to install any anymore. It’s a reason to be cautious, but there’s definitely advantages to using packages as well. For starters, if you use packages, it can save you a lot of time by not reinventing the wheel. Others have already solved problems that a lot of developers are running into and chances are they have done it in a way that is more solid, with better browser compatibility and maybe even better security practices than your quick fix for this problem. Besides, even frameworks like React and Vue come in npm packages and they will have their own dependencies on other packages, so there’s not really a way around it in modern development.
I would say we can broadly divide packages into two categories, core packages and trivial packages. Examples of core packages at OpenClassrooms are things like React, Material-UI, Redux and Webpack. Those are packages that come from trusted resources and other people have put a lot of time and effort into them. They will often have an impact on your architecture and your way of building your project, so you choose them based on that. Once you decide you want to use one of those core packages, install them, they can absolutely make your life better. The trivial packages are up to personal preferences whether you’ll want to add them to your project or not. They won’t impact your architecture, they just might save you time. So do you think you’ll gain or lose more from adding them?
A note about using Git packages
Including git repositories as packages in your project should mostly be used as a last resort, especially if you don’t have ownership of the repository. Git is not a package management tool, so it will give you less control over what happens to the package. A Git repository can easily be deleted or updated with breaking changes.
When there is a bug in the package you would like to use, it’s usually better to first try to make a pull request to the original package than to fork the project. If you do however want to use a Git package, try to set it to a certain release, tag or if those are not available, a commit hash. That way it could still get deleted, but it at least won’t surprise you with a breaking change.
Which package to pick out of npm’s 1.000.000+ packages
Once you have decided that you have a need for a package, you’ll have to choose which package is your best option. Anyone who is willing to make an account on npm can publish packages there, so the quality of the packages will vary greatly. While no method will be waterproof, there’s a few tips that can help you choose the best one:
- See if it’s used by a lot of developers. Npm publishes statistics about the amount of weekly downloads (note that every install counts here, even installs by a build server, so take the number with a grain of salt). A high number of downloads is probably an indicator it’s a useful package to other developers.
- Check if it’s still actively maintained. Look at the last release date on npm and go to Github to see if some activity is taking place in the open and closed issues. For a small trivial package it might be fine if it’s a little older, but as soon as the package would have a somewhat bigger role in your project, it is important that it still works optimally for the latest browsers and it should work together well with other newer packages. A classic example is that some old packages won’t work with a newer version of Node.
- Verify the size of the package if they are used client-side. With an eye on performance, it’s best to make sure your bundle doesn’t become too big. Some packages will add a lot more weight to your application than others. One way to find out is to check on BundlePhobia. Also check the number of dependencies the package has, the less the better.
- Make sure the documentation is clear enough to you. When it’s not directly clear from the documentation, it can be a good idea to read the source code of any package you add to your project. Often that will help you to understand how it works. If it’s doesn’t quickly become clear to you from seeing both the docs and the source code, it will probably be a pain to use the package and it might be better to move on to another one.
- Since I tried so hard to make you scared of adding packages to your project earlier in this article, I’ll have to add this one. Check the security issues and vulnerabilities of the project. These days however that’s much easier than a few years ago. Github will indicate any security issues it detects in your package.json and even npm install will check for vulnerabilities. Having said that, these are only the known security issues, things can go undetected at first.
Dependencies vs devDependencies
When you install a new package in your project, you have the option to either save it in dependencies or in devDependencies. As explained by npm, dependencies are “Packages required by your application in production” and devDependencies are “Packages that are only needed for local development and testing”. However, this distinction is more important when you’re making an npm package yourself, than when you are making an application for the web. When someone installs an npm package, the dependencies of that package will also be installed, but the devDependencies will be ignored. In the case of making a web application, you’re probably using a bundler like Webpack, which doesn’t look at whether packages are in dependencies or devDependencies. Webpack will start in your entry file, follow all the imports and only include the packages that were imported in your code.
If we take Webpack as an example, it’s not used in production, just for preparing the bundle for production, so it would be included in the devDependencies. If this would be the case of an npm package, it means it wouldn’t be included in the project of someone using your package so he/she still has the option between using Webpack or another tool like Parcel. In the case of building an application, Webpack would be installed everywhere, on your computer, on your colleagues computer, on the build server, so everyone can make a bundled build of the application. This build is the only thing the user of the application gets to see, the exact contents of your package.json won’t make the final difference for your user.
One last thing to note is that, even for a web application, separating your packages into these two categories can still be helpful for organisational purposes. It shows you how many packages you are using in production. Or it can help other developers to quickly scan if some package they could use is already included.
Unused packages
The same thing applies to unused packages. Because Webpack never finds an import to them, it won’t include them in your final bundle. By all means I’m not saying you shouldn’t remove them, clutter is never good, but they won’t harm your bundle size.
What about that other one, peerDependencies?
There’s one other category of dependencies that you see at times, peerDependencies. This kind of dependencies is only intended for packages that are released on npm, you won’t ever use them in a web project. You would include a peerDependency in your npm package when a) you expect the projects that include your package to also have that package as a dependency, b) you need a specific version of that package and c) your npm package would break when the importing project has a dependency for another version.
The value of the lock file
After you’ve installed packages with one of the modern package managers, you’ll come across a lock file. When you use npm it’s package-lock.json, for yarn it’s yarn.lock. Before these lock files existed, you were never entirely sure which versions of packages would be installed and what the differences would be between the project on your computer, your colleagues computer and the server. Even when your package.json only had fixed versions, no carets or tildes, if the dependencies of your dependencies didn’t have fixed versions, the versions of packages could differ from one install to the other if one of those packages deeper down the tree had been updated.
These days we won’t experience these problems anymore as lock files came to the rescue. Lock files make sure that the whole dependency try is locked, it will save the version of the dependency of a dependency of a package in your dependencies, just as far as your dependencies go. Everyone who runs an install for the project will get exactly the same versions (as long as you commit them). Only after an update they will be changed, which leads me to the next subject, updating…
Keeping your packages up-to-date
One well known problem in the javascript world is the feeling of having updated your packages only yesterday and they already released a new major version again. Updating is often not an easy task, especially as some breaking changes will have you digging deep in the code to make this new version usable again.
In an ideal world you would keep all your packages up-to-date by running a general npm update or yarn upgrade on a regular basis. By default packages will be installed with a caret, which means that such an update should not have breaking changes. Major versions would still have to be handled manually.
In reality for any project that’s a little bigger, I’ve seen developers usually say “If a package is not a big part of your project and it currently works well, then why update it and run the risk of breaking things?” For a lot of packages you can trust that semver system is being used properly, but for some packages it happens that breaking changed sneak into a patch request. A lot if times the gains from updating all your packages minor or patch versions don’t outweigh the risk of breaking things and the time you have to spend on fixing them. It’s a matter of what is personally important to you, but from my experience, it works well to stick to the following cases in your choice of which packages to update:
- The new version has features you want to use. For example when React released the version with hooks, we (here at OpenClassrooms) were really quick to upgrade this one.
- There’s a certain bug you are experiencing that can be solved by updating the package that’s causing it. To find out about this, the best way is usually to go to the GitHub page of the package and search for the error you are experiencing. If you’re lucky there’s a closed issue with details about which version to update to (or any other solution).
- When you’re using another package that’s asking for a newer version of an already installed package. Usually this will be indicated in the peerDependencies.
- Sometimes a package will publish a newer version with significant performance improvements. This can for example come from modern browsers releasing features that natively implement certain behaviours that previously needed a workaround.
- The one that speaks for itself, you found out about a security issue in a package.
One important rule is of course not to wait too long before updating the packages that fit in the above list. The longer you wait, the more breaking changes might be released. Sometimes developer add warnings to the code that certain functions are going to be deprecated in the next version, which can make the experience of updating much more pleasant.
On the other hand, when a new major version has just been released, you might want to wait a little while before you update. Even though the developer will probably do their best to make it bug free, there’s always a risk that some overseen problem comes up just following a release. It’s hard to test one’s package against all possible use cases after all.
There is not simply one good solution to package management, a lot of it comes down to personal experiences or preferences. I hope though that this article helped you forming your own opinions on it.
One thing I would still like to experiment with is more actively fixing some versions in my package.json and giving some flexibility to the ones that come from trusted resources like React. It might be a relatively easy way to keep some core packages more up-to-date with smaller risk of running into breaking changes unexpectedly.
<joke>One more thing, never forget the golden rule of what to do when your application doesn’t work anymore. Just remove and reinstall all of your node modules. Works even better than turning your computer off and on again. </joke>
