Mastering NPM Versioning
Not so long ago, software development was similar to the world depicted in George Orwell’s 1984 — dominated by major vendors like IBM, Microsoft, SAP, Sun, and Oracle. Life was simpler back then. Most developers worked for governments or large enterprises. Senior management would pick a supplier, who would then provide nearly everything required. After all, nobody got fired for buying IBM. Those days are now a distant memory.
Today, developers not only code using multiple languages, they assemble applications by combining hundreds of code components. These basic building blocks, which snap together like Lego bricks, include open source and proprietary packages, third party APIs, and a dizzying array of cloud-based technologies. Developers download the components and write additional code to integrate them. Each code component brings with it dependencies on yet other code components.
For those of us who cut our development teeth on the Microsoft world, the code component model seems eerily familiar. In those days, code components were called Dynamic Linked Libraries (DLLs). DLLs were both a blessing and a curse. Any mismatch between the DLLs in your app and those already installed on a user’s PC could trigger a cascade commonly referred to as “DLL hell”. Some would argue that what goes around comes around and, in the brave new world of package and dependency management, we are now experiencing its return.
This article explores the pros and cons of the code component revolution, and delves into best versioning practices that can prevent applications from breaking down when updating code components.
The Component Revolution: Coding Unleashed
For most developers, the advantages of the code component model far outweigh the disadvantages. In fact, it has revolutionized the way we work. For a start, the code components revolution is tied closely with the rise of open source. With a few exceptions, most of our frameworks, libraries, and languages are not proprietary black boxes, but open ecosystems. Open source is no longer the preserve of outliers like Richard Stallman and Linus Torvalds. Today Fortune 500 companies like Apple, Amazon, Google, Microsoft, Samsung, and even Oracle, support open source projects and GitHub repos.
Furthermore, iterative development and agile methodologies have changed the way we write, test, and release our code. Container technologies have made it easier to both recreate production environments locally and deploy a developer’s local environment into production. Recent developments in cloud computing have led to the rise of serverless computing, which makes it possible to concentrate on writing code and ignore how it is deployed.
NPM and its Challenges
The code component model emerged in parallel with, and in many ways was facilitated by, the rise of Javascript. It was further accelerated with the release of NodeJS, which made it possible to use a single language for both backend and frontend development. Node has made it easier to write code components, and the Node Package Manager (NPM) gives us a way to bundle and distribute them. Since its release, NPM has become the de facto standard for distributing most Javascript projects, from servers to frontend frameworks; and overall, it has vastly simplified developers lives. Unfortunately, it also introduced a number of challenges regarding versioning and dependency management.
Now, let’s say you are developing a new project and you are using NPM to find and manage your packages. Your framework will provide a command line interface (CLI) that lets you create a new project with a single command. Some tools will install an initial set of packages or generate a list for you to install at some point.
As you start writing code, you will start to add packages to provide useful tools or support new features. These packages, in turn, will install additional packages that they depend on. The result is that, not long after you start an entirely new project, there may be hundreds — and possibly thousands — of packages installed. Many of them without your knowledge.
NPM Dependency Graph: Top 100 dependent upon npm packages and their dependencies in 4 levels of depth.
One of the many great things about NPM is that it lets us take care of version updates with one simple command: `$ npm update`. Not only does this command let us update existing packages in one go, it also lets us do basic cleanup and remove old and unwanted packages.
Just like updating the operating system on your smartphone or laptop, running NPM’s install or update commands is a drama-free experience. However, when you update a device’s operating system, the device’s vendor has first made sure that the existing, updated, and new components work together. By contrast, in the NPM world there is no single, definitive source of truth and new versions and patches are released independently. This means that there is always a risk that your app will break when an updated component won’t work with an existing one. In addition, many components may use the same subcomponents, but require different and incompatible versions.
Overcome NPM’s Challenges: Master Component Versioning
Before we look at solutions, we need to understand how NPM manages project dependencies. Let’s start our journey by looking at package.json. Many CLI framework tools create the file each time you create a new project, based on a project-specific template. For example, if you create a React project with the create-react-app command, the file will include the project name, an initial version, three react dependencies, and four basic scripts.
If you are running NPM 5 or higher, each time you install a local package as part of your project, NPM automatically updates the package.json file. Unless you have a specific reason for touching package.json, most of the time you can safely ignore it. For more information, take a look at Working with Package.json
Understanding NPM Version Notation and Semantic Versioning
The first step in avoiding component incompatibility issues is to understand how versions are managed. When you open package.json and scroll to the dependencies section, you will notice that each component has been installed as a key:value pair (“package”: “¹.0.0”) indicating the package name, followed by the version number. SemVer, the semantic version numbering scheme used by NPM, uses these three digits to indicate the major, minor, and patch versions.
By default, the package version is prefixed with a ^ (caret) character, which instructs NPM how to handle the next package update. According to the NPM documentation, the caret “…allows for changes that are presumed to be additive (but non-breaking), according to commonly observed practices.”
The caret allows changes that do not modify the left-most non-zero digit in the indicated version. In other words, it will allow patch and minor updates for versions 1.x.x, patch updates for versions 0.1.x, and no updates for versions 0.0.x. Note that while the caret provides you with tremendous flexibility, it can also add additional complexity. In other words, most of the time you’ll be fine, but there’s always a chance that something could break.
Using Package Locks
One way to manage dependency versions is to take advantage of package locking using the package-lock.json file. This file specifies a version, location, and hash for every module and each of its dependencies, with the hash being used to verify the package integrity. NPM creates this file the first time you install a package, or recreates it if you accidentally delete it. NPM modifies and updates this file each time you run a command that modifies the package.json or node_modules, such as install, update, or remove.
A side benefit of package locking is that it lets you propagate your version of the module tree across your organization. In other words, if you commit your package-lock.json to a version- controlled repository, any team members who merge from the repo should be able to replicate the changes you made to the package map. It also ensures that whatever your operating system or how many developers pull the code, exactly the same packages and dependencies are installed. If you want to know more about this topic, take a look at Everything You Wanted To Know About package-lock.json But Were Too Afraid To Ask.
Installing Specific Package Versions
For many of us, the best way to avoid problems is to follow the KISS (Keep it Simple Stupid) principle. The simplest way to get the exact package you want is to install that version. NPM also lets you install a specific package version by appending the @ symbol to a package name, followed by the version number, for example:
$ npm install package@1.1.2
In this case, you can use NPM install to download version 1.1.2 of your package. After you run the command, NPM will automatically make the necessary changes to the package.json and package-lock.json files.
Best Practices
In addition to the mechanisms outlined above, here is a short list of best practices that will help you manage your installed packages and will prevent NPM versioning issues. The list contains a number of useful NPM commands that run version management related utilities.
1. Update Frequently and Use Package Locking
I already discussed these guidelines in detail, but I repeat them here because one of our human foibles is that we frequently overlook the simple and obvious. Either way, you should frequently run the NPM install and update commands to refresh your project dependencies. In addition, if multiple developers are working on the same project, make sure you use package locking to ensure that everybody is on the same page.
2. Know More About Your Package
When installing NPM packages, it is very easy to just fire and forget. In other words, once we’ve located the package we need, we install it without a great deal of thought and only remember that it exists if something breaks and it’s mentioned in a cryptic error message/trace. It is important to do your homework before installing the package.
3. Read the Home Page
Although the quality of the information may vary, most packages include a link to a project homepage. Also, a quick rule of thumb: if it doesn’t have a home page, this is a sign that the project is not maintained, and you should avoid it. To view a project home page, use the following CLI command:
npm home <project-name>
4. Review the Project Readme
High quality projects usually have a well-maintained home page. If it doesn’t, the next best thing is to view the project’s readme. Most developers will update their project’s readme and commit it to their GitHub repo when they release new versions. To view the project repo/GitHub page, type:
npm repo <project-name>
5. Get the Issues List
NPM also has a handy command for display package issues. Using the following CLI command opens the project’s GitHub issues page.
npm bugs <project-name>
6. See Who’s Responsible
Although not strictly version related, NPM packages include a list of the team responsible for maintaining it. To display the list, use:
npm view <project-name> maintainers
7. Find Outdated Packages and Dependencies
If you are considering updating some or all of your packages, use the following command in your project directory to see a list of outdated packages:
npm outdated
Since most projects can easily contain hundreds or even thousands of packages, to see when a specific package was updated, run the following command to get the exact time and date the package was last updated.
npm view <package> time.modified
8. Read these Articles and Blog Posts
For a through introduction to NPM version management, take a look at A Beginner’s Guide to npm — the Node Package Manager. To learn more about semantic versioning, read Semantic Versioning: Why You Should Be Using it, and Understanding the NPM Dependency Model. Finally, to learn some really useful tips and tricks, see 10 Tips and Tricks That Will Make You an NPM Ninja.
Conclusion: Just the Start of the Journey
Code components have revolutionised software development, but they also introduced challenges related to dependency and package management. Understanding how versions are managed by NPM and employing some package versioning best practices can help you avoid common pitfalls when updating packages in an app. The practices that we described here are just a drop in the bucket, and if you want to know more, below is a list of articles used to research this topic, which will help you get the most out of the code component revolution.
Originally published at www.datree.io.
