Why my team uses npm instead of bower
tl;dr
My team moved away from Bower because:
- Programmatic usage was painful
- Its interface kept changing
- Some features, like the url shorthand, are entirely broken
- Using both bower and npm in the same project is painful
- Keeping bower.json version field in sync with git tags is painful
- Source control != package management
- CommonJS support is not straightforward
Substantive discussion
My team builds an SDK for creating and deploying web apps at Opower. A basic requirement for us is providing some sort of package management. We originally gravitated towards bower because it was simple and “for the web”, but since switched to npm, which provided several key advantages (at the time — this is not intended to be a side-by-side comparison of bower versus npm today). I am not recommending everyone use npm instead of bower regardless of the circumstances — I only wish to describe our evolution.
When we first started using Bower in early 2013. Compared to using no package manager at all for web projects, our honeymoon was blissful. You can install things on the command line! You can declaratively specify your dependencies, and programmatically inspect them! Because there are no shared dependencies and the dependency tree is flat, the mental model is very simple. When you don’t need those features, not having to worry about their added overhead is quite nice.
But after a while, when the warm glow of a new relationship faded, we started noticing some issues.
Programmatic usage is painful
Our SDK requires heavy programmatic integration with our package manager, and Bower was a little rough around the edges there. For instance, calling the API would return user-formatted strings instead of a js object, and commands did not emit meaningful events. (To Bower’s credit, they accepted PRs of mine fixing those issues.) Additionally, before the Big Rewrite, logic about parsing and normalizing bower config was locked up inside bower. After the rewrite, they fixed this with modules like bower-config.
Tool interface is unstable
Through Bower’s development, they changed interfaces and standards. This is perfectly reasonable for a pre-1.0.0 tool to do, and I don’t expect anyone to get it right the first time. However, it did add churn to our process to rename component.json to bower.json in all our packages, and our .bowerrc ended up bloated to support multiple versions of bower being used:
Do I think that bower is a bad tool because they are evolving their design? No. Do I think that the benefit the this new design happened to bring is justified by our distraction to update our code to meet the new standard? Marginally at best. On the other hand, npm is fanatical about not breaking backwards compatibility. At times, this can be irritating, but overall I believe it’s a sound strategy.
Some features are entirely broken
Some aspects of using bower just didn’t appear to be fully thought through. For example, the shorthand-resolver feature appears to be a nice syntactic sugar for specifying where to find dependencies. However, this feature takes effect across the entire dependency tree, which may include packages that you don’t control and don’t want to use your resolver setting. This totally breaks the feature, as people have pointed out with bower#1369 and bower#1342. If a package requires their own shorthand resolver, and you install that package as a transitive dependency of one of your own, you would see the same resolver mismatch issue.
Not everyone wants a flat dependency tree
The flat dependency tree is a good idea in many cases for web development. It’s a best practice not to send three different versions of a dependency like lodash to the browser just because you have a tree like this:
However, this can be useful as well. It’s very nice to not require all your deps to agree on one global version of a dependency. You’ll never say, “Well, I’d like to update to the next version of E, but not every single one of my dependencies is compatible with it, so I’m stuck”. If the E in question is a very small package, then the performance hit from having multiple similar copies might not be too bad. And if you don’t control B, C, and D, you might not have the option of changing which version of E it uses.
My team didn’t want to make a one-size-fits-all decision for the users of our SDK. However, with Bower’s flat dependency tree, we were forcing them into the approach of never having different copies of the same dependency on the page.
Bower added complexity for our users
Speaking of our users, Bower made their lives harder in other ways. When you specify a version of a package to install, Bower used that spec to find a git tag that matches. The version field of bower.json wasn’t actually used in this process. If there was a mismatch between bower.json and the git tag, great confusion could result. Our team had to develop tooling around this that would read bower.json and tag the repo automatically. With npm, this isn’t a problem, because when you run `npm publish`, whatever is in your package.json version field is what’s used. You’re free to tag your repo as well, but that’s just for your own workflow.
An additional burden on our users was our requirement that they use both npm and bower. Bower would bring in runtime dependencies for their web apps, and npm would bring in the dev dependencies, like grunt, grunt tasks, and testing frameworks. This adds another bit of complexity to the dev experience in a system where we’re trying to optimize for our users being able to crank out content very quickly. “Did you remember to npm install AND bower install?” was a common response to people asking for support. Beyond the added overhead to the workflow, it made the mental model more cumbersome because when you wanted to work with a dependency, you first had to figure out which of the two package managers brought it in. Moreover, duplicating config between package.json and bower.json is a huge bummer. We added more tooling to ensure that those files stayed in sync, but this is a case where the tooling is just papering over a fundamentally bad situation.
Bower conflates source control with published package management
The Bower server is essentially a url shortener. It allows you to specify a dependency like so:
And it will resolve to the git repo where lodash lives. This is delightfully simple, but if you want a build step in your workflow, you have two options: run the build step and check the resulting artifacts into git, or run the build step at a deferred stage (such as right before deployment). Neither of these options were palatable to us.
Building at a deferred stage is not good. When you are authoring the code, you know exactly how it should be built, and you’re testing the built artifact given a current set of external conditions (like which versions of third party dependencies are available, or what the underlying OS is). We wanted developers to be able to QA their product, and then freeze it in time by doing the build step, so there is no question what you get later is bit-for-bit what you tested. (npm does a great job of explaining why pre-publish is a better time to build than post-install.) We experimented with writing various manifest files at checkin-time and using those down the line to do the build, but it turned out to be a fundamentally error-prone system.
Checking build artifacts into git pollutes the repo with files that people shouldn’t be editing by hand, but there’s room for error if someone new or desperate is working in the repo. And how do you create these build artifacts into git? Does the developer have to run a command every time they are about to commit? What happens if they forget, or if they run the build command, change the source files, and neglect to run the build command again? Do you build CI tooling to check out the repo, run the build, then check the artifacts back in? This means that your history will be a series where only every other entry is by a human, and you only want to deploy those entries built by machines. Saying “only half of the points in our git history are actually useable as-is” violates least surprise.
The solution is not to conflate source control with package management. Running an npm registry can be an incredible headache, but having it has freed us to do so many more interesting tasks in our build step, and to radically transform our web apps after checkin but before deployment.
Bower doesn’t make CommonJS as easy as npm
Finally, we wanted our developers to be able to use commonjs modules. Browserify (or, more recently for us, webpack) and npm work together smoothly to achieve this. If you wanted to do AMD, npm wouldn’t be such a slam-dunk, but we felt comfortable dictating this choice for our users.
Where are we now?
Because of the aforementioned issues, my team was curious about alternate solutions, such as Browserify and npm. In Fall 2013, we did a prototype, and since the first time we wrote “var _ = require(‘lodash’);” and saw it working, we never looked back. There are many modules on npm, and having a simple way to share code between the server and client is super nice. Because we’re using CommonJS, we are no longer dictating what choice our users make on whether or not to have only one version of a dependency in their tree.
Instead of telling our users they have to think about two different package managers, they just run “npm install” and are done. The tools that were built to keep package.json and bower.json in sync have long since been given the Ol’ Yeller treatment and marked as deprecated.
Instead of awkwardly building artifacts on deployment or checking them in to git, we have an npm registry. It is a total pain in the ass to run sometimes, and mysteriously npm Inc has not been responsive when we’ve reached out to them and asked if we could trade USD for their hosting services, but it’s still a net gain. The additional complexity of the deferred build rippled through our entire system and made it less manageable and more error-prone.
Unfortunately, npm is not any easier to use programmatically than Bower. @searls totally nailed it when he pointed out that npm is not designed with tools that build other projects in mind. I don’t blame npm for not focusing on that, but it would be misleading to suggest that it’s better than Bower here.
npm has given us trouble as well. A full treatment of that would be a discussion of its own. But a lot of that trouble comes from us trying to push npm to its limits with massive install trees over slow connections. In one dark moment, we considered ditching npm entirely. However, we were able to work with the npm core team and resolve many of the issues we were seeing, and have since returned to stability. Despite all of this, the thought has never crossed our mind to go back to Bower.
