How We Simplified our Tooling Setup for Node.js Projects

tldr; We suffered from JavaScript fatigue and created something like create-react-app for our Node.js projects. By wrapping tools like Babel, ESLint, or Prettier with an additional CLI we can set up and maintain tooling much easier.

Photo by Scott Webb on Unsplash

Earlier this year I heard about a new tool in the JavaScript ecosystem, a code formatter called “Prettier”. I was very excited about it for numerous reasons. For the first time, we would be able to enforce a consistent formatting style across our entire team. No more unnecessary code style changes would mean easier code reviews. And also, it would (hopefully) save us a lot of time because there would be no need to fix formatting issues when ESLint formatting rules were violated.
I pitched Prettier to my team, and we agreed to give it a try. We removed our previous ESLint formatting rules, added Prettier to our workflow, installed editor integrations, and the rest is history. Prettier delivered on its promises, and now we can’t imagine working without it. We wanted all our code to be “prettier” and all our projects to have Prettier configured. The only problem was, we just had too many projects. Installing and configuring a new tool across many projects would definitely take too much time. So we asked ourselves: How can we integrate a new tool into over 60 projects in a reasonable amount of time?

Our project landscape

Almost everything we do, we do in JavaScript. It’s currently our go-to language and for most of the problems we’re facing it fits our needs. But if you are familiar with the JavaScript landscape, you’ve probably encountered the same issues we did when it comes to tooling. Setting up a new project is nothing but straight-forward. In most cases, we would set up a compilation step, a static code analyzer, a code formatter, a test runner and a testing framework. And the way we did that was usually by copy-pasting some boilerplate code around and tweaking configs until we got everything working. The problem with that is that you don’t always get your configs right at the first time, or you’ll need to update them later on, add additional tools, switch out one for the other in a way that you haven’t thought of when setting everything up. When you have tools and configs living and evolving individually, it’s hard to keep them consistent and change them in multiple places. For example, here are a couple of real-world problems we were discussing recently:

• How can we add a new language feature or a specific Babel plugin to our projects?
• What if we wanted to add types with Flow or TypeScript?
• Our test-runner takes too long to start, how can we improve that?
• How can we make source-maps work in our tests?
• How do we introduce a new tool like Prettier?
• What if we wanted to use a tool to auto-generate changelogs?

The answers to those questions weren’t complicated and introducing the required changes for a single project wouldn’t have been an issue at all. But doing it for multiple projects is time-consuming. We need to stay productive, solve problems for our customers, and can’t spend most of our time on meta-programming issues. Unfortunately, as a consequence, we found ourselves in a state where we were too afraid to change the status quo.

Moving towards “zero-configuration”

The problem really breaks down to configuration. Or more precisely, the fact that we have separately configurable tools in each of our projects allows them to diverge in different places. If we want to enforce a consistent behavior for our tools, we need to restrict their configuration options. Likewise, if we don’t want our developers to spend too much time configuring different tools just to set up a new project, our tools should be useful with as little configuration as possible.

Dan Abramov talks about this in his talk “The Melting Pot of JavaScript”. If you’re thinking about creating a new tool/library/framework I highly recommend checking it out.
Projects like create-react-app are a good example of this approach. react-scripts is the CLI used in create-react-app, and it exposes a minimal API to the outside world. It comes with subcommands like build, start, or test which internally map to the use of tools like Webpack, ESLint, or Jest. The nice thing is that the developer doesn’t need to care about the underlying technologies and how they are configured. It’s a huge productivity boost when you can just start writing React code immediately instead of spending time learning how to setup Webpack. Similarly, developers don’t need to worry about updating configs because there are no configs to update. When everything is encapsulated in another layer of abstraction like create-react-app all you have to do is update one dependency.

Of course, there’s a downside here as well. Taking away the power to customize the behavior of our tools means that they might not be useful in all places and not everybody will be happy with the choices we make. Every project is different, and there can’t be a one-size-fits-all solution. create-react-app is a very opinionated and takes this approach to the extreme. You may not be able to write your styling in SASS, you may miss out on some specific Babel plugin that you really like, but that’s the cost you pay to not worry about configuration anymore. There is, however, a way to provide great developer experience and still allow customization. Kent C. Dodds created something called paypal-scripts which by default works without any configuration but once you define config files, they take precedence over the internal ones. It’s a nice escape-hatch but also a bit dangerous. If more and more projects start to create configs to override your defaults, you might end up with the same problem you actually wanted to solve. It also gets harder to stay compatible and move quickly when your API surface area is not just a single command anymore.

Introducing blogfoster-scripts

blogfoster-scripts is our attempt at creating a “toolkit”, a thin wrapper around some of the tools we like and use a lot. It turns out that it’s not that complicated to write such a tool on your own. We got a working prototype of it ready in just a few days. You simply create an executable that wraps other tools by either using their Node API or running them in a child process. Now, to get started with a new project, all we have to do is to run npm install --save-dev blogfoster-tools. There’s no need to spend countless hours configuring tools again and again. We just install one dependency and have access to the following subcommands:

• lint — to check code for linting errors with ESLint
• format — to format code with Prettier
• build — to compile JavaScript and copy static assets with Webpack and Babel

It’s super exciting that we can be productive again with just the following package.json file:

{
  "scripts": {
    "lint": "blogfoster-tools lint",
    "format": "blogfoster-tools format",
    "build": "blogfoster-tools build"
  },
  "devDependencies": {
    "blogfoster-tools": "^0.2.0"
  }
}

Conclusion

Although it’s an early implementation and probably has some rough-edges (we’re missing a command to run our tests), I think just creating this project is a step in the right direction. For the first time in a long while, I feel like we took back control of our tools. We can fix any tooling-related problems in one go without worrying too much about how to take those changes into all of our projects. The only thing we’d need to do to then is just upgrade to the latest blogfoster-tools.

So next time you are starting a new project, I highly encourage you to go with tools that come pre-configured and don’t spend unnecessary time on your setup. If you’re interested in creating your own toolkit, either for personal projects or for your company, read through the react-scripts repository. The code is super easy to read and excellently documented. Also, check out what we did with blogfoster-scripts here on GitHub. Feel free to contribute to it, or if you don’t like the choices we made, just fork and customize it to your own needs. If you have any questions or other feedback, let me know and thanks a lot for reading!