Scripting Bits
Published in

Scripting Bits

Re-Evaluating My Node.js Stack

It’s been a while since I’ve last done a new Node.js project. I usually work on browser-based applications supported by a server and build and test tooling. Sometimes, I work on tooling that tries to improve the developer experience of such applications. Sometimes, I distill libraries that are reusable across such applications.

Since React was released, web application developers have been locked into a single tooling stack. React applications would be compiled using webpack and Babel for the browser and also compiled using Babel for automated testing; naturally, any server-side code would be compiled using Babel as well. This led to a lot of confusion over the many moving pieces that needed to be configured and sped up.

The JavaScript ecosystem has come a far way since then, and new options have arisen. Today, browsers and Node.js support more and more of the new JavaScript features (although they’ll never support JSX), which reduce the need for compilation and bundling. CDNs like unpkg and alternative compilers like Snowpack fill in the gap of getting npm packages to work in the browser. Alternative runtimes like Deno go as far as to execute non-standard code natively.

I think that now is a good time to reassess all of the moving parts I’ve been keeping around and see which parts I no longer need.

Type Checking

I’ve been using Flow since its first release. It’s very useful for finding errors in data-intensive libraries, above and beyond automated test cases. I’d say that it becomes more cumbersome than useful for simple applications or for any code that has a lot of side-effects.

I think that any usecase complex enough to need type checking should leverage a language with a really good type system, like ReasonML. That code can be distributed as a library for consumption by applications whose code is simpler and doesn’t require type checking.

On the other hand, TypeScript, aims to provide a lot more than just type checking. It also provides tight integration with editors and enables refactoring and code navigation. Deno even natively executes TypeScript.

Writing an NPM Package with Minimal Tooling

So, I’m going to try rewriting one of my libraries with less compilation tooling and see how it shakes out.

I’ll rewrite passing-notes, which is a library that provides a side-effect free middleware abstraction for serving HTTP requests.

Using Native ES Modules

The process is pretty similar to what I’m used to. After skimming through the Node.js documentation on ES modules, I do the usual:

  • Run git init
  • Create a README.md
  • Create a package.json:
{
"name": "passing-notes",
"version": "0.0.0",
"type": "module",
"dependencies": {},
"devDependencies": {},
}

The notable inclusion is the "type": "module", which instructs Node.js to treat files ending in .js as ES modules. A lot of tools have the implicit assumption that JavaScript files end in .js (or .jsx).

After that, from a different codebase, I’m able to install this new package by running:

yarn add file:../passing-notes

No Compiler

Previously, testing packages locally was tricky. I would have to think about how and when to compile the code so that it could be consumed from a different location with a potentially different compilation workflow.

@babel/register is the main way to compile code in Node.js. It’s relatively easy to configure compared to Webpack and imposes a much smaller, but still noticeable performance hit. In the past, I’ve tried to use @babel/register in multiple threads, but the performance degradation became pretty annoying. Eliminating the need to use it altogether is nice.

Hot Loading

When I’m developing server code, I enjoy being able to modify my code and try it without having to restart the server process. Depending on the codebase, start up time can be prohibitively long.

I always try to find a seam, usually in between jobs or requests, to re-import potentially changed code. Because Node.js caches imported files, if I just invalidate the cache for files that change, I can achieve pretty efficient hot loading. Only files that change, and the files that depend on them need to be re-executed.

Previously, to implement this, I could just delete the relevant entries in require.cache. With ES modules, Node.js no longer exposes a cache object. They do, however, provide hooks that allow us to control the module importing workflow. In short, I can modify the URL of files that have changed, adding a ?version=1 query string paramter that I increment with each change.

I inject the following loader by running:

node --experimental-loader ./loader.js server.js

There are a lot of moving parts in which I implement and keep track of a dependency graph as well as the current version of each file. But, the salient bits are on line 66, where I build the dependency tree, and line 42, where I do the actual invalidation of changed files via breadth-first traversal of the dependency graph.

Because the --experimental-loader flag takes a string that seems to be fed into an import statement, I should be able to package this loader into its own npm package.

Automated Testing

I expect that, since I no longer need to configure a compilation step, it’ll be much easier to use any automated test runner.

My test runner of choice is AVA.

It’s always interesting to add tests for the first time to a piece of code. Having the additional constraint of being to reuse a piece of code within a test case drives many good coding practices.

Where previously I only needed to start an HTTP server in one place, now I need to also start a server in a test case. And, since I may want to write multiple test cases, I need to be able to configure the port.

Now, my code looks like:

AVA works out of the box with no configuration. It starts up and runs the one test in 0.8s.

Linting

Nowadays, all of the popular linting tools descend from ESLint, which is essentially a plugin-based framework for formatting code. Lots of people have published their own plugins and configuration files for ESLint.

In my experience, it’s pretty cumbersome to have to remember 10 different plugin packages to install every time I start a React project. That’s why I moved to Standard, which bundles ESLint together with an opinionated set of plugins. Eventually, I wrote standard-esnext, which bundles Standard together with a bunch of other plugins, including Prettier.

Today, I’ll try out XO, which is more of the same.

Enabling Prettier and getting it to allow file extensions in my import statements require the following to be added to my package.json:

"xo": {
"prettier": true,
"rules": {
"import/extensions": "off"
}
}

Configuring it to look more like what I’m used to requires:

"xo": {
"prettier": true,
"space": 2,
"semicolon": false,
"rules": {
"import/extensions": "off"
}
}

Overall, not bad.

Less Tooling While Developing

In past years, I’ve focused on making the compiling/bundling tools we have to use, easier to use. After all, while we develop code, we have to interact with the compiler/bundler once every couple of minutes.

Now, with ES modules natively supported both in browser and Node.js, we can reduce the number of tools we have to contend with, at least during development.

Of course, to maintain compatibility with older JavaScript runtimes, we may need a compilation step prior to release. But, the frequency of releases are on the order of days or weeks.

Overall, I’ve gathered a lot of insight into the current possibilities. I think I’ll have a go at actually rewriting passing-notes, and moving it over to this new stack!

--

--

--

Join me in my short coding experiments where I dive into the rabbit hole of what-ifs!

Recommended from Medium

An event-driven MVVM alternative to the Flux front-end architecture

Three No-Code JavaScript Libraries You Can Use In Every Project

How to Develop a Native PGP Encryption Suitelet in NetSuite

Javascript prototypal model and inheritance

Laravel Advance | Delete record using ajax request in Laravel Example

What is the Difference between foreach and forelse in Laravel

How To Build A Slider In React

Kendo UI for React in 2017

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Vinson Chuong

Vinson Chuong

More from Medium

Remix VS Next JS

Pith instructions to Deploy a Next JS application to your own server

Just Another NextJS Guide

Why Next . Js