The future of web deployment without bundlers or compromises — ES Modules, NodeJS and HTTP/2 Push
Next generation web development
Introduction
Every time I create a new project for a web application I start by setting up a bundler. Even though bundlers have come a long way since the early days I consider them a hassle. In my experience webpack configs tend to get larger and more complex as the project progresses and over time require more in-depth knowledge of webpack and the project itself.
Setting up and maintaining bundler configs is my least favorite part and for some time now I was looking for ways to deploy web applications without bundling like I can with NodeJS applications.
ES Modules to the rescue
Nowadays most browsers support ES Modules or JavaScript Modules, making it possible for a JS file to use the import/export syntax, similar to the NodeJS support for ES Modules. When a module imports another module, this will result in an HTTP request. After all imported modules are loaded from the server and executed, the importing module can be executed, which opens up the possibility to deploy web applications without bundling them first. Here is a good example of how such an application could look like:
Open DevTools and have a look in the network tab — you will see how every JS file is loaded separately. If you look at the Initiator column you can see which module requested the loaded one and even see the explicit import statements.
The problem with loading ES Modules
An attentive observer might have noticed in the above waterfall diagram, that some modules are loaded in parallel while others are loaded sequentially. This is due to the fact, that the browser finds out it needs to load todo.js after it has loaded and evaluated (not executed!) app.js. Ultimately this means that every row of the dependency tree is loaded sequentially, delaying the app execution by one RTT (round trip time) per layer. In comparison: because bundled applications are loaded as a single file there are no multiple loaded files and therefore no additional delays.
NodeJS resolving our problems
A couple of months ago I had a closer look at how NodeJS does its module loading, for commonJS and ES Modules. CJS modules are executed top-down, meaning the entry file is executed first and when execution hits a require statement it will pause execution and execute the loaded module (if not already loaded and present in cache). ES Modules are fundamentally different as they get executed bottom-up. This means, that NodeJS has to recursively build up the whole dependency tree, before it can start executing code. This got me thinking — can we use the dependency resolving mechanism without having to execute the actual code?
Well, yes! Long story short: I dug into the NodeJS codebase, connected some dots and managed to implement a function that returns all dependencies for a given ES module. If you want to see how I did it, have a look here.
Preload the dependencies
Now that we know of all needed dependencies we need to somehow get them to the client.
HTML Preload
One way would be to use the HTML Preload feature. We would need to inject all dependencies as preload instructions in the HTML header and let the browser load them before the JS runtime knows it needs them. Unfortunately this approach is not able to solve the problem of dynamic imports and only works for browsers, not for JS runtimes like node or deno. This leaves us with HTTP/2 Push.
HTTP/2 Push
Basically HTTP/2 Push is a mechanism to deploy (push) resources to the clients cache before the client even knows, that it needs them. This fits our needs perfectly and lets us push the dependencies preemptively to the client, so they are already present when the JS runtime realizes that they are needed.
Bring it all together
Now that we have resolved all issues we can build an application that uses ES modules without the penalty of additional delays. I have built an example implementation. You are welcome to have a look and play around with it:
This is how ES Modules would be loaded classically. Again we see how all modules are loaded one after another.
Here you can see how the application loading changes if we push the dependencies. Modules A and B are loaded together with main instead of sequentially. dynamic.mjs is imported imperatively by main, but imports C declaratively, so C is pushed when the dynamic module is requested.
Summary
We use ES Modules so our runtime can load and execute our application files unbundled. To avoid loading delays we let a NodeJS based server resolve the dependency tree beforehand and use HTTP/2 Push to deploy the dependencies to the clients cache.
Conclusion
ES Modules support is spreading and they are here to stay. Resolving dependencies on the server-side and pushing them is just the next logical step towards a new generation of web applications and will bring us closer to a more universal JavaScript across browsers and runtimes. I believe with the rise of dynamically executed applications (deno example) this technique will become the standard way to serve ES Modules in the future.
