A Brief History of Web App Loading

Image from Wikimedia Commons

In the beginning, there was the <script> tag. We managed dependencies manually by carefully arranging our scripts in our HTML. You had to be careful to load jQuery before you loaded your plugins, your app code after your libraries. This started to get out of hand as we began building more interactivity and evolved from web sites to web apps. We needed a way to manage and share dependencies, because large projects had complex waterfalls of requests that were difficult to manage and optimize. We had defer and async attributes, but they only help in some circumstances.

The first step forward was when we began concatenating our scripts together. This reduced the total number of HTTP requests that needed to be made and helped guarantee execution order, but it remained a manual process. Scripts needed to be concatenated together in the correct order in order to work. We can concatenate different groups of scripts together to balance each file’s size against the total number of requests, but we had to specify the order and grouping. This is about the time that the concept of having a build step for your Javascript gained popularity, and CoffeeScript entered as the first popular alternate syntax.

Enter Require.js and Bower. Require.js introduced “asynchronous module definitions,” or AMD modules, a packaging method still used by some apps. They were loaded into the browser on-demand, which was super cool! No more manually shuffling script tags. The syntax was a little clunky,

// from http://requirejs.org/docs/api.html
requirejs(['jquery', 'canvas', 'app/sub'],
function($, canvas, sub) {
//jQuery, canvas and the app/sub module are all
//loaded and can be used here now.
});

but that’s fine! It’s much better than manually managing the order ourselves. Bower was initially a complement to npm, before npm had many modules that supported running in the browser. Eventually Bower was deprecated in favor of npm, and Require.js added the option of passing a require function to emulate commonJS modules.

So now we had something that automatically managed which scripts to load and in which order to load them. Life was good. Slowly, a new problem began to develop: it was so easy to add dependencies that we began to use a lot. Because each dependency was loaded as a separate script, loading a web app would kick off dozens — or even hundreds — of HTTP requests for tiny .js files, which would block each other from loading.

There were several fixes developed for this. The problem was taken into consideration for the design of HTTP2, which added multiplexing to help alleviate the problem. Require.js added an optimizer tool that would bundle up these modules up into a single file or group of files, but it wasn’t suitable for development and was tricky to configure. HTTP2 rolled out very slowly, and ultimately didn’t help as much as people hoped it would.

Developers began experimenting with alternatives, and the number of tools for bundling dependencies exploded. Browserify, Broccoli.js, Rollup, webpack, and surely other that I never heard about. There are still more being created, with Parcel being the most recent addition I know of. They all had slightly different takes on API and features. Ultimately webpack won mindshare for apps because of its excellent code splitting features and flexibility, and later iterations significantly improved usability. Rollup became the go-to tool for bundling libraries because it produced the smallest bundle in most cases.

This focus on tools for resolving dependencies revealed some shortcomings with CommonJS’ require function. require was created as part of Node.js, and had some semantics that made it more difficult to use in the browser. Taking lessons learned from CommonJS and AMD modules, TC39 standardized a module definition specification, ES modules, that better meets the different use cases in Node.js and the browser. It’s still evolving — Node.js recently released version 10 with experimental support, and the dynamic import() function hasn’t quite landed.

That brings us to today. I didn’t even touch on Yarn vs npm vs pnpm, services like unpkg, or any of the drama and arguments that got us where we are today. npm has taken off into the stratosphere after hitting a billion downloads a week in 2016, with the numbers at the beginning for 2018 dwarfing those. The challenges we have today are around when not to use dependencies, and keeping an eye on the total amount of code we’re shipping.

This is just a representation of what I’ve experienced firsthand in the past 6 years of writing code that runs in the browser. It’s a short period of time in the history of the web, but the amount of innovation and evolution has been incredible to watch.


I’m Carl Vitullo, I’m on Twitter as @cvitullo, and I help run a Discord server for React developers, Reactiflux, and another for node developers called Nodeiflux, where I’m @vcarl#7694.