AMD is not the problem, your Web tech stack is

The AMD debate is back, and I have something to say.

It’s half past 2013 and Web development is still a mess. Javascript still has no sensible way to import and export names by default. Still virtually no one is talking about how to package up Web components for the client-side and the JS community is as fragmented as ever.

First let’s begin with some history. Skip to the AMD section at the bottom for the TL;DR; version.

The <link> and <script> tags

For more than a decade since the introduction of CSS and Javascript, people have had to make do with manually managing CSS and Javascript dependencies by juggling <link> and <script> tags. This is not a problem at the beginning, since webapp was still not a word (my dictionary tells me it still isn’t, but it should be) and few people had websites with more than a dozen pages, most likely all include the same flat CSS file with 100s of rules, and a JS file with 100s of functions. People still had no idea how to write CSS and Javascript back then, let alone organize them.

But then things started to improve. CSS come out with an @import statement and is supported first in IE 5.5. Javascript has had virtually no improvement on this front even to this date.

In 2001, the Dot-Com Bubble bursted and Mr Crockford enlightened the world with his reverse-engineering/reverse-understanding of Javascript. At around 2004, 2005, Google released Google Suggest. The term Ajax was coined. Soon after, Prototype was released and bundled into Rails, then jQuery came along in 2006. YUI invented the module pattern also in 2006. We’ve entered the Web 2.0 era, where the browsers had started playing host to a new wave of Web apps that seek to emulate desktop apps. It is at that time that the amount of unoptimized CSS and Javascripts started to explode. It caused not only logistical problems, but also performance problems.

Steve Souders told us to concatenate and minify the CSS and Javascript files in 2007 to speed up the Web.

Now we have a new problem — how do we find out the dependencies of all the open-source Web components, linearize the assets into one big file for each kind, and pass them to the minifiers without doing too much work?

Enter asset management solutions.

Existing Solutions and their problems

Sprockets & Friends

In the years that followed, various server-side libraries, most prominently Sprockets, sprung up to help us deal with this somewhat, but there was still not a centralized place or agreed-upon method to distribute and consume web components. You still had to manually manage the file dependencies outside of CSS and Javascript mostly because these tools don’t understand CSS syntax, so they can’t linearize the CSS files just by following the @import dependency graph. The reason for that escapes me, but I suppose most tools chose to consume dependency metadata externally because they’d have to do it for Javascript anyway. Also, it’s 2013. Lots of newer webapps have moved most of the UI logic to the browser, some even have attempted to write single page apps that completely decouple the UI from the server. The Web needs a new asset management toolchain designed solely for front-end developers for an environment they are most familiar with — Javascript.

CommonJS & Node.js

A first true attempt at solving the module definition and importing problem is the CommonJS effort (Actionscript 3 notwithstanding). At around the same time Sprockets came about, a group of well-meaning renegades started the CommonJS and Node.js project. Finally we have a sane way to distribute and consume Javascript with require, exports and npm.

var sanity = require(‘sanity’);
export.happiness = “npm is awesome”;
export.doubleHappiness = “I can export more than 1 thing too!”;

require looks and works like the require/import statements from other languages you are used to. exports is a little weird, but no weirder than the static and extern keywords in C.

This should lead straight to a straight-forward implementation of a sensible asset management system completely decoupled from any server-side frameworks right? Not so fast.

Client-Side Web Components

At last, CommonJS is a server-side solution. The browsers don’t speak CommonJS. Specifically, while there are various solutions such as Ender and browserify and that can wrap CommonJS modules seamlessly for the browser, they are in some ways even worse of a solution than Sprockets because they are completely useless for packaging up CSS/LESS/SASS. CommonJS also currently does not define any package.json extension that allows components to expose anything other than Javascripts for other components to consume.

AMD & require.js

I’m just going to equate the two here because for all intents and purposes, require.js is the de-facto reference implementation of AMD.

Require.js is by far the most popular attempt to bring some order to the world of client-side asset management, but based on my couple of trials with it, I think it suffers a number of serious flaws which I think will render it a passing fad fairly soon. (Do correct me if I’m wrong because I’ve only tried a couple of times and I still feel I’m not totally comfortable with it)

  1. Too many ways to define or require a module. Too many patterns that are essentially work-arounds and too many options. Too much boilerplate code.
  2. Still can’t define a Web component that exports more than one kind of resource. You’d think with the number of options require.js gives you, you’d be able to do this, but it doesn’t.
  3. Hard to debug. This is the most annoying issue that I keep running into. require.js fails silently when there’s a circular dependency . Errors are often not detected until somewhere deep down the call chain where I reference that file. Often times, I can’t even find out which file couldn’t load or which file was trying to load it based on the error messages.
  4. Pointless asynchronous loading. This has a tendency to lead people believe that they could just require modules on the spot whenever and wherever, this is just wrong. This leads to callback hell and if you are not careful, lots of choppy incremental rendering on the screen. Making lots of requests also goes against common Web performance best-practices.
  5. r.js has yet more inscrutable configuration options. (50+ options for essentially a thin layer of asset management code on top of the minifiers, many options are duplicates of require.config).
  6. r.js doesn’t compile require.js into the output bundle by default. Why must the loader be separate?
  7. Reinventing the wheel of module definition syntax, but not compatible with CommonJS tools and ES6 syntax. This is the most important reason of why I think require.js will fade out in a few years time because it is significantly more complex than ES6 or CommonJS. Even with the ES6 to AMD transpiler, it’s still only a temporary solution until the libraries and tools catch up with ES6.

There are many more idiosyncratic problems with require.js based on my experiences, but I’m too lazy to list them all here.

The Community and The Way Forward

AMD is an admirable attempt to solve a number of issues plaguing client-side Web development, but it is also true that it remains an admirable attempt at best. Its complexity and deficiencies are actually a reflection of the state of affairs for front-end Web development — everybody has been busy reinventing the wheel, nobody asks what the wheel is suppose to roll on, and nobody can agree to combine their efforts to work on anything.

What this ends up with is of course a number of mutually incompatible, half-solutions with varying community sizes.

Is There a Solution?

Well, in the long term, I think the solution is definitely for the browser vendors to agree to what a Web component constitutes of, and come up with a cross-platform packaging solution that enjoys an overwhelmingly wide acceptance. Ideally, visiting a URL will automatically read a dependency metadata file, download the dependencies with WebRTC peer-to-peer, or load them from the cache. This way multiple websites/apps can share library resources. Essentially, a distributed CDN + NPM on steroid, on the browser.

In the medium term, I think quite certainly that ES6's module syntax needs to be finalized as soon as possible so browser vendors can start shipping it, and developers can solve half of the problem first.

Short Term Band-Aid

As far as I know there is only one tool that is close to a solution to packaging Web components — Component. It not only can manage and build the assets for you, it also has a clear vision and definition of what a Web component should consist of. It also has a number of fairly well done plugins that can manage your LESS and SASS files and more. While I’m still waiting for it to support CSS sprites/icon fonts, and image optimization, they shouldn’t be a serious enough problem to hold you back. If you have a large modern day webapp with lots of static assets, I encourage you to check it out.

Like what you read? Give Jimmy Yuen Ho Wong a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.