Breaking the Web with JavaScript

A month ago Firefox tried to implement an ECMAScript 7 proposal for Array.prototype.contains and people at Mozilla quickly realized that it would break the Web. While rapidly declining, MooTools is still the second most used old-generation JavaScript library. What happened?

The Wild West of Building JavaScript Frameworks

First, to give some context, here is what the JavaScript community and ecosystem was like from 2006–2010. Browser development was slow. Microsoft just released the first new version of Internet Explorer in five years. ECMAScript 4 was viewed as a backwards incompatible version and due to its complexity it was abandoned in mid-2008. The JavaScript community as it exists today was still in its infancy – the first JSConf took place in 2009; CommonJS and NodeJS were started around the same time but it would still take more than a year to gain any significant traction. In early 2010, Google Chrome had a 0.10 % market share. The common way to install a JavaScript library was to manually download it from a website and drop it somewhere into your project.

I started using MooTools right when it was first released in 2006 – I was 17 at the time and I built games and worked on fun projects in almost all of my free time. The frameworks back then solved an entirely different class of problems compared to current libraries and since then many of them have become obsolete. There was this little group of people who was enticed by MooTools. We cared as much about the beauty of the code we were writing as we cared about the things we were building with it. MooTools was elegant yet powerful, it very much had Valerio’s personality encoded within it. Later complemented by Jan’s, Harald’s and Tom’s contributions, then Scott’s and mine and later Arian’s and some others – we all encoded our personalities within this neat little library, with many other contributors taking care of the things outside of the core and side-projects like Slick that Thomas and Fabio worked on. We had different generations of owners, each subsequent one learning tremendously from the previous one.

In our idealist world we saw MooTools as an extension to JavaScript rather than just a library on top of it. The same way that React now is becoming an extension of the DOM and JavaScript. This prophecy has become true, every single core component of MooTools is part of one standard or another, mostly ES6. The best abstraction is the one that doesn’t exist, which ties in nicely with Sebastian’s JSConf talk. MooTools is not a failed project, it very much succeeded in what it was set out to do – cease to exist altogether. MooTools was just an ECMAScript-future shim. This is why it was a great framework.

<Builtin>.prototype.* is verboten

With our ambitious long-term goal of shipping a library with no code in it we were required to modify the environment as much as we could. Eventually JavaScript engines would just fill in the gaps for us. We had global state everywhere, custom attributes on DOM nodes and most importantly we extended all native builtin objects, such as Array or String. Of course we didn’t know what would eventually become a standard but we attempted to implement the most beautiful framework core that should be part of every web programming environment.

There is two things we missed:

• When the update process for a library is manual, good luck on distributing that update all over the Internet.
• Extending builtins is the worst thing you can do if you don’t control the environment your code runs in.

Now, the first one can be resolved by being smart about package management and not always breaking compatibility. We had a package manager but it was hard to use and we didn’t encourage people to use it as part of their build system. In fact, most people didn’t have a build system for JavaScript back then. Our desire for writing clean and easy to read code lead us to rewrite the core features all the time and we broke compatibility. We ended up in a Python-like situation. It was the worst. There are still websites out there running versions of MooTools from 2008. We didn’t care about this at all. Just look at the breaking changes in these three versions: 1.1 to 1.2 and 1.2 to 1.3, ignoring all the subtle differences that made upgrading even harder.

MooTools was heavily inspired by another library called PrototypeJS. One difference was a simple function called Function.prototype.bind. One of the two versions eventually became part of ECMAScript 5 and it was not ours. Well, fuck. Now, how did we fix it?

In my memories it was my own ingenuity that came up with this insane hack but Sebastian and I actually came up with this solution together. This is the commit and it fills me with joy that the people who commented inline are all people who I work with every day now at Facebook. We solved this by overwriting the native implementation in the old version of MooTools (“our Python 2"). In the new version we used a simple variant of the standard but in compatibility mode we would fall back to the non-standard solution. This meant that you could run one version of MooTools with different implementations of a function based on a build-step setting. Oddly enough, this hasn’t caused any problems in practice, although I did imagine people would be gasping for air when they saw this. Yes, there was a lot of internal drama leading up to this solution but at that point in time, it was really the only thing we could think of to fix this and it was practical enough.

This is similar to what happened a month ago. This time the fix was much simpler, but given that most MooTools websites nowadays are mostly unmaintained the chances of them being upgraded at all are low – the MooTools team has not even released a new version. In fact, the previously mentioned super-old version of MooTools is still being used and has even caused trouble with React.

Move Fast with Stable Infrastructure

Extending built-ins is a great concept. If you control the environment your code runs in and you have the ability to deploy your code often, it should be used to fill in missing features to provide future APIs today. It is still impractical however – even if you control all parts of your application – you will never be able to control all possible browser environments. This is an interesting observation:

“While every version of a library we ship is final, the environment that library will run in is not. In fact, there are countless environments and they are all changing, constantly diverging and eventually converging again.”

Browsers have become the new wild west — prototyping potential standards in the wild with no opt-in/opt-out controls. If you have been using a polyfill for Promises for a while and some browser suddenly decides to ship a broken version of Promises then you are out of luck — you have to constantly stay on top of the game and watch out for this.

How do we avoid such problems in current frameworks and libraries? First off, always create wrapper modules, don’t use polyfills directly. These can feature-detect native implementations but it is better when the use of the native implementation can be toggled by a whitelist of browser versions with a known working implementation, ideally being controlled by a server side setting.

As Sophie said, “Constraints are liberating” in a talk at this year’s @scale. We have way too few constrains in JavaScript – but this is really important – with MooTools we didn’t know where the language was going but we were free to imagine it the way we wanted. More constrains on the browser side of things seem desirable at first but per-feature opt-ins would be terrible for complexity and fragmentation reasons. There is one way without actually removing some good parts of JavaScript like the ability to extend built-ins:

“Framework developers should create their own language based on future language proposals. It should be compiled to different versions of the implemented standard in different browsers.”

While we pretend we are using ES6/ES7 in React and at Facebook, we are really not. We have our own version of the language with custom extensions added on top.

If we constrain ourselves to only use language features that are in some way backwards compatible we can now prototype potential new features without first implementing them in the browser. New features can now be tested by framework developers, arguably the people who should be pushing the standards the most. In addition, it gives developers immediate access to new syntax features across all browsers.

If we also throw in a static type system we get the ability to add functionality to built-in objects. Instead of extending Map.prototype, one could transform function calls on Maps to wrapper calls. This requires a smart code-mod tool that can transform ASTs to ASTs that can also be used to quickly rewrite language modifications that don’t work out in practice.

As a community we shouldn’t be building JavaScript libraries to extend the standard library any more, but rather build dialects of JavaScript for experimentation.