SmooshGate: The ongoing struggle between progress and stability in JavaScript
On March 6, 2018 at 1:17pm PST a bug report was filed on Mozilla’s Firefox nightly build about a broken 14-day forecast on German weather site wetteronline.de. In the ensuing days, this very faint and seemingly routine canary in the coal mine would set off an increasingly loud and contentious chain of events in the JavaScript community, played out in Hacker News comments, GitHub issues, and not-so-subtweets. Wounds that have never completely healed in the ongoing battle between the need to not “break the internet” and the desire to advance the JavaScript language were ripped open as people argued, often with themselves, about how to move forward in a world where we are forever tied to the decisions of the past.
TC39–the committee that decides the future of the JavaScript language specification–has a difficult job to do. Software written in JavaScript and executed in the browser does not have the luxury of deciding what version of the language it prefers. No, browser users (all 4.1 billion of them) decide which version of JavaScript to run a website in when they download a browser. On top of this, modern browsers self-update bringing new features to the language without users ever being the wiser.
Contrast this with developing a server application. I, as the author of a server application, can choose and freeze the version of Node that my application runs in. Though the language may advance, my application can continue to exist in perpetuity using the version of JavaScript that existed at its publishing.
Thus TC39 must thread the needle of advancing the most popular programming language in the world while not losing sight of software that was written more than a decade ago.
Standardization of the general purpose, cross platform, vendor-neutral programming language ECMAScript. — ECMA Technical Committee 39 Scope of work
This particular needle surrounded the stage-3 proposal for a new array method calledArray.prototype.flatten. The method would take nested arrays and “flatten” them into a single array.
const myArray = [1, [2, 3], [4]];
console.log(myArray.flatten());
// [1, 2, 3, 4]This functionality is fairly intuitive and has precedent in other languages. So predictable was this feature that the authors of the JavaScript utility library MooTools anticipated it nearly a decade before it would be introduced into the language. The problem arises from the method by which they did this anticipating.
MooTools, and other libraries, popularized a method of extending native JavaScript types by using a technique often called “monkey patching”. This is now widely considered to not be a Best Practice™ (and there were plenty who advised against this at the time), but nevertheless the technique was popular and code that relies on it is still running in browsers the world over.
To explain how monkey patching works in JavaScript, let’s look at example. Imagine we want to augment our JavaScript runtime to allow us to capitalize the first letter of a string. We could do this by defining a simple utility function:
const capitalize = (s) => s[0].toUpperCase() + s.substr(1);However, we then need to refer to this function anywhere in our program when we want to use it.
const capitalName = capitalize('jacob');In the age of ES modules and sophisticated build tools, this is not a problem. In a time before these things (like when MooTools was first created) it was much more portable if the functionality was attached directly to the native String prototype.
String.prototype.capitalize = function() {
return this[0].toUpperCase() + this.substr(1);
}Now, anywhere I define a String, I can also capitalize it:
const capitalName = 'jacob'.capitalize();It is easy to see why this method is convenient. What is maybe less obvious is why it could be dangerous. For one, any library can modify the prototype. This means if another library I am using also implements a String.prototype.capitalize method, but does so slightly differently, it could cause unexpected behavior in my application.
Note: monkey patching at will shouldn’t be confused with the more common practice of polyfilling, which uses the same technique to add specification features to runtimes in which they don’t yet exist (for instance, old browsers).
The real trouble doesn’t come from the fact that MooTools monkey patched Array.prototype.flatten alone, but from the fact that their implementation differs from the one that arrived as a draft in the JavaScript specification 10 years later in one key way.
MooTools’ version of the flatten method defaults to infinite depth flattening. The authors of the specification on the other hand, anticipating performance concerns and common use cases, defaulted the method to a single level of flattening. This implementation difference is what was thought to have materialized as a bug on a forecast weather widget when using the Firefox nightly build and in turn ignited a passionate conversation amongst professional JavaScript developers.
The problem was evident, the solution, far from it.
1. Break the internet
The first potential solution is the one that feels right to most who have weighed in on this topic: ship Array.prototype.flatten as is, old MooTools sites be damned. Why, they argue, should the progress of a language be beholden to the (perhaps ill advised) decisions of a library created over a decade ago? Developers who chose to indulge in this naughty behavior will now have their chickens come home to roost and will have to spend the time updating their sites.
There are a couple of potential problems with this line of thinking. First of all, many of the sites that a change like this would impact are not actively maintained. There is no developer that will go in and update the version of MooTools. Instead, the people who are punished by the decision are the users of the websites that will break.
This leads us to the second reason why this scorched-earth approach might be problematic for the TC39: a specification is only valuable when it is implemented. Any change to the language spec that would cause breakage in a non-negligible number of sites might cause browser vendors to refuse to implement a feature for fear users might flee their platform. Once TC39 loses the browsers on a single feature it is a slippery slope back to the wild west of non-compliant browsers making life extremely difficult for front-end developers.
2. A Great Renaming
What really made this whole ordeal go viral was the suggestion from Michael Ficarra to rename the proposed flatten method to the light-hearted name: smoosh. The reaction to this suggestion was swift and predictable
There are many good arguments as to why this is a bad idea aside from the somewhat absurd sounding name that has no basis in other programming languages and is not at all obvious to non-native English speakers.
The most convincing of these arguments goes something like this: if we codify a policy that demands we never break any library that some non-trivial number of sites are using, then anyone can create a library and sit on the global namespace. This means that the language will constantly have to dance around libraries that are engaged in “bad behavior”.
To illustrate this point, RxJS developer André Staltz did just that by creating a package that monkey patches Array.prototype.smoosh itself.
There may be room to compromise in this solution though. There are, for instance, other names like flat or unwrap that both don’t conflict with MooTools and also have some precedent in other languages.
3. “Backwards Compat Hack”
Another class of solutions is what might be dubbed “backwards compatibility hacks”. The name alone probably sets off red flags, but this is probably about as close to a compromise as can be made.
To understand how this solution would work, we need to dive a bit deeper into why MooTools of >8 years ago breaks, but newer versions of MooTools do not. All versions of MooTools patch flatten in the same way, but until 8 years ago this patching was conditional. That is, it would only patch this behavior if it wasn’t already present on the prototype.
if (!Array.prototype.flatten) {
Array.prototype.flatten = function () { /* Do flatten */ };
}This means that once the browser supplies this function, MooTools would no longer patch it, but the applications using MooTools would expect it to have infinite depth recursion.
The most obvious “compat hack” is to simply state in the ES specification that, though flatten is a function, it should be falsey when used in a conditional. This way MooTools would think that flatten was not defined and thus would patch it with its own behavior.
Though this behavior may seem gross and unintuitive (and it is), it is not without precedent. In fact, there is even a term for this type of object: undefined-like exotic object (ULEO). The primary (only?) example of one of these in the wild is document.all. document.all is an HTMLCollection, but when it is coerced to a boolean it is false (relatedly, its typeof is undefined). The reason for this is twofold: 1. though document.all was never part of the HTML specification it was used in many legacy applications written to target old versions Internet Explorer and 2. many legacy applications would use its “truthiness” to determine whether or not the code was executing in IE.
if (document.all) { /* run IE specific code */ }If document.all were removed altogether, it would break applications in category 1. If instead it were a normal object, it would break applications in category 2. Thus, the ULEO was born: an object that acts as if it is undefined.
For obvious reasons, many are hesitant to jump to this somewhat extreme “hack” to solve this problem. But there are potentially more clever approaches that might be more palatable. For instance, Teddy Katz has a proposal for a flatten implementation that would only be defined if the JavaScript is executing in “strict mode”.
4. Compromise on Functionality
Finally, another potential compromise would be to simply change the specification’s default depth to match that of MooTools. On its face, this seems like both a clean compromise and a reasonable tradeoff. However, as you might imagine, there are detractors from this argument as well.
Some might argue that if changing the name is surrendering to past poor decisions, changing the functionality to match those same decisions is a much worse and impactful concession. That being said, in this particular situation the default depth is something that was debated prior to this issue coming to light with plenty of reasonable people on either side.
To further complicate this entire ordeal, as people dug further into the issue, it was discovered that the implementation difference was not actually what caused the original bug. Yes, it is likely the implementation difference would cause bugs, but there were other issues with how MooTools patched native types that would also have to be dealt with.
Ultimately, what is at play here is something bigger than Array.prototype.flatten. In fact, it is bigger even than the discussion of how to navigate implementation of language changes in an ecosystem with over two decades of history and executing code. The reason that this argument struck a nerve boils down to frustration over how the language is governed and voicelessness some feel they have in that process.
Though it has historically been a less-than-transparent organization that has at times thrust its decisions upon the lay developer, this discussion shows that it is headed in the right direction. The smoosh saga is not over and how it is handled will set the tone for the next era of JavaScript decision making; let’s hope it is one that we can all have a voice in.
Hello! I’m Jacob Friedmann, a software engineer working on the front-end platform team at Rover.com. I write about React and React Native. I love teaching about JavaScript and Front-end development and have led classes at General Assembly. I enjoy talking about, building with, and diving deep on JavaScript topics of all kinds, so reach out if you do to! I have an amazing pup named Bogart and live in Seattle where I can indulge in all of my hobbies: exploring the PNW 🌲, eating Pho 🍲 and drinking all the coffee ☕️.
