Progressive Transpilation
In recent years, it’s been common for JavaScript (aka EcmaScript) programmers to use Source to source compilers, also known as “transpilers”, to convert a more enhanced or modern version of the language to an earlier version that can run on older browsers.
Transpilation: purpose
In the past, developers had to wait until all of the browsers they’re targeting would get the support for a particular language construct before they could use it. For instance, if you wanted to use the .map() method on an array, you’d have to make sure that every browser that loads your web page supports at least version 5 of the EcmaScript language. Otherwise, you’d have to include a “polyfill”, which is an implementation of the missing feature in terms of older constructs of the language.
When the EcmaScript committee began work on version 6, several transpilers became available which would transpile it down to version 5. This means that you could simply code as if all the browsers support version 6! Then, in your build process, you’d transpile your code down to a version of the language that the target browsers understand. Two of the most popular transpilers are Babel and TypeScript.
Worthy of note that “transpilation” is also used to convert source from other languages to JavaScript, such a CoffeScript, Dart or C++. In this article, we are only focusing on the version transpiliation aspect which gives us the ability to use newer versions of EcmaScript not yet supported by the target browsers.
Is this temporary?
When one of the most popular transpilers, Babel was first conceived, it was to convert version 6 to version 5 of EcmaScript. In fact, Babel’s original name was “6to5”. One might think that these version transpilers that convert from a newer version to an older version of the language are solving a temporary problem in that, once all of the browsers you care about support version 6 directly, the transpiler would not be needed. Although it’s true that once all the browsers that you care about support version 6 you no longer need to transpile to version 6, it is not the case that these types of transpilers will go away. The reason is that once version 6 becomes generally ubiquitous, version 7 will be standardized but not yet supported by new browsers. That means, that any given time, there will be some newer version of the language, along with new features, that you may want to use, but cannot directly. In fact, it was for this reason that “6to5” was renamed to Babel, to reflect it’s more long-term goals. So, no, version transpilers are here to stay.
Static and Dynamic transpilation
A transpiler is invoked similar to a compiler in that it is a tool that takes as input a set of source files and produces output. Generally, a programmer using a transpiler will set up their development environment such that they can write using new language features and have tools automatically transpile to the target version. Sometimes this integration is via the IDE, where saving the file automatically invokes the transpilation process for that source file. Other times external tools watch the file system for files that are changing and invoke the transpilation process behind the scenes. In addition, a build process, which may also minify and concatenate files will also transpile the code. All of these are considered static transpilation in that it is done during the development process. Once the browser is involved, it only sees the final transpiled version of the scripts, such as pure EcmaScript 5.
On the other hand, dynamic transpilation is when the development tools do not perform any transpilation. Instead, special code loaded in the browser detects that the source being served is of an older version and invokes the transpiler automatically. This is a nice feature during development, but not something you want to do in production. The reason you don’t want to do this in production is because the tranpilation is done every time the browser loads the page. You wouldn’t want every user to transpile your code every single time, since it can be done once, up front, ahead of time.
The problem with static and dynamic transpilation
In either case, there is a problem with transpilers: which version of the language do you target?
When EcmaScript 6 was new, most browsers hardly supported any version of the language, but all the browsers that most people cared about did support EcmaScript 5. So the question as to what version to target was simple: you target version 5 if you can target modern browsers, for instance.
However, as more and more browsers support version 6, you start wondering, when can we just start using version 6 directly, or targeting version 6 instead of 5, because we’re using version 7 features. First of all, it’s important to mention that there’s overhead in transpilation output. If you look at the EcmaScript 5 produced by Babel when the source language uses certain features, such as modules, you will see that there’s quite a bit of boilerplate code that gets inserted into the target scripts (similar to polyfills). As of this writing, “Chrome 50” supports most if not all EcmaScript 6 features you may care about. However, I wouldn’t necessarily be able to switch my transpiler to target version 6 because I am still targeting very old browsers, such as in old Android phones. Incidentally, these phones are still being sold, and their software isn’t always updated by the vendors.
What if only 5% of my customers are using browsers that require version 5?. Do you simply abandon those users? …Tell them to get a new phone. Or do you continue to target version 5 even if most of the customers are running browsers that don’t require the overhead? These are decisions that we’ll have to start making soon.
Progressive Transpilation
Progressive transpilation addresses this issue by proposing to serve a different versions of your applications scripts depending on the capabilities of the user’s browser. This differs from polyfills in that unnecessary code is not downloaded to the target devices.
There are several ways this can be done, each providing finer level of granularity at the cost of complexity.
First, can be easily described using the above example of the “5% android market”. Assuming you need to target some old phones that still support EcmaScript 5, the rest can be served EcmaScript 6. You could simply create two sets of source files and load the appropriate page depending “a client-side test” to determine the version of the language required, by way of a feature test. You might have a client side shim that loads a small script to make a browser-local feature-test. And based on that, dynamically load a different version of a script. It’s also possible the test is done up front, at the home page, and serve entirely different pages that have embedded within them direct links to the correct version in the script tags. This is similar to how a web site may serve mobile pages when it detects a mobile browser.
Now let’s make this a bit more sophisticated. What if there’s a wide range of browsers that you need to support, all with varying degrees of support for EcmaScript 6. And lets say that you knew up front all of the browsers you are going to support and exactly which EcmaScript 6 features they support or don’t support. You could run a transpilation process for each unique version of required compatibility and produce a different set of scripts for each browser type that will load your scripts. The newest version of the Babel transpiler has an “opt-in” model where you tell it which new features you are using. These are specified in terms “Transform Plugins”. So now, let’s assume you have 50 (or 100) or so versions of your scripts available. Every time one of these browsers attempts to load your web page, the shim checks the local set of available features and then requests the correctly target transpiled script that’s specific to it.
We’re already running into a major problem with this approach. In the previous example we just needed to test to see if EcmaScript 6 is supported or not. It would have been a single test. Perhaps it would be a test to see block-level function declarations are supported, and based on that assume EcmaScript 6 is available. If not, we assume none of it is available, and server EcmaScript 5. But what if to determine that a particular version is needed requires dozens of feature testing on the client side. That seems a rather time-consuming and wasteful process. And worse, we’re now sending more scripts to the client to do such testing.
No, what we really need is a way to map a particular browser version to a set of EcmaScript features it supports. Let’s create a script for progressive feature detection. The script does an exhaustive set of tests as to which EcmaScript features are available. The granularity of the feature-set would match the plug-in model for opt-in features for your transpiler, such as Babel. This tells you which set of Transform Plugs to select in the transpilation process for that browser. The result of the script is stored in a registry database and mapped to that browser’s “user-agent” strings.
Let’s assume we can magically run this script against all known browser versions out there. If we do that, then we know which browser version supports which features based simply on their user agent string. A script based feature detector may no longer be necessary since the user-agent string is passed to the server by the client during the initial web page fetch.
Okay, but how do you initialize the database of “features vs. user-agent-strings”? Any one with a unique version of a browser can visit a web page and run the tests for their versions, which then gets stored on the server (barring security issues). In reality, I think this would be done as part of “system testing” using a service such as BrowserStack. It’s also very possible that a global registry can be created and shared by everyone and populated by the community. Of course, every time a new “Transform Plug” is created, it needs to be tested against all the browser versions that we care about.
On the off-chance that a browser with a user-agent string visits your site that you do not have in the registry, you can simply assume that the browser does not support any EcmaScript 6 or later features at all, and send down a version of the script that only requires EcmaScript 5, or the least-common-denominator at the time. So the worse case scenario is actually no worse than what we deem to be state-of-the-art today.
