Selective transpilation for modern JavaScript environments
Background
This post is a fairly deep dive into some advanced transpilation configurations using Gulp, Babel, and SystemJS; the end goal is to ship native ES6 to environments capable of running it, and transpiled ES5 everywhere else.
I’m basically going to outline what it was like setting all of this up, which, tl;dr, was far more work than I imagined, for a very, very modest benefit.
Target audience
This post is already long, and explaining every tool and piece of code herein would make it unbearably so. If you’re new to these utilities, I’d urge you to start more simply. Set up some basic transpilation tasks with introductory tutorials as your guide, set up some basic gulp pipelines, and build incrementally. Trying to implement what’s in this post without a decent, working knowledge of these tools will likely end in unhappiness.
Viewing the code in this post
All the code below is from my side project here. I’ll reference code snippets by the file it’s in, so curious readers can find the full, updated code in question if desired.
Goal
With more and more environments supporting ES6 in full (minus modules, of course) it should in theory be possible to transpile only the experimental or non-standard ES features you’re using, and then ship actual ES6 to your users on modern browsers. Of course you’ll also want to run a pre-release process which transpiles everything to ES5 bundles for older engines like Internet Explorer. Lastly, when your app loads, you’ll need client-side JavaScript to detect whether the browser your user is on is ES6 compatible, in order to load either the ES6, or ES5 bundles.
The advantages of doing this include debugging against simpler code, smaller code being shipped to your users, and perhaps a warm and fuzzy feeling that comes with doing something new and cool.
A note of caution.
ES6 is still new, and browsers may still have some kinks and bugs, to say nothing of the fact that these features may not be fully optimized yet. Oh, and ES6 support for minification is not done yet; the UglifyJS2 harmony branch is not released, and does not yet support all ES6 features. I didn’t realize this when I started, in June 2016, but by the time I realized I was already in fairly deep, so figured I’d finish. So the end product will only minify the fully transpiled “IE” bundles. Given these factors, I would urge caution before attempting any of this on code that matters; I don’t plan on shipping ES6 professionally for at least another 6 months.
Babel 6
Babel 6 is fully and beautifully configurable. It’s slightly less convenient to use out of the box than Babel 5, but it’s certainly for the best. I had previously struggled with appreciating these upgrades, but I was wrong — mea culpa.
I won’t belabor configuring and installing all the presets and plugins. The docs are pretty good, and plenty of other people have written at length about this.
CJS / ESM interop coming from Babel 5
If you’re still running Babel 5, be forewarned that upgrading to Babel 6 may introduce many bugs, from how CJS and ES6 modules interoperate. I haven’t researched exactly what the causes are, but it seems Babel 6 turns an ES6 module into an object, with properties representing the exports. This causes problems if it’s `require`d later — for example React won’t be happy to get an object with a default property on it containing your component, rather than just the component. Just use ES module syntax everywhere.
Decorators
Decorators are not really supported in Babel 6, since the proposal is in flux. As such, if you’re using them for anything, like I am, then either stay on Babel 5, or be prepared to have a separate transpilation process just in the location where you’re using decorators — as I do for my controllers directory
Transpiling for Node 6
Node 6 ships (essentially) with full ES6 support! The future is now!
So I upgraded to Node 6.
Since I’m using object spread and async, my initial thought was to just configure Babel to transpile those experimental features, and ES6 modules, and leave everything else alone. My configuration looked like this (this is from gulpfile.js at the root of my project)
.pipe(babel({
presets: ['stage-2'],
plugins: ['transform-es2015-modules-commonjs']
}))Let’s see how that worked out. This code
class SubjectDAO extends DAO {
constructor(userId){
super();
this.userId = userId;
}
async loadSubjects(){
let db = await super.open();
try {
return await db.collection('subjects').find({ userId: this.userId }).toArray();
} finally {
super.dispose(db);
}
}
}Becomes
class SubjectDAO extends DAO {
constructor(userId) {
super();
this.userId = userId;
}
loadSubjects() {
var _this = this;
return _asyncToGenerator(function* () {
let db = yield super.open();
try {
return yield db.collection('subjects').find({ userId: _this.userId }).toArray();
} finally {
super.dispose(db);
}
})();
}
}Ostensibly it looks great. Our class was left alone, and our async function was translated into a nice generator, with a thin co wrapper to process the yields. But look closely: do you see the problem?
It’s here
let db = yield super.open();super has no meaning inside the generator. This is a known issue with Babel, and it’s really not their fault; we told Babel to not transpile our class, and so it didn’t.
The only real fix is to transpile the whole class. At this point, given that async wouldn’t transpile, and since I was getting graceful-fs deprecation warning everywher, I just downgraded back to Node 4, and decided to transpile everything for Node.
The future’ll have to wait a bit.
Now my babel call looks like this
.pipe(babel({
presets: ['stage-2', 'es2015']
}))We’ll still be selectively transpiling for browsers, just not for Node; gulp makes this easy — stay tuned.
But first, back to our async method. With full ES6 transpilation, our transpiled async code now looks like this
var SubjectDAO = function (_DAO) {
_inherits(SubjectDAO, _DAO);
function SubjectDAO(userId) {
_classCallCheck(this, SubjectDAO);
var _this = _possibleConstructorReturn(this, Object.getPrototypeOf(SubjectDAO).call(this));
_this.userId = userId;
return _this;
}
_createClass(SubjectDAO, [{
key: 'loadSubjects',
value: function () {
var ref = _asyncToGenerator(regeneratorRuntime.mark(function _callee() {
var db;
return regeneratorRuntime.wrap(function _callee$(_context) {
while (1) {
switch (_context.prev = _context.next) {
case 0:
_context.next = 2;
return _get(Object.getPrototypeOf(SubjectDAO.prototype), 'open', this).call(this);
case 2:
db = _context.sent;
_context.prev = 3;
_context.next = 6;
return db.collection('subjects').find({ userId: this.userId }).toArray();
case 6:
return _context.abrupt('return', _context.sent);
case 7:
_context.prev = 7;
_get(Object.getPrototypeOf(SubjectDAO.prototype), 'dispose', this).call(this, db);
return _context.finish(7);
case 10:
case 'end':
return _context.stop();
}
}
}, _callee, this, [[3,, 7, 10]]);
}));
function loadSubjects() {
return ref.apply(this, arguments);
}
return loadSubjects;
}()
}]);
return SubjectDAO;
}(DAO);`super` is now gone, and everything is translated to valid ES5.
Selective transpilation
gulp-if for selective transpilation
So how do we transpile a file one way if it’s a Node file (since it potentially has async methods, needing full transpilation), and another way if not? Since I’m using gulp, I used gulp-if.
function isClientFile(file){
return file.path.indexOf('\\react-redux\\') >= 0;
}// .......pipe(gulpIf(isClientFile, babel({
presets: ['stage-2', 'react'],
plugins: ['transform-es2015-modules-commonjs']
}), babel({
presets: ['stage-2', 'es2015']
})))
The check is extremely crude, obviously; your mileage may vary. If the file is from anywhere within react-redux, it’s a file for the browser, and so we’ll transpile stage 2 (and higher) experimental features, react jsx, and of course ES6 module syntax. If it’s a Node file, we’ll run full ES6 transpilation to make sure async class methods are transpiled, and also still Stage 2.
Creating our bundles
I’ve previously written about SystemJS, a module loader / bundler I use extensively. Essentially, this stack gives you free module loading out of the box, but leaves you to put together your own bundling process. Webpack is sort of the reverse: it requires a build-step up front which you configure, but the build-step for production is more or less built in (likely just some configuration changes). I’ve found success with SystemJS since I like running with no build step, and I, for some strange reason, enjoy hand-crafting my own build processes.
So all of our (browser) code has been transpiled with just the ES6 modules, stage 2 features, and JSX converted; we’d now like to bundle for production. We need one set of bundles for modern browsers, and another set of bundles for older browsers, which we’ll pipe through babel with full ES2015 transpilation.
The relevant code is here — note the full transpilation connected to the first call to gulpIf, in conjunction with lazypipe. Also note the conditional minification which is applied only when full transpilation happens; I could have just attached this to the lazy pipe, but at some point uglifyJS2 should be updated to be compatible with ES6, at which time I would be able to just remove the gulpIf check and minify everything.
Basically, I’m creating my bundles twice, once with no transpilation, and once with full transpilation, and sending them to dist-es5 or dist-es6 directories. If we are doing the full transpilation, then I also apply minification. It’s worth noting that the SystemJS bundler is pretty fast, which is why I didn’t bother to go through the trouble of making the bundles only once, and copying them to the second directory to (only) transpile there.
The full code is in react-redux/build/build-es6.js
Running the bundles
The rest of the build code gathers up the names of all the modules in each bundle (SystemJS Builder gives you that in the resolution of the bundle call) and then pieces together a basic mapping file in the format SystemJS wants. It looks something like this
var gBundlePathsTranspiled = {
'dist-bundles/modules/scan/scan-build.js':
['modules/scan/scan.js', 'modules/...'],
'dist-bundles/reactStartup-build.js’:
['util/responsiveUiLoaders.js', 'util...']
}To configure SystemJS to use bundles you give it a map like the above, where object keys point to the bundle, and the corresponding array of modules tells SystemJS which modules are contained therein. If any of those modules are requested, SystemJS knows to grab the appropriate bundle. Rather than keep two sets of bundle maps, and apply the one or other based on ES6 support, I just made one, (the bundle names and contents are identical for ES5 or 6), with the fake name of dist-bundles. Later I’ll use the ES6 feature detection to simply map the path dist-bundles to the right address. Did you catch all that? If you’re thinking this is more work than it’s worth, either stay tuned or just skip to the conclusion where I say basically that.
The client-side SystemJS configuration looks like this
I do a crude check to see if I’m running locally (ie localhost). If I’m not (ie I’m deployed and running live) then I use Kyle Simpson’s featuretests.io to see if the browser supports ES6 (and I give it half a second to finish, otherwise I just assume no, and move on).
Then I load the right SystemJS configuration (production configuration has minified react builds, and so on) and map dist-bundles to the appropriate location. Obviously if we’re local then we don’t set any builds, so each individual file can be loaded as needed, without needing to run a build after each edit.
Does it work? Yep. Here’s some valid ES6 running right in Chrome. Notice the class definition, with concise method syntax.
Please don’t judge SystemJS by all of this
SystemJS is usually much, much less work to use. My build process was significantly simpler before I decided to implement dual ES5/6 bundles. Which of course raises the question of whether any of this was worth it…
Conclusions
It was a lot of work.
For my setup, I needed to pipe my code through two different babel pipelines depending on where the code lived, which I was using to crudely surmise which experimental features it (likely) contained, to say nothing of the old-school babel 5 process to which I alluded, for my decorators.
And of course we still need to run the browser bundles through a full transpilation for non ES6 browsers, which led to bootstrapping / ES6 feature detection, and dynamically directing the module loader to the right bundles location.
Is it worth it? What are the savings?
A 90KB ES5 bundle shows up as 78KB un-transpiled, for a meager 12KB savings. And these are unminified numbers (you can’t minify ES6 at the moment, remember), and so the 12KB savings would surely shrink once we add minification and gzipping.
I imagine Webpack users will have a slightly easier time of all this: if we pretend for the moment UglifyJS2 was ES6 compatible, I imagine Webpack users would simply re-run their build, with different transpilation settings, and then figure out a way to dynamically switch your WebPack’s entry point.
Takeaways
Think long and hard before taking the time to do this. I’m glad I did; it was interesting and I learned a lot. But it’s hard to imagine this paying off with enough value to justify the effort (unless of course the effort is to see what it’s like, and play with JavaScript build tools, as it was for me).
Of course in the reasonably near future it’s easy to imagine modern engines (and Node) supporting these Stage 2 features, and UglifyJS2 should be up to date even sooner. And it’s not outrageous to imagine IE usage dropping low enough in the next year to not support it for some apps. So this should get a bit easier as time passes.
Do what’s best for you
Do whatever’s best for your application. If your existing babel 5 transpilation process is working great, and you don’t feel like disturbing it, then don’t. At my regular job we’re still on Babel 5. That said, our application does not use React, we use CJS (with some old AMD still lingering) instead of ES6 modules, and we’re using no experimental ES features at all: as a result, we’re in the process of shutting our transpiler off altogether, and coding directly in ES6 during development, with a simple Babel 5 transpilation task producing ES5 production bundles for all users. I likely won’t trust browser ES6 implementations / UglifyJS’s ES6 support with our paying customers for at least another 6–12 months, and by then, who knows, maybe the browser landscape will be such that we can drop IE support and ship ES6 to everyone.
Again, do whatever’s right for your application, and your users.
