Interoperability between ES Modules and Common JS is a mistake.

You’re going to have to change your filename extensions to .mjs if you want to use the ECMAScript’s import/export syntax with Node.JS. Here’s why it doesn’t have to be this way:

Let me say up front that I have nothing but respect for the folks involved in the standardization process in TC39, as well as for those undertaking to implement those standards in browsers and Node. Building consensus is difficult work, and often returns nothing but complaints. These people deserve our thanks for their efforts in trying to advance the platform upon which so many of our livelihoods depend. There are no villains or bad actors here, only the results of exhausting hours of trying to find solutions that will make most people happy.

Most of us

The problem with compromise, however, is that sometimes we are driven to sacrifice too much in the pursuit of agreement. Who among us hasn’t ended up eating pizza and watching YouTube all evening because none of our friends could decide what they wanted to do? And so the Node community finds itself with the phone in its hand, looking up the number for Little Caesars, because it’s the only thing everyone is okay with.

You’ve never used ES Modules

Let’s try to understand the difficulty in working out interoperability between Node’s implementation of Common JS modules and ES Modules as defined by the ES2015 specification. Superficially, the two seem similar enough. CJS modules look like this:

var foo = require('foo')
function bar (x) {
  return foo(x) || 'blergh'
}
module.exports = bar

And ESM looks like this:

import foo from 'foo'
export default function bar(x) {
  return foo(x) || 'blergh'
}

So what’s the big deal, right? Didn’t Babel do this with a syntax transform like a year ago? Well, yeah, it kind of did. That’s part of the problem.

If you hand Babel the above snippet that uses ES Modules, by default it will transform it into something like this:

'use strict';
Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.default = bar;
var _foo = require('foo');
var _foo2 = _interopRequireDefault(_foo);
function _interopRequireDefault(obj) {
  return obj && obj.__esModule
    ? obj
    : { default: obj };
}
function bar(x) {
  return (0, _foo2.default)(x) || 'blegh';
}

I know, semicolons, right? You may recognize that extra _interopRequireDefault function as not anything you wrote. It’s there to paper over one of the differences between the two systems: the default export. ES Modules allow for a single default export to be defined alongside any number of named exports. Node’s CJS implementation sort of allows for doing one or the other of these, but not both. You can either export a single value, or an object with a bunch of keyed values. Sometimes people try to approximate this by exporting a function and appending “named” exports as properties of the function object, but whatever.

What Babel is doing under the hood is exporting an object with a key literally named “default” and throwing a helper function in front of your require()’s to handle CJS modules that it didn’t compile. It’s a handy trick, but it isn’t part of the ECMAScript specification. In fact, when Babel’s developers were implementing this transform, the specification for actually loading the modules requested by the spiffy new syntax didn’t exist yet. (More on that in a minute) They did what seemed best at the time, and were understandably focused on how they could make it work with existing module formats.

Sounds great, right? I’ve got nothing but love for Babel’s maintainers by the way. I’m all about love.

Everything Just Worked

Okay, so Babel got really popular in 2015 and a lot of people started using ES Module syntax. I was one of them. Sure, we weren’t really using ES Modules, but we felt like we were! It was the future! The best part was we didn’t need to do anything special to use existing packages in our new code. Everything Just Worked, mostly. If you published libraries that were compiled with Babel, you might have encountered some awkwardness when attempting to use Node’s require() to load your library’s default export somewhere else. Instead of the default export, you got a weird object:

// Somewhere else...
var bar = require('bar')
// bar = { default: function (x) {...}, ...}

Annoying… but okay. No problem. You just do this instead:

var bar = require('bar').default
// haha! I won!

Something like this was the first hint many of us detected that there was a family of squirrels living under our wallpaper. It’s not a big deal in and of itself, and if this were the whole issue there’d be no issue at all. Unfortunately, this is only the tip of the iceberg of the semantic differences between the two systems.

Harmony

Remember when I said that the ES Module Loader specification didn’t exist when Babel implemented a syntax translation? It still doesn’t! It’s not part of ES2015. It’s not even a proposal for ECMAScript proper anymore. It was moved out of TC39 and taken up by WHATWG. They have this document at http://whatwg.github.io/loader/, which has the very friendly sounding subtitle: “A Collection of Interesting Ideas.” I think we may be on our own here?

TC39’s official position on the Module Loader Specification

Browser vendors are moving ahead with implementing module loaders to back the ES Module syntax anyway. They can just do that! This puts pressure on Node.JS’s technical committee to figure out how ES Modules and CJS can live together in harmony. There are some challenges.

One of the things the ECMAScript specification does say about ES Modules is that their import and export bindings are static. That is to say that their dependencies and exports are analyzable before the code is actually running. This makes build features like tree shaking and static language analysis tools like editor autocompletion easier to implement. It also means you can’t do things like this:

var yolo = require(Math.random() + '.coffee')

You all know where I’m going with this

No, seriously, dynamic importing is cool and there are sometimes good reasons to do it. Language features always have tradeoffs.

The Name of the Rose

There are other, more arcane but important distinctions between the two systems. For example, ES Modules are supposed to support asynchronous loading. Node’s CJS modules are loaded synchronously, and — since they’re generally loaded from a local filesystem — there’s not a lot of need to support asynchronicity. But, the really thorny problem is that ES Modules entail a fundamentally distinct parsing mode from non-ESM JavaScript.

All the JavaScript you’ve ever written or run has been parsed by your JavaScript engine under a particular set of rules, which until very recently was just “the way you parse JavaScript.” Under the ES Module specification, Modules are a new top-level parsing target and everything that isn’t a Module is a “Script”. Scripts get parsed the same way JavaScript has always been parsed. Modules, however, have some special rules:

1. strict mode is always on
2. They have their own top-level scope that is neither a function nor the global scope.
3. Perhaps obviously, they are able to use import and export syntax

Go read https://www.nczonline.net/blog/2016/04/es6-module-loading-more-complicated-than-you-think/ if you want to better understand why this is a big deal, but it is. The different semantics between Scripts and Modules make it essential that the parser know which its dealing with at the beginning of parsing the file. The ECMAScript specification has no opinion on how this information is to be communicated, but it is not provided for within that language itself.

Pride and Prejudice

If you’re still reading, hopefully I’ve convinced you that CJS and ESM are actually pretty different things, and perhaps I’ve also engendered a bit of empathy for the folks that have to sort all this out for us.

Anyway, as Node’s maintainers began to decide how to implement ESM, two options quickly rose to the top:

1. use a new field in package.json to record a glob to indicate which files in a package should be parsed as Modules
2. use a new file extension to indicate the same

In very general terms, “front-end” type people seemed to favor the former, while “Node Core” members preferred to the later. There’s no easy way to tally which proposal had more community support, but, as of right now #2 is the plan and the new file extension is .mjs. My understanding is that it’s a tribute to the late King of Pop.

Epitaph

This a difficult problem and every suggested solution entails compromises that will make someone unhappy. Some problems have intrinsic constraints that make pleasant solutions impossible in the mathematical sense. When one has to decide which compromise is the lesser evil, there’s often little criterion other than personal preference.

That being said, sometimes when faced with extremely unpleasant conclusions, the proper response is to go back to the beginning and examine your assumptions. Errors often hide under the cover of “self-evident” assumptions that everyone knows are correct. In this case, I believe the erroneous constraint is the insistence that ES Modules be able to interoperate seamlessly with the existing CJS ecosystem. It was decided very early in this process that interoperability was a “must-have”, but letting go of that single constraint would avoid almost all of the unpleasant compromises the proposed solutions entail. In their place, we would have two simple rules:

1. If you load something with import it is parsed as a Module,
2. If you load with require(), it is parsed as a Script

To be clear, this would mean that you could not use import with CJS modules, and you could not use require() with ES Modules. You would need to know what type of module you were loading. Packages would need to bump their major versions when switching from one module system to another, and would need to communicate the change to their consumers. There’s no question there would be pain points. People would be angry on Twitter and would call other people bad names.

And it would all pass.

My belief is that it would pass more quickly than anyone thinks, and that generally things would be Pretty Much Okay. Not every technological transition is Python 3. No existing code would break. No platform backward incompatibilities need to be introduced. CJS modules would keep working the same way they always have, and developers would have the option of migrating to the ES Module system whenever they felt like it. This is why Semantic Versioning has major version numbers.

Testimonial

I think we’d all like to get to a place where we can author JavaScript in one way, and have all our tools know how to consume it in an appropriate way for whatever platform it’s running on. There are no bad actors here. People are trying their best to do what they think is right for the community. However, given the relatively small number of people who actually understand what’s going on right now, I do worry about the potential for an echo-chamber effect to make this conclusion look more inevitable than it has to be.

I could be wrong, but I strongly believe that the community has a much greater tolerance for temporary inconvenience than anyone is giving it credit for. It’s easy for the loudest voices to drown out the silent multitudes, but observing the lengths to which people are willing to go to with transpilers leads me to believe we are far more eager for change than is commonly presumed. And really this shouldn’t surprise anyone. Most of us are in this business because we are captivated by discovering new ways to do the same old thing. As a group, we love reinventing the wheel. It’s what gets us up in the morning.

And so I submit this post as a respectful suggestion that those responsible for navigating this decision reconsider the necessity of their accepted constraints. Consider, at least for one more moment, the possibility that implementing ES Modules in the most straightforward possible way will really be okay. Allow yourself to hope, just for a second, that most people are smart, and will be able to navigate the extra complexity. Give us the implementation we’ll want to be using in 5 years. It’s going to be alright.

P.S. I’ve tried to represent the reality of the technical constraints involved in this problem as accurately as I can. If I’ve badly misrepresented anything, please let me know.

P.P.S. Also thanks to Nick Niemeir (@nickniemeir) for his input. (Get mad at him too)