Tree-shaking in real world: what could go wrong?

TL;DR

• Tree shaking is not as easy as it seems
• You can only use tree-shaking if you’re using ES modules
• Babel doesn’t produce tree-shakable modules by default — you have to specify modules: false first
• Webpack 4 configuration docs are a mess and there’s no obvious way to examine what parts of your bundle are actually detected as unused ES module exports
• If you want to create a tree-shakable library and publish it to npm, you can’t do it with webpack — you’ll have to use rollup instead
• The fact that some of the ES6 exports are detected as unused doesn’t automatically mean they will be removed as dead code — some of the things will make minifier think that export can’t be removed safely because it has side effects

Let’s say you’re making a front-end application. It’s 2018 out there, so you use the most commonly used tools for your app — React, webpack 4, babel for modern ES syntax features, stuff like that.

One day, you notice your bundle is pretty big. But there’s this buzz-word you heared sometime around the release of webpack 2. The one that promises salvation, reduction of bundle sizes and massive speed-ups of your app.

Tree-shaking.

Happy with your good memory, you start googling and finding guides and writing configs and tirelessly try to shake those trees or whatever.

But somehow, for reasons unknown, it all just doesn’t want to work the way it should.

Tree shaking 101

Let’s consider a small front-end app:

Our components.jsx file exports 2 simple React components — FooComponent and BarComponent. library.js file exports 2 functions — foo and bar.

Now, let’s say you need to create a bundle for this simple application with main.js as your entry point.

Tree-shaking is the process of detecting and marking dead code in your bundle based on ES modules’ usage of import and export statements. In our example, only foo from library.js is imported in main.js, so we’d expect bar to be marked as dead code and later removed by a minifier that supports DCE (Dead Code Elimination). Likewise, only BarComponent and not FooComponent is imported in main.js, so FooComponent should be omitted automatically as dead code.

Of course, dead code elimination algorithms need to be very careful about what code can and cannot be safely removed. The general idea is that any unused code without side-effects (i.e. modifying external variables or global state, polyfilling and stuff like that) can be safely removed, unused code with side-effects has to be kept because we can’t guarantee those side-effects are unwanted or undesired, and in any dubious situations it’s better to play safe and leave the code as it is.

Official webpack guide on the topic promises that tree-shaking would work out of the box, and we’ll see unused harmony export comments in our bundle near things that are exported but are not used anywhere in the app. Let’s see how it works for us.

Tree shaking with webpack

Let’s create a typical webpack.config.js for React application. We’ll need to create a folder for it, initialize a package.json and npm install several dependencies first:

npm init --yes
npm install --save-dev react prop-types webpack webpack-cli babel-core babel-loader babel-preset-env babel-preset-react

Here’s what our webpack.config.js will look like:

All in all, a pretty typical config for any app that uses React. Note that we’re marking all dependencies from our package.json — right now that means react and prop-types — as externals so that they are excluded from our bundle and taken from global instead. We only do it to make the bundle easier to examine, and you really wouldn’t want to do that in a real-world app.

Now we can build it:

npx webpack --config webpack.config.js

So, let’s examine the bundle!

Soo… No tree-shaking, it seems. No magical unused harmony export comments. What did just go wrong?

…

See it already?

No?

Let me give you a hint — it looks like, when bundling our code, webpack had no idea those files are ES modules at all. They look more like CommonJS modules — they use exports.something to export stuff. And CommonJS modules cannot be tree-shaken because their imports and exports are not static — you could add whatever you want, in any manner you see fit, to module.exports.

…That better?

Well, you see, webpack really was bundling regular CommonJS modules. It’s because our ES modules were actually transpiled into CommonJS — i.e. import/export statements were replaced with require and module.exports.

Tree shaking with webpack — the first thing that could go wrong

If you haven’t guessed already, the one guilty here is Babel, namely babel-preset-env. One of the things env preset does is transpile any module type into CommonJS module by default.

That shouldn’t be too hard to fix — we’ll just update our babel-loader options and tell env preset to stop transpiling modules:

use: {
  loader: "babel-loader",
  options: {
    presets: [
      ["env", {modules: false}],
      "react"
    ]
  }
}

So, that’s the first problem you could bump into.

Tree shaking doesn’t work with CommonJS modules and only works with ES modules. To make it work with Babel, you must specify modules: false in your babel-preset-env config.

Let’s see how it goes now. I’m omitting all the boring webpack stuff from the bundle and just showing you how our modules look like:

Those harmony import and harmony export commets sure look promising, but the guide promised us it will also detect mark unused exports with unused harmony export comment, which it clearly doesn’t right now. So it doesn’t look like it’s going to tree-shake anything.

Tree shaking with webpack — the second thing that could go wrong

I won’t even keep you guessing here because it’s pretty much impossible.

One important thing that guide fails to mention is that ES imports and exports usage analysis is disabled by default, and is only enabled if you specify mode: “production” in your webpack config — but then, since production mode also minifies by default, you won’t see any comments at all. What’s even more weird, as of right now, official webpack docs fail to mention that the option exists in the first place — I had do dig into the sources just to find out it’s possible. The settings we’re looking for are optimization.providedExports and optimization.usedExports, they are disabled by default and we can enable them by adding the following to our to our webpack.config.js :

optimization: {
  providedExports: true,
  usedExports: true
}

That’s the second thing that might go wrong.

In order for webpack to analyze the bundle for unused exports (and see unused harmony export comments, as webpack tree shaking guide promises you would) you need to enable undocumented optimization flags — optimization.providedExports and optimization.usedExports in your webpack config.

Here’s how components.jsx module looks like in the bundle:

Yay! Now it looks like it’s going to be tree-shaken just fine, right?

Right?

(spoiler: no, not fine at all)

But we’ll get to that a bit later.

For now, let’s just think it all works as we expect, and consider a different situation. Let us assume that we would like to extract our components.jsx to a separate library to be used, say, by a couple of other similar apps. Of course, we would like those apps to be able to only bundle the components that they actually need, in other words, we want to…

Create a tree-shakable library with webpack

There is a guide for creating a library with webpack.

It says that, in order to build our app as a library, we need to correctly specify externals so that they don’t get bundled (which we already did for the sake of brevity in our example), and specify output.library and output.libraryTarget to create a module that could be used as a library.

Like we already discussed, only ES modules can be tree-shaken, so, in order for our library to be tree-shakable, we need to specify output.libraryTarget with value of…

Wait a moment.

var means that return value of our library entry point will be assigned to a single variable. assign means that same return value would be used to reassign existing variable. commonjs, commonjs2, amd, umd… Those are all fine and good, but they don’t provide imports and exports statically, and cannot be tree-shaken.

And… that’s it?

So, here’s the third thing that could make you bang your head against the wall.

If you want to create a tree-shakable library, you cannot bundle it with webpack.

There is currently an open discussion of a proposal for webpack 5 that should make it possible but we’re not there yet.

Webpack library authoring guide mentions that you could specify your library entry point in your package.json module field, and that webpack would recognize that as ES module. But what if, in our library, we decide to use some babel plugin that enables a barely known language feature that is still in strawman status? In fact, even React JSX would already break the consumers of our would-be library.

What we need of the library is to have everything except import/export statements transpiled. And we can’t do it with webpack, because it doesn’t support this particular library target yet.

In fact — and I had to dig for a while to find it out — the recommended practice is to use webpack for apps, and rollup for libraries.

Create a tree-shakable library with rollup

Thankfully, rollup does exactly what we need in our case.

Let’s write a simple rollup config and try to bundle our components.jsx into a tree-shakable ES module. We first need to npm install --save-dev rollup rollup-plugin-babel. Our config will look like this:

Let’s bundle our components with npx rollup --config rollup.config.js and see what we get in output bundle:

That looks promising!

Let’s move our components.jsx to a separate folder ../library, create package.json for it and move our rollup.config.js there. package.json for our library would look something like this:

Notice the module field instead of main — that should tell webpack to try and use that file as ES module first. Let’s build it again with npm install and npx rollup --config rollup.config.js. Our bundle should remain unchanged, so in our primary library we can now create a symlink to emulate package installation with npm install ../library. Let’s also change texts in our components.jsx to something like Hello from FooComponent living in separate library, {name}! to distinguish between old and new components.

Now let’s make sure that our library is tree-shakable. Let’s replace the code in our main.js with the following:

We will also need to make sure that some-awesome-library is actually bundled, so it shouldn’t be marked as external:

externals: Object.keys(dependencies).filter(package => package !== 'some-awesome-library')

We can try to bundle it all with npx webpack --config webpack.config.js and examine our bundle:

Nice! Webpack correctly detected that FooComponent is unused. Now, all that is left is to minify the output to see if unused exports are actually removed from our code. We can now also remove externals from our webpack.config.js and bundle react along with everything else.

We can do that by running webpack in production mode:

npx webpack --config webpack.config.js --mode production

That produces a minified build. If we try to re-format the output bundle with prettier, we could find the following troubling lines of code in there:

So, here’s what’s happening: we created a tree-shakable library, built it without minifier and saw that webpack correctly marked unused component with unused harmony export comment, and we expected that unused export to be promptly removed from our bundle as dead code. But here it is.

Tree shaking — the third thing that could go wrong

And that’s the final, and probably the most troubling thing, that could go wrong.

Do you see who’s guilty here?

I certainly didn’t when I first encountered it, and, through tedious trial and error, I’ve managed to narrow down and reproduce the problem with the following piece of code:

(function() {
  var someDeadCodeFunction = function() {return 'dead code!';}
  someDeadCodeFunction.foo = 42;
})()

When you run npx uglifyjs --compress --verbose -- input.js with the above contents of input.js, you’ll get the following:

WARN: Dropping side-effect-free statement [input.js:3,2]
WARN: Dropping unused variable someDeadCodeFunction [input.js:2,6]
WARN: Dropping side-effect-free statement [input.js:1,0]

If, however, you add a line of code so that your input.js looks like this:

(function() {
  var someDeadCodeFunction = function() {return 'dead code!';}
  someDeadCodeFunction.foo = 42;
  someDeadCodeFunction.bar = 43;
})()

…the result is the following:

!function(){var someDeadCodeFunction=function(){return"This is dead code!"};someDeadCodeFunction.foo=42,someDeadCodeFunction.bar=43}();

So the one guilty for that is, as you guessed, UglifyJS.

A lot of the things you can do with export-ed variables, functions and classes (like adding two static properties to it) are considered side effects and prevent that particular export to be detected as dead code by UglifyJS, even if webpack marks them as unused.

So, in our case, tree-shaking didn’t work because we added propTypes and defaultProps properties to our components, which is considered to be a side effect. The reasons for that are not that obvious, and I’ve been unable to find any useful info about that. I’m not versed enough in UglifyJS to make assumptions here, but if I would, I’d think UglifyJS a-priori assumes that property access can mean calling a getter or a setter function, which could have side effects. There’s even a compress option for that, but that still didn’t work for me, for some reason.

You could try rollup for your app in such a case, but it still doesn’t work when you’ve got even a single property added to your exports.

So, cool as it may sound, tree-shaking is still pretty far from being that silver bullet and working as you would expect. There are several factors at work here, not the least of them being an utter mess that webpack documentation has been since version 4 release.

Dynamic nature of JavaScript also means that it’s very hard to say whether this code is used or unused — there are a lot of things that could go wrong, and you can’t really blame minifier developers for playing as safe as possible, because loosing actually valuable code could be disastrous.

Still, it’s pretty important to remember what can go wrong, should you decide to implement tree-shaking in your project —but you still totally should try it, because it can reveal things about your project that you didn’t know, not to mention reduce your bundle sizes big time.