Getting the most out of Webpack(er), Part 2
- Only import what you need
- Don’t implicitly trust your package manager
- Keep dependencies as up-to-date as possible
- Proactively strip out what you don’t need
- Split aggressively… but also thoughtfully
Setting the stage
As we optimize the commons chunk, the metric we’ll use to track our progress is gzipped kilobytes. At the start of our journey, the commons chunk weighs in at a hefty 1,216 kB.
When we first fire up the Bundle Analyzer, our first issue immediately jumps off the screen. Either someone’s spiked the office water cooler, or we need to go get our eyes checked:
If you can’t tell from the impossibly tiny text in the above GIF (we’re software engineers; not graphics experts), that’s:
- Two copies of Moment.js, including all of its locales.
- Two copies of jQuery.
- Two copies of Backbone.js.
- Three copies of Underscore.js.
We don’t have many math majors on the team, but by our unscientific count that’s approximately 2–3 times more packages than we need.
We’ll start with the double jQuery, which leads into our first Big (if obvious) Takeaway™:
1. Only import what you need
require statements are necessary, especially over time. Properly-configured code quality tools like ESLint can help alert you to unused imports, but they aren’t perfect.
jquery-ujs package as an import everywhere it’d previously been available via a Sprockets manifest.
When we added the package, Yarn looked at
jquery-ujs's dependencies, saw
"jquery": ">=1.8.0" as a peer, and checked whether it already had an identical version expression:
When it didn’t find an exact match, Yarn resolved
>=1.8.0 to jQuery 3.3.1:
And thus were our beautiful twin jQueries born.
Ultimately, we undertook a more thorough search and determined that we could remove
jquery-ujs entirely, remediating the problem. However, the fact that it was an issue at all leads into our second Big Takeaway™:
2. Don’t implicitly trust your package manager
Yarn is a very cool piece of software created and maintained by very smart people. Yarn’s sole purpose isn’t to collaborate seamlessly with Webpack; it’s to manage packages. It’s a reasonable assumption that when we add a brand new package into an ecosystem full of old packages we’d prefer to use the latest and greatest versions of that new package’s dependencies. It’s an equally reasonable assumption on Yarn’s part that it’s probably best to let sleeping dogs lie — just because a sub-dependency could be resolved to a higher version doesn’t mean it must, and it’s better to wait on proaction from the user to do so.
However, the upshot of Yarn being so darn reasonable is that we were left packaging up two copies of jQuery in our commons chunk for a short while. We’d ducked action in the first instance by simply removing the problematic package, but we’ll have no such easy path out with the second: Moment.js.
A couple months ago, we bumped the version of our Moment.js dependency from
~2.18.1. Prior to the bump, our
yarn.lock file looked like this:
>= 2.9.0 is a clear superset of
~2.18.1, Yarn decided to play it safe and left good enough alone. Since the resolution of
>= 2.9.0 to
2.10.6 only exists inside of the
yarn.lock file, we could try deleting the offending lines (7–9 in the above gist) and re-running
yarn. Upon not finding an existing resolution for
"moment@>= 2.9.0", Yarn grabs the latest version,
2.22.2. Uh oh!
Ah, yeah, we forgot about that pesky warning at the top of every
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
Trying to manually bend lock files to our will is a risky prospect, especially if the package in need of bumping is tangled in a twisted web of peer- and sub-dependencies. And, anyway, Yarn already provides a nice API for us to accomplish exactly what we want.
If you’re feeling adventurous and don’t have to operate within a codebase that contains some hasn’t-been-touched-in-years, keep-everything-locked-to-the-exact-versions-that-work-and-don’t-change-a-thing legacy code, try out
--flat option, which will walk you through the process of selecting a single version per dependency. We do have to maintain a fairly sprawling Backbone / Marionette / Alt / jQuery-driven chunk of code, so we’ll take the scalpel approach and set a single, targeted dependency override for Moment:
We’re now down to a single copy of Moment.js in our commons chunk, and, together with the jQuery de-duplication, we’ve managed to trim the chunk down to 1135 kB, a full 6.7% decrease! We’ll come back to deal with the duplicate Backbones and Underscores still floating around, but first let’s take a brief detour to talk about the importance of updating dependencies.
3. Keep dependencies as up-to-date as possible
Upgrading Moment.js caused an unfortunate temporary regression in bundle size, but, in general, keeping everything up-to-date should be a top priority. For an active, well-maintained package, upgrading can prevent historical dependency cruft from creeping into your bundles. For an example, let’s turn back to the Bundle Analyzer and take a look at the updated commons chunk:
What even is all that?
readable-stream is an extraction of the Streams API out of Node.js’s core.
It turns out that version 0.13.6 of the Sails JS client, which powers our legacy real-time chat on Learn.co, depends on Request, an HTTP client for Node.js. The dependency was changed to a dev dependency in the 0.13.7 release, and the call to
require('request') was removed from the distributed code.
However, we have a strict version requirement that has prevented us from receiving even the 0.13.7 and 0.13.8 patch releases:
We use Sails in a number of places across our monolith, and Webpack dutifully heads further down the rabbit hole with each additional nested
node_modules all the way down! Sails 0.13.6 requires Request ~2.34.0, which requires… well… a solid chunk of the Node.js ecosystem.
After a thorough QA process, we were able to bump our Sails dependency to 0.13.7 (and then 0.13.8 for good measure). By relaxing our strict dependency and jumping up a single patch version, our commons chunk shrank by 11%, down to 1010 kB. Once again for the people in the back: absolutely-useless-in-the-browser Node.js-land cruft made up over a tenth of our entire commons chunk.
4. Proactively strip out what you don’t need
Webpack (via Webpacker, in our case) has a ton of plugins that can be used to optimize your output bundles in various ways. Some of them, such as the
UglifyjsWebpackPlugin, have an extremely broad purview, mangling code left and right across the entire codebase. Others, such as the
IgnorePlugin, can be used in a much more targeted manner, only affecting a well-scoped sliver of the mass of code processed by Webpack. That targeted approach comes in handy for dealing with Moment’s locales, an ever-growing series of extensions to the core library that localize date-time representations for different audiences around the world.
Not all libraries are set up for seamless integration with a modern, tree-shaking build tool like Webpack. When Webpack arrives at an
import moment from ‘moment' statement and starts processing the library, it loads every single file in the
./locale/ directory. By the time Moment makes it into our commons chunk, the core library has been mangled and compressed down to 16.4 kB, but that child directory housing locales weighs in at 46.1 kB, almost three times the size. At this time, we don’t perform any date-time localization on Learn.co, so every single locale file is unused.²
The bloat added by Moment’s locales is a common gripe, especially within the bundle size-conscious Webpack community. It’s so common that one of the two sections on the
IgnorePlugin in Webpack’s official documentation is dedicated to how to use the plugin to strip out Moment’s locales. And that’s exactly what we’re going to do:
The first argument passed to the
IgnorePlugin tells it to ignore any request for an asset (via a
import statement) matching
./locale, and the second argument scopes it down to only ignore matches that originate from a module directory ending in
When we fire up the Bundle Analyzer again, look how much smaller Moment.js gets:
The commons chunk shrank by another 4.6% to 962 kB, and, short of massive refactors to remove legacy package dependencies, we’re running out of slim-down targets. But even though there aren’t any packages left to bump or unused imports to ignore, there’s one final optimization to make that will have as much impact as the previous four combined:
5. Split aggressively… but also thoughtfully
Our commons chunk setup should:
- Minimize the amount of code browsers have to download by preventing duplicate code from being packaged up in multiple bundles.
- Avoid including modules that are frequently updated in order to take advantage of browsers’ asset caching abilities.
The first bullet point is a bit more complex than it may seem at first blush. We have a large vendor asset that is version-locked. However, it’s only used in a single place in the application. How do we weigh whether to include it in the commons chunk? The answer differs if that single place is on a high-traffic page versus a lower-traffic feature that perhaps requires some user action to activate.
When we first moved to Webpack, we set up our
CommonsChunkPlugin to split out vendor assets into a separate chunk exactly like this example from Webpack’s documentation:
There are a few issues with that setup, but the one we’re going to focus on now has to do with the way this setup decides which code belongs in the commons chunk and which does not.
The code above says that every imported module will be passed to the
minChunks function during Webpack’s build process. If the passed-in module lives in the
node_modules directory, it will be moved into the commons chunk. Simple as that.
That setup is a bit naïve, as it doesn’t distinguish between a vendor module that’s used in a single, isolated place within the application and a vendor module that’s used in every single bundle. It also doesn’t take into account non-vendor modules that rarely change and are used widely across the app.
As it happens, three of the four largest modules in our commons chunk fit that first narrative to a T. The
c3 library and its chief dependency,
d3, are, respectively, the third and fourth largest modules, and they’re used to graph student progress in a single, admin-only interface. Not only is the page restricted to admins, but the graphs aren’t even visible on initial page load — the admin must perform an action for them to appear. It’s legacy code that will likely be removed in the not-too-distant future, but it makes little sense to include both libraries in the commons chunk that’s downloaded by every single site visitor.
While it sees significantly more usage than
d3, it’s equally difficult to justify including the
brace module in the commons chunk. It accounts for over a third of the commons chunk’s size on its own, checking in at nearly five times the size of the second largest module.
brace is the editor component of the in-browser IDE, which is only available on lesson pages under certain conditions. And, just like the graphing libraries, the IDE is never visible when the page loads — students must click a button to open it.
Because both the graphing libraries and the IDE editor package require users to take an action before they are needed, they’re perfect candidates to be split out into separate chunks that can be loaded only when they’re needed. Let’s adjust our
CommonsChunkPlugin setup a bit to create three separate commons chunks — one for the IDE, one for the Organizations app (where the graphs live), and one for everything else in
node_modules. The ‘everything else’ chunk has to be defined first:
The ordering of the other two chunks doesn’t matter, but they must come after the main chunk because they’re only going to extract modules that have already been bundled up in the main chunk. We’ll do the organizations chunk first:
tablesorter is another vendor library that’s only used within the organizations app, so we’ve split it off into the
And then the IDE chunk:
react-ace is the actual editor package that
brace wraps, and
lodash.isequal is a deep comparison function that
react-ace depends on. Both were logical choices to package up with
brace in the
Before spinning off those specialty commons chunks, a student’s browser would have to download a 962 kB commons chunk the first time the student navigated to the site. After the change, that figure is down to 490 kB, a 49% drop, with the IDE commons chunk weighing in at 351 kB and the organizations chunk at 122 kB.
Here’s a complete gist showing everything we’ve talked about laid out in Webpacker’s
And here’s a final look at our three commons chunks via the Bundle Analyzer:
Thanks for reading!
- Deprecated in Webpack 4 in favor of the SplitChunksPlugin.
- Even if we only supported a small subset of languages, we could use the ContextReplacementPlugin to whitelist the subset and discard everything else. Or, if we wanted to support every locale, we could probably send up an array of the choices and then load the actual locale files as they are requested by the user.