Webpack 3 to 4: Facing the known unknowns and unknown unknowns

Chris Liu
Chris Liu
Oct 11, 2018 · 7 min read

This post will sketch out what a Webpack 3 to 4 upgrade looks like in a large modern web application. We hope this is either an entertaining recollection or helpful for your own future upgrades.

Why upgrade?

As we were on Webpack 3, this resulted in an uncomfortable situation: CommonsChunkPlugin, the mechanism for ensuring that code common to many split points is extracted to a common file, is not part of the deal in Webpack 4. We made the decision to upgrade to Webpack 4 to avoid sinking time into a removed plugin, while also hoping to get some of the benefits.

Just follow the instructions, right?

Current State of the World

In development mode, we use Webpack dev middleware along with a custom dev server to emulate the production edge server. The custom dev server forwards requests to a local copy of the SSR server to serve requests coming from developers’ laptops.

Coursera front-end infrastructure

Known Unknowns

  • We had custom code that used the Webpack plugin API for injecting data into HTML templates. This post by the maintainer of ts-loader and this guide by the creator of Webpack were enough for us to understand what changes were necessary. Specifically, the plugin API now used the hooks property to tap into Webpack compiler hooks or the developer’s custom hooks.

For us, this involved changing

compilation.plugin('html-webpack-plugin-after-html-processing', (htmlPluginData, callback) => {
...
}

to

compilation.hooks.htmlWebpackPluginAfterHtmlProcessing.tap( 'withInterpolatedTemplateContext', htmlPluginData => {
...
}
  • We used a variety of libraries that required bumping up versions to support Webpack 4 [e.g., html-webpack-plugin, happypack, webpack-dev-middleware …]. This required reading through the changelog or release notes of each library to find which version was Webpack 4 compatible.
  • Webpack 4 has a new mode field that applies certain optimizations by default. This post by Webpack founder Tobias Koppers gave us all the necessary information. For instance, we removed extraneous plugins like NoEmitOnErrorsPlugin.
  • We needed to migrate our old usage of CommonsChunkPlugin over to SplitChunksPlugin before investing more into code splitting. This required reading this gist describing the difference between the two plugins and then reading the docs. It took us several rounds of reading and playing around with the code for us to understand the new concepts. Our old setting looked like this:
new webpack.optimize.CommonsChunkPlugin({
name: 'app',
async: true,
children: true,
minChunks: 3,
})

Which translates to “create a single async common chunk that contains code common to 3+ async splits.” This translates to the following SplitChunksPlugin setting in config.optimizations.splitChunks.cacheGroups:

{

commons: {
chunks: 'async',
name: 'asyncCommonJS',
minChunks: 3,
// Ignores `minSize`, `maxSize`, and other defaults.
enforce: true,
priority: 0,
},
// The next 2 lines disable the 2 default `cacheGroups`, which are specified in
// https://webpack.js.org/plugins/split-chunks-plugin/#optimization-splitchunks
default: false,
vendor: false,

}
  • We needed to change require paths for various Webpack internal files that had been refactored to import the correct files. For instance,require('webpack/schemas/webpackOptionsSchema.json') -> require('webpack/schemas/WebpackOptions.json')

Unknown Unknowns

JSON modules

Source from Webpack 4 changelogs

This turned out not to play nicely with our usage of bundle-loader, which we used for lazy loading JSON. See this issue for more details. The fix is to force files with .json extensions to use json-loader, which does work with bundle-loader:

{
test: /\.json$/,
loader: 'json-loader',
type: 'javascript/auto',
}

CSS Bundling

This issue was a common theme with extract-text-webpack-plugin [e.g., see here], so we bit the bullet and migrated to mini-css-extract-plugin.

The config changes are as follows:

new ExtractTextPlugin({
filename: [name].[chunkhash].css,
allChunks: true,
disable: isDev(),
})

is now

new MiniCssExtractPlugin({
filename: ‘[name].[chunkhash].css’,
})

along with the following option in optimizations.splitChunks.cacheGroups:

{
name: 'allStyles',
test: (m: Module) => m.constructor.name === 'CssModule',
chunks: 'all',
enforce: true,
priority: 1,
}

We needed to use the test property to test on CssModule rather than looking for the .css extension because we use Stylus in our codebase.

After this change, we also faced this issue with the plugin generating an additional Javascript file stealing the original entrypoint. We modified our usage of html-webpack-include-assets-plugin to also include the additional Javascript file to unbreak module loading.

Larger stats.json output

modules: true,
chunks: true,
timings: false,
source: false,
reasons: true,

which still gave us the relevant information while keeping the file small enough to be produced.

Webpack-multi-output and undocumented hooks usage

Webpack 3 jsonp script hack for webpack-multi-output

Access to this hook is no longer allowed in Webpack 4, so we had to use the mainTemplate.hooks.render hook along with an undocumented stage flag to make this work. It now looks like this:

webpack-multi-output jsonp script hack for Webpack 4

We want to stop using webpack-multi-output in the near future — this hack reduces bus factor, makes future upgrades hard, and disempowers non-experts who want to make changes.

What did we get out of this upgrade?

  • Bundle sizes are roughly identical.
  • We’ve reduced our dependency on deprecated, unmaintained, or outdated plugins.
  • We are now confident building on top of Webpack — through reading the source code of Webpack, keeping up with recent developments, and having a better mental image of Webpack internals [e.g., chunk graph algorithm, split chunks plugin], we now feel empowered to take full advantage of Webpack.

That’s all — we hope you enjoyed following along with Coursera’s journey upgrading from Webpack 3 to Webpack 4. We went through trials and tribulations, but think the final outcome was worth the effort.

Coursera Engineering

We're changing the way the world learns! Posts from @Coursera engineers and data scientists!

Chris Liu

Written by

Chris Liu

Passionate about education and solving hard problems in a collaborative fashion.

Coursera Engineering

We're changing the way the world learns! Posts from @Coursera engineers and data scientists!