How to Incrementally Switch to webpack

This is the second of a two-part series on why and how we switched our JavaScript bundling system from an ad hoc system of Grunt tasks and PHP, to a declarative webpack configuration. Click here to go to Why We Switched to webpack to find out why we switched to webpack.

This article is for you if your codebase’s JavaScript build system has problems similar to ours in Why We Switched to webpack, and you want to migrate to webpack in an incremental fashion by dividing it into smaller steps. Some parts of this article are specific to AngularJS 1.x, but the rest can be applied to any framework.

To make the migration as painless as possible, break it down into these steps:

1. Introduce webpack alongside your existing build system without changing your application code.
2. Use webpack during development for a period of time, and resolve issues as they come up. Deprecate usage of the old build system, but continue to use it in production.
3. Remove the deprecated build system, leaving only webpack.
4. Update the codebase with improvements which are now possible with webpack.

By breaking the migration down into these steps, you avoid spending the initial time that would have been spent testing everything and covering all edge cases to ensure the new build system works in production; simply continue to use your existing build system for production deploys.

During development, you immediately benefit from certain webpack features, like fast rebuild times, and sourcemap-based debugging, and if there’s a problem with the new build step, you can always fall back to your old one.

1. Introduce webpack alongside your existing build system

Start by replicating your old build system’s core features necessary to build your application:

• Creating a JavaScript bundle
• Creating a CSS bundle
• Rendering JS / CSS asset paths to your HTML template

Note that you can exclude more advanced requirements from this list, such as minification, bundling for karma tests, or bundling translated language strings. These can be handled during step #2.

Creating a JavaScript bundle

Your existing build system probably involves a concatenation step to combine many scripts into one, such as with grunt-contrib-concat:

grunt.initConfig({
  concat: {
    js: {
      files: [{
        src: [
          'vendor/lodash.js',
          'vendor/jquery.js',
          'vendor/angular.js',
          'vendor/angular-cookies.js',
          'app/scripts/**/*.js'
        ],
        dest: 'build/js/'
      }]
    }
  }
});

Luckily, webpack is flexible enough to replicate this behaviour using imports-loader, exports-loader, and context:

To make your webpack JS bundle behave identically to your old script-concatenation bundle, create a new .js file to serve as your webpack entry point, and use imports-loader and exports-loader to import dependencies and export values for your vendor scripts, e.g.

// app/app.js
// Import legacy vendor scripts in the correct order
window._ = require(
  '../vendor/lodash'
);
window.$ = window.jQuery = require(
  '../vendor/jquery'
);
window.angular = require(
  'exports?window.angular!../vendor/angular'
);
require(
  'imports?angular=>window.angular!' +
  '../vendor/angular-cookies'
);
// ... application scripts

Since each vendor script has different dependencies and exported values, each must be inspected manually to determine whether & how to use imports-loader and exports-loader. Be sure to assign global variables to `window` to ensure they are available in your application code.

If your application code also needs to be executed in a particular order, you can simply `require` each file in sequence, e.g.

// app/app.js
// ... vendor scripts
require('./scripts/moduleA');
require('./scripts/moduleB');
require('./scripts/moduleC');
// ...
require('./scripts/main');

Or, if your application code can be executed in any order (e.g. if you use AngularJS modules), you can make use of webpack context to `require` all of these files at once:

// app/app.js
// ... vendor scripts
/**
 * `require` all modules in the given webpack context
 */
function requireAll(context) {
  context.keys().forEach(context);
}
// Collect all angular modules
requireAll(require.context(
  './scripts',
  /* use subdirectories: */ true,
));

For handling your JS files, that’s all there is to it! No changes to the vendor scripts or application code needed to be made; you only need to modify the webpack entry point. It’s important that you don’t modify the application code itself, to ensure that you can continue to build using your previous build system.

Creating a CSS bundle

Your CSS build step probably involves a preprocessor step, such as with Stylus via grunt-contrib-stylus:

grunt.initConfig({
  stylus: {
    compile: {
      files: {
        src: 'app/css/main.styl',
        dest: 'build/css/main.css'
      }
    }
  }
});

webpack offers a set of loaders to handle CSS preprocessing and output:

• stylus-loader / sass-loader / less-loader: runs the given preprocessor on your CSS file(s), returning the plain CSS
• css-loader: resolves `@import` / `url(…)` and returns the resulting CSS
• url-loader / file-loader: inlines or emits file contents in webpack’s output, like font or image `url(…)`s in CSS
• extract-text-webpack-plugin: moves your CSS to a separate output file, rather than inlining it in the HTML

Configuring webpack with these loaders is left as an exercise for the reader, but once it’s configured, adding your stylesheet to the webpack build is as simple as adding a `require(…)` to your entry point:

// app/app.js
require('./css/main.styl');
// ... vendor scripts
// ... application scripts

Note: the CSS build step doesn’t necessarily have to be part of the webpack build; for example, we could have opted to switch to a simple npm script invoking the `stylus` command. But we get some benefits by using webpack:

• Only webpack needs to be run to build both JS and CSS; no need to run a separate command to build CSS.
• Image / font assets referenced in CSS are automatically included in webpack’s output; no need to copy them to your output folder and manage asset paths via some other method.
• file-loader will automatically generate hashed filenames for long-term caching of the CSS file and included fonts / images.

Rendering JS / CSS asset paths to your HTML template

The final step to be able to run both webpack and your old build system on the same codebase is to render the new webpack JS / CSS assets to your site’s HTML template.

You can capture the paths of all of webpack’s output assets using the `Stats` object that is returned when the build completes. A simple way to pass that data to your HTML template is via stats-webpack-plugin:

// webpack.config.js
const StatsPlugin = require('stats-webpack-plugin');
module.exports = {
  // ...
  plugins: [
    // ...
    new StatsPlugin('webpack-stats.json', {
      chunks: false,
      modules: false,
      children: false,
      cached: false,
      reasons: false,
      source: false,
      errorDetails: false,
      chunkOrigins: false,
    })
  ]
};

To easily switch back and forth between webpack and your old build system, you can simply render webpack’s asset paths if webpack-stats.json exists in your output folder, else fall back to your old build system’s asset paths.

For example, using PHP:

Top: HTML template before adding webpack. Bottom: HTML template rendering webpack assets only if ‘webpack-stats.json’ exists

Those are all of the pieces necessary to get a minimal webpack build working! At this point, your project should be in this state:

• Run your old build step (e.g. `grunt build`), which will create JS / CSS assets in your build folder, which causes your site’s HTML template to render asset paths in the old way.
• Run `webpack`, which will create JS / CSS assets and webpack-stats.json in your build folder, which causes your site’s HTML template to render asset paths using webpack-stats.json.

2. Use webpack during development and deprecate usage of the old build system

During this stage, tell other developers on the project to use webpack instead of the old build system. If any problems arise (such as a misconfigured loader), you can resolve them safely without affecting production, since production continues to use the old build system for now.

At this stage, you should also configure replacements for any other tasks that the old build system does, such as asset minification, karma test execution, or language translations.

If you use a task runner like Grunt or Gulp, then npm scripts are often a simpler alternative to Grunt/Gulp tasks.

Here are some Grunt task alternatives that we use:

• grunt-karma: Karma’s command-line interface (`karma start`)
• grunt-angular-gettext: angular-gettext-cli + angular-gettext-loader
• grunt-angular-templates: ngtemplate-l0ader

At the end of this stage, you should have:

• An npm script to replace each build task, e.g. `npm run bundle:watch` to bundle & watch during development, `npm run bundle:production` to bundle & minify with language translations, `npm run karma` to run Karma tests
• Thoroughly tested your entire application to ensure it works properly: watch out for missing scripts or stylesheets as these can cause your app to break.

3. Remove the deprecated build system, leaving only webpack

This part is easy: simply configure your CI build script to run your new npm build scripts (like `npm test && npm run bundle:production`), delete your old task runner configuration (Gruntfile.js / gulpfile.js / etc.), and remove now-unused dependencies from package.json.

You’ll also want to delete the code in your HTML template that falls back to your old build system if webpack-stats.json doesn’t exist.

If you’ve thoroughly tested your application in the previous stage, this stage should go off without a hitch. Of course, you should first deploy to a production-like test environment first to be safe.

4. Update the codebase with improvements enabled by webpack

Now that webpack is the only build system in place, we can address the issues originally described in Why We Switched to webpack:

Resolving dependencies

Remember how you setup the webpack entry point to import scripts globally, in dependency-first order?

// app/app.js
require('./css/main.styl');
// Import legacy vendor scripts in the correct order
window._ = require(
  '../vendor/lodash'
);
// ...
require('./scripts/moduleA');
require('./scripts/moduleB');
require('./scripts/moduleC');
// ...
require('./scripts/main');

It’s not ideal to continue importing them this way: ensuring the order is correct can be error-prone, and polluting the global namespace can be problematic.

Instead, you can now import dependencies in the module where they are used, e.g. if main.js depends on lodash.js, moduleA.js, and moduleC.js:

// app/scripts/main.js
var _ = require('../../vendor/lodash');
var A = require('./moduleA');
var C = require('./moduleC');
// ...

And if moduleB.js depends only on moduleA.js:

// app/scripts/moduleB.js
var A = require('./moduleA');
// ...

Once a module is imported explicitly by all of its dependents, it can be removed from the webpack entry point: transition one module at a time, and eventually your webpack entry point will be left with just:

// app/app.js
require('./css/main.styl');
require('./scripts/main');

Using npm packages

Now you can start using npm packages instead of copying 3rd-party vendor modules to the repository: upgrading packages becomes easier, shared transitive dependencies reduce code duplication, and you no longer need to create custom UMD bundles.

It should be as simple as doing an `npm install <package>`, then updating references to the module, e.g. change this:

var _ = require('../../vendor/lodash');

To this:

var _ = require('lodash');

Now that dependencies are explicit and we’re using npm packages, we’ve achieved (at least somewhat) the goal of making the codebase easier to understand. And we have improved dev-prod parity too, since webpack is now handling our module dependencies the same way in both development and production: the only difference being that the production build is minified.

Of course, we also made the codebase ready for a future where it’s easier to start using ES2015+ by turning on babel-loader, and using React and Redux becomes easier with the ability to use JSX and npm packages.

This was the transition that EventMobi’s primary event app codebase recently made successfully. If your project is in similar shape, I hope this has provided some insight on how to improve it!

[If you liked this article, click the little ❤️ on the left so other folks know. If you want more posts like this, follow EventMobi publication with the (Follow) button on the right!

Finally if solving problems like this seems up your alley, we’re hiring! Checkout our current open positions at http://www.eventmobi.com/careers/]