From Require.js to Webpack — Part 2 (The How)

This is the follow up to a post I wrote recently called From Require.js to Webpack — Part 1 (the why) which was published in my personal blog.

In that post I talked about 3 the main reasons my team decided to move from require.js to webpack:

1. Common JS support
2. NPM support
3. a healthy loader/plugin ecosystem.

Despite the clear benefits in developer experience (DX) the setup was fairly difficult and I’d like to cover some of the challenges we faced to make the transition a bit easier.

From paths to alias to NPM

The first thing you do when you’re converting from require.js to webpack is you take your whole require.js configuration file and convert it to a webpack.config.js file.

In practice for us this meant addressing three areas:

1. Tell webpack where you keep your JS files
2. The enormous paths list in our require.js config
3. All of the shim config

Module Path Resolution

The first thing you need to do is tell webpack where your JS files are. Require.js could usually infer this based on the <script> tag you used to set it up or you might have configured it using the baseUrl option. This is super easy to setup in webpack by adding the following to your config:

{
    resolve: {
        modulesDirectories: ['public/js']
    }
}

If you forget to set this webpack will look for your files in the node_modules directory or in a directory called web_modules.

Migrating Require.js paths to webpack aliases

Initially the conversion process is really straight forward.

Start with a require.js config like this:

requirejs.config({
    paths: {
        "backbone": "lib/backbone-1.1.0",
        "jquery": "lib/jquery-1.10.2",
        "underscore": "lib/lodash.underscore-2.3.0",
        "jqueryUI": "lib/jquery-ui.min"
    }
});

And it translates into the following webpack configuration:

module.exports = {
    resolve: {
        alias: {
                "backbone": "lib/backbone-1.1.0",
                "jquery": "lib/jquery-1.10.2",
                "underscore": "lib/lodash.underscore-2.3.0",
                "jqueryUI": "lib/jquery-ui.min"
        }           
    }
}

A Loader for Every Shim

The next thing, which took some special care, was fixing up our shim config. This was slightly harder than it looked, because it’s easy to forget what the shim’s are actually doing. Let’s revisit that for a moment.

A require.js shim takes modules that are not AMD-compatible and makes them AMD comptible by wrapping them in a little bit of code which will pull in the appropriate dependencies.

Let’e examine exactly how that works with the follow simple example (require.js config):

{
    shim: {
        "underscore": {
            exports: "_"
        },
        "backbone": {
            deps: ["jquery", "underscore"],
            exports: "Backbone"
        }
    }
}

Here we are applying shims for both underscore and backbone. For underscore the shim will wrap the library and then return the value of the _ variable for any scripts using it as a dependency. The backbone case is slightly more complicated:

1. It wraps the library and exports the value of the Backbone variable.
2. It makes sure that when evaluated backbone has access to both jquery and underscore

Let’s see how we would get the same setup using webpack loaders:

{
  module: {
    loaders: [
      { test: /underscore/, loader: 'exports?_' }
      { test: /backbone/, loader: 'exports?Backbone!imports?underscore,jquery' }
    ]
  }
}

Conceptually it’s not very much different than the previous version. The main difference being the way that you configure the loaders in a single line.

A few things to note:

1. A webpack loader is more dynamic than a require.js shim and is actually a lot more like a require.js plugin.
2. The test is a regular expression which matches against the full file path, so be careful to be specific!
3. In order to use these loaders you need to install them npm install exports-loader imports-loader

Pretty much all of the common use cases you might bump into are covered in the webpack guide to shimming modules.

The Fun Part: NPM

So now that you’ve migrated all of your shim and paths config, let me suggest something: delete it.

Now go to NPM and install underscore, jquery, backbone, react. Whatever else you're using is probably there as well and more than likely will work out of the box without any special loaders or aliasing.

That’s the magic of webpack. If you get your dependencies from NPM you don’t need any of that!

They will just work and you can go on building great apps not spending time carefully maintaining each dependency in your config files.

To add support for NPM to webpack, just make sure this is included in your webpack config:

{
    resolve: {
        modulesDirectories: ['public/js', 'node_modules']
    }
}

Migrating the Router

The heart of our app is this fancy router. When a new route comes in we would dynamically pull in the view associated with that path, instantiate it and re-render the main app view with the new page. It works pretty well. The main nav and everything are part of the main app, but each tab in the navbar had its own separate JS bundle and we’d only pull that in on an as-needed basis*.

This was by far the most challenging piece, mostly because my misunderstanding of how different the splitting/bundling technique is between webpack and require.js.

* This has changed for us somewhat recently, but hopefully the lesson will still be valuable for you!

Goodbye Manual Splitting

So we’ve talked a lot about our require.js config. Now we’re going to talk about our r.js (the require.js optimizer) config. All of the special information needed, in addition to what was already mentioned, to create our javascript build. Here’s the bulk of it:

{
    baseUrl: 'public/js',
    mainConfigFile: 'public/js/config.js',
    dir: '.build/js',
    modules: [
        { name: 'config' },
        { name: 'view/summary/index', exclude: ['config'] },
        { name: 'view/activity/index', exclude: ['config'] },
        { name: 'view/transfer/index', exclude: ['config'] },
        { name: 'view/wallet/index', exclude: ['config'] },
        { name: 'view/settings/index', exclude: ['config'] },
    ]
}

Most of this file is concerned with all of the different bundles we are making. With webpack we can simplify that greatly.

At first my instict was to replace this with the webpack entry concept. I spent a lot of time going down the wrong path worrying about entries. You probably don’t need that.

There is another approach that worked better for us:

• Using async require() to create split points in our app and then letting webpack create the bundles automatically.

The Routing Code: Old and New

Here is the gist of the old require.js code:

function handleRouteChange(path) {
    require(['/views/' + path], function(PageView) {
        app.setView(new PageView());
    });
}

The first thing I tried to do when moving this to webpack was just leaving it. It seemed to work! The problem is that it only created a single bundle for all of the views. This means a signicantly larger payload for the browser.

Here’s our improved webpack solution:

function loadPage(PageView) {
    app.setView(new PageView());
}
function handleRouteChange(path) {
    switch (path) {
        case 'settings':
            require(['/views/settings'], loadPage);
            break;
        case 'transfer':
            require(['/views/transfer'], loadPage);
            break;
        case 'wallet':
            require(['/views/wallet'], loadPage);
            break;
    default:
        // you could either require everything here as a last resort or just leave it..
    }
}

Each time you use the AMD-style require([]) or require.ensure() webpack will see what you passed in there and go and create a bundle with every possible match. That works great if you pass in a single file, but when you use a variable, it might end up bundling your entire view folder. That's why you need to use something like a switch statement to make sure that you declare your split points on a route-by-route basis.

You should probably grumble about this a bit, but just remember because you do this you don’t need any module or entry logic in your config. It will create all the bundles you need automatically, mostly automatically anyway :)

Entry or Async Require

Let me do my best to explain when you want an entry and when you want to rely on webpack to do the splitting for you:

1. Use an entry if you want to include a new <script> tag in a page.
2. Use AMD-style require([]) or require.ensure() if you want webpack to dynamically pull in additional modules.

I recommend reading Pete Hunt’s excellent webpack how-to on the subject of async loading for more information.

The trouble with our CDN

One of the last things that got us was our CDN. Everything worked fine in local development mode, but when we got to our live testing environments we noticed that it was trying to pull additional bundles off of the webserver by default and not the CDN where our JS actually lives in production. require.js handles this automatically by parsing the URL from the <script> on the page. Webpack doesn't have this magic. If you have a non-standard place where you want it to pull down additional dynamic bundles you need to tell it:

__webpack_public_path__ = document.body.getAttribute('data-js-path') + '/apps/';

More info can be found here: https://github.com/webpack/docs/wiki/configuration#outputpublicpath

If we’d used webpack to handle our total build process we could let webpack apply the hash automatically, but since we’re still only using it for our JS build, we need to handle this programmatically. Probably one of my least favorite things about the setup. Not terribly hard, just a bit annoying to remember.

Build Size & Time

At this point we were pretty excited about everything. It was actually working, but we noticed that the file sizes were a bit higher, maybe even a lot higher than our old require.js builds.

Enter Display Reasons

webpack --display-reasons is one the most helpful things a build tool ever provided.

Run webpack with that flag and you’ll see:

1. Every module that was added to your bundle
2. How much space it’s taking up
3. The filename and module format that included it

Here is the partial output from a recent build:

This is a goldmine of information. Which modules took up the most space? How did that get included in this bundle? All answered with this information!

Note: By default all of the files in NPM are excluded from the output. If you want to add that simply add the --display-modules flag to your query (as above) and now you can see every file that requires jquery (node_modules is shortened to ~) or other shared modules. It's pretty awesome and it helped us find a little issue with moment.js.

Moment.js and the Ignore Plugin

When we first started this process we noticed moment.js was pulling in 900kb of files. In the minified output it was still around 100kb. After poking around a bit we came across this stackoverflow article: How to prevent Moment.js from Loading Locales with Webpack.

Becuase you’re now in a CommonJS environment Moment thinks it’s safe to load all of its locale information. It presumes that you’re on the server, not bundling JS for someone’s Windows XP box. Thankfully there’s a webpack plugin that will strip out those extra requests prety easily.

{
    plugins: [
        new webpack.IgnorePlugin(/^\.\/locale$/, [/moment$/]), // saves ~100k
    ]
}

Source maps

These can cause also your builds to appear huge. They are great and can really help with debugging, but make sure you know what you’re doing with them and make sure you’re not counting them against your total bundle size. Most browsers only download these if the developer console is open, so they don’t affect your customers.

However, if you set your source-map type to eval or eval-source-map the source-maps are bundled directly into your JS. Do not use this type of source map in your production build. Here's our development config:

{
    devtool: 'source-map'
}

There is this whole huge section in the webpack docs about all the various kinds of source map options you can use here:
http://webpack.github.io/docs/configuration.html#devtool

Minification

To add minification with uglify js all you need to do is add this config:

{
    plugins: [
        new webpack.optimize.UglifyJsPlugin({minimize: true})
    ]
}

Conclusion

It was a long and storied journey to from require.js to webpack, but ultimately the entire team is reaping the benefits. One of the best features this article didn’t mention is the excellent babel-loader which made it very easy to start using ES6 features in our code today. The whole ecoystem and support for webpack have been fantastic. I hope this little guide helps your team avoid the hurdles we ran into. If you need any help feel free to leave me a note on twitter or checkout the excellent webpack chat room on gitter. Have a great day!