My experience writing a webpack’s child compiler plugin.

This is me sharing my experience of writing my first webpack plugin and the things that came as learning out of it. Since its an experience-based blog post, I do not intend this to make a statement towards webpack’s DX. This was my first ever webpack plugin so everything down below might sound like big disaster to you. Please feel free to point me in a better direction.

A few weeks ago I took up a task of writing a webpack plugin. I wanted this plugin to spit out mirror copies of the javascript assets generated by webpack but with a different babel configuration.

More context: With browsers supporting script module/nomodule we should ship less transpiled code to users today. Read more in this article from Phil Walton https://philipwalton.com/articles/deploying-es2015-code-in-production-today/

So my idea was to have a webpack plugin which could sit in your webpack configuration and transpile your files using the babel-preset-env with the target set to esmodules. So all that’s left will be to add a <script type=moduletag.


I started by reading a few blogs, some articles(some outdated/some still relevant) to figure out how to build a webpack plugin. I found a few webpack hooks which are called for every asset that webpack encounters while parsing your code etc. While doing the reading exercise I stumbled on the concept of childCompiler, this sounded like a subprocess kinda thing and “sounded” to be the thing that I’d want. But this had no dedicated documentation on what it is, its pros/cons, when to use etc on the webpack docs site. Since I couldn’t find anything else I decided to give it a try. The only help from the internet was this stackoverflow explanation from Arthur Stolyar https://stackoverflow.com/questions/38276028/webpack-child-compiler-change-configuration.


So after realizing that Child compilers are exactly like sub processes inside the main webpack process I was convinced that this the thing I could use to re-run the entire codebase through a different babel setting.

So I just tried following the above stackoverflow explanation and created a basic child compiler as follows:

Step 1: Create a class with apply method to tap into webpack’s plugin cycles

class MyPlugin {
constructor() {...}
apply(compiler) {
compiler.hooks.make.tapAsync(PLUGIN_NAME, (compilation, cb) =>
{
// all logic goes here
}
}
}

In the above example we hooked our plugin into the plugin system of webpack and our apply method will be called with the main compilation as the argument.

Step 2: Using main compilation to instantiate a childCompiler and execute it.

//inside apply function
const childCompiler = compilation.createChildCompiler(PLUGIN_NAME, outputOptions, plugins);

The above method takes 3 parameters,

  1. The name of the compiler
  2. The output options for the files generated inside the child compilation(same as the webpack config).
  3. The array of plugins you’d want to execute on this childCompilation.

This here is a pretty powerful API, I used this to rename my generated files from x.[chunk].jsto x.[chunk].esm.js. Also one could add or remove the plugins from the actual config for the child compilation. e.g. In my usage I add a WebpackDefinePlugin to add an env variable isESM so that I could add conditionals for the other build.

Note: In order to access the parent configuration use compiler.options from the compiler object passed in the apply method params.

Step 3: Add entry points

If you look closely, in the createChildCompiler compiler we only gave output options of our config. This means that the child compiler has no knowledge of the entry points, thus we’d have to specify it explicitly.

There is a webpack internal plugin exactly for this known asSingleEntryPlugin. So the following code could just take all the entry point from the parent config and add them as is to the child compiler.

Object.keys(compiler.options.entry).forEach(entry => {         
childCompiler.apply(
new SingleEntryPlugin(
compiler.context,
compiler.options.entry[entry], entry
)
);
});

If you run this childCompiler at this point you’ll see that you are generating mirror copies of the original assets.

IMPORTANT: Do not forget to change the filename in the createChildCompiler call, else you’ll override the actual files.

Step 4: Discover that the chunks are not being fetched.

If you try to run an app with the new files generated, you’ll observe that the entry point bundle works fine but it is not fetching the chunks over the network!!!

After some searching I came to know about another internal plugin which actually adds the code to fetch the chunks over the network JsonpTemplatePlugin. And here’s the catch: simply adding this plugin to your plugins array will not make it work. Why? I really don’t know the reason and couldn’t find any documentation about these special plugins anywhere on the documentation site.

So in order for this plugin to work you need to apply it to the compiler:

childCompiler.apply(new JsonpTemplatePlugin());

And that should make the chunks working.

Step 5: Tell parent compiler about errors, assets and named chunks

So even if you don’t do this it might work fine with you as the files are generated and you are free to use them. However if any other plugin(say HtmlWebpackPlugin) is trying to know the output, your additional assets will not be visible to it. In a nutshell you have to tell the parent compiler about the assets and named chunks you generated

compilation.assets = Object.assign(childCompilation.assets,              compilation.assets);

Please use the additionalAssets hook of the parent compilation to make sure you dont end up in a race condition with other plugin.

Step 6: Performance plugins

After playing with this plugin I realized any plugin in my config in the optimization section is also not being applied. Thus I went back and looked for specific plugin. Seems like webpack’s main compiler goes over the specified config and apply the plugins one by one see here https://github.com/webpack/webpack/blob/9f0056b10d6997fd8e1472fec4b8c84b5d6b1db7/lib/WebpackOptionsApply.js#L322-L373

The above code goes through your config and applies it to the main compiler. It did kinda felt weird though that the same was not copied over from parent compiler to child compiler or there is not a friendlier function to which I could pass the original config and expect the same plugins to be applied on my compiler.


Hope this would have be useful for anyone who is writing a child compiler for the first time.

Here’s the github link for the finished work: https://github.com/prateekbh/babel-esm-plugin#readme

Cheers!