Hot Module Replacement for Chrome Extension

When developing a Web page in the old days. We need a full reload to see the changes after editing the source code. HMR is a common feature of the development tools nowadays that allows developers to see the changes without fully reloading the page.

Our product, Cooby, is a Chrome Extension that helps users organize their WhatsApp chats, co-work better with their team, and even integrate with popular CRMs to bring more productivity. To improve the user experience and bring features to our users rapidly and frequently. We need a higher development performance and HMR is a must-have feature.

Unfortunately, this outstanding feature does not work well when developing a Chrome Extension.

TL; DR

Enable the HMR feature for Chrome Extension Development would encounter many issues. After some research, there are no easy config or Webpack plugins that achieve it. So we decided to face the problem on our own and luckily made it. Check out https://github.com/cooby-inc/crx-load-script-webpack-plugin to get started with HMR for the Chrome Extension.

Chrome Extension Development without HMR

Let’s say we have a typo during our extension development where the crm should be CRM; how long would it take to apply our fix and see the final change from the page?

Typically, we can utilize the Webpack’s watch mode with the following steps:

1. Update the text crm to CRM in the code editor

2. Waiting for Webpack recompilation

3. Navigate to chrome://extensions/ and manually click the reload button.

4. Reload the page where our extension runs.

Though tools like “Extension Reloader” can simplify the last two steps, you may find that every tiny change could take many manual steps to make it visible on the final page, which may significantly slow down our development process.

Fix the typo without HMR

It takes over 10 seconds to see the update. It is exhausting when it comes to building new components and the need to check the styles and layout frequently.

How does HMR work?

Speaking of HMR, a well-known scenario is the HMR feature provided by Webpack. It provides the HMR feature by the hot mode of webpack-dev-server. When the hot mode is enabled, it will attach an HMR client to the bundle js file and establish a WebSocket connection between the client and the Webpack dev server. When the developer updates the source code and causes the compiled code to update, the following steps happen:

1. The Webpack dev server notifies the HMR client of an update via WebSocket.
2. The HMR client asks the Webpack runtime to check for updates, and then the Webpack runtime fetches the updated manifest. The manifest’s filename ends like `.hot-update.json`.
3. The Webpack runtime checks the manifest and fetches the updated chunks. The filenames of the chunks end like `.hot-update.js`.
4. The Webpack runtime applies the updates.

The above steps can be enabled simply with a few lines of Webpack configuration.

// webpack.config.js
devServer: {
  hot: ture
}

Why can’t Chrome extensions use HMR?

Before we jump into the issues, let me give you a rough intro to our project and the environment of Chrome Extensions

• Content script: It contains all of our React UI code and it is the script which we take the most advantage of HMR. The content script runs in an isolated world, sharing the DOM with WhatsApp but not sharing the js variables or functions created by WhatsApp’s js bundle.
• Background script: It runs as a service worker. It focuses on reacting to relevant browser events exposed by chrome’s extension APIs, like the extension icon action, creating or focusing a tab, etc.

Let us turn on the Webpack dev server’s hot mode. Immediately, some errors were logged to the console. In short, we can categorize these errors into the following three major blockers.

Blocker 1: Unable to establish the WebSocket connection between the content script and webpack-dev-server

By default, the publicPath from which Webpack runtime fetches the output files, is /. That means Webpack will assume all the output files and assets are served from /, And the WebSocket URL it is trying to connect to is ${publicPath}/ws. It’s web.whatsapp.com/ws in our case. To fix that, we can change the publicPath to the location of the dev server, which is localhost:8080.

After setting the publicPath, a new error is invalid Host/Origin header. After some research, it turns out that it happens because the option allowedHosts should be set to the page where the extension runs.

// webpack.config.js
output: {
  publicPath: ‘http://localhost:8080/’,
},
devServer: {
  hot: true,
  allowedHosts: ['web.whatsapp.com'],
}

After configuring this, the good news is that the WebSocket connection is established. The bad news is that after modifying some code, new errors show up.

Blocker 2: Failed to load the update chunks with Chrome Extension

Currently, the output.publicPath is set to http://localhost:8080 for WebSocket to connect to the webpack-dev-server. This will cause problems below.

1. publicPath is where the Webpack runtime gets the output assets from, e.g: bundle js or the images loaded by file-loader. Setting this to localhost:8080 may work when the dev server is running, but it will fail in the production environment. The better configuration is setting it to chrome-extension://<extension_id>/.
2. The error message shows that it was a CORS error. But even if we adjust the response CORS header of the dev-server, loading a script with src from localhost:8080 will be against WhatsApp web’s Content-Security-Policy. This is another reason we should set the publicPath to chrome-extension://<extension_id>/.
3. So the WebSocket URL should be localhost:8080 instead of publicPath. And luckily there’s a config option for this.

// content_script.js
// chrome.runtime.getURL('')
// will return `chrome-extension://<extension_id>/`,
// and you can set publicPath at runtime like this:
// ref: https://webpack.js.org/guides/public-path/#on-the-fly
__webpack_public_path__ = chrome.runtime.getURL('');
// webpack.config.js
output: {
-    publicPath: ‘http://localhost:8080/’,
},
devServer: {
    hot: true,
    allowedHosts: ['web.whatsapp.com'],
+   client: {
+     webSocketURL: 'ws://localhost:8080/ws',
+   }
}

After fixing the issue above. The most severe problem showed up. Webpack loads the update chunks with the HTML <script> tag. And the <script> tag will run in the WhatsApp web’s js context and will not affect the content script.

We need another way to load the update chunks. The first thought is to use eval. Unfortunately, eval is no longer allowed for manifest v3.

Although Chrome provides another API named chrome.scripting. However, I triedregisterContentScripts the script won’t run and executeScript kept throwing errors.

After some trial and error and research, I found this page says that besides scripting, the host_permission is also needed. I added it and tried again, executeScript finally works!

Because the scripting API is only allowed in the background script, below are the steps I designed to load the script. Once the HMR client receives which update chunks should load, it passes these chunk names to the background script and asks it to executeScript.

When the Webpack runtime receives the update.json. Instead of loading update.js with a <script> tag, It sends a message to the background script and tells it to load the update.js with executeScript .

Blocker 3: How to override the default load script mechanic of Webpack

So now I got the new API for loading scripts. The next question is how to override the mechanism of Webpack for loading scripts. Luckily, we have source maps enabled in dev mode, so it is easy to find out which line of code caused the load script errors by clicking the link in the console. To throw the load script error. I use localhost:8000 for the publicPath and let the browser throw a content security policy error.

Follow the link at the right top and search for the code in thenode_modules folder. It is the LoadScriptRuntimeModule, which is injected by an internal Webpack plugin named RuntimePlugin. If it is injected with a Webpack plugin, it can also be overridden by a Webpack plugin.

So here it is, the new LoadScriptRuntimeModule and the CrxLoadScriptWebpackPlugin. Here is a rough walk-through of them. You can find more details in the repo.

The new LoadScriptRuntimeModule sends a message to the background script with the script’s filename when it tries to load the scripts. And the CrxLoadScriptWebpackPlugin replaces the original LoadScriptRuntimeModule by tapping into the same compilation hook.

// LoadScriptRuntimeModule.js
'chrome.runtime.sendMessage({',
Template.indent([
  "type:'load_script',",
  'payload:{',
  Template.indent([
    `file: url.replace(${RuntimeGlobals.publicPath},'')`,
  ]),
  '}',
  ]),
'},...
// CrxLoadScriptWebpackPlugin.js
const LoadScriptRuntimeModule = require('./lib/LoadScriptRuntimeModule');
class CrxLoadScriptWebpackPlugin {
  apply(compiler) {
    compiler.hooks.compilation.tap(
      'CrxLoadScriptWebpackPlugin',
      (compilation) => {
        const { RuntimeGlobals } = compiler.webpack;
        compilation.hooks.runtimeRequirementInTree
          .for(RuntimeGlobals.loadScript)
          .tap('CrxLoadScriptWebpackPlugin', (chunk, set) => {
            compilation.addRuntimeModule(chunk, new LoadScriptRuntimeModule());
            return true;
          });
      }
    );
  }
}
module.exports = CrxLoadScriptWebpackPlugin;

Results

With this configuration and the CrxLoadScriptWebpackPlugin. It takes no more than 1.5s to see the updates. It is almost a 10x speed up!

Conclusion

This plugin is working but is far from perfect and still needs improvement. I hope someone may think it is helpful and even help improve it. If you have any thoughts, feel free to leave a comment, create issues on Github, or email eng@cooby.co.