How we’ve solved an issue in Webpack at HiBob using a binary search algorithm

Recently I worked on an issue at HiBob related to Webpack hashing. Since we solved the issue in an interesting way, I thought it was worth sharing.

Our vendor file (generated with Webpack 4.25.0) for production gets a new hash (chunkhash) in each build. That leads to performance issues for our clients, as each build of the app code for production causes clients to download the vendor javascript file, even if we didn’t change anything related to it (like adding a new framework in package.json).

After searching the web, I found that many other developers have struggled with the same issue. While looking for solutions, I also came across a couple of Webpack plugins such as assets-webpack-plugin, webpack-md5-hash that I thought would give me an easy solution. After trying some Webpack plugins, I realized it would not help me.

I examined each build locally and realized that the app bundle hash doesn’t change. This lead me to the conclusion that each build of the vendor chunk creates different content. From the minified+uglified file, it was hard to understand what was causing it. I knew I had to find something else to understand the root of the problem. Binary search came to help me.

Here is the configuration for output in webpac.config.prod.js:

output: {
  filename: '[name].[chunkhash].js',
  chunkFilename: `[name].[chunkhash].${app}.js`,
  publicPath,
},

Here are the output file names:

vendor.cde0f9555dc9542214d5.mobile.js
mobile.4ab223053b3c16d85ccc.js
vendor.d8c01b0b780c58ad61a9.mobile.js
mobile.4ab223053b3c16d85ccc.js

Although I didn’t make any changes related to the vendor, the hash was different.

I decided to look into which of the 3d party libraries inside node_module caused this change. I did this in binary search technique and played with the vendor regular expression inside splitChunks configuration:

optimization: {
  namedModules: true,
  concatenateModules: true,
  splitChunks: {
    cacheGroups: {
      vendor: {
        // Play with this regular expression =>
        test: /node_modules\/[a-j].*/,
        chunks: 'initial',
        name: 'vendor',
        enforce: true,
      },
    },
  },
},

Each time I changed the regular expression in it and examined the result, I looked to see when the vendor hash was different. When it was different, I dived deeper into the binary search algorithm.

Here are some of my steps:

test: /node_modules\/[a-i].*/
vendor.f8b06d871ee62553951a.mobile.js
mobile.f86f0ba8a2d1200e0eb2.js
  
vendor.f8b06d871ee62553951a.mobile.js
mobile.9350aa79df709503a51c.js
--- vendor hash is the same, continue ---
test: /node_modules\/[j-z].*/
vendor.9663994de993c08ffa17.mobile.js
mobile.b50f0e88c6c4ddf78755.js
vendor.9663994de993c08ffa17.mobile.js
mobile.61b9cbef2d5ca11859db.js
--- vendor hash is the same, continue ---
test: /node_modules\/[j-z].*/ 
vendor.f85ddc7b8a3889ccdd7b.mobile.js
mobile.5274e6ef86c7226fb911.js
vendor.098b372b9f9f802cec06.mobile.js
mobile.5274e6ef86c7226fb911.js
--- vendor hash is the different, go deeper ---
...
--- Finally find what change my hash each build ---
test: /node_modules\/[u].*/
test: /node_modules\/((@uirouter)\/).*/

Within half an hour I realized that the content of one of the libraries (uirouter) inside was generated differently each WebPack build (I am still not sure why). However, in order to solve the vendor hash that caused our client performance issues, I just ignored this anomaly and packed it together with the app bundle that was changed anyway. That way, I could keep the vendor hash loading experience (which was super important to our client app) in the price of 50kb~ to the app bundle. Here is my final configuration (until we find what caused the uirouter to change on each build):

optimization: {
  namedModules: true,
  concatenateModules: true,
  splitChunks: {
    cacheGroups: {
      vendor: {
        test: /node_modules\/(?!(@uirouter)\/).*/,
        chunks: 'initial',
        name: 'vendor',
        enforce: true,
      },
    },
  },
},

In the following chart (generated with webpack-bundle-analyzer), we can see that, for now, uirouter library is packed together with the app:

The uirouter library is now packed with the app

after searching more for a similar issue, I came across the following issue at Webpack:

contenthash changing even though file remains identical · Issue #8419 · webpack/webpack

Conclusions:

If you feel stuck on an issue, try taking a different approach. This seems obvious but sometimes we insist on sticking to what we know. It’s sometimes hard to stop and change the way you think about an issue.

From an architect’s point of view, this is still not the best solution since the app bundle now contains a 3rd party, but the impact on the performance was a high priority. Hopefully, when we move all our client code to Angular v7, this issue will be resolved.

In short:

• Sometimes it’s worthwhile to stop and re-analyze the problem.
• You don’t need to wait to for the perfect solution. If you find a way to improve your current issue, start with it, and continue searching a better one.

Now our clients are happy since the vendor file is cached and loading time is significantly improved.

the vendor chunk is returned from cached and not change on each commit to production

I hope this post helps you or gives you some ideas. Thank you for reading and feel free to reach out!