Concurrent Webpack Builds and Webpack's Node API
I've been working with Webpack for the last month, digging into its deepest functionalities and trying to improve my life trough this useful tool.
Is very common that applications grow fast, and like mentioned in my previous article, we don't want to lose time while building applications and delivering code not only in development but in production phases.
This time I want to show how to configure your Webpack files in order to run them in parallel. I don't need to say that this will make our application faster because of a simple math's formula.
Let's say that we have two builds that are not necessarily dependent. Let's call them build A and build B. In order be more precise and didactic, the time that the application will take to finish whenever I have synchronous builds is
buildTime = A + B
And the time that the application will take to finish whenever I have asynchronous builds is
buildTime = Max(A, B)
Parallel Tasks in a Single Thread World
I won't spend time going deep into this topic because it requires a full article to explain how processes work in an operating system and how NodeJs allows us to run multiple processes.
If you want to understand better this topic, you can read the following article that gave me a very good perspective about how does Node Js handle processes.
How to use spawn(), exec(), execFile(), and fork()medium.freecodecamp.org
I made a little research and I found two possible solutions for my problem:
- Reading this thread, I found that you can return more than one Webpack configuration object, and every object will be run in a separate instance.
- I found this package that was made by a company called Trivago. I didn't try it honestly, but if someone is interested, this is another option.
Finally, I decided to try it by myself and that is the motivation of this post.
What does this article cover
- Build the sass code into a file called styles.css
- Use the Webpack Node Api to build the previous files
- Generate the stats.json file
Npm Script and Entry Point
webpack — config ./webpack.production.config.js — progress — profile
We will have something like
Webpack Concurrent Build
In this file, I declared the pipeline needed to finish the build in which I included two configuration files
I used the fork method for parallelising the build. It's quite simple, it just receives the path of the process that you want to run as a child_process and that's all.
Inside each configuration file, you will have a normal Webpack configuration file. The only thing that change here is the need of calling directly the compiler instance from Webpack, passing the configuration object and a callback to know whether the build finished successfully or not.
The compile method is a method imported from a utils file that I created in order to be reused from multiple build files and it's the only different thing.
After calling the compile method, the program will wait for it to finish to send a message to its parent process to indicate that it has finished and now can be killed. (It's a suicide message x.x)
As mentioned implicitly multiple times before, Webpack has the ability to be called from a Node Js API.
In summary, you can get an instance of the compiler that runs inside Webpack, and call it with your configuration objects.
As described before, just by requiring the webpack lib and calling it with the configuration object and a callback, the compiler is called as it was called by using the webpack command from the npm scripts.
I used the default webpack method that calls the compiler if a callback is passed as a parameter.
I created the method compile just to wrap the pre and post processing of the compiler. A promise is returned in order to let the caller know when does it finish.
Inside the callback, error checks are made and if everything work as expected, the promise is fulfilled and the result is printed in the console.
Is important to say that there's a lot of things that need to be called again, like the console output with colors and statistics.
For further understanding, read the official documentation.
The styles build was made using the same concepts explained before so it’s straightforward. Feel free to look the code at github.
Generating the Stats.json file
You can find a method called generateWebpackStats inside webpack.utils.js.
The same way as the compiler method, I created it to wrap the logic related to check whether the output dir exists or not, and create it if it does not exist.
The information needed for this file is returned at the compiler's callback and passed as a parameter to this method.
Pros and Cons
You need to ask yourself (please do it!) if it's better to implement all this logic again or you can go with another approach. Not everything is beautiful and easy and it took me a good time to join all these pieces. Things like the Hot Module Replacement Middleware and console outputs need to be considered and "reimplemented" by yourself. I put reimplemented between quotes because you won't implement it, but you will have to configure and call everything by yourself. In this article, I covered the Node Webpack API for production mode only, but another article will be published considering the development mode.
Thanks for reading, and hope it helps you! :)