Webpack with WebAssembly: GSoC 2018 — First Step (1)
After a month of GSoC work, we release our first WebAssembly version of webpack-sources , the core package to manipulate and create source maps for your codes. We use Rust to develop our project and bundle the WebAssembly binary with a JavaScript interface, so you can easily load it as a Node.js module. We are going to share our work with you in this and following posts.
The project repository and npm package:
Early adopter: webpack-sources in WebAssembly
The APIs of our package are 100% compatible with currently used weback-sources , means that you can plug and play our npm package without changing any piece of code in Webpack!
To replace webpack-sources with our package, you can get it via yarn using:
$> yarn add webpack-sources@npm:wasm-webpack-sourcesMake sure you are using Node.js version 8 or above to support WebAssembly module. The backward compatibility, falling back to pure JavaScript implementation when using earlier Node.js, has not been completed in this release (^0.1.2).
Important notice: This package is still under development and do not use it in production.
Why webpack-sources?
As a module bundler, Webpack manipulates your codes from different files in an efficient way. When processing your codes, Webpack stores them as strings in different “sources”. The source in Webpack can represent a code snippet, content and associated file name from a file, or even a manipulation on codes, like concatenation of codes or replacement in certain positions.
The sources are implemented as classes according to different purposes, you can find sources like RawSource , OriginalSource , ConcatSource , and ReplaceSource in the package. Moreover, webpack-sources creates source maps for your files with its two major dependencies: source-map and source-list-map . When you specify devtool in your Webpack configuration, webpack-sources creates useful mapping information for enhancing the debugging process.
If you forget about source map, here is a small reminder.
There are several reasons we choose webpack-sources as our first step:
- Performance-sensitive: Just as what aforementioned, many string manipulations in Webpack are done with
webpack-sources. Also, when generating source maps, we need to encode the codes’ position information by Base64 VLQ, which is computationally expensive. - Simple input-output: WebAssembly modules are fast, but transferring values between WebAssembly and JavaScript is not trivial. When the arguments and returns are too complicated (like objects and array of string), it takes tons of time to move these values to WebAssembly's memory and reconstruct the data structures, leads to so-called “boundary crossing issue”.
webpack-sourcestakes your codes as input, and output the processed code with corresponding source map. The input and output are straightforward: String in and String(s) out, and that is exactly what we want. - Simple project structure:
webpack-sourcesis now released as an npm package and depends on two inner packages:source-mapby Mozilla andsource-list-mapby Webpack team (they are also released as npm packages, but are not used directly in Webpack). When converting the target, it will be better to convert all of its dependencies to reduce the times of passing values between WebAssembly and JavaScript. Thus dependencies being inner means that we don’t need to worry about exposing APIs of these packages.
WebAssembly with Rust
We choose Rust to rewrite the package and its dependencies and compile to WebAssembly binaries. You may ask why we prefer Rust instead of C++? There are some reasons for us to come to the decision:
- Memory safety: As you may know, Rust guarantees memory safety by introducing some strict compiler rules. This feature is so important for low-level programming language without garbage collection and will save us a lot of trouble in future works.
- Package management: As a developer spoiled by npm, cargo, the default package manager of Rust, is entirely sufficient to let you choose Rust instead of C++.
- WebAssembly Toolchain: There is no doubt that many mature tools are there to help us develop WebAssembly in C++, like
emscripten. However, more and more tools are now published for Rust with fantastic features! For example,wasm-bindgenis a tool to create a JavaScript interface to load WebAssembly binary. It builds functions in JavaScript to wrap the WebAssembly exports and provides high-level API for passing and returning complicated data without demanding developers to write those memory management codes manually. Thus, with the help of these tools, we can develop our WebAssembly module in a more high-level way.
There is a speech by Lin in JSConf EU 2018 talking about Rust with WebAssembly. You can refer to his post “Baby’s First Rust+WebAssembly module: Say hi to JSConf EU!”.
What’s next?
If you have given our project try, you may find out that the performance is worse than the original JavaScript implementation. As our first step in introducing WebAssembly binaries to Webpack, we focused on “making it work” in our first month. Indeed, we agree that performance is a principal objective. We have found some possible issues, and we will address the problem in future works.
In this series of posts, we will introduce our conversion work with Rust and wasm-bindgen . We put efforts on solutions of JavaScript interface for API compatibility. At the same time, the “boundary crossing issue” due to transferring values between WebAssembly and JavaScript is targeted out in this step. We will elaborate the details for you in later posts.
