MOSS and WebAssembly (week 17)

2017/07/24–2017/07/30

Published in

webpack

5 min readAug 1, 2017

This week we released version 3.4 including a bunch of performance improvements and a lot of small bugfixes and improvements. This also includes a bugfix for a crash when using Scope Hoisting.

Note that Scope Hoisting is still experimental as contains at least one known bug.

Next to the release I also upgraded most of the dependencies of webpack including devDependencies. This means CI runs with the latest versions again and dependencies badge is green again:
https://david-dm.org/webpack/webpack

After the release I updated and merged some older PRs (a bit PR list cleanup).

There are also some small new features on the master branch (not yet released):

stats.excludeAssets to filter the assets list like with the modules list.
import( /* webpackMode: "weak" */ "..." ) which loads the module if it’s available (won’t do a network request, but throw instead).
require.resolveWeak with expression.

MOSS

Also make sure to read the news regarding webpack + WebAssembly:
https://medium.com/webpack/webpack-awarded-125-000-from-moss-program-f63eeaaf4e15

We received a grant from Mozilla to work on great support for WebAssembly (WASM). I know there are already some open source tools/loaders for WASM, but the idea of this native integration is to make WASM modules feel like ESM. webpack will hide all async loading/compiling details in a way that allows the developer to use WASM with ESM import:

// using import to access wasm modules
// this is still compiled async
import { createVNode, calcDiff } from "./vdom.wasm";// Passing object, strings
const node = createVNode({ name: "h1", content: "Some text"));// Passing pointers
const diff = calcDiff(oldNode, node);

Of course this is the long term plan. As first step we want to support something like this:

import { add } from "./addition.wasm";const c = add(1, 2);

You may wonder how a sync import can cause a async wasm compile. The trick is to hoist the compilation process to the next async import() in the dependency tree. That’s something you can’t do with a loader. That’s why native support is so important.

Native support includes the download+compile hositing, exposing WASM exports to ESM and importing other modules from WASM (the wasm spec has it’s own notation of imports which are similar to ESM).

On top of this native support you can implement all these wonderful features via loaders:

Compile WAT/C++/Rust/… to WASM
Generate wrappers to support strings and objects as arguments/results and optionally wrap pointers.

And there is also much work involved in the ecosystem:

Go away from the emscripten way of compiling all native code into a big WASM blob. Instead preferring a modular way in which each (i. e. c++/rust) file is compiled into a wasm module importing other wasm modules. This may involve a lot of changes in the native compiler infrastructure
Providing basic WASM modules like memory which provides a shared memory for multiple WASM modules. This way pointers can be passed between modules and are still valid.

Some questions you may have:

Can WASM modules import ESM? Yes. This is i. e. needed to access Web APIs: You super fast vdom.wasm can import dom.js to apply it’s diff to the real DOM. WASM can’t access the DOM, but JS can.

What’s about browsers that doesn’t support WASM? We may automatically compile your WASM modules into asm.js and load this if WASM is not support. If we do so, it’s transparent for the developer if WASM is supported or not. There will be additional download cost for this feature if WASM is supported, we’ll download chunk.wasm or chunk.asm.js depending on support.

What if I import the same wasm module in multiple places? The some semantics as of ESM or CommonJS will apply here. There is only a single instance of the module in the module cache. Importing the module will always return the same instance.

I don’t like ESM. Can I use WASM with require? Yes. You can require() WASM modules and compiling will also hoist to the next require.ensure. You can also mix these. Internally these are all sync or async dependencies. webpack don’t make a big difference here. ESM may allow some additional optimizations (like export name mangling, Scope Hoistings), but this is transparent of the developer.

Can I import() WASM modules directly? Sure. There may be additional optimizations possible when doing so.

What happens when there is not import() to hoist compilation to? There will be a warning in this case, because runtime behavior will not be optimal in this case. I guess we still allow this, can either fall back to sync compilation or delay the application start until all WASM compilation has finished.

Is Scope Hoisting for WASM possible? Yes, but this is more a stretch goal. It’s possible to link multiple WASM modules together into a single WASM file.

Can I debug my Rust code in the browsers devtools? Not in the initial steps, but Mozilla already experiment with WASM SourceMaps and in long term this is something webpack will support. It’s not part of the MOSS, so no time frame yet.

I want to write my code in rust now. How long does it take until this is implemented? The MOSS aims for a year, but basic functionally will be available soon (a few months).

How is the money handled? We have a list of milestones for the project. A part of the money is paid when milestone are reached. We plan to distribute the money to the contributors of the milestones. Example: If the milestone includes Rust support and you contributed to the rust-loader, you can get a piece of the cake.

Mozilla and Segment are not listed as sponsors in the list. We will include them with the amount when the money is paid. As both paid the money over time, it will take a bit until they climb up the sponsor list.

< week 16 week 18–19 >

webpack is not backed by a big company, unlike many other big Open Source products. The development is funded by donations. Please consider donating if you depend on webpack… (Ask your boss!)

Special thanks to these sponsors: (Top 5)

ag-Grid (DataGrid) donated a total of $12,500
Capital One (Bank) donated a total of $12,000
AngularClass (Training) donated a total of $4,018
Google Angular (Framework) donated a total of $2,500
OpenCollective (OS funding) donated a total of $2,394
Full list

MOSS and WebAssembly (week 17)

2017/07/24–2017/07/30

Sponsors

MOSS

Some questions you may have:

Written by Tobias Koppers