A better way to work with WebWorkers (part 1)

Working on a new PWA (progressive web application) I decided I want to have most of the computation and IO work (rest API calls, data manipulation, PouchDB etc.) run in a WebWorker so the main thread remains free to only deal with user interaction and UI rendering.

Actually I need more than that as this document based application is likely to be open across multiple browser tabs and by using a SharedWorker I can keep memory usage down (by having a single worker serve all tabs) but for the moment lets just consider a simple Dedicated worker scenario, serving a single instance of our application.

Talking to web workers is done using async messaging so you would typically see code that looks like this:

That works, but I don’t like it!

It’s perfect for ‘fire-and-forget’ scenarios but if what I want is to hand off some processing to the worker AND GET BACK A RESULT then this paradigm suffers from three major concerns :

1. The code flow is broken into separate areas of the client calling code
2. The context of the call is lost.
3. Its very very hard to compose.

Lets look at an example to illustrate what I mean: fetching a document from the database and rendering it.

If we aren’t not using a worker we might see something like this:

If we are using a worker we now have to do something like this:

Now if all the worker did was respond to this one type of message and the declaration of the handler that handles its responses was just next to code that calls it (as above) that would kind-of be OK but… its never like that is it?

We’d have a lot more going on in the worker which means that we’ll have lots of different message types coming out of it, which means that these switch statements that determine how to handle each message type are going to be much longer. It is also likely that the incoming message handler (from the worker) might actually live in a completely different file from the code that initiates the process:

 worker.postMessage({type: ‘FETCH_DOC’, docId: 5})

So what we are going to have is code that works but is hard to follow and reason about. Something as simple as fetch a document and then do something with it, now requires us to jump between different parts of the code (and probably different files) to read through and we’re forced to keep more in our heads and that’s hard and I’m lazy (in a good way) so I don’t like it.

So that was my first gripe and while relevant it can still be considered a marginal ‘readability’ concern by those who don’t think (as I do) that readability is probably the most important metric for the quality (and future proofing) of your code but lets now look at my second gripe with the typical worker setup which is even worse: loss of context .

Building on the example above, lets now say that our application has a new use case where it needs to load a document from the DB but NOT for rendering, we just need to read it and process something based on it’s contents.

Can we use the existing code we already wrote? Mmmm… not really! On the worker side everything is fine, we can receive requests for documents (by id), fetch them from the database and return them to the main script. But on the client side we now have a big problem: the handler has no idea WHY this document carrying message has just arrived and what to do with it. Should we render? should we do something else?

A typical solution would be to start using more fine grained message types to differentiate between intents and thread them through the worker. For example:

And this is now not only ugly but also dirty. The worker shouldn’t have to care about providing the context for our calling code or take responsibility for it.

Our readability concern from our previous point been compounded (even more switch cases and complexity to chase through to figure out what happens next and where does the next code to run actually live).

Now is a good time to bring in a linguist to help with synonyms as we’re going to have to come up with a lot of meaningful names for all these message types that DO the same but MEAN something different.

I don’t like it!

It gets worse when you start needing composition. Lets say we now need to load 2 documents and combine them in some way.

Using the current setup that would have to post two ‘FETCH_DOC’ messages to the worker and then wait for the two documents to come back and then run some work using both. OK, but how can we do that? The document carrying messages that come back are separate events and have no correlation at all except for the fact that some calling code in our app really thinks they are related and needs them together.

We’re probably going to start doing things like:

1. Create a new worker method (and message type) that returns 2 documents so we don’t really have to compose at all.
2. Create some kind of imperative semaphore on the calling code side that maintains state to track when the second document has arrived (probably using even more specific and verbose msg type name). Ah, the joys of complexity

note: I realize that option 1 is actually a good idea in this case (i.e have your worker api accept bulk requests for multiple docs at once) but the overall point here is about the difficulty to compose separate worker-bound actions.

So, I hope I’ve managed to illustrate my frustration with what communications with a WebWorker does to my code.

Lets switch hats a second and think about how I would like it to work.

Well, to be honest what I’d like to see is that you can look at the client code and have no idea whatsoever that some of the functionality actually lives across the world on the isolated async island of a WebWorker:

This looks much much better doesn’t it? It also means that I can be selective about what I choose to move to the WebWorker and my calling code doesn’t have to change at all to accommodate it.

But how do we get there?

Well, as you might have inferred I might have a suggestion or two and here’s a spoiler: Its even better than the imaginary code above, it doesn’t use promises and its lazy (just like me).

Read all about it in the next post.