A better way to work with WebWorkers (part 2). Enter RxJs

In a previous post I presented my frustration with what working with WebWorkers does to my code structure and composition and have dared to imagine a better world.

Ready? Lets go!

To begin with, the 3 main points (broken code flow, loss of context, difficulty with composition) all stem from a single source:

We have no way to correlate the invocation of a worker bound process to its result. They are completely separate things (in this case messages).

That’s lovely from a de-coupling POV but has significant detrimental effects as discussed previously, and in a real life application can actually lead to ever more subtle coupling in the form of an explosion of fine grained message types that carry the context of the call around.

To make thing a bit clearer, think about how we typically work with ajax:

This is clearly an async world but using either callbacks or promises we get a construct that ties the invocation of the process and its result together. The ajax API knows takes care of channeling the result of the call back to the provided callback (or promise) and so you have a construct that has locality.

So how can do bring this to our WebWorker bound scenario?

Well, I think we need to start with giving an identity to each message so we have something to correlate the invocation and the response messages and recognize that they are two parts of the same thing:

OK, now that we have message identity lets see how we can actually tie messages together.

There are various ways we could do this but I think this a perfect fit for one of the most useful tools in my toolkit: RxJs (or any other FRP library).

Yes, I realize this is a heavy dependency to bring in but as I’m already using on in my project (and so should you, seriously) that isn’t a problem.

Let’s see:

TADAA!!! This looks much better, we have locality, implicit context and composition in one go, and to make things even better the whole thing is lazy so we can setup a complete pipeline of functionality upfront and only run it when we want to!

I’m starting to feel better.

Still, there’s a list of things that are still not right: the whole observer creation code is an infrastructure concern and boilerplate code that I‘d rather not have to repeat in each function that is worker bound, the manual manipulation of the ids is brittle and we haven’t even yet looked at the worker side of things.

Let’s refactor! First thing we’ll do is split the infrastructure concerns into a generic helper function:

and we use it like this:

Its starting to feel really nice. The calling code is fluent and I can see what I’m doing easily without having to jump through hoops. But its still not quite there.

We got so focused on the RPC style scenario (action => result) that we neglected those case where we really just want to send a message to the worker and leave it at that without expecting a result. We could just fall back to using postMessage directly but since we’re writing such a nice helper lets have it accommodate us:

OK, all together now:

and we use it like this:

We could do much more, for example right now we support processes that have a single result but since our underlying infrastructure (RxJs) supports streams we can take benefit from it and have a stream of correlated messages represent an ongoing process. For example we could have a sync to remote db action (in the worker) that raises multiple progress events (all using the same id) and our client handler subscribed to this process now fires with each progress report so that we can put to use the full semantics of RxJs subscriptions:

But, doing this will require a few more meta-properties added to our commuter and also to our worker and that reminds me we haven’t even looked at the worker side of things at all yet so lets do that now:

and now that all of our generic setup is in place lets use it:

This is quite pleasant and clean. I like it!

Now the code that actually does the work in our worker (no pun) has got no clue it lives in a worker and that output is going to be serialized into a message etc. All the infrastructure concerns are setup in one place upfront and the rest of the code just goes about its business.

Some notes:

1. On the worker side we could easily do without RxJs at all but as noted earlier I’m already using it anyway so …
2. There are various bits here that are naive, for example the error handling or the way we check for and thread results from handlers that return promises or observables.

I’ve setup a github repo that takes this concept a bit further ahead and adds support for the kinds of ongoing actions suggested above (with progress reports) and for SharedWorkers which are a bit trickier. I haven’t yet had the time to write another post that walks you through but I think you’ll find the code quire readable and if not just drop me a line.

I hope you find this concept useful for your own projects.