Web Worker easier than you thought
In this article, a DIRTY, unsafe, unstable and scary <em>eval</em> method will be described. So, if you are uncomfortable with that, stop reading right now.
First off, some issues with convenience remained unresolved: in code sent to web web workers, closure can’t be used.
All of us like new technologies, and all of us like new technologies to be convenient to use. However, it’s not exactly the case with web workers. web workers accept files or links to files, which isn’t convenient. It would be good to be able to put any task into web workers, not just specifically planned code.
What do we need to make web workers more convenient to operate? I believe, it’s the following:
- A possibility to launch in web workers any code at any moment
- A possibility to send to web workers complicated data (class instances, functions)
- A possibility to receive a promise with a reply from a web worker.
Let’s try to write it. For starters, we’ll need a communication protocol between a web worker and the main window. In general, a protocol is just a structure and types of data used for communication between a browser window and a web worker. It’s pretty straightforward. You can use this or write your own version. Every message will have an ID and data typical of a specific message type. Initially, we’ll have two types of messages for web workers:
- Adding libraries/files to a web worker
- Launch.
A file that will be inside a web worker
Before writing a web worker, we need to describe a file that will be inside of it, supporting the protocol described above. I like object-oriented programming (OOP), so this will be a class named workerBody. This class has to subscribe to an event from the parent window.
self.onmessage = (message) => {
this.onMessage(message.data);
};Now we can listen to events from the parent window. We have two types of events: those which imply a reply and all the rest. Let’s do events: \
Libraries and files are added to a web worker using importScripts API.
And now the scariest part: for launching a random function, we’ll use eval.
... onMessage(message) {
switch (message.type) {
case MESSAGE_TYPE.ADD_LIBS:
this.addLibs(message.libs);
break;
case MESSAGE_TYPE.WORK:
this.doWork(message);
break;
}
}
doWork(message) {
try {
const processor = eval(message.job);
const params = this._parser.parse(message.params);
const result = processor(params);
if (result && result.then && typeof result.then === 'function') {
result.then((data) => {
this.send({ id: message.id, state: true, body: data });
}, (error) => {
if (error instanceof Error) {
error = String(error);
}
this.send({ id: message.id, state: false, body: error });
});
} else {
this.send({ id: message.id, state: true, body: result });
}
} catch (e) {
this.send({ id: message.id, state: false, body: String(e) });
}
} send(data) {
data.body = this._serializer.serialize(data.body);
try {
self.postMessage(data);
} catch (e) {
const toSet = {
id: data.id,
state: false,
body: String(e)
};
self.postMessage(toSet);
}
}
The method onMessage is responsible for receiving a message and choosing a handler, doWork launches a sent function and send sends a reply to the parent window.
Parser and serializer
Now that we have the web worker’s content, we need to learn to serialize and parse any data, so they could be sent to the web worker. Let’s start with a serializer. We want to be able to send to the web worker any data, including class instances, classes and functions, while the web worker’s native capacity enables sending only JSON-like data. To go around that, we’ll need _eval_. We’ll wrap all data that JSON can’t accept into corresponding sting structures and launch them on the other side. To preserve immutability, received data will be cloned on the fly, replacing whatever cannot be serialized by ordinary methods with service objects, which will be replaced back on the other side by a parser. At first sight, this task isn’t difficult, but there are many pitfalls. The scariest limitation of this approach is the inability to use closure, which leads to a slightly different code writing style. Let’s start with the easiest part, the function. First, we need to learn to distinguish a function from a class constructor. Let’s do that:
static isFunction(Factory){
if (!Factory.prototype) {
// Arrow function has no prototype
return true;
}
const prototypePropsLength = Object.getOwnPropertyNames(Factory.prototype)
.filter(item => item !== 'constructor')
.length; return prototypePropsLength === 0 && Serializer.getClassParents(Factory).length === 1;
} static getClassParents(Factory) {
const result = [Factory];
let tmp = Factory;
let item = Object.getPrototypeOf(tmp); while (item.prototype) {
result.push(item);
tmp = item;
item = Object.getPrototypeOf(tmp);
}
return result.reverse();
}
First, we’ll check if the function has a prototype. If it doesn’t, this is certainly a function. Then, we look at the number of the prototype’s features. If it only has a constructor and the function isn’t a successor of another class, we consider it a function.
When we discover a function, we just replace it with a service object with the fields __type = “serialized-function“ and template corresponds to the template of this function (func.toString()).
For now, we’ll skip class and look at class instance. Later, we’ll need to distinguish between regular objects and class instances.
static isInstance(some) {
const constructor = some.constructor;
if (!constructor) {
return false;
}
return !Serializer.isNative(constructor);
} static isNative(data) {
return /function .*?\(\) \{ \[native code\] \}/.test(data.toString()); }
We believe that an object is regular if it doesn’t have a constructor or its constructor is a native function. Once we’ve discovered a class instance, we’ll replace it with a service object with the following fields:
- __type: ‘serialized-instance’
- data is data contained in the instance
- index is the class index of this instance on the service class list.
To send data, we have to create an extra field, in which we will store a list of unique classes that we send. However, there is a challenge: discovering a class, we need to take not only its template, but also the templates of all parent classes and save them as independent classes, so every parent class gets sent only once, also saving instanceof proof. Discovering a class is easy: this is a function that failed our Serializer.isFunction proof. When adding a class, we check the presence of that class on the list of serialized data and add only unique classes. Code that assembles a class into a template is quite large and is available here.
In the parser, we initially check all classes sent to us and compile them if they haven’t been sent. Then, we recursively check every data field and replace service objects with compiled data. The most interesting part is class instances. We have a class and data that were in its instance, but we can’t just create an instance as a constructor request may require parameters that we don’t have. We get that from the nearly forgotten Object.create method, which creates an object with a set prototype. This way, we avoid requesting a constructor, get a class instance and just copy properties into the instance.
Creating a web worker
For a web worker to operate successfully, we need a parser and a serializer within the web worker and outside. So we take a serializer and turn it, parser and web worker body into a template. From the template, we make a blob and create a download link over URL.createObjectURL (this method may not work for some “Content-Security-Policy”). This method is also good for launching random code from a string.
_createworker(customworker) {
const template = `var Myworker = ${this._createTemplate(customworker)};`;
const blob = new Blob([template], { type: 'application/javascript' }); return new worker(URL.createObjectURL(blob));
} _createTemplate(workerBody) {
const Name = Serializer.getFnName(workerBody);
if (!Name) {
throw new Error('Unnamed worker Body class! Please add name to worker Body class!');
} return [
'(function () {',
this._getFullClassTemplate(Serializer, 'Serializer'),
this._getFullClassTemplate(Parser, 'Parser'),
this._getFullClassTemplate(workerBody, 'workerBody'),
`return new workerBody(Serializer, Parser)})();`
].join('\n');
}
Outcome
So, we got a simple-to-use library that can send any code to the web worker. It supports TypeScript classes, for instance:
const wrapper = workerWrapper.create(); wrapper.process((params) => {
// This code in worker. Cannot use closure!
// do some hard work
return 100; // or return Promise.resolve(100)
}, params).then((result) => {
// result = 100;
}); wrapper.terminate() // terminate for kill worker process
Future development
Unfortunately, this library is far from ideal. We need to add support of setters and getters for classes, objects, prototypes and static features. Also, we need to add caching, an alternative script launch without eval, using URL.createObjectURL instead. Finally, a file with the web worker content needs to be added to the assembly (in case on-the-fly creation is not available) etc. Visit the repository!
Read Waves News channel
Follow Waves Twitter
Watch on Waves Youtube
Subscribe to Waves Subreddit
