DOM to canvas in just 7 years

Published in

Exokit

5 min readAug 7, 2018

You know what’s harder than creating an alternate reality with Javascript? Drawing a <div>.

If you don’t know what DOM-to-canvas is, I’d recommend reviewing the classic by Robert O’Callahan, and possibly one in which he talks about security of the scheme.

But here’s my TLDR:

Browsers — the thing designed to draw web pages — don’t let developers draw web pages. For reasons.

You’d be 👌 right to suspect this is one of the most requested features in a browser. Especially in a world where Rob’s predictions about 3D objects being drawn with HTML are coming true with the whole WebXR thing.

Yet it’s 7 years later and not much has changed with what the major vendors have done with DOM to canvas. It’s still not happening, for reasons.

Of course, the reason given by browsers for not implementing such a feature is security. The browser uses state such as cookies when rendering a web page, which is exposed when you draw it.

It’s the same story as cross-origin policies, iframe issues, and CORS. You might recognize this as another list of web developer banes.

To say we can’t DOM-to-canvas because of the cookies is like saying we can’t take the car because of the elephant.

I just want a browser in my browser

The impetus for this post was that I wanted to run Shane Harris’s A-Frame in-app Browser in WebXR.

And all I could think was:

Cool, I remember haxing this before!
I want to use this!
I wonder which hack and caveat this requires to work…

That’s why I decided to write this post to round up the techniques I’d used and seen used.

27 ways to kinda sorta DOM-to-canvas

Since browsers didn’t implement this much-asked for feature, developers took it into their own hands with varying degrees of success. Here are some hacks you’ll see in the wild:

The Mozilla <svg>

Drawing DOM objects into a canvas

Although it's not trivial (for security reasons), it's possible to draw DOM content-such as HTML-into a canvas. This…

developer.mozilla.org

This technique involves packing your DOM into an XMLSerializer to get an XML representation which you can wrap in a <foreignObject> that you put into an SVG, which you load into an HTMLImageElement, wait for it to load, and then finally draw to a <canvas>.

Also if you’re using images or certain styles you’ll have to compile them to data URLs and maybe insert a delay to make sure they’ve loaded before you try to draw your SVG.

This is the recommended API by the premier proponent of the web.

The capture tab

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/tabs/captureTab

Extension authors would flip if they couldn’t capture the contents of the browser, so browser vendors acquiesced here.

You can capture a tab screenshot if you are an extension. Presumably as an extension you are more trustworthy than a site so they are blessed with the privilege.

The idea is you call a browser-internal API and get back a data URL for a png or jpeg, which you then waste time decoding and drawing to a canvas. You’ll get double-digit FPS out of it too!

The catch is that you need to be an extension, or write one.

Unsurprisingly, I have seen sites in the wild that require you to install an extension to unlock this functionality on the site with a hack.

The fat companion

https://github.com/shaneharris/aframe-in-app-browser/blob/57c3fb4367381725735193b750692b353f94d701/server.js#L9

This is the technique used by shaneharris/aframe-in-app-browser.

The idea is that you start an extra browser somewhere. And that browser does the drawing for you, and sends you back the results.

Generally this browser will be running on some server, so this limits usefulness to the cases where you are running a local app or don’t mind the server/bandwidth bills for your app.

Again this has been used in production.

The webkit.js

http://trevorlinton.github.io/

This one is ingenious. Especially for its time in a pre-WebAssembly world.

The idea is that you literally take WebKit (the rendering engine for Safari and formerly Chrome), and you compile it to Javascript with emscripten. And then you run it in the browser, so you can finally draw things freely.

The most shocking thing is that it works, if you don’t mind 0.1 FPS.

The time capsule

How did we get here?

How did we get to a point where instead of browser vendors solving a problem so many developers want solved, end developers are inventing new modes of computation inception to compensate?

In a gem from 2011 Paul Irish, we get a time capsule into what web developers were thinking back in the pre-ES5-toric times.

Here’s some of the cream of the crop.

Our friend DOM to canvas makes an appearance. As does rethinking the DOM, performance, and straight up throwing away the browser.

A step back

I can empathize with browser security, and I wouldn’t advocate breaking it. By and large browser vendors have accomplished the impossible with an arbitrary code execution platform that works (tm).

But the browser has also dug itself an immense hole: it has to support so many APIs and paradigms and interactions that sometimes things just fundamentally don’t mesh. And so this is why we can’t do a fast DOM to canvas. Because reasons.

And as a WebXR developer I’m caught in the middle. I just want a browser engine that gives me APIs and gets of the way. Maybe I’m running in a container. Maybe I’m shipping to an app store. Maybe all of the DOMs I’m drawing are local files. But the browser doesn’t know.

So that’s why we have open source.