Non-Intrusive WebGL
Part 1: Context Loss & Preloading
Kicking off from my last article, The Era of WebGL, in this post I hope to explore some of the more technical details on making efficient and non-intrusive WebGL applications. Ideally, “non-intrusive” applications look and feel like any other browser experience, or DOM container, but under the hood, WebGL is employed for full control over rendering.
This series will cover what I’ve learned from my work on kami, Pixi.JS, and the apps I’ve built on top of these tools.
Firstly, and most importantly: know your audience, and pick your battles. Most users would not be pleased to see content sites like Facebook or NYTimes hogging their GPU just for a dazzling banner animation. On the other hand; if you are developing a rich-media site, aiming to meet the high expectations of Awwwards and The FWA, adding a flair of WebGL may be more appropriate. Also keep in mind: if your experience already runs fast on 2D canvas, you probably won’t need WebGL (as was the case with YouTube AdBlitz).
Before I begin, I’d recommend checking out Brandon Jones’ article on Holistic WebGL, as it covers some of the major points of non-intrusive WebGL applications. Here, I’ll be going into a few more details on the nitty-gritty.
Context Loss Hell
A truly smooth WebGL experience will handle context loss in all the right ways. This is much easier said than done: even libraries like ThreeJS and Pixi.JS tend to have trouble handling context loss, as it’s usually something that needs to be built into the engine from the very ground up.
Typical scenarios that could introduce context loss are: the user is furiously switching tabs, plugging in external monitors, hitting the “Home” button on their mobile device, playing other OpenGL games, etc. It won’t happen very often, but when it does, the user is greeted with an ugly message (Chrome) and often a black and unresponsive WebGL canvas.
To avoid this situation, you first need to call preventDefault() on the “webglcontextlost” event. This tells the browser that you’re going to, daringly, try to recover yourself from context loss.
The next step is to re-initialize all of your resources on the “webglcontextrestored” event. This is the tricky part, especially if your app uses dynamic textures or a lot of async operations (e.g. WebWorkers). At this point, all of your handles from createTexture, createBuffer, and such will have been thrown into the wind. A short list of things that need to be done:
- Re-initialize your WebGL context via canvas.getContext(“webgl”)
- Recompile shaders and setup uniforms
- Reload and rebuild meshes, vertex/index buffers, Frame Buffers
- Reload textures and re-upload them to the GPU
- Re-initialize your GL state (glClearColor, glEnable, etc)
There is plenty more info and code samples here.
In my own WebGL utils (kami), I’ve put a great deal of effort into ensuring the context can be restored easily with no extra work on the developer. This is done by “managing” GL objects, and re-initializing them on context restoration. However, there is still lots of room for improvement.
You can simulate context loss with the WEBGL_lose_context extension, like so:
https://gist.github.com/mattdesl/9995467
Preloading the Context
If your app’s preloader is grabbing WebGL textures and meshes, it would be ideal to re-use this to also preload resources after the contextrestored event. This gives the developer a unified interface for preloading the WebGL app regardless of whether we are at the start of the experience, or returning to the experience after a contextlost event.
This was another consideration of kami; you can test its asset loader and simulate context loss in the following demo:
http://mattdesl.github.io/kami-demos/release/assets.html
The source is available here. Perhaps over time we will see more efforts to handle context restoration on top of current tools, like PreloadJS and PxLoader.
Preloading Textures
Speaking of preloading, this is something that could be improved in many applications. Currently, most ThreeJS and PixiJS apps tend to preload images up-front, during the start of the experience (e.g. with PreloadJS). This means the next time they are requested (i.e. by the library handling WebGL), it will be faster since it’s from cache. However, the second request can still cause slight hiccups, especially with larger images, and ideally our libraries should be designed to only request the image once. This is only a minor nuisance, as the delays are generally quite small.
A more serious concern I see among current WebGL tools is how and when the texture upload occurs. To get images into WebGL, we need to “upload” them to the GPU by passing the Image object to gl.texImage2D. This is a synchronous operation which can cause animations to block, and may make your application feel unresponsive. Most current tools do not give the developer much control over this operation; it simply happens “some time in the future,” perhaps all in a single frame, or perhaps over the course of several frames, depending on how the texture is being used.
This is a real issue that we experienced when building Heart of the Arctic, which is built on top of Pixi. The game, albeit beautiful, is by no means a “non-intrusive” WebGL experience. It’s very processing- and memory-heavy, but it illustrates the issue of synchronous texture uploads.
After the game’s preloader is finished, Pixi uploads all the images to the GPU in one step. This caused a few seconds of total unresponsiveness, during which we can’t perform animations. We had to hide the hitch with a fade to pure white.
Jam3, Cossette, Tendril
One solution I examined to fix this hitch was to throttle texture uploads, i.e. no more than N per frame. This may work for certain experiences, but for us it led to occasional hitches and textures “popping in” as they were uploaded.
A better solution, which we didn’t end up exploring, would be to integrate the texture upload into the preload step. This ensures that all textures will be in GPU memory by the time the preloader is finished, and there is no extra wait time or unresponsiveness. Libraries that consider this would also be able to remove the second and unnecessary Image request I mentioned earlier.
Going Further
To take it a step further; a complex preloader may also be able to handle operations such as linking shaders, filling vertex buffers, generating simplex noise, and creating Frame Buffer objects. These operations, alone, are very fast, but together in a large application they may start to add up and cause hitches. This is particularly important with the emergence of mobile WebGL, where things like shader compilation and linking is not as fast. Such a preloader could also tie in nicely with the context restoration step.
I’ve only scratched the surface here, but careful context management and preloading are both integral to creating seamless and non-intrusive applications of WebGL. My next article will examine some common pitfalls and general performance considerations when integrating WebGL into an online experience.
