Graphics, WebGL, and Projector

Projector is a collaborative, multimedia graphic design platform that’s accessible from your browser. Its capabilities and interfaces were built for people who want to create beautiful visuals — social media posts, GIFs, explainer videos, presentations — but aren’t likely to invest in highly-specialized tools targeted at professional designers.

Despite its web DNA and accessible user interface, we didn’t want to compromise Projector’s graphics capabilities when it came to visual fidelity or performance. It’s truly amazing how far you can push the browser with HTML5 Canvas and CSS, but we want our users to be able to achieve outcomes traditionally only possible with desktop tools like After Effects, Photoshop, and Illustrator.

To get advanced, native-desktop-app-levels of visual fidelity and performance, we need to harness the power of the GPU. To be sure, HTML5 Canvas and CSS do leverage the GPU under the hood. But there is no practical way to composite DOM elements at 60 frames per second, apply animated Instagram style filters to images and videos, or draw antialiased animated vector shapes on high resolution displays without making more direct, lower-level use of the GPU. Our first prototypes of Projector were created using HTML5 Canvas and CSS, carefully crafted to squeeze every ounce of visual fidelity and performance we could out of those technologies. While expedient for us to develop, the task was a bit like trying to paint a masterpiece with a paint roller….while wearing mittens.

Enter WebGL. WebGL gives us the flexibility to write individual programs that operate on each pixel in the scene; billions of them per second. If trying to achieve the kind of visual fidelity and performance with HTML5 Canvas and CSS was like working with a paint roller, working to achieve it with WebGL is like painting a mural with a toothpick held in a pair of tweezers. Ultimate flexibility and precision, but a process that can easily devolve into an exercise in painstaking pedantry. It’s no wonder drawing a single, humble triangle is a right of passage for even the most experienced graphics programmers on each new platform we tackle!

Using our experience developing console game engines and tools, we’ve built a powerful set of frameworks, tools and technologies for Projector that help us move past the pedantry, and harness the power of the GPU to get the kinds of outcomes our users deserve.

We’ve developed a shader and material system that dynamically assembles fragment and vertex shaders from a library of hundreds of shader components for ultimate flexibility without sacrificing performance. Our animation system supports animating literally any property of any object in the scene from water ripples to image warps to letter spacing. Speaking of text, we eschew the built-in browser text rendering for a custom font layout and rendering engine that lets us support arbitrary user uploaded fonts that can be filled with everything from solid colors to high resolution video, all at 60 frames per second and beyond.

While we’ve developed many custom graphics systems to power Projector, we look for places where we can use best-in-breed existing open source libraries instead of building from scratch. Not only does this save us time, but we get to leverage some of the brightest and best minds in graphics. Examples include Google’s Skia C++ path operation computational geometry library that underpins Android and Chrome, the industry standard harfbuzz C++ text layout and shaping library, or the excellent libimagequant C library that magically reduces true color images to impossibly high quality paletted GIFs.

While Projector is predominately implemented in the TypeScript programming language, we make judicious use of custom WebAssembly builds of open source C and C++ libraries to help us bring computationally sophisticated capabilities to the browser. Capabilities usually reserved for high end native desktop apps.

Interested in joining us? Send us a note at careers@projector.com!