Zelda 3D isometric, Sir Carma, 2016

Pixels and voxels, the long answer

Retronator Stray Pixels

Earlier this year, somebody asked on Quora:

What is the difference between pixel and voxel?

I sometimes can’t help myself, so instead of giving a straight answer, I wrote a whole article on the topic.

I get the question. Society serves you things like the movie posters for Pixels and you are genuinely confused. Is it a pixel? Is it a voxel? Is it a bird? Is it a plane? They sure don’t make it easy on you.

I still can’t decide if because of my love for old video games I should definitely watch this movie or definitely *not* watch this movie.

Fear not, you’re in safe hands now. By the time you’re done reading this article you will know everything there is to know about pixels and voxels and everything in between. Here, take a cookie.

(This is a long article, but I promise you, you’re reading Retronator Magazine which means there are more images than text.)


I’ll first give you just a bit of a background so you understand the full picture. There are two main ways of representing graphics on computers: vector and raster.

Mathematical precision of vector graphics (left) and discrete nature of raster graphics (right).

Vector graphics describe the image with mathematical equations, usually representing things such as lines, curves and shapes. Raster graphics instead describe the image as an array of color values that are positioned one after the other into a grid pattern.

The second distinction in computer graphics is between representing 2D and 3D space. Together with vector/raster divide this gives us 4 quadrants to look at:

Everybody loves quadrants!

Vector graphics

In 2D vector graphics, each point on a line or a shape is described with a vector that has two components (x and y). That’s what makes it 2D (two components—two dimensions).

This is how 2D vectors describe all points in 2D vector graphics.

Below is an example of a so called low-poly 2D vector image.

Uluru the Mighty Dreamer, Anh Tran, 2015

It is constructed completely out of 2D polygons (in this case triangles). The term low-poly means the amount of polygons used to make the image is relatively small, low. This makes the triangles easily noticeable.

Let’s add a dimension. In 3D vector graphics the situation is the same, but each vector uses three components (x, y and z). Three components—three dimensions.

Let’s take a look at a 3D low-poly artwork.

Racetrack iOS Game Concept, Timothy J. Reynolds, 2013

The big difference between the 2D image of the Ayers Rock above and the 3D racetrack here is that we can look at the racetrack from any place we want.

Racetrack iOS Game Concept (alternative views), Timothy J. Reynolds, 2013

To show the racetrack on your screen (which is a 2D surface), we have to select one particular viewpoint and project the 3D geometry to 2D from that viewpoint.

The transformation from a 3D to 2D is called a projection.

That’s how we get to one particular 2D image.

But we can use a trick to show the volumetric nature of 3D geometry even in 2D—we can make an animation where we move the viewpoint around the object (or retain the viewpoint while rotating the object itself, as we see below).

Wagon, Timothy J. Reynolds, 2013

Yay, we can see it’s really 3D, no glasses needed!

Raster graphics

This was just warm-up. Let’s move on from vectors and look at how raster graphics deal with 2D and 3D.

In 2D raster graphics, the image is divided into a number of evenly sized rows and columns:

Turbo Esprit Sprite, Matej ‘Retro’ Jan, 2014

Each cell is called a pixel (from picture element). Besides its 2D coordinate within the grid (x, y) its main property is the color that is placed at that coordinate.

We’ve seen how low-poly vector art uses big, noticeable polygons. If we do the same in raster graphics (use big, noticeable pixels) we get pixel art.

Tribute (cropped), Matej ‘Retro’ Jan, 2005, 2012

With 2D pixel artworks, even though they try to represent 3-dimensional objects (a Lotus Esprit or an X-wing), they are drawn directly onto the 2D grid of pixels. You can’t rotate this image around as we did with the 3D vector wagon above. Similarly, the image of the Ayers Rock at the start of the article also can’t be rotated. Even though it is made of polygons, they weren’t placed in 3D space, but in 2D directly.


So far we’ve covered 2D and 3D vector graphics, as well as 2D raster graphics. The last step is 3D raster graphics.

Exciting stuff ahead!

In 3D raster graphics, the volume is divided into evenly spaced rows and columns, covering all three different directions (up-down, left-right, in-out). This divides the 3D space into cubes, also known as voxels (volume elements or volumetric pixels). Each voxel is defined by a 3D coordinate and the color at that coordinate.

Just like pixel art (which is art with very deliberate placing of pixels), we now have voxel art, where each cube is carefully considered.

StarWars Scene, Sir Carma, 2015

It’s a lot like LEGOs, don’t you think?

Note that because we are in 3D space, voxels too can be observed from any angle. Here is another look at voxel Tatooine:

StarWars Scene (alternative view), Sir Carma, 2015

We can even do animations! Here is an example of one of Sir Carma’s animated voxel characters:

Knight Run, Sir Carma, 2015

Compare this to a 2D pixel art character:

sprite from Final Element, Glauber Kotaki, 2015

You can see how in voxel art, the animation changes the presence (color) of little cubes—voxels—and in the pixel art the change of color occurs on little squares—pixels.

And now you know the difference between pixels and voxels (and much more … haha, sorry about that).

But I won’t stop here. See, the reason why I explained the vector/raster, 2D/3D nature is that on our modern displays, every graphics type eventually ends up being displayed as a 2D raster image.

The reason we care about this in a pixel art magazine is that we can use these types of transforms to create modern styles of pixel art that use art assets from non-pixel art quadrants.

“I can make pixel art out of voxels or 3D models?” say you? Yes indeed! Clever shading and rendering techniques allow us to create uniquely looking visual styles that are taking pixel art into the future.

Vector displays and projection

The chart above is not entirely true. There is a way to display 2D vector art directly, with a small caveat.

When you have a 2D vector image it can be directly displayed only on a vector monitor, such as those used by Atari’s arcade game Asteroids.

Asteroids, Atari, 1979

Here is how it would look like in practice (an Asteroids-like game displayed on an oscilloscope):

Space Rocks (game), Autopilot, via Wikimedia Commons [CC BY-SA 3.0]

We can also display 3D vector images (usually called 3D models) this way.

Like mentioned before, 3D models first need to be projected to two dimensions, resulting in a 2D vector image that can be displayed on a vector monitor.

VEC9, Andrew Reitano & Todd Bailey, 2013

I highly recommend watching the VEC9 trailer as well for a huge dose of 80s badassery:

VEC9, Andrew Reitano & Todd Bailey, 2013

Rasterization

These days you’ll have a hard time finding a vector monitor outside a museum. Instead we use displays that are made out of … pixels!

RGB LCD, Luís Flávio Loureiro dos Santos, via Wikimedia Commons [CC BY 3.0]

A modern LCD display will create different colors by turning on and off (or somewhere in between) little red, green and blue LCD (liquid crystal display) or elements. In case you were wondering, it was pretty similar even in CRT (cathode ray tube) days, just that they used three phosphor types in various configurations that would glow red, green or blue when hit with the electron beam.

CRT phosphor dots, via Wikipedia [CC-SA]

So what do we do if we have a vector image and only a raster display to show it on? Well, to go from a 2D vector image to a 2D raster image, the image needs to be rendered or rasterized. Each polygon (usually a triangle) gets rendered into a grid of pixels.

Rasterization using sampling, a slide from Making WebGL Dance, Steven Wittens, 2013

This can be extended to rendering 3D models on a raster display. First, 3D triangles get projected into 2D triangles. Second, 2D vector triangles get rasterized into pixels.

Star Fox, Nintendo, 1993

Triangulation

Now what about voxels? The most common approach with voxel art nowadays is to represent each voxel as a 3D vector cube. We do this by creating a 3D model with triangles placed along the sides of voxel cubes.

World’s most exciting 3D model, Matej ‘Retro’ Jan, 2016

Just like before, 3D triangles then get projected into 2D image space and are finally rasterized to display a 2D raster image.

Spinning cube (tech demo), Matej ‘Retro’ Jan, 2016

This is how we get to the most common voxel art style today, created almost exclusively with the free modeling tool MagicaVoxel.

Rapunzel tower, Thibault Simar, 2016
untitled, Argo San, 2016
Pokemon Voxel, Playiku, 2016
Cat vs Voxel, Stefan Smiljkovic, 2016
Trench Run, Gabriel de Laubier, 2015
Voxair balloon, Gabriel de Laubier, 2015
Sky Chase, Sir Carma, 2015
Talaak village, Sir Carma, 2016
Latica Cliffs, Sir Carma, 2015

Ray casting

It’s not necessary to take the cube approach though. Each voxel can be thought of as a point in 3D space, a blob of volume at that location. You can draw each voxel in 2D space directly by placing a pixel (or a few) at that 2D location. Or the reverse—you take a pixel on the screen and see which voxel in the scene appears at that location.

This reverse approach is called ray casting. You cast a ray from the viewpoint into the scene and see which voxel you hit. In fact, you cast many rays to sweep the entire field of view.

Simple raycasting with fisheye correction, Kieff, via Wikimedia Commons [public domain]

The technique was pioneered by Wolfenstein 3D where its voxels were entire room blocks, so it was just another way to render the voxel-is-a-box approach. It was very speedy though because you only needed to cast one ray for a whole column of pixels on the screen. This made it an essentially 2D process, which is why we also sometimes call this type of 3D graphics 2.5D (the third dimension is kind of fake as it just allows extruded 2D surfaces).

Wolfenstein 3D, id Software, 1992

We don’t usually think of Wolfenstein as drawing voxels though. We have to make the cells small enough and allow them to have different heights. That’s how we come to classic voxel graphics engines from the 90s.

Comanche: Maximum Overkill, NovaLogic, 1992

Voxels were at first used only for terrain. They had to simplify things so the whole volumetric information was just elevation of the terrain stored in a 2D image (also called a height map).

A height map (left) tells how high the vertical columns of voxels are (black is low, white is high).

Limiting voxel information to a height map means you can’t have overhanging cliffs for example. But jolly gosh, the terrain had the amount of details unlike anything we’ve seen in games before.

Delta Force, NovaLogic, 1998
Outcast, Appeal, 1999

The end of voxels

Ray casting wasn’t the only way games in the 90s rendered voxel information. Other solutions cropped up, each with its own advantages such as destructible terrain or support for vehicle rendering and characters. It was cutting edge stuff! Ironically, this creative diversity would be one more reason for the technology’s end.

Vangers, K-D Lab, 1998

The problem was, by the year 2000, graphics accelerator cards took off. These were specialized pieces of hardware for projecting and rasterizing 3D polygons (today we call these chips graphics processing units or GPUs). They were really fast at rendering triangles, but that’s also all they did. Custom voxel rendering algorithms, including ray casting, were outside their scope.

Hexplore, Doki Denki Studio, 1998

Voxel engines remained to be implemented on the CPU (central processing unit), but the CPU also had to take care of all the rest like physics, gameplay and artificial intelligence. The point for doing graphics on the GPU was to separate rendering onto a dedicated chip, making both rendering faster and freeing up CPU to do more complex simulations. Voxel engines couldn’t keep up with the performance of polygon graphics and so voxels died …

… until about ten years later when one game took them to a whole new level of popularity. The game left the old ways behind and paved the way for the voxel-as-a-cube approach. By now this could be efficiently rendered using GPUs and the rest is history.

Minecraft, Mojang, 2009–present

Definition

Let’s recap what we’ve learned and answer the original question, what is a pixel and what is a voxel, with some mathematical precision.

A pixel is the smallest unit when dividing 2D space into discrete, usually uniform (equally sized) regions.

Each pixel can be addressed by a two-component vector where x and y are both integers. That’s what makes the pixel space discrete compared to continuous nature of vector graphics where each coordinate is a real number (represented with floating point numbers).

Similarly, a voxel is the smallest unit of volume when dividing 3D space into discrete, uniform regions.

There you have it.


Am I done now? No I’m not done.

With a definition as general as this, pixels and voxels can appear in many different ways and we can creatively push the concepts into diverse visual looks. In particular, we’ll look at combining the 4 quadrants of raster/vector, 2D/3D.

Pure 2D graphics

In the old days, if you wanted to draw a 2D sprite onto the screen you would have to directly copy the bits of memory that stored the colors of the sprite into the memory that stored the colors displayed on the screen (this copying is also know as bit blit or bit BLT, which is short for bit block transfer). Today almost nobody renders 2D graphics purely like that. The fantasy console PICO-8 is a modern example as it pays homage to the good old days when blitting was the only way to go.

PICO-8, Lexallofle Games (and respective authors of featured carts), 2014–present

Textures in 3D graphics

Nowadays most graphic engines work with vectors at the lowest level since that’s how GPUs work. The main way to make 2D images appear on the screen in this system is by drawing them on the polygons using a technique called texture mapping.

Textures are 2D raster images that get placed (or mapped) onto 3D polygons.

That is how the vast majority of 3D video games (and 3D images in general) are created at their simplest.

For example, this is a what it looks like when we have a high-poly 3D model and we add a high-res texture to it:

“Mirror’s Edge: Catalyst” Keyart, Per Haagensen, 2016

Since we’re using smooth polygon shading and texture mapping we don’t even need that many triangles to create nice looking characters. Here is a low-poly 3D model with a high-res texture:

Low Poly Peon, Mark Henriksen, 2015

When we also switch a high-resolution texture to a low-res version, we get something along the lines of this low-poly 3D model with a pixel art texture:

Drift Stage, 2014–present (work in progress)

The most prominent example of this is of course Minecraft. Although Minecraft blocks are voxels by definition (they are the smallest discrete volume units in the game), they come in a wide variety of types represented by low-poly models with pixel art textures. Even though they are blocky, many of them are not simple cubes.

Minecraft, Mojang, 2009–present

This covers all 3D model cases (there’s not much need for a high-poly 3D model with a low-res texture combination, but correct me if I’m wrong).

Mirror’s Edge (top-left), Max Payne (bottom-left) and Minecraft (bottom-right)

Textures in 2D graphics

Onto 2D! When we apply texture mapping to 2D rectangles we get the usual modern 2D games. With today’s hardware, each 2D image (in this context often called a sprite) is displayed by being placed on a rectangle formed by 2 triangles. The two triangles (together also called a quad) get rendered with the sprite mapped onto them, making the image appear in their place.

Image parts of the character (left) get texture-mapped onto animated quads (right). Badminton, Matej ‘Retro’ Jan, 2006

Things are pretty straight-forward with high-res images …

Braid, Number None, 2008
Limbo, Playdead, 2010

… but get a bit more complicated with low-res, pixel art textures. It all depends on the display resolution we’re rendering our sprites at.

Braid (top-left), Path to the Sky (top-right), Kingdom (bottom-right)

We’ve seen that we can have a pixel art texture applied to a low-poly 3D model, but still render it at a very high resolution. Think Minecraft. Low-poly models, low-res 16×16 pixel textures, placed in a scene that gets rendered at something like 1920×1080 display resolution.

Minecraft, Mojang, 2009–present

The same can be done with 2D polygons. We can take a pixel art image, put it on a 2D quad and render it to a high-resolution screen so that each pixel in the source image covers multiple pixels in the display.

Hotline Miami, Dennaton Games, 2012

We call this the big pixel art style. Each sprite pixel gets rendered bigger than the pixel in the display so it appears as a bigger square in the image.

Each source sprite pixel being rendered onto 3x3 display pixels, Moonman, Ben Porter, 2011–present (work in progress)

Big pixel style becomes apparent when sprites get rotated or skewed:

Path to the Sky, Johannes ‘Dek’ Märtterer, 2011–present (work in progress)

Look at the foliage above and compare it to a rotating sprite rendered at low resolution:

Kingdom, Noio & Licorice, 2015

Do you see how the pixels of the water wheel stay aligned horizontally/vertically whereas in Path to the Sky the foliage, bird, and bridge get their big pixels angled and transformed?

Kingdom achieves this by rendering the whole game at low resolution and then enlarging (upscaling) only the final image to the display resolution. Path to the Sky, Hotline Miami and Moonman on the other hand render sprites directly to a high-resolution display.

Back to 3D

Kingdom is a 2D game, but its approach can be taken into 3D as well.

We can use 3D models with pixel art textures, but render them at low-resolution to get something like this:

Pixel Art Academy tech demo, Matej ‘Retro’ Jan, 2016

You can see we have correct 3D shading with projected shadows. Even though the result looks like some sort of 2D pixel art it is actually a 3D scene rendered at low-resolution using pixel art textures:

Pixel Art Academy tech demo (scene view), Matej ‘Retro’ Jan, 2016

Animations that are vector based (they use a skeleton rig) can also take advantage of this. Here’s how they look in big pixel style:

Animation rig with reference, Matej ‘Retro’ Jan, 2016

But when they get rendered at low resolution they appear more like pixel art, with aligned pixels similar to Kingdom above.

Pixel Art Academy animation test, Matej ‘Retro’ Jan, 2016

It’s far from lovely hand-crafted, frame-by-frame animations, but hold a certain aesthetic nonetheless, reminiscent of rotoscoped animations of the 90s.

Prince of Persia, Jordan Mechner, 1989

3D effects

Back at high resolution, one game that takes full advantage of being 3D is The Last Night by Odd Tales.

The Last Night, Odd Tales, 2014–present (work in progress)

Their art assets are essentially 2D, but are layered out into a 3D world with all the modern graphics effects you can think of (dynamic lighting, bloom, depth of field, cinematic camera angles, reflections …).

WIP scene from The Last Night, Tim Soret, 2016

In this way they are constructing a very much three-dimensional world that you look at from different sides.

3D scene construction in The Last Night, Tim Soret, 2016

Another example of dynamic 3D lighting was the beautifully atmospheric, but ill-fated Confederate Express:

Confederate Express, Maksym Pashanin, 2013–2014 (unreleased)

While the art assets are still only 2D, they also include images of shading from multiple directions. These get processed by a tool such as Sprite Lamp and produce smooth illumination of sprites from any position of the light source.

Pixel art from voxels

The problem with the above approach is that only the shading of the sprites can be done precisely while the shadows that they cast lack the necessary 3D geometry to be produced correctly. To do that you eventually have to have volume information. Enter voxels!

A great example of this approach is the recently announced Pathway:

Pathway, Robotality, 2016 (work in progress)

Graphics look completely like they’re using pixel art sprites, but the information behind them is actually volumetric. Unlike the voxel engines from the 90s that tried to look as modern and realistic as possible, developers at Robotality don’t have to go further than matching voxels to the size of display pixels. This creates a clever disguise of looking like pixel art, but behind the scenes they have all the 3D information with which to create completely correct dynamic lighting.

The use of voxel geometry to produce pixel art isn’t completely new though. The pioneer here was FEZ with its so-called trixels (3d pixels). Trixels are just voxels that get combined into 16×16×16 triles (3d tiles).

Development screenshots from FEZ, Polytron, 2007

When a FEZ scene gets rendered in-game it is mostly seen with a 2D orthogonal projection, which is how they achieve the traditional pixel art look but allow for FEZ’s trademark view rotation.

FEZ GDC ’09 trailer, Polytron Corporation, 2009
FEZ, Polytron Corporation, 2012

Pure voxels

Finally, since we’ve arrived full circle back to voxel geometry we can leave the world of pixel art behind and just render pure, discrete voxels in 3D space (using the cube approach without any textures).

Lexallofle’s Voxatron is the name of the game in this space.

Voxatron, Lexallofle Games, 2010–present (work in progress)

Do you notice a theme in Lexallofle’s virtual consoles? Pico-8 has a pure 2D graphics engine while Voxatron does the same for 3D voxels. They are the perfect companions.

Voxatron, Lexallofle Games, 2010–present (work in progress)

Voxatron is one of the few examples (if not the only) that stay true to the pure discrete division of 3D space. But its close cousin—the analog to the big pixel style—found its place in many games, especially in the mobile space.

Crossy Road, Hipster Whale, 2014
Shooty Skies, Mighty Games, 2016
PAC-MAN 256, Hipster Whale, 2015

We come full-circle back to Sir Carma. After rising to one of the most well known voxel artists, he’s now using Unity to push the voxels-only aesthetic to further heights with numerous visual effects, similar to what Odd Tales are doing for pixel art.

The Way Back, Sir Carma, 2016 (work in progress)
The Way Back, Sir Carma, 2016 (work in progress)

Voxel Zelda/Atic Atac anyone?

The Way Back, Sir Carma, 2016 (work in progress)

And there you have it, we covered all 2D/3D/raster/vector/low-res/high-res combinations I can think of. I’m sure I forgot something and I’m even more sure we’ll come up with even more interesting approaches in the future, but for now, this is as good as it gets.


I hope you enjoyed reading — kudos for coming all the way to the end! I hope it gave you a deeper understanding of pixel and voxel graphics as well as some creative ideas of where to take them next. For more pixel art features you can follow Retronator on Tumblr, Twitter and Facebook. Inspiration served daily!
—Retro