How to create hyper-realistic VR food using photogrammetry

Recreating food in 3D is not the easiest thing to cook up. Using photogrammetry, we successfully delivered a VR experience that portrayed the award-winning onboard product and service that Air Canada offers.

Photogrammetry

From the beginning, it was clear that Air Canada wanted to emphasise the high-quality food they serve on board. But to model convincing 3D food is a quite challenging task, especially taking into account the need to sustain the illusion when shown in VR, where the user has a very good spatial perception.

We used photogrammetry to deliver a hyper-realistic replication of dishes. Photogrammetry is the technical process that allows obtaining volumetric information about physical objects and environments through capturing, measuring and interpreting photographic images and patterns.

Replicating organic elements such as food in photo-realistic 3D is not an easy task. Our brain is very well trained to know how real food looks, so it’s hard to trick it with a 3D replica of something we interact with so frequently in the real world. In aesthetics, the uncanny valley is a hypothesised relationship between the degree of an object’s resemblance to a human being and the emotional response to such an object. To a lesser extent, the risk here was that the same phenomenon would occur, at least in part, with 3D food. Our goal was to stay away from this ‘fakeness’ as much as possible.

To achieve a strong user immersion and to convey the client’s message, the food needed to look as realistic as possible, and that’s exactly why we used photogrammetry. The main purpose was to create truly delicious-looking food, in line with Air Canada’s superlative standards.

The process

After doing some tests in the office, our team travelled to Canada and carried out a shoot of all the dishes in a lightbox, which was offering the neutral lighting setup required to produce the ideal image material. We used a DSLR camera and decided to go with a new piece of software called Reality Capture, which computed everything on the GPU, making it way faster than any of the alternatives.

But the results straight out from the photogrammetry software were not the final step — it only helped with the volumetric information. Only then did the intricate craft start. Some elements needed resculpting, and then we had to texture and repaint a lot of it to achieve the desired results. For this, we used Zbrush and Substance Painter.

In total, we created around twenty dishes, ranging from a small salad to lasagna and a chocolate cake. Each of them came with unique characteristics (shininess, roughness, etc…) which we kept and highlighted as much as possible.

If we had tried a conventional approach to building these assets, it would’ve taken more time and the results would have almost certainly been less realistic. Photogrammetry sped up the whole process and allowed us to achieve superb results.

But this is no magical solution — the mesh re-sculpting and texturing still required a fair bit of manual work and the trained eye of a 3D artist. Eventually, when the environment is required to look quite realistic, high-quality 3D assets coming from scanned material are key in building immersion and true interactivity, especially in VR.

Overall, team effort and the combination of the right skills, craftsmanship and great technologies made these recipes possible. Now that you have seen the results, perhaps you’d like to tuck into a slice of this VR cake?