Canada at Expo 2020 Dubai — Virtual Experience Case Study
Traces
Traces is a multisensorial exhibition designed for the Canadian Pavilion at Expo 2020 Dubai.
We worked in close collaboration with KANVA to fulfil our mission, which was to transcend the physical works while translating their messages and artistic direction into immersive experiences.
All 3D compositions were created with Blender and exported as video or integrated using Three.js if the interactions required it.
You can see these experiences live at pavilion.thefutureinmind.ca
The Jewel
For The Jewel, we centred the digital experience on the physical aspect of the work, using simple click & drag interaction.
We worked in Blender to create a shader that would be as similar as possible to the original work. It is lit from the inside using yellow tones, with imperfections within it highlighted. The yellow lighting on an orange-ish tone was reproduced using a ColorRamp, and the cube’s internal “breathing” was re-created using a particle system.
The refraction effect produced by the amber didn’t allow us to bake the texture, so we had some constraints when it came to reproducing the shader in Three.js. The Blender rendering served as a reference point in terms of expectations for the rendering and compositing, and we were able to reuse the compositing for the flocks of birds by instantiating them, instead of having to place them manually in Three.js.
The Nearness
The experience we created for The Nearness is a dream-like voyage within the vertical landscape of the original work. The imposing form of the work created an immersive effect in and of itself. All that was needed was to bring it to life.
With this work, for the sake of time, development, and performance, we opted to create a pre-rendered video using Cycles. Scrolling launches the video and the flight of the bird, which represents the user within this environment that is frozen in time.
The journey loops infinitely, moving from a high-contrast scene to a dark environment nearly devoid of life, before returning to light and life — as with the variations presented in the physical work.
Our first approach involved a journey outside of the world, with entrances into the monolith and exits out of it. The monolith itself appeared in the distance, like a mirage. This involved having two monoliths, one in front of the other, in order to create the loop — and that caused technical concerns. To avoid suddenly exiting the first monolith, we progressively lowered the alpha value of the shader, which caused the exterior scene to flash when passing from 1 to .999. This problem was fixed with After Effects by rendering the loop without the second monolith, and simply adding a still image of it in AE so that it would crossfade as the original sequence of images disappeared.
But these transitions often resulted in colour banding, the nemesis of h264!
The other problem was that we were spending a huge amount of time outside the monolith (more than a third of the entire journey).
It was much more creatively satisfying to just travel within the monolith itself, thereby reinforcing the impression of acting within this universe, and this also allowed us to avoid a lot of problems. The final version is longer, with much more varied phases.
The Seat
The Seat is an elementary school desk which brings us face to face with our individual responsibilities. The physical work opens on a tragic scene: the image of a rigid bird, struggling to survive. The idea of a scene that plays out in accordance with the users’ actions was very appealing to us, and we decided to once again use the concept of a curtain being pulled aside, revealing the reality behind it.
The mesh used was originally designed for 3D printing, and its topology was very rough, made up of a lot of n-gons and overlapping faces, for a total of 60,000 vertices.
This made the creation of a rig impossible, even after skinning. So we spent a fair bit of time reworking the mesh. A Decimate Modifier allowed us to greatly reduce the number of vertices, and then we made adjustments using Blender’s sculpting tools. After this, we quickly had a clean enough mesh to apply a rig.
The final mesh was made up of 2,345 vertices with a rig/skin sufficiently flexible for the demands of animation.
The early versions used black tones to reinforce the dramatic/oil slick.
The idea behind the animation was to make users feel the fatalism of the situation — with them serving as mere spectators. The bird struggles quietly, as though it knows what fate awaits it, and that it is pointless to fight it.
In the end, we decided on a less dark and quiet version focused on the message.
The Memorial
The Memorial went through a number of different iterations. Our physical reference is a topography inhabited by flocks of birds. It is fixed and timeless, a kind of homage to the fragility and beauty of our ecosystem.
Our first idea was an experience offering an endless flight over the topography, where we would see before us artifacts of species in danger of disappearing, as this work-in-progress video shows.
This idea was quickly set aside, because it did not capture the sense of a memorial.
The next proposal was to offer a distant and sanctified view of The Memorial before approaching it and overflying the work in its entirety. Its distinctive characteristic was that a single point of illumination sprung from the users’ cursor — an element that would be taken up again in the final version.
That final version is an endless flyover of the topography — as in the first iteration — but with the difference that the user is the source of light, the moving force behind this homage. In addition, an interaction was added to the flyover, in which the birds disappear, as if to show their fragility and our impact on this sensitive ecosystem.
We got the topography as a Rhino asset consisting of Bezier curves forming layers. We re-created the contours based on these curves with Rhino, then re-created the volumes by extruding the faces with Blender.
The challenge was to then quickly transform the topography to make it loop. A simple loop was created with Blender by modifying the meshes at the extremities in order to have short seams. Then a layer of clouds was added in Three.js to mask the mirror pattern joining the two instances of the topography.
The bird disappearance animation was created in Blender as a reference (Particle System, Explode Modifier, Turbulences and Wind Force Fields), and then re-created in Three.js.
The Sanctuary
The Sanctuary is a contemplative experience in which humans do not exercise authority. The physical work takes the form of a cubic aviary in which birds come to life through the mechanical efforts of machinery. The only audio that accompanies the work is a raw and simple mechanical sound.
We knew right from the start that we would have to mirror this aspect of a sanctuary protected by humans in the digital experience. Our solution was a cursor permanently stuck outside the experience’s cube. The physical work is in a format that lends itself well to digital media, so we tried to re-create it as faithfully as possible. Compositing was done in After Effects, and consisted of animation in 2D layers within a 3D scene, forming an infinitely repeatable loop.
Emphasis was placed on the following three points:
- Mechanical development in the animation. The lamps light and heat, and the background starts to slowly rotate. As soon as the main animation of the bird begins, a stop-motion animation is activated, as it would on a projector with sprite animation.
- The scene includes multiple perspectives. The foreground with the flock of birds is fixed, as though set on a stand, while in the middle ground animated birds move in a circular arc around an axis. Meanwhile, the background moves very slowly and much less asymmetrically.
- The audio supports and reinforces this mechanical atmosphere.
The earliest versions included more subtle animation of the lamps gradually illuminating and heating. These were withdrawn because of concerns with colour banding when exporting to h264.
The Awareness
The Awareness is a visual and aural experience in which spectators become participants. As viewers take their seats on the four chairs, the sound around them changes, going from an urban soundtrack when the chairs are empty to one inspired more and more by nature when they are occupied.
We drew inspiration from the message the physical experience conveys, and interpreted it as a poetic experience based on typography. The audio launches as background sound, and as we hover over the text some of the letters gradually disappear, allowing a message to appear.
The Gathering
The Gathering inspires collective action. A tangible disturbance featuring flashes of light and sound occurs inside this amber-coloured world filled with birds whenever people approach it. It is only by coming together and sitting on the chairs that the birds are calmed, revealing their visual and aural universe.
This is what we sought to re-create in the digital experience. The world hides within a cube that opens as visitors hover over four different elements. We move from an agitated aural ambience to one that is calm, as a luminous world unfolds in front of us.
The Forgotten
The Forgotten demonstrates our collective indifference to the beauty that surrounds us. These amber cubes, containing fossilized birds and habitats, are pile up in a corner, completely abandoned and ignored. Gradually, their radiance fades.
We wanted to cast users as the agents of change in this work. The digital version focuses on the installation’s message and invites us to interact with indistinct forms, maintaining the intensity of their illumination and keeping hope alive through our interest in these shapes — which represent memories and thoughts. We become increasingly aware of that which surrounds us. Each of the forms is associated with its own distinctive sound, which plays as background sound for a period of time, until users move their cursor and the sound gradually fades away.
You can see these experiences live at pavilion.thefutureinmind.ca
Check out our other works at akufen.ca
Traces is the result of a collaboration between KANVA, Creos, Étienne Paquette, the National Film Board of Canada (NFB) and Global Affairs Canada.
