Autonomous Vehicle Research Lab Case Study


Our collaborator is researching new concepts for displays for autonomous cars- what becomes possible when there’s no longer a driver and everyone is a passenger. They were looking to develop an avatar that lives on the car dashboard and compliments this new type of automobile experience. The avatar should have the ability to change form and to animate in new ways as it responded to passengers’ moods or chosen themes.

Gray Area staff members Josette Melchor and Chelley Sherman offered to work on the project along with 3D artist and developer, Ray McClure. Ray is a member of the Gray Area Cultural Incubator program and his interactive studio, Dreamboat is located at the Gray Area Theater.


Our collaborator desired an organic look and supplied examples of an artist mixing paint as a form of performance or video art. Over the course of 3 months our teams met weekly to develop a digital experience that blends the artistic references and shape shifting requirements.


The primary challenge of the project was to develop a system for changing the avatar’s shape and animation style. We tried different approaches including a blobby liquid form before coming up with a solution involving skinned blend shapes that enabled a wide variety of interesting possibilities. We would set a series of components that could individually change form and arrange themselves in different configurations. This would allow it to move from an organic animated 3D form to a 2D plane that displays information or moving images. In a 3D modeling software we created basic strips of geometry that could blend into pipes, cones and cloth. These objects were bound to a simple skeleton for articulation.


Unreal Engine was selected as the platform to develop the experience. Ray has been using Unreal Engine for the past year to create audio reactive projection art and VR experiences. Ray felt Unreal’s 3D editor and visual programming language were ideal for rapid prototyping and transfer of knowledge upon completion but it was primarily chosen for its beautiful lighting and environmental effects. Unreal’s virtual reality features allowed Renault to experience and review the ongoing progress in an exciting and immersive way.

Once imported and set up in the Unreal editor we instanced 30 of the shape-shifting objects and explored different arrangements and mixes between the blend shapes- from 2D planes into undulating tentacles to balls of crumpled paper and cloth. Using Unreal’s layered materials we could combine different looks and material properties including video projected onto the surface of the avatar. The audio and driving data were represented in different ways for each form such as glowing emissive tentacles that danced to the music while swaying to the driving direction and wireframe relief maps that bounce. Finally each form was mapped to a keyboard key allowing viewers to morph between states. A GUI overlay was set up to for controlling the influence of driving inertia and mixing textures and materials.

The blend shape solution worked out well both in capability and presentation. We continued to create new forms until the day the project was presented including a standout moment where the avatar resembles a dancing ring of fire.


The result of 3 months of collaboration and development was an interactive experience that allows passengers to select from 8 distinct avatar forms that transition between each other and animate in response to music, driving data and the color of the car environment.

Next Steps

With more time we would like to develop a minimal representation of an autonomous car in 3D so the avatar could be seen in the dash as intended. The future automobile experience that our client is dreaming of is ideal for virtual reality since it requires a bit of imagination to visualize how things will look in 5–10 years as the promise of autonomous driving becomes a reality. VR would allow a short trip across an imaginary town of the future where passengers can enjoy the sites and augment their experience with this new technology, potentially shared with a partner online.

The project’s development is a great model for collaboration between digital artists at Gray Area and corporations seeking creative inspiration.