How Unity + Volvo Cars built a 3D Test Track
At the Volvo Cars Open Innovation Arena, one of the tools that we use and enable the rest of the company to use is real-time 3D. Through the partnership we have with Unity, we built an internal simulator — “Product Simulator” that is a representation of a complete virtual vehicle.
We are able to take the latest design models, add the required functions, and use them to validate and verify functions while also conducting user experience research. This enables various organizations and stakeholders to collaborate, explore and develop our products and services.
Unity is a cross-platform game engine developed by Unity Technologies, first announced and released as a Mac OS X game engine at Apple Worldwide Developers Conference in June 2005. Since then, the engine has been gradually expanded to support desktop, mobile, console and virtual reality platforms. Among beginners it is known for its ease of use, and also has popularity in indie game development.
The engine can be used to create three-dimensional (3D) and two-dimensional (2D) games, as well as interactive simulations and other experiences. The engine has been adopted by industries outside video gaming, such as film, automotive, architecture, engineering, construction and aeronautics.
At Volvo Cars, we are using Unity and other real-time 3D engines for big variety of use cases. (https://unity.com/case-study/volvo)
With the help of this technology and using XR headsets such as Varjo, we are able to evaluate the design, from the starting sketch to the road-ready early in the process. This allows us to collaborate remotely and do more iterations.
What we are able to achieve with Unity :
· Improved design-engineering communication and collaboration
· Faster design iteration and behaviour validation
· Reduced reliance on physical prototype vehicles
· More immersive and effective buying experiences
After completing our virtual vehicle, we added the car model in context and simulate scenarios to improve our safety features. We were able to train our cameras and sensors and develop the lidar sensors and other technologies to experience and cover the scenarios that are too rare or too dangerous to test in real life.
We use virtual and accurate models of our test tracks or virtual twin of Gothenburg, where we can compare and analyze how our virtual models and features perform compared to our physical tests on the track. Together we can fuse the findings to deliver the best results.
The “Product Simulator” made in Unity acts as back-end support and connects the system modules, such as vehicle dynamics, user interface, product functionality, and digital interaction. Unity is then used for rendering, animation, dynamic content, and scene creation at various stages of production.
Unity — Realtime 3D
Through our collaborations with partners such as Unity, Nvidia, Varjo and others, we are often asked to share some of our APIs, 3D models and datasets.
The first time we tried to share the product simulator tool with developers and designers was after the MIT Reality hack (https://www.mitrealityhack.com/). We decided to start with various stakeholders and asked around who wanted to share some of their work and tools with the world. That’s how our Developer Portal (https://developer.volvocars.com) started.
First, we released a template together with Unity of an Autoshowroom with the XC40 Recharge, highlighting Unity’s HD rendering capabilities, high-quality materials, and light. This template enabled millions of Unity developers to have an out-of-the-box, ready-to-use car model to try developing car configurators, AR and VR experiences, and games.
With a big interest in the 3D model and pull from our communities, we have decided to also launch a test track template that is now available through Unity hub. We’ve put the Volvo XC40 Recharge on a digital twin of our test track in Hällered where based on a point cloud scan and with the help of the creative team from Unity, we were able to have an artistic yet accurate representation of our track.
Unity test track template
Built using 29 packages, our track is rendered using the High Definition Render Pipeline (HDRP) built by Unity to target modern (Compute Shader Compatible) platforms. The track is designed using advanced material types and physically accurate lighting, based on a point cloud scan of the track.
This template is the first time when a company has openly released both an accurate 3D car model, and a representation of test facilities.
Assets
· A sample car with its controller (Volvo XC40 Recharge).
· Props and decals to populate the roads.
· A sample of trees built especially for this template with Unity’s speedtree solution.
· A realistic sky with animated clouds.
Technical details
The template is highly optimised with performance and size in mind.
· Supports Windows and MacOs.
· High-Definition Render Pipeline
· Template size: 476.46 MB.
Use cases
· This template scene is equipped with riggings and a controller script to serve as an example of how a vehicle can be set up for simulating driving on virtual roads.
· Learn how the car controllers and physics are configured.
· Learn how we optimise the project in order to achieve performance.
Use the template
Prerequisites
1. Have the desktop version of Unity Hub 2021 installed (or higher). Download it here
2. Have Unity 2021.x.x LTS version installed (or higher). Download it here
3. Have a valid Unity license.
Get your template
1. Create a new project in Unity.
2. Select Test track as the template to build your project.
3. You are now ready to build your new 3D application with the Volvo car and the track!
The Unity Test Track Template is designed as a starting point for creators who want to:
· Customize the rigging and controller for a car.
· Use curves to generate and dress roads in the Unity Editor.
· Use scattering tools to dress terrain.
· Understand optimizations you must make in Unity, in order for the test track to run smoothly.
Tools
Scattering Tool — dressing the terrain with foliage and trees. Create rules for excluding areas, such as roads.
Road tool — import point cloud lidar data, and generate road surfaces, with different physical materials, such as asphalt, gravel and grass. Edit curves and waypoints, as well as resolve intersections.
Car Controller — Customize the rigging and controller for a car
Based on the Product Simulator setup, the car controller allows you to equip the vehicle for driving. In order to access and change different values via script, we used Unity Data Items, which allows us to store data, and customize the values.
One example would be to create a sensor that enables the car to detect the road marking lanes, train camera sensors to detect objects, and other vehicles as well, or create your own 2D lidar sensor.
We can use Unity Sensor SDK — SensorSDK is a package built on top of SystemGraph designed to emulate photosensors, such as lidars and cameras. It offers a list of custom system graph nodes to create sensor systems. It also contains Prefabs of existing sensors.
SensorSDK is designed to help users do the following:
· Simulate existing and prototype photosensors in Unity.
· Provide specialized SystemGraph components that are easy to configure, extend and modify.
· Offer users more accurate simulation data.
Here you can find a small tutorial on how to model your own lidar sensor:
Below you can see an example of using a lidar model with our Product Simulator, integrated with software in the loop, where we could simulate thousands of scenarios, and train our sensors to detect pedestrian pose estimation, which saved us thousands of hours of manual annotation and training in the physical world.
As there is more and more tech and software in the car, not everyone has access to the actual product, when they develop or simulate features. By connecting to the virtual twin though, they are able to emulate those features in this tool. For us, it is important to include users, developers, and potential customers in our research and development.
Virtual road trip
If we move a bit from simulation to entertainment, one idea I have been playing around with is having a virtual road trip: driving on the test track of virtual Gothenburg, inviting other users to join you in the car, queuing your favourite song, exploring the car and building your own surroundings.
To do this, I will start with the template. Now I need a character. One of my favourite plugins is ReadyPlayer Me, a versatile and easy-to-use avatar system. You can configure your own avatar and use it on over 3000 compatible games and experiences. The avatar system is compatible with VR, AR, mobile, PC and many consoles.
I think this looks quite close to how I look :)). Now I can download the ReadyPlayerMe Unity SDK into my project, and check the documentation: https://docs.readyplayer.me/ready-player-me/integration-guides/unity-sdk
Import Avatar to Unity
Now I have my avatar in Unity, with a few animations.
To add a character controller, I went to Unity Asset store, where I found this template with a Third Person Character Controller.
To add more animations, such as entering the car, driving. I went to Mixamo by Adobe, a library with thousands of animations compatible with Unity and Ready Player me avatar system.
Now I can download the animations in Unity:
And add them to the Character Animation Controller:
Now I can walk around, enter the car, drive around the test track:
To be more accurate of a test track in Sweden, I added an animated moose from the asset store to cross the road.
In this case we can test the Large Animal detection safety function.
To add multiplayer capabilities, I downloaded Photon PUN2 plugin for Unity. Powering up most multiplayer PC and mobile games.
Photon comes with a variety of samples, that enables you to start fast, and create a simple server/client integration, as well as multiplayer matchmaking functionality.
Now we can build the project and have a multiplayer virtual road trip with the XC40 Recharge.
We would encourage the community to share what you create with us and engage in potential collaborations. Have fun!
