Automotive Infotainment Prototype in Unreal Engine 4

Introduction

Big news! Epic Games has officially entered the Automotive HMI industry. Why should you bother? Because this announcement carries a great promise of bringing the designers and developers closer when building user interfaces for the cars.

As you might know, we have started playing around with UE4 at Siili_Auto a while ago. We already published a short article about our Automotive Instrument Cluster Prototype in Unreal Engine 4 in March 2020 but this one is about our next step.

Fig. 1 — Automotive Instrument Cluster Prototype in Unreal Engine 4

This time we aimed for something a bit more advanced. We wanted to create an Infotainment system in UE4 and check its promise of the improved design-to-development workflow.

While building it, we set ourselves 3 goals:
- follow “zero code” approach,
- run the infotainment on an automotive hardware,
- use micro-interactions in the UI.

Before we go any further, check out the video below to see the infotainment in action along with the story behind it.

Automotive Infotainment in Unreal Engine 4

Visual Design

We’ve decided to build the infotainment prototype around 5 screens: Today, Music, Movies, Audio and Battery Settings. That gave us big enough room to create non trivial interactions and explore how well UE4 supports making automotive UI components. The interface is mainly flat 2D with a high degree of responsiveness. This provides a clean look and at the same time feels alive and focused on the things that matter. We also wanted to use UE4’s great 3D capabilities in a way that enhances the user experience and goes beyond just nice aesthetics. We’ve decided to use 3D animations to add a feeling of freedom and open space while navigating the infotainment.

Fig. 2A “Today” screen

Fig. 2B — “Today” screen, weather widget

The “Today” screen displays information that we think the user will find useful on a daily basis. It is here where you can check the current weather, calendar, reminders, and missed calls. All these widgets are interactive and will expand on-click to reveal more details. If you scroll down this view you will see recommended TV Shows and Music Albums.

Fig. 3 — “Music” screen, music player

In the “Music” screen you can play around with the Music Player and currently active playlist. Below the player itself, you can find recently played artists, and some recommendations. As before — all tiles are interactive and will expand to provide more information when needed.

Fig. 4 — “Movies” screen, last show

At the very top of the “Movies” screen, you can find your last TV Show. Episodes are marked with three possible states (watched, in progress, and not seen). If an episode has already been started (but not finished) the dark overlay on the episode’s cover shows where you stopped (based on the percentage).

Fig. 5 — “Movies” screen, “Discover” section

Below the last TV Show, there are two additional sections: Discover and What’s hot. You can expand each Movie’s cover to read a short plot summary and get some additional movie info.

Fig. 6A — Unreal Car Controls, “Audio” view

Fig. 6B — Unreal Car Controls, “Battery” view

The “Unreal Car Controls” is more or less a car settings screen. For now, we have implemented Audio and Battery views which should be pretty self-explanatory.

The last visual feature which I want to mention here is the actual transition between the screens. Please check the following quote explaining the main idea behind it:

The heart of the concept is a tile-based HMI with a cube-like feature overlay. Think of it as if the user was placed inside of a minimalist room where each feature of the system occupies one full wall. When the user chooses to change the feature by swiping or pressing a button, a 3D animation transitions one screen into the other with widgets drifting between the walls like a river of pixels.

Fig 7. — cube-like screen transition

Implementation

If you are reading this article carefully you already know that this project was made without a single line of code. How is it possible? I already mentioned “blueprints” in my previous article but here is a quick reminder from the official documentation.

The Blueprints Visual Scripting system in Unreal Engine is a complete gameplay scripting system based on the concept of using a node-based interface to create gameplay elements from within Unreal Editor. As with many common scripting languages, it is used to define object-oriented (OO) classes or objects in the engine. As you use UE4, you’ll often find that objects defined using Blueprint are colloquially referred to as just “Blueprints.”

Thanks to the blueprints our designers have more power and can work hand in hand with developers on the same, final project. It makes the whole development process more efficient and the iteration time is way shorter. Most of the implementation was made by one Technical Artist and one Software Engineer so it should give you a hint of how much work can be done with just a few people.

Fig. 8 — widget for last show (design) created in Unreal Motion Graphics UI Designer (UMG)

In Fig. 8 you can see three movie covers (snowboarder with yellow pants). If you remember my Movies screen description from the previous chapter, each movie can be in one of three states: watched, in-progress, or not seen at all. If the movie is in-progress then the dark overlay will be displayed on it to show how much has already been seen (based on the percentage).

Fig. 9 — UMG widget for last show (blueprint)

In the Fig. 9 you can see how easily such behavior can be achieved in blueprints. The function “Set Icon and Style” carries information about the New Movie Status. This function calls the Sequence node which helps us split the logic into two separate chunks: setting the correct icon and enabling dark overlay if the movie was seen.

Fig. 10 — UMG widget for weather button (blueprint)

Fig. 10 is an example of how you can bind the events together. When the weather button is clicked (red node) it will trigger an animation responsible for expanding the weather widget. If this widget is already expanded then we want to play the same animation backwards to collapse the view. We also use some event dispatchers (nodes with the envelope icon) for more advanced blueprint communication.

Fig. 11 — UMG widget for Music Player (design)

The last example presents a simple animation from UMG’s Designer view. At the very bottom, you can see a timeline with small, red circles. These circles are called keyframes and store values for specific timestamps. We use them to change the heart icon scale over time. This animation is responsible for a “heart pulse” effect when someone clicks the heart icon in the Music Player.

Automotive Hardware

You can create state-of-the-art graphics with all the newest technology achievements used by AAA games but it will not matter in Automotive HMI unless you will be able to put it in the car and run on an automotive hardware.

This is exactly why one of our main goals was to run our application on Qualcomm’s SA8155P board. It’s running Android Automotive 10 and our application was packaged as Android App.

Fig. 12 — Qualcomm SA8155P running our application

Thanks

I would like to thank everyone involved in this project. It was an internal effort and there were a lot of people helping us from time to time between their commercial work. Once again, thank you for all your hard work.

Thanks to Dawid, Mariusz Ostrowski and Monika Nowak for the awesome designs, Mateusz Górski for integrating our application with Qualcomm’s board, Mateusz Zdziebko for preparing the beautiful video, Damian Zygmuntowicz for the website.

Special thanks to Tomasz Synowski for help with blueprints, UMG widgets and our 3D screen transition.

Outro

As always — if you are interested in such content and would like to know more then please let me know.

Make sure to check out the case study of this project on our official website.

Thanks for reading!

Links

Project’s Website: https://auto.siili.com/cases/unreal-engine-4
Project’s Video: https://vimeo.com/463793652
Previous Article: Automotive Instrument Cluster Prototype in Unreal Engine 4
Epic Games’ Announcement: https://www.unrealengine.com/en-US/blog/usa-s-largest-car-maker-builds-hmi-systems-in-unreal-engine
Epic Games’ Landing Page: https://www.unrealengine.com/hmi