Dash: Driving The Future Of Automotive Training

How might mixed reality transform automotive training to empower the next generation of technicians?

The Problem (and Opportunity)

Automotive repair is complex, dynamic, and multifaceted. With this, was an opportunity for Mixed Reality to: 1) bring scattered pieces of information together in more useful and usable ways; 2) support mechanics in keeping up with rapid technological changes; and 3) bridge the gap between theory and practice.

Overview Of System

Dash is an educational platform that integrates contextually relevant information into dynamic Mixed Reality environments that promote critical problem solving and data driven learning in automotive repair.

Concept Video for Dash

Key Features

  1. Streamlined information from multiple sources (schematics, scan tool data, vehicle specifications, etc.) through contextual overlays on user’s AR field of view.
  2. Remote collaboration and view sharing between instructor and students allows them to see from each other’s point of view. This is especially helpful when instructors want to show parts of the car that are difficult to access and when students need remote assistance.
  3. Manipulation of virtual components gives students the opportunity to interact with theoretical content, such as 3D models of car components. They can zoom in and take components apart to better grasp abstract course material.
  4. Realtime and archived work-log tracking allows instructors to view the steps taken by students while fixing a problem to make suggestions for improvement and share teachable moments with class.

System Map

The Dash system is composed of three main parts; the Augmented Reality safety glasses, a tablet backend to customize courses and AR settings and an internet enabled scan tool that sense real car data and makes it available via Dash’s AR system.

Navigation & Menus

The system’s information and tools can be accessed in three ways:

  1. Primary Menu: consists of options, like resources, scan tool, work order, settings, search and call.
  2. Contextual Menu: pulled up when user taps on a car component to access resources and scan tool data pertaining to that part.
  3. Customizable Dashboard: users can pin frequently accessed information to their field of view for easy reference.

Interactions With The System

We could see Voice, Gaze, Touch, Tap and Gesture being used at different times (depending on the context) and have imagined a solution where the user can largely customize their preferences for interaction. For example, during times when users have their hands full, they have the option of using voice to converse with Dash. To pull up the contextual menu, users touch the physical components in front of them. And, to quickly navigating through the menu, we expect air-tap and gaze as the primary means of interaction.

Dash For Instructors

Tablet Interface for Instructors

Through the backend dashboard, Dash allows instructors to build practical lessons around work orders and link relevant material like diagrams, schematics and instructional videos for students to access in the workshop.

The system also allows easy tracking of a student’s past work and growth in the course. Instructors can make a note of anything they want to discuss with students and save teachable moments to share with the rest of the class.

Tablet Information Architecture

Dash For Students

For students, their augmented reality goggles allow them to bring theory and practice together in meaningful ways. With Dash, they can engage with abstract concepts via interactive models, for example, and dive into the details of how things really work.

Dash brings together information from various sources to give students everything they need to work through complex problems. They can access sensor data via the internet enabled scan tool, view schematics in context, and can customize their dashboards to easily view the information that is most important to them. The system also allows students to call the instructor for help.

Check out our final presentation for more details.