Stage 5: Testing Prototypes With Instructors

Learning gestures and voice interactions on Hololens

The instructors we’ve spoken to at Rosedale tend to be pretty tech savvy. Due to the nature of their work, they work hard at keeping their knowledge of current trends and technologies up to date. So, when we talked to them about mixed, augmented, and virtual reality, they had some idea of what we were talking about. Having said that, there’s nothing like having them experience it for themselves.

Before we showed Chuck, our resident expert automotive instructor, the prototypes we made, we had him go through the Hololens “Learn Gestures” tutorial. Once he was familiar with how interactions, like gaze, voice and gesture, actually worked we thought it would make for a more fruitful discussion about the prototypes — especially since they were somewhat low-tech in comparison.

Feedback on prototypes for AR and tablet interfaces

Work Orders As Lesson Prompts

  • Instead of using an open text field for instructors to type up lesson prompts, Chuck suggested that we allow them to upload a work order instead.
  • Chuck uses work orders (shown below) to give his students instructions for the class. The work order typically encloses the details of what is wrong with the car followed by steps taken by the mechanic to fix it.
  • In doing so, students build the habit of documenting their work at every stage even before they go out into the industry.
“There’s a little bit of a generational gap, but I see the potential here. It is not taking away the smell, senses and real experience of working on a car.”

Team Mode

  • Chuck liked the idea of having individual and team modes, but wanted us to dive deeper into how team mode would work. E.g. how do you designate who is in charge? How do you stop team members’ actions from conflicting with each other’s? How is information shared across each student’s view?


  • Apart from specific resources that the instructor can upload, Chuck also wanted an option to link to a database of videos and other resources available online. It was important to him that these resources are approved by him as many resources on the internet can be unreliable.
  • There is a patented highlighting system for color coding schematics. It would be good for the schematics on our system to abide by this convention.

View/Workflow Capture

  • Okay to record on car all the time but not when they’re not working on the car.
  • We suggested allowing view capture on when students are synced to a car and are in close proximity to it. This way, the system can avoid other moments that are not pertinent to the lesson and workflow.
  • Chuck suggested having a simple video editing program as part of the backend instructor dashboard to help instructors, like him, edit clips to save/share teachable moments.

Experimenting with gesture, touch, gaze and voice


  • Great for when hands are free


  • Feels natural to look at something when you want to evoke an action from it
  • When coupled with gesture, provides a good way to interact with the system when the shop gets noisy.


  • “If I’m inside the car, I might be more inclined to use voice”
“I wouldn’t eliminate any of the interaction options. They are useful in specific contexts.”