IBM Design Challenge

Nov 2017

Prompt

Design a digital solution to provide predictive maintenance insights for an airline mechanic. Your solution should allow a mechanic to access multiple data sources in real time to predict asset failure or quality issues to avoid costly down-time and reduce maintenance costs. The early identification of potential concerns helps your user deploy limited resources more cost effectively, maximize equipment uptime and enhance quality and supply chain processes, ultimately improving customer satisfaction.

Initial Research

Understanding the Problem

To begin, I drew something of a service blueprint to understand the front and back end interactions that occur to get a plane from manufacturing to takeoff. This diagram is not comprehensive, but it helped me get a rough understanding of where AI might fit into the system.

Opportunities for AI/IoT:

  • language processing
  • advanced content analytics — examine maintenance history to find connections between various factors
  • IoT sensors in aircrafts deliver real-time data to ground crews — gives them heads up to be ready
  • single access point
  • preventative measures — avoid turbulence, minimize flight path changes
  • can it analyze notes left by crew and previous mechanics to find patterns and predict problems? (text processing)
  • digital library of resources for repair manuals

Research on aircraft mechanics:

  • work environment: hangars, airfield, repair stations
  • loud environment due to engines and equipment
  • hands full/busy
  • mechanics might be spread out over the aircraft — each should be able to update others on his task status
  • repair procedures questions can come up during maintenance — be able to look things up in the middle of another procedure

Existing Technologies

From my research, I was able to set some constraints on the form of the digital repair tool. I considered voice, gesture, and touch as ways of commanding the tool. I felt that touch and gesture were tricky because the mechanic’s hands would be occupied. I also considered the mechanic wearing the manual on his wrist, since his hands would be in front of him while working, but the display might be too small.

In the end, I liked the idea of smart glasses, which could superimpose how-to instructions on what’s in front of the mechanic. A highly visual display, it also allows for hands-free voice command, which can pick his voice apart even from the loud background noise, and doesn’t require (or allow for) complex touch gestures.

“The Glass display rests just above your natural line of sight, allowing people to experience the world and access Glass when they need to.” — Google. It’s important that the display doesn’t impair the mechanic’s view, because he needs to see what he’s working on.

I consulted Google’s documentation to understand the types of inputs that currently exist for Google Glass, which would inform elements of the user interface I would design.

Controls and Interaction

  • Slide finger on the touchpad to scroll
  • Tap the touchpad to select something
  • Head gestures to turn on/off
  • Voice command

Storyboard

Sketches

  1. Weather monitoring alerts crew of a storm approaching Boston. The crew checks the schedule for flights passing through the storm.
  2. Manager issues work orders to his team. Some are deployed to ensure soon-to-depart flights are safe to fly, some to prepare to meet landing flights based on their real time data.
  3. Experience 1: Tim the Mechanic checks his work orders—he starts with priority #1, flight DL12345 located in Bay #11, scheduled to fly in 12 hours.
  4. Experience 2: He checks the IoT sensor logs, which give him a list of items to inspect. He looks at the centrifugal compressor first, the most pressing matter determined by the AI, and finds what is causing the leak. He identifies the part that needs to be replaced, sees it’s in the warehouse and puts in a request to bring it over.
  5. Experience 3: The parts are brought to Tim. He consults the digital repair manual as he repairs the leak, finishing quickly using step by step videos and on-demand information. He enters some notes, and updates the central system about his completion status. In turn, his manager and the team are updated on the status of the centrifugal compressor.
  6. Flight DL12345 is ready to take off on time.

Low-Fi Designs

I used this rough storyboard to understand what interfaces might be needed to support this narrative. I identified:

  • Glass UI
  • Tablets/mobile for viewing more text-heavy information, or videos on a larger display
  • Dashboard to visualize centralized AI data (for managers)

Knowing what devices informed what kinds of screens I would design for, and what visual components I would need to consider. For example, how do I show navigation? How do I indicate what’s clickable? For different devices, how would I use layout and typography to convey information consistently?

Journey Map

Blue: new journey map; brown: old journey map.

Addressing Pain Points:

Mountains of paper manuals makes it difficult for a mechanic to identify and diagnose a problem with a plane.

  • Manuals are digitalized — no paper
  • Hands-free
  • Mechanic can follow the steps for a procedure easily on a screen
  • Can also find how-to’s for specific and relevant information with voice command Web search. Useful for unexpected problems that come up

Not having a centralized digital diagnostic tool slows down the process, requiring mechanics to rely on experience and intuition rather than data.

  • IoT sensors record flight data and mechanical history, logs are kept in a central system and sent to individual devices
  • Sensor logs and previous notes give mechanics tips on where to look first
  • No out of date manuals, and maintenance procedures are optimal and standardized

No current use of AI to optimize supply chain.

  • Use of AI optimizes supply chain by ensuring needed parts are in stock
  • Predictive measures based on which parts are likely to need repair based on weather, flight conditions, and other factors
  • Fewer delays on repairs, fewer aircrafts pulled from circulation

Solution Features


Visual Design Process

Mood Board

My Pinterest mood board (a secret board)

I chose a visual style that was a mix of the engineer drawings (4th column bottom), holographic wire model (4th column top) and simple line vector art (bottom middle).

I wanted the visual style should have a serious and professional feel, because the mechanic’s work holds real weight in ensuring the safety of passengers and crew. At the same time, I wanted to simplify it from full on engineer drawings to make it look more approachable to a mechanic who doesn’t necessarily need all the measurements.

Grid

I used Glass’ existing style guide as a basis for my Glass UI grids and type scale.

Building off of Google Glass’ scrollable “cards” interface, I hypothesized that a mechanic could use a similar interface to keep track of work orders, digital manuals, etc. I used their style guides here and here to understand more about the scale and legibility of text, image and icon usage.

I was the most unfamiliar with Glass interfaces, so I tried several different layouts to see which column widths felt most comfortable to me.

I liked the balance of content space and margins the best in the last one. I also laid out corresponding mobile and tablet grids.

Colors

Color studies to find a color palette with neutral and popout tones. This is keeping in mind that augmented displays like Google Glass would show up against objects in the real world.
I wanted colors that would work on both dark and light backgrounds.

Components, Style, Treatment

Diagrams & Maps

Exploring how to visualize plane diagrams, what fidelity and style, to best highlight parts to make it easy for the mechanic to find the problem.

Iconography/Visual Components

Left: exploring icon styles, and whether icons would be useful. Right: exploring ways to show navigation on mobile.
Exploring navigation indicators that users can’t touch to interact with

Typography

Left to right: Helvetica Neue, Source Sans, Maison Neue

Iterations

Mockup of mechanic’s view with Glass display:

Style Guide

Design Components

Glass

Mobile

Day view
Night view; TOC

Tablet