AR for early-prototyping: beyond Hololens, digital brochures and assisted repairs

Heard about enterprise Augmented Reality? I bet it was not as your next day-to-day design tool but as a digital sales & marketing app, or maybe to perform guided maintenance and repairs. Whereas Augmented Reality (AR) is very much present downstream of the product development lifecycle for sales and operators, it is still pretty absent of the design process. It could bring tremendous value there.

Overall, here’s AR’s presence in the product development process:

Consult any AR ecosystem mapping (e.g. here), you’ll find a lot of AR apps for sales, or Maintenance, Repair and Operations (MRO). However, for the design phase far less. Sure, you have mixed reality headsets such as Microsoft Hololens, when those are used for design review sessions it’s a huge “WOW”. That being said the hardware, software and know-how required to use those devices restrict them by nature to the in depth and punctual design reviews i.e. when you have a high-end 3D model, a prepared session, a technical installation, servers ready, technicians, some dedicated place to display, etc.

Those in depth design reviews happen quite late in the design process, right before the physical prototype. Before that, it seems it’s an AR desert.

Let’s check first about the AR presence in the product development process, and then how it can better serve design specifically.

Mapping AR in the product development lifecycle

Examples of AR use-cases. Credits: IKEA, Microsoft, PTC

Here’ s breakdown of AR applications accross the product design lifecycle steps:

• Discover: None
• Design: nothing else than mixed reality headsets — mainly used for in-depth onsite design reviews, with several amazing examples of Microsoft Hololens deployments at major manufacturers, e.g. in the automotive industry at Ford (see this Wired article) AR helps to “create much more intuitive designs, as the concepts will be evaluated in real time as they are being developed”. It also bridges the gap between designers and engineers, gathering them around the same visual representation of a design.
• Sell: many branded AR catalogs — from apps catalogs editors such as Augment, DecorMatters, Neybers, to manufacturing Majors’ own apps like IKEA Place, Amazon’s AR View, or smaller furniture players such as Extremis, mobile-AR let you view products in your home before you buy them. You can see how they fit, and test how they look. Amazon captured it perfectly: “Curious whether a throw pillow is the right shade of blue, or whether a mixer you like will fit under the counter? Now just pull out your phone to find out.”
• Planification and Production: a few here and there — software editors suchas PTC Thingworx, or Re-Flekt, to name a few, bring maintenance work instructions to operators and assembly work instructions or online performance dashboard through smartglasses, projectors or iPads.
• Maintenance, Repair & Operations (MRO): many MRO apps — generally associated with smart glasses, editors and AR devices manufacturers such DAQRI, DIOTA, Microsoft Hololens (again!), aim to empower workers and customers with instant expert knowledge to train, install, operate, maintain or fix anything, anywhere.

Focus on design: the “anytime in less than 15 minutes” rule

A designer friend recently stated: “our customers generally have not more than 15 minutes to review a design concept. I work with customers in Russia and China, they’re abroad, sometimes on-the-road, at an airport, I have no more than 15 minutes to get feedback on a concept. So forget about headsets, even more if I have to set up a system, guide them, etc. it doesn’t fly”.

Credits: Microsoft, example of Design Review withMixed Reality headsets such as Hololens

Indeed, exchanges before design execution are day-to-day, express. Iterations are short. Although they have design review capabilities, the mixed reality headset do not fit because it implies to be at a specific place, at a specific moment, with a specific piece of hardware and specific know-how from your stakeholders. So, Designers will still prefer exchanging around renderings, or the many DIY alternatives you can hear, like projecting a 3D model on a paperboard and marking-up on it with a pencil (yes, pretty fast actually!).

To be more specific, here two major design use-cases where AR has room to grow:

• Iterations around design concepts: as described above from this designer friend, at this stage designer has an idea about the design to build, prepare several sketches varying from one another, and refines over the course of various short loop iterations with customers, coworkers, etc. Today, it mainly happens with static 2D renderings with the issues it bears (see a specific post on the topic here).
• Design validation: designer has several final concepts and want to check which ones the targeted customers prefer. Today, focus group mainly provide feedbacks over static 2D renderings (again with the issues it bears), or physical prototypes (with the long turnaround associated to it).

Mobile-AR brings design iterations anywhere, anytime, to anyone

Apple, Google or Facebook bet the Future of AR will be on your phone, turning any into “a lens to a virtual world”, Moreover “Compared to the likes of Magic Leap or Google Glass, [mobile] apps are simple, almost trivial” (see a Wired post on the topic here).

Mobile-AR looks like this:

Credits: Apple, extract from latest WWDC, Sept. 12th.

Think this is “gadget”, or “good for gaming”? This has actually the potential to change the cumbersome iterations around design concepts.

Imagine the following from the above picture:

• In the center is your design as a digital mock-up at scale
• People around (or connected remotely) are your coworkers or customers, they can experience your design and mark it up interactively

This turns your iPhone into a reviewing tool allowing you to provide precise review output anywhere, anytime.

Some precursory attempts, promising mobile-AR perspectives

Recently, several CAD software editors started investigating on the topic (see for instance this EnTiTi AR/VR exporter for Autodesk Fusion 360), some independent app makers too have started mainly in neighboring business sectors such as the construction industry (see: Building Conversation for instance).

But it appears designers haven’t seen yet the app that will meet with their minimum requirements, and haven’t found a way to replace their good old 2D renderings.

This is the challenge Augmented Review has been working to address.