UX in a Hybrid World — From Dinosaurs to Diagnostics

We look at how mixed reality (MR) is shaping a hybrid world where reality and digital reality coexist, and what this means for UX designers.

It starts like this. You step into the time machine and strap yourself in to one of the many pods before you. The pods start to rattle and shake, flashes of light and loud noises create a sense of chaos around you, and your journey back in time begins. Before you know it the doors open and you have arrived 67 million years into the past and right into the time of the dinosaurs.

No, this isn’t Jurassic Park, but it’s the closest thing you’ll likely find to it.

Welcome to Dinosaurs in the Wild. In this immersive experience, the audience (the users) is guided on a safari surrounded by living dinosaurs. Of course these dinosaurs aren’t actually living. The experience is a combination of theatre, theme parks and technology that comes to life through high resolution screens, virtual reality, 3D technology, CG animals and holograms, animatronic dinosaurs, lighting, sound, and a team of actors playing the roles of guides, scientists, laboratory workers and more.

But for 70 minutes, the audience suspends disbelief. They become lost in the experience. They are able to escape reality in favor of a mixed reality.

It is an exaggerated example of what is becoming an increasingly hybrid world — one where the lines between reality and virtual reality begin to blur, coming together to create new environments and experiences. It’s a conglomeration of augmented reality, virtual reality and, well, actual reality.

MR, the T-Rex of AR and VR

Known as “mixed reality,” or MR, this hybrid technology is often positioned as the next big thing. However, the key to making these experiences successful and believable exists in a tradition as old as time: storytelling.

“I think one of the most powerful tools in your kit bag is story,” said Tim Haines, creative director of Dinosaurs in the Wild.

Haines knows. The idea for Dinosaurs in the Wild first came to him more than 15 years ago. A science journalist by trade, Haines is best known for his BBC documentary-style television series Walking With Dinosaurs, which also lead to the production of other programs including Walking With Beasts and Walking With Monsters. He has made a career out of telling stories about dinosaurs.

Much of the team that helped bring this vision to life, including producers Jill Bryant and Bob Deere, come from a journalistic background, having worked with Haines at BBC on Walking With Dinosaurs. Story is intrinsic to their approach, but sharing this story with a diverse user-base presented a design challenge that involved some creative thinking.

Though he may have taken slight offense when I likened him to Jurassic Park’s John Hammond, prompting him to say that he’s “not nearly as old and famous…or dangerous,” he did admit that creating complex experiences like this is a difficult line to tread.

“You want to capture a large group. Adults will say, ‘couldn’t we have more gore, couldn’t we have louder sounds,’ then when you see a little one crying you think of dear, I don’t like that,” he said. “I think we’ve got the combination right. The little ones are excited and the adults appreciate that it is made with care and detail, and the story is complete.”

It wasn’t until four years ago that Haines’ dream began to come into fruition. To create an experience of this magnitude, he needed the support of investors. What better way to win them over than by bringing a mini version of the experience to the pitch with them?

“We presented to possible investors by sitting in a room and changing one of the windows to a screen and they didn’t notice,” Haines said. “We pulled up the blind and the T-Rex came down the street outside and said, ‘hello.’”

Hybrid Entertainment

Though Dinosaurs in the Wild is a UK production, there are examples of this type of experience sprouting up in America. Sea World recently unleashed a ‘Kraken’ VR Roller Coaster, and improved 3D and holographic technology has made its way on to the big screen, such as in the sci-fi film starring Scarlett Johansson, Ghost in the Shell.

These experiences tend to exist in the sphere of entertainment, but their place extends far beyond it and the potential for this type of hybrid experience is presently unmapped, only now its possibilities are beginning to be explored. These entertainment examples, however, can be used as case studies for what makes a good user experience in hybrid technology. How can the technologies be used effectively and what can we learn from them in these instances?

“I think this combination of information and entertainment, which is often given horrible names like infotainment, is actually a very potent area to get it right,” Haines said.

In an article that appeared in Creative Review, writer Mark Sinclair spoke with studio Territory about the production of Ghost in the Shell.

In the piece, the studio’s director David Sheldon-Hicks says, “Futuristic tech design really has to be developed as though it is to be used in the real world, and requires UI/UX experience, a strong aesthetic understanding and a passion for storytelling to get it right.”

Part of the approach to Dinosaurs in the Wild and Ghost in the Shell was an exploration into the concept of the screen and how screens — or lack there of — shape our experiences, and the role they play in advancing a story.

In DITW, Haines’ team used the latest screen technology in a way that made the screens look like windows into another dimension, whereas in Ghost in the Shell, Sinclair writes that the team looked at, “how people are likely to engage with tech in a world where screens of any form are obsolete. Instead, interactions based on gestures, voice and thought replace traditional screen-based interfaces.”

In both cases, screens were approached in unconventional ways even though the screens themselves are a crucial component of delivering both experiences. Each gives designers much to consider.

Hybrid UX in Healthcare

The technology involved in these experiences is the same technology that is being used to advance other industries such as the medical field.

In an interview with Medical Device and Diagnostic Industry Online, Brandon Bogdalek, a healthcare design consultant at Worrell, talked about how mixed media combines virtual and augmented reality with the introduction of the technology’s ability to detect space.

“The mixed reality technology that we use here at Worrell is the Microsoft HoloLens. It actually has the capability to track the room that you’re in and will place holograms relative to what’s tracking,” he said in the interview.

“The use case for a surgical setting, for example, would be a mixed reality approach, where I’m using a head-mounted display — let’s use the Microsoft HoloLens for example here. So, let’s say a patient has a tumor that’s located in the liver and I input that CT or MRI data into the HoloLens. It should be able to pretty accurately detect where that tumor is in approximation to the body and overlay that information.”

As the way we experience and interact with technology evolves and changes, becoming interlaid over our material existence, designers face a whole new world of design challenges and opportunities that shape shift across various industries.

Back to Reality

When it comes to entertainment, however, Haines said experiences like Dinosaurs in the Wild mark the beginning of a new era. Never before have so many technologies come together to tell such compelling stories in real time.

“I think the idea of using more and more mixed technologies to entertain people is something that is going to get more common, and you’ll see a lot of different forms of entertainment moving together,” Haines said. “There are lots of possibilities for new storytelling, one of which has to do with things like VR, very good graphics, and of course MR and MA, which will come along and I think be even more useful than VR for shared experiences.”

Designers, don’t listen to what Michael Crichton wrote in Jurassic Park the novel when he said, “Entertainment has nothing to do with reality. Entertainment is antithetical to reality.”

The line between entertainment and reality has never been more blurred.

Sheena Lyonnais is a Toronto-based writer, editor and digital specialist. She works in content marketing by day, studies digital strategy by night, and practices yoga somewhere in between. Follow her on Twitter @SheenaLyonnais.

Originally published at blogs.adobe.com.

Learn about Adobe XD, our all-in-one design and prototyping tool:

• Download Adobe Experience Design CC (Beta)
• Adobe XD Twitter account — also use #adobexd to talk to the team!
• Adobe XD UserVoice ideas database
• Adobe XD forum