Microsoft HoloLens, image via microsoft.com

The New Reality: Microsoft HoloLens and the Rise of AR

Written by: Leigh Bryant

Augmented and virtual reality (AR & VR) are starting to enter mainstream use in a way they haven’t before. Neither concept is new, but advances in both fields have long been confined primarily to the game-play space. Until recently, both AR & VR were thought of as little more than novelty tech — an interesting thing to play around with, but unlikely to have any real value for day-to-day applications.

As the technology has continued to progress, however, we’ve started to see inroads being made into more unexpected use cases for AR & VR. And at Myplanet, we’ve been keen to see how these latest iterations of the technology could be given more tangible, real-world uses for our clients.

“We are focused on more assistive and automated interfaces. We see AR as ready for prime time, and an incredible way to deliver a more assistive experience,” says Myplanet CEO Jason Cottrell.

Fortunately for us, we were given the chance to experiment with that question not long ago, when we began working on a project involving the Microsoft HoloLens. Originally conceived as a proof of concept for a client interested in learning just how this new technology could be applied to their workplace setting, we have since expanded the work to see just how far we can take this emerging technology.

The Challenge

Unlike a typical project, which has at least a general objective, for this project our team had an almost entirely open-ended assignment: discover what is and isn’t possible with the HoloLens. A fairly onerous task, even for experimentally-minded designers and developers. In spite of that open-endedness, the first steps were easy enough for the team to define. On the development front, step 1 was get up to date on the latest in HoloLens advances.

HoloLens technology is brand new, which means it’s constantly evolving. What was known and possible even a month ago is different from what’s known and possible today. Rarely do we get the chance to play in such uncharted territory. That kind of liberty is fun and exciting, but it also meant the top priority was to sort out exactly what the full range of possibilities with the HoloLens were so we could begin applying them to our scenarios.

On the design front, we faced a similar challenge: how do you design an experience for a technology that has barely even been defined yet? Fortunately for our team, Myplanet’s design process is one that leans heavily on experimentation, uncertainty, and small, ongoing adjustments as a project evolves. Using those principles, the design side got to work.

“Our initial milestone was to have the system recognize specific markers,” says Andrew Semuschak, Associate Director of Visual Design at Myplanet. “While development was busy creating a program that would meet this objective, design was creating a narrative that would align that (and other milestones) to a story our customer would understand.”

A few of the wireframes created to bring the HoloLens experience to life

From there, the team continued to engage in a process of rapid iteration, adding and adjusting scope as they continued to define the project. Actively creating the project as you’re working on it is no small task, but the team made it work.

Key Learnings

One of the most interesting learnings for the team in working with this new tech, was the ways our current modes of designing and developing overlap with what we need to do going forward — and the ways they differ.

“The thinking that goes in to MR/AR experiences is definitely more complex and sensory-driven than a static application or feature-driven site,” notes Andrew, “but a lot of the same principles driving experience design and user centrism remain as supports to the overall process.”

In other words, putting the user first remains the cornerstone of what we do. Just because the technology changes, doesn’t mean our mission or our primary end goal (to build a useful, usable experience for the user) should.

That being said, it is a new technology. While many of the patterns and general concepts can be maintained, a lot of the specifics of designing and developing must be updated. As Andrew points out, designers have the foundations already, but there are things you can’t anticipate unless you’ve spent some time immersing yourself. “I think that any designer working on this platform would benefit from downloading Unity and experimenting with 3D modelling,” he says.

And it’s not just the designers and developers that will need to grapple with these new experiences. Users have a learning curve ahead of them as well. One advantage more “traditional” tech experiences have is that our users broadly understand how they work at this point. As Andy Peverley, Lead Developer on this project, notes, “People have a lot of preconceived notions about AR, about what the HoloLens does, and about what each interaction is supposed to be, but no real familiarity. Overcoming all that meant we had to work hard to communicate the intent of each step and follow a consistent approach throughout.” Again, none of that differs substantially from how one would approach a more traditional development cycle, but the heavy lifting up front requires a much more explicit approach than most of us would be used to at this stage.

One of the other key insights we gleaned from this proof-of-concept work was that it’s not just the HoloLens and the AR technology itself that is still in its nascent stages. It’s all the associated technologies as well. Things like voice and gesture controls are themselves still emerging technologies and only just starting to break through to enter common usage — and their interactivity with HoloLens has a big impact on the usability of the HoloLens itself.

“Getting voice right will be a huge boost to this form factor,” notes Jason. Since the device has a key benefit in the hands-free power of voice control, vocal recognition technology is paramount.

The world is shifting that way already, with CUI and accessibility concerns being two of the hottest areas of focus in tech this year. “It ties to our growing focus on conversational interfaces, both at Myplanet and throughout the industry. I think Siri, Google Assistant, and Cortana — already making significant inroads into our homes and lives — will see a big increase in use as this technology evolves,” adds Jason.

What’s Next

So where do we go from here? We’ve put the HoloLens through its paces, created a dynamic potential use case for a client, and pushed the boundaries of what the device is capable of at this point as we think through other worthwhile applications outside of this experiment. But what is the next step?

“Technology should help us at work — far more than it does today,” says Jason. “AR lets us bring in data, guidance, and assistance in a way that feels very natural to our daily interactions.” The HoloLens is a monumental first step in this, but it is just a first step. It changes and updates every day, and those advances will happen exponentially as more companies explore the technology and we start to see the ROI of employing AR experiences in everyday settings.

As gesture and voice controls continue to expand and improve alongside the AR experiences offered by things like HoloLens, we expect to see a rapid expansion in the opportunities for applying AR to daily interactions. The opportunities from an accessibility standpoint alone are staggering, with hands-free and voice-controlled interaction offering options we haven’t had historically. But there is still so much more to uncover.

“Understanding the limitations of these devices and technologies leads us to make better use of their known capabilities. Just from the few weeks spent creating this demo, it’s clear that the HoloLens would be incredibly well suited to be part of an instructional or training toolkit. What it can do for immersive learning as part of a comprehensive plan is endless,” says Andy. By giving easy entry to complex data, the HoloLens and experiences like it will offer incredible support for businesses in training staff, as well as in understanding their users and how to offer the best services and products for them.

Simple, straightforward interactions that allow us to make use of data in easy-to-understand ways we haven’t yet experienced are the next step for business and technology, and AR will be enormously important to that effort going forward. Things like Myplanet’s Digital Marketing Assistant Q are already laying the foundation of these experiences with conversational assistants that make using data a more streamlined aspect of doing business, and AR will take those experiences to new places with more immersive and tangible uses.

For now, we’ll continue to experiment with potential use cases and to seek out new opportunities to push the boundaries of what the technology can do. The good news for us, is every day it can do something new, so the limit to what’s possible does not exist.


Interested in learning more about how we’re using tech solutions to make everyday experiences more enjoyable? Contact us here.