The Augmented Doctor — What Microsoft HoloLens can do for healthcare

Augmented reality has been the brain child of science fiction writers for many years. However, that once distant pipe dream is becoming our reality. So as we begin to blur the lines between the physical and simulated, it is essential we understand its applications to healthcare — ensuring we continue to make today's science fiction, tomorrow’s reality.

What is augmented reality

Many often confuse the terms augmented reality (AR) and virtual reality (VR). Virtual reality is exactly that, a completely virtual space, with no inclusion of the real world (think Ready Player One, for the nerds amongst us).

Augmented reality, on the other hand, is the physical world that surrounds you, with computer-generated graphics laid on top. So your reality is augmented, not rewritten as in VR.

There are many companies working on advanced AR technology, but in this article we will focus on one that is making strident efforts into the field of medical related AR. The HoloLens from Microsoft.

The HoloLens

The technology behind the HoloLens produces a form of mixed reality. In mixed reality a user is able to interact with objects that are projected into their view rather than just observe them. This concept is being used to revolutionise the way we teach our students, assess our patients, and formulate surgical approaches to complex cases.

Medical Education

Much like your mobile phone, the HoloLens runs off of apps. Two apps that have been developed for the HoloLens are HoloPatient and HoloHuman, a joint venture between Microsoft and Pearson. You can see HoloPatient working in this video:

HoloPatient fixes a problem often experienced by medical schools around the world — getting high quality actors to act as unwell patients for medical students and nurses to learn how to manage these situations, before being thrown into the real thing. The HoloPatient is completely reproducible, adaptive, and perhaps most importantly, will not tire (unless they’re meant to).

In addition to these added advantages, simulations have been shown to improve retention substantially when compared to conventional educational methods. Therefore, many will argue HoloPatient has the potential to train the doctors and nurses of tomorrow faster and better than those of us unfortunate enough not to have had this tool at our disposal. Though, this has yet to be proven.

A poster child of the augmented reality evolution within healthcare is its application to anatomy. HoloHuman is the HoloLens answer to this, replacing 2D anatomical drawings with a 3D interactive human model.

HoloHuman allows you to dissect your way through the human body, with no need to crowd round some unfortunate cadaver. A perfect solution for the anatomically inquisitive, but dissection squeamish amongst us.

Surgery

There are few more obvious applications for augmented reality than in surgery. And this has not been overlooked by developers and reseachers within the AR community.

A team at Imperial College London are looking inside of patients before commencing surgery to help them plan operations. The team has done this by combining CT images taken prior to surgery and mapping these onto the patients’ body. This gives you an exact 3D recreation of the patients anatomy before ever having to pick up a scalpel. Using this technique is making operations safer and speeding up the rate with which procedures can be performed in practice.

Limitations of the Augmented Doctor

Although the HoloLens shows great promise in healthcare, the technology is young and some are wary to adopt it so readily into the healthcare space without further study. A recent paper published in IEEE Transactions on Biomedical Engineering has found it is difficult for humans to perform AR-assisted tasks that require a high level of precision when the objects are in close proximity (within 2m). This is clearly a huge issue if the ‘precise task’ involves a scalpel and the femoral artery.

The reason for this is something called ‘focal rivalry’. Focal Rivalry refers to the fact that humans find it very difficult to focus on both simulated and real objects simultaneously in augmented reality, particularly when these objects are nearby to one another — leaving room for error. An unwelcome revelation for those wanting to improve the accuracy and safety of surgery using AR assistance. It is possible more advanced technology could bypass this issue, however, as it stands, no solution has been found.

The future — HoloLens 2

The HoloLens 2, the upgraded version of the HoloLens, has been released this year. The new headset boasts an increased processing power, wider field of vision and increased battery life when compared to its predecessor. The release of this more advanced headset has prompted large companies to begin developing in AR.

When presenting the HoloLens 2 at an event in Barcelona, Microsoft closed by showcasing a new concept developed alongside Philips for image-guided minimally invasive procedures. The tool they’ve developed with Microsoft allows a practitioner to see the real world superimposed with live data and 3D medical imagery helping to guide precision therapy. In addition to this, procedures can be controlled with voice recognition, eye tracking and advanced gestures.

The tool is not yet commercially available, and does not describe how it over comes the issue of focal rivalry. However, given the minimally invasive nature of the procedures involved, it is possible that focal rivalry may not be such an issue, as most of these procedures involve looking at a screen in order to perform them. Therefore, augmented reality in this case is likely to improve focus, as oppose to hindering it.

Can you develop for the HoloLens?

For the talented developers among you, Microsoft has published detailed documentation on how you can build apps for HoloLens 2, go here to learn how to build in AR today.

Conclusion

Augmented reality clearly has an important role to play in the health tech revolution we are currently undergoing. From education to direct patient care, the potential applications have far from been explored, and the opportunities for young healthcare professionals within this field are tremendous. Although the technology has teething troubles, it is undoubtable that these devices will be a mainstay of our care within the next decade and it is important we embrace them, although with a healthy caution.

If you enjoyed this article, make sure to follow Doctors in Tech, as well as on Twitter and Facebook.

To see all my content, check out my website.