Learning to Love the Hololens at my First AR Hackaton

Proptech @ Realities Center

Recently I had the opportunity of participating on the Hololens Hackaton at Realities Center London. It was a pretty cool experience and I’m happy to say my team took the #1 spot in the (super friendly and not-at-all-serious) competition.

When I first tried it, I was very skeptical of this version of the Microsoft Hololens. I immediately noticed the very small Field of View and the transparency of the “holograms” under natural light. I wasn’t confident I could build something useful with it. The technology was very impressive, but it felt like an alpha version, reminding of an Oculus DK1 for AR.

At the same time AR is really exciting for me. In an AR/VR world 10 years from now I believe AR will be where people spend most of their time. So having an opportunity to build something for it so early was great.

Our Goals at the Hackaton

We were tasked with finding a way the Hololens could help meet the 2025 efficiency targets set for UK construction.

  • 33% Lower Costs
  • 50% Lower emissions
  • 50% Faster Delivery

Our super talented group included me, Claire Sena, Florian Rathgeber and Kwame Edum-Fotwe (who already had experience with the Hololens and ended up doing most of the work with the prototype).

That’s us

As a team we went in knowing very little about the construction business and what kind of problems there are to solve in it. I focused on speaking to our mentors and learning about what kind of problems they face. I took the hackaton as a real assignment, and tried to figure out how we could get the Hololens to solve a real problem.

One surprising thing was how waste kept popping up as an important problem in construction. Speaking off the cuff our mentor said something like 25% of construction was actually waste. This was due to many issues, and I’ll list some of the most important ones we heard about.

  • Coordination and flow of information on a construction site is an immense problem. There can be 10–12 contractors working on a single build. These are independent companies with separate workforces and contracts. This means…
  • Site Managers are easily overwhelmed. At a construction site today, there are too few of them to keep track of what is actually being built. 50% of the work of a Site Manager is making sure Health & Safety regulations are being followed. They are left with too little time to validate the work of so many contractors. So…
  • Validation of work done isn’t happening. Site Managers have little ability to validate everything that’s being done, or to keep up-to-date records. And up-to-date records are important, because…
  • Product replacement happens very often in the construction industry. Contractors have fixed time-cost contracts, and they are responsible for sourcing their own materials. They might promise to get Material A, and that goes into the manifest. But when they actually arrive at the site months later, their supplier informs them they’re out of Material A. They have Material B, which has close to the same characteristics. Now their contract says they have to install isolation in 2 weeks, and they can’t source Material A on time. And they’ll get penalized if they don’t keep their schedule. What ends up happening is the Contractor purchases and installs Material B. They inform the Site Manager, but he’s overwhelmed and he doesn’t have the time to keep track of everything. And…
  • Changes like this impact the Environmental Modelling of the building, which is done by the architect’s office. Information like product replacement can take up to 3 or 4 days to leave the construction site. Then up to one more week to have the new environmental modelling come back from the architect. By the time that’s done the contractor has installed Material B and left the site.

The BIM or the Building Information Model

The construction industry already uses software as a key part of their pipeline. Thanks to BIM, or the the Building Information Model, detailed plans of the building are actually made in advance of construction. The problem is they are rarely updated. Often by the time the building is finished the blueprints for it are outdated, because so many changes were made during construction.

As we continued to speak to our mentors, we started seeing waste as a problem that is directly related to:

  • The difficulty of keeping track of what’s being done
  • Poor maintenance of records during construction

“What they need version control…”

Just as we finished speaking to our mentors, Kwame mentioned version control. Construction is based on large, complex, many-moving-parts projects that are shared between multiple people or workgroups working on it at the same time. We were all Git users, and we all realized that the problems we were talking about seemed very similar to the ones we see in software.

We started riffing on the idea of version control for buildings, and it started making more and more sense. The Hololens captures very rich data by default, which means you could use it to keep very accurate records of a construction:

  • It creates a 3D mesh of the space it’s in as part of its inside-out tracking
  • It can capture video and audio
  • And it can do speech-to-text conversion using Microsoft’s technology, making any audio notes searchable

Version Control seemed like solution to many of the problems we were hearing about. We imagined how it could work:

  • A contractor coming in would ask for a “fork”, or an authorization to change the BIM. The Site Manager would then be given the option to approve any changes. This would mean both the Site Manager and the BIM would be kept up-to-date on every change.
  • At the end of each workday, the contractor would put on Hololens again and make a quick note of what he’s done, essentially doing a “commit”. That commit would include 3D, audio and video data, which would then be transcribed into text making it machine-readable.
  • Other contractors that would come in could see the work that’s been done and make any adjustments necessary.

There were a few Hololens-specific advantages to this:

  • It would be fast and hands-free costing the workers and supervisors a minimum of their productivity.
  • It would be spatially aware so the records of any changes made would be located precisely where they were made.
  • It would fit the roadmap. The UK Government is now working on BIM “level 3” a new specification. Our mentors seemed convinced our idea of using AR was a good fit for the Government’s objectives.
How “Version Control for Buildings” would work.

We made a small prototype and we ended up winning the hackaton. Here’s the presentation on Slideshare.

Learning to love the Hololens

The hackaton slowly changed my outlook on the Hololens. It still feels like a device from the future, crippled by the limitations of the present. But making this AR prototype made me realize how easy it is to find rich uses for AR devices.

On our group, the more we used the Hololens, the more we liked it. Looking at someone wearing a Hololens never felt as weird or distant as people wearing VR headsets, and to me, that gut feeling alone marks a remarkable difference between the two technologies. After working with VR for two years, AR feels like the exciting new thing that’s about to arrive.

I am currently freelancing in London in VR/360 Post-Production and VR Design & Prototyping. I’m always on the lookout for cool and interesting projects, so drop me a line if you think I would be a good fit for your project.