Building the ‘AR-Cloud’: Part One
Why we’re building a realtime, 3D map of the world that machines can understand
Note: This is part one of a multi-part series on what we believe is THE biggest and most exciting challenge in computing today. Read part two here.
Over the last century, computing has transitioned from server-rooms, to desktops, to our pockets. ‘Spatial computing’ is the next step, as we move from content hidden behind screens, to a ubiquitous digital layer that exists in the world around us.
One of the first industries to tap into this ubiquitous ‘digital layer’, is augmented reality.
Why is this such a big deal?
Instead of looking at 2D diagrams in textbooks, Medical students will be able to visualise the body in A.R., allowing them to gain a deeper understanding of how the body works. Designers will be able to rapidly prototype new ideas and see those new ideas materialise before their eyes. Engineers will be visually guided through day-to-day tasks, allowing them to fix machinery that they’ve never even seen before.
A.R. will do for skills & understanding, what the internet browser did for information.
However, there are a number of challenges left to solve before this vision of the future can become a reality. First and foremost, is understanding your precise location.
Up until now, the majority of mobile-AR experiences have been so-called ‘table-top AR’ i.e. experiences designed to be viewed within space no more than a few meters in front of you. In these examples, the location you are in, has no significant impact on the content viewed.
However, to tie digital information to specific physical spaces at a global scale, it’s vital to know the precise location and orientation of the person viewing that content. This is because, unless we know exactly where you are, digital information and the physical environment won’t align.
On most mobile phones, GPS is only accurate to within 7 meters. What’s more, in urban environments, the signals sent by GPS satellites can bounce off buildings, further adding to the uncertainty of where you actually are.
If you rely on GPS for location & compass for orientation, two people in the same space will have totally different reference frames for where they are. Consequently, this means it would be impossible to have a multi-user experience, as everybody would have different frames of reference for where the AR content should appear.
If people are unable to pin content to a particular location & collaborate with others in the same shared space, A.R. will never achieve its full potential.
We need something better.
Introducing the AR-Cloud
In answer to the limitations with existing technology, industry veterans and AR researchers have long proposed something called the ‘AR-Cloud’, described as “the single most important software infrastructure in computing, far more valuable than Facebook’s social graph or Google’s PageRank index”.
In the same way that Google indexes the web, the AR-Cloud (aka Machine-Perception Cloud) serves as an index of the real world, which can be used to align both content and devices to the same reference frame.
Rather than relying on GPS, the AR-Cloud uses computer-vision to provide a precise ‘anchor’ of where devices are in the world, with a higher degree of accuracy than GPS. Instead of your devices knowing roughly where they are, devices using the AR-Cloud will be able to determine exactly where where they are, down to the centimeter, allowing augmented content to have spatial persistence in space. The AR-Cloud allows people to collaborate with augmented content within the same reference frame, as they would with physical objects.
However, the AR & Machine-Perception cloud is far more than just determining your location.
After informing devices ‘where they are’, a second component of the AR-Cloud lets devices know ‘what they are looking at’. We call this second component, the ‘contextual layer’.
This second component of the AR-Cloud allows devices to understand the physical structure and semantics of what’s nearby. This ability for devices to understand what is in the scene around them, will help inform how information should be displayed.
Finally, on top of letting devices know where they are and what’s around them, a third component of the AR-Cloud keeps up-to-date with what’s changing. This is important because, while certain objects like buildings and roads don’t change often, people and things are far more dynamic. The role of this third layer is to track of these dynamic objects within the environment, predict what will happen next and ensure this information can be accessed by those who need to.
Together, the three components of the AR-Cloud allows camera-enabled internet devices to tap into a collective understanding of the world around them.
Over the next few years, we will see a shift from computing devices which we access from behind rectangular screens, to computing devices which have a physical place in the world around us. Ultimately, the AR-Cloud is the system which connects the physical world to digital infrastructure. Any machine or device that needs to understand where it is and what is around them, will benefit from the same core infrastructure.
By allowing devices to understand where they are and what’s around them, we believe machines can become more collaborative, more intelligent & ultimately, more useful. In the age of autonomous cars, advanced robotics & drones, it’s increasingly clear that this new type of cloud mapping infrastructure will be an invaluable component in helping these new industries to thrive.
At Scape Technologies, we’ve been quietly building parts of this ‘AR Cloud’.
From our offices in London, our team has been constructing a 1:1-scale mapping infrastructure for computing devices, that allows content or services to be tied to specific places in the world, outside & on a massive scale.
Over the next few months, we’ll be releasing a running series of articles designed to share our team’s thoughts, challenges-faced & progress-towards building this fundamental infrastructure.
If you are keen to learn more about the work we are doing at Scape or what you can do to partner with us, please, get in touch.
This article is part one of a series on building the AR or ‘Machine Perception’ Cloud. Read part two here.
Over the next few months, as we gear up for public release, we will touch upon what makes our technology unique and demonstrate more of what our system is capable of. If you have questions you would like answered, please leave them in the comments and I will address them in future posts.
Edward is co-founder & CEO of Scape Technologies, a computer vision startup in London, working to build a digital framework for the physical world.
Interested to learn more about Scape Technologies?
We send a newsletter every couple of months, making sense of the AR industry and sharing our progress.
Sign Up to our newsletter.
