Illustrations by Danny Deruntz

LANDING THE CLOUD

How Augmented Reality, Virtual Reality and the Internet of Things are pulling the “Cloud” down to earth

Danny DeRuntz
IDEO CoLab Ventures
6 min readOct 14, 2016

--

In virtual reality we shot a bow & arrow and “felt” the tension in the string. In an augmented reality, we caught an unsuspecting Pokemon at a dog park. At 1am. We surrendered the climate of our house to an internet of things. Soon we’ll surrender our commute. Any one of these technologies is worthy of the attention they receive. But we’re most excited thinking about these technologies woven together. For that to make sense, first we’re going to rethink this thing called the cloud

Fluidly here, not Safely away

The cloud, as it was originally introduced, was about scalability, reliability and security — all excellent advantages for one of its initial use cases, and its very popular buddy: storage. I’m not sure if it’s the word “cloud,” or the idea of de-localized data, but it all evokes the idea of things being kept safely away and remote. That is until we call that data back down to earth.

Now we have an estimated internet of 6.4 billion things connected, streaming to and from this cloud. Constantly. It simply shouldn’t be thought of as a disconnected entity floating up in the sky. Better to think of it as something existing and woven all around us, like another dimension.

The Cloud Dimension

The cloud captures and keeps our memories. It senses and tracks us and our environment. It learns how to imitate the way we think (no wonder we’re sending it more and more of our jobs!). Like all good alternate dimensions, the cloud dimension is connected to and reflects our reality.

I recently got my hands on a Microsoft Hololens and proceeded to pack my house full of virtual objects and applications. It’s so easy to think of these things spatially. I still know where all the items I placed are, even without seeing them. It’s the same when my wife plays Pokemon Go: she’s aware of pokemon without her phone out. She expects to find Pikachus whenever she spots electric towers.

As humans we use our physical space to organize our world: the rooms in our house, the papers on our desk. And so we’re constantly making spatial links between our hard and soft realities. As the links increase, this cloud dimension is becoming more like a living and breathing world: ready to be explored, understood, and shaped!

Let’s talk about three mediums that are going to propel this exploration — Augmented Reality, Internet of things, and Virtual Reality — and we’ll talk about the complementary roles they play.

TEMPORAL RIFTS

by Augmented reality

We have a universe of fluid data all around us. Augmented reality is a practice of us tapping into it, letting it leak into our context, on-demand. To illustrate: Think about shopping for something as basic as an apple. There are many invisible factors that could help us make better choices. Are there chemicals on it? Is it local? How nutritious is it? Will it taste good? While shopping, we automatically make a lot of assumptions about these questions, but real answers to these kinds of questions exist. Soon:

When an apple is picked, it’s scanned and given a unique identity in the cloud dimension. We inform that dimension of its nutrients and other key pieces of information. It connects it to time, to places, and everything it knows about those places. The data realm then tracks and directs its every move all the way to the grocery store. We now have a way to know the true quality of an apple: its nutrient content, how old it is, how much energy it took to get here, its taste profile. And we can access any of it with a glance. We can make the truth of an apple spill out before our eyes.

We’ve just begun to explore this and we’re definitely excited about tapping in with AR and letting these two worlds mix. It’s hard to guess what will be too much or too little augmentation for different people. Because AR is so deep and spatial, it has more affordances than many other mediums. We’ll probably feel like guinea pigs for a while, but ultimately we’re going to find a powerful mix!

MODERN MEDIUMS

by Internet of Things

There are points where the flow between realms wants to be constant, where things should share their presence with one another and communicate. This is the role that the internet plays in connecting things. To illustrate: Today, our homes are getting better at being efficient by coordinating usage around our living habits. But, imagine if we could coordinate and share our energy consumption with our neighbors or with people who share our values or behaviors? This could not only lower cost, but also optimize our infrastructure. Soon:

When a house is built, it’s given a “connected” fusebox (its soul living in the cloud dimension). Most of the houses in the neighborhood will have one. They can detect and identify any device drawing from them, and they know what we humans need, and what our devices need. They also know how to coordinate to avoid energy spikes which drive up cost. While our homes are turning the lights on and off in our realm, this translates into lending and borrowing “energy” in the data realm. And they’re doing it directly with one-another on our behalf.

This IOT is becoming more thought-like and thought-full every day. Soon, if a tree falls in the forest, the forest is going to hear it, and maybe even do something about it. Are we making life easier for people? Or are we creating new “natural” cycles for the entire planet? Lots to explore!

CROSSING OVER

Virtual Reality

As often presented, VR (Virtual Reality) is about bringing our personal activity into a realm made of data, without natural constraints. But we don’t always need to send a human along with an activity, especially when an activity has its own intelligence.

VR can peel back the lid on our smart cloud and let us jump wholly in — helping us see what we otherwise can’t. To illustrate: Think again about homes trading and debating their own energy usage. Keeping the lights on at the right times is handled well by an internet of things. Deciding which neighbors to share energy with — or what kinds of companies to buy power from — these are decisions we will want to understand and participate in. But how and where? Soon:

Your connected house has been terrific. You and your neighbors have lowered both your costs and usage. Through AR you see an invitation to the quarterly town-hall. You slide your glasses to full immersion (VR) and your house peels away. You are looking at your neighbors and your neighborhood’s lifelines. It’s time to vote on energy sources. You see the energy veins leading off to your primary energy provider, but notice some interesting visuals. Air sensors show carcinogens around that power plant. Clearly your energy has “hidden costs.” You notice a group of your neighbors have placed flags at a different plant just beyond, voting for a shift. You fly over and place your marker seeing that it will only cost you $2.74 more per month. But, by joining, you lowered the cost a few pennies for everyone. The health and pollution reports update their projections for next year. Now it’s time to look at transportation…

When can we make better holistic decisions by removing ourselves from our physical constraints? Will we see the forest by floating above the trees? Will immersing ourselves in simulated consequences affect our ethical choices? It’s exciting to think about simultaneously giving people massive context and micro understanding. You heard us right. VR, AR, IoT: they will unlock our humanity. Just roll with that for, like, two minutes.

It‘s going to be fun watching all this cloud talk come down to earth in the next few years. Of course, the best part will be getting out there and helping it stick the landing.

See how the IDEO CoLab is navigating the clouds of AR/VR/IOT here

--

--