The Spatial Computing Exercise

Robert Scoble
Jul 13 · 10 min read

Republished from: https://www.linkedin.com/pulse/spatial-computing-exercise-robert-scoble/

Since Irena Cronin and I are writing a book and doing research for our company, Infinite Retina, we have been interviewing dozens of people and talking to them about augmented and virtual reality and how they are using such in their lives. I realized I needed to look at the world differently by doing that work (the book will come next year). VR and AR add massive new capabilities, but not always in the places I was expecting. Part of that is due to the cost. A HoloLens is $3,500 so enterprises are only using them where they will see a very high return on investment. During surgery, for instance, or dangerous work on an oil refinery. As costs come down, in VR, we are seeing more training projects opening up, too, which shows that as the industry matures new opportunities will open up.

Dreaming about a world a decade from now, when the cost and size and complexity will come way down, requires thinking differently. I find myself often holding my glasses up and dreaming about what I want on a digital display at that time. That’s what the picture above is all about.

So, lately, I’ve come up with a new exercise for myself. I started parting up my day and thinking about where would I be open to using new kinds of computing. I have an advantage, I already have to wear glasses simply to see the world (that’s a photo I made during this exercise on one of our family walks).

The day; the night

We only have 24 hours a day. That’s 1,440 minutes. I sleep around eight hours a day. That’s 480 minutes taken out of the day. Will computing play a role during sleep? Possibly. It might watch me sleep, making sure I am breathing correctly, etc, but it won’t play a major role in most people’s sleep. Yet Siri and Alexa and Google Home are all listening, ready for questions like “hey Google, what time is it?” or “hey Google, can you set an alarm for 6 a.m.?”

Working

That leaves us about 1,000 minutes for the rest of the day. I spend another seven to 10 hours in front of my 4K screen up in my office, or on a laptop, sitting at a coworking space or a local coffee shop. There goes another 480 to 600 minutes of my day. Here computing is hugely important. Everything from answering emails to doing zoom calls to working on 3D stuff.

Driving

About 60 minutes of my day is spent driving. Sometimes more, sometimes a little less. I pick up kids, that takes 25 minutes each time, and if I visit a local startup or go to a meetup my time on the road goes way up. Here computing plays a role, I listen to music on Spotify or a book on tape or a podcast and my Tesla navigates me around town and I use Waze to get warned about objects in road and cops with their radar guns out. My Tesla drives a lot of that time with its rapidly improving Autopilot feature but that hasn’t really changed my computing habits or needs much. It isn’t good enough for me to stare at my phone for very long, and while I can see a day coming in the next decade where it will fully self drive (I don’t believe Elon Musk when he says it’s a year away, I believe it’s closer to five) I still am not sure that I will do that much computing. While my wife drives I’ve tried to work on a laptop and it’s too bumpy to get any real work done and concentrate the way I am while I am typing this on my Mac and its 4K display on a solid surface that never moves. So, I bet even in an autonomous world I mostly will do what I already am doing with the exception that I’ll watch more video and play more games.

Exercise

Some of us exercise, or go to the gym. That takes an hour away, and, when I’ve been running or swimming I might want some digital displays, mostly to keep track of my performance, but I can’t see doing something more serious like working on a 3D model for a VR game at that time. Although BlackBox VR shows you can totally change the act of exercise. This week a new pair of AR goggles were announced for swimmers. That’s cool, but those goggles only work while swimming, and can’t be used to improve any of the other minutes of your day.

Entertainment and family nights

That leaves the evening. A few hundred minutes to either play some VR on our new Oculus Quests, or watch some lame TV to turn off my brain, or buy a movie. All that competes with friends coming over and conversations with my wife or physical games with my kids or reading books. I often end up spending a lot of those minutes looking at social media infrequently while running around the house doing family stuff and chores. Dishes, laundry, homework with the kids, etc. That time also gets eaten away by other people’s dinners, parties, and chores like shopping for groceries or going to the hardware store for a tool or something.

Now some of you have different things. Some of you go to church a couple of hours a week. Some of you go to school, like my son did, for 10 to 20 hours a week.

“So, Scoble, where are you going with this and what does it have to do with Spatial Computing? You know, VR or AR?”

Well, if I was a product designer at, say, Magic Leap or Apple or Facebook, I would do the same for a large number of people, then pick what part of the day I’d want to change.

See, if you are building devices for using on the face you have to make tradeoffs.

  1. Will they be very lightweight? If so, you can’t have big GPUs and big screens.
  2. Will they be very cheap? Then you will have to give up a lot of things. (Like you will probably need an advertising supported model, so you will have to give up some of your privacy, and I know developers at Facebook and Google, and others, are looking at putting compute in the cloud, which will require new 5G infrastructure).
  3. Will they be aimed at putting as much screen quality into them? Then they won’t be cheap or lightweight.
  4. Will they have long battery life? Then they won’t be lightweight.
  5. Will they have lots of GPU for video games? Then they will need to be tethered with a wire to a PC with an NVIDIA card. If you try to put high-speed, low-latency wireless inside then you will need to add weight and cost.
  6. Will they have lots of sensors on the front to give you great room tracking and sensors inside to enable new kinds of experiences, like eye sensors? Then you have to give up privacy.

For instance, want to change my work hours? That’s the juiciest to change since it represents the most ROI and most number of hours and is the place we all are most likely to use computing? Well, then, you may decide to replace that 4K monitor in my life. But that will require a bigger device than glasses. Something closer to what Oculus Quest is today, which is a big, fairly ugly, device. Here’s my son playing on ours.

If you want “glasses” then you will need to go with something like the Focals by North. These are far more attractive outwardly than, say, our Oculus Quests, and can be worn while walking around, because they made tradeoffs for their users to see the real world, but come at a huge cost of functionality. You won’t be playing Beat Saber on these. They aren’t even close to replacing the monitors in our lives except for the small one on, say, a watch face.

Another tradeoff is our social contract with each other. If I can’t see your eyes then I don’t like talking to you. Talking to my son while he plays the Quest is uncomfortable even though we’ve had them on for dozens, if not hundreds, of hours so far in the first six weeks of having them. Talking to a wearer of Focals by North is far far far nicer. You won’t be wearing something like a Quest on a date night.

Which gets us to the rumors of the week. We heard Apple is stopping development of its glasses. Now, remember, I used to hear the same thing about the iPhone before the first one shipped. My brother-in-law worked at Apple and I used to work at Microsoft back then. He’d say “oh, Steve Jobs killed the phone project.” Then a few months later he’d say “Steve Jobs started the phone project back up.” So, I take these rumors with a grain of salt. I know many people inside Apple who are still working on head-mounted displays that do augmented reality and some do virtual reality too. Apple hasn’t given up and anyone who claims such simply isn’t rational. There are multiple products being considered and multiple projects underway at Apple. Even today.

But when I look at what I want Spatial Computing for it’s not usually for walking around the real world playing something like the new Minecraft Earth that’s coming out. At best that only affects an hour out of my day, and usually a lot less. What I want is monitor replacements. When I watch YouTube in my Quest, which only has a 1K display per eye the experience totally blows away watching YouTube on my iPad (which is the biggest one available) or my laptop, which has a 4K display, or, even, watching it on my office computer. Why is that? Because the screen virtually wraps around you and because you can put that virtual screen wherever you want. It is also more private, so I can watch an R-rated movie on mine while my son watches, say, Sponge Bob on his. Personalized viewing will be a HUGE new market. Just think about all those business travelers who need to work on intellectual property, or who want to watch a movie and not on the small screen on the seat in front of them.

We will be happy to wear something that’s a bit uglier, a bit more expensive, and a bit heavier to do that and many other tasks. Unfortunately the devices out there so far, whether the Quest, or Magic Leap, just don’t have optics and screens inside that are good enough to replace the physical monitors for what I do most of the time, which is look at text. The 1K displays in my Quests are blurry when I look at, say, LinkedIn, and the HoloLens and Magic Leap, because they traded off seeing the real world while seeing the virtual one, don’t work well in bright lighting conditions. Yes, new optics and screens are coming. Lumus in Israel makes some waveguides that are way better than the ones in the current HoloLens or Magic Leaps but they still have tradeoffs. The projectors driving those lenses can’t be fit into a small pair of glasses, like the Focals by North. It’s unclear how long a small battery can drive them, either. Our Oculus Quests only get two to three hours out of the battery in those. Not long enough to work for eight hours. We solved that problem by getting two, so one can always be charging, but that only works if one person is using them and, obviously, doubles the cost to $800 for a set.

To wrap this all up into a TL;DR: no matter what happens in the next 24 months none of us will get exactly what we want: which is lightweight pair of glasses that gives us huge, wrap around screens, that can “mix” the real world and the virtual world. That is an engineering problem that won’t be solved in the next 24 months.

That said, the Oculus Quest is an amazing product that we are having a boatload of fun with. Shows this industry can do a LOT without the expectations of fitting all that into a pair of glasses. If you look at the industry that way, we will get a bunch of new devices soon that will really change lots of the minutes of our days and those devices are coming in the next 24 months from Apple, Facebook, Microsoft, Magic Leap, Nreal, Tilt Five, and many others.

Why don’t you take the same exercise? Keep track of what you do every day and then dream about how you would change those minutes of the day with various devices, whether glasses, or larger head-mounted displays.

Recently I talked with Tilt Five’s founder Jeri Ellsworth. She looked at all these tradeoffs and decided to just go after the few minutes of entertainment in the evening that families have together and is building a pair of glasses, and experiences, for doing games on coffeetables and living rooms. That, to me, is smart to go after the home entertainment time of the day. It is easier to get that done than to try to be the perfect device for driving, shopping, walking, learning, working, exercising, etc. The problem for her, and others, who are going after those minutes is that Apple and Facebook are coming with devices that are aimed at that too. It’ll be interesting to see what kind of customer base some of these specialized devices get and whether they can withstand devices released by the bigger companies.

Robert Scoble

Written by

Spatial Computing Research at Infinite Retina.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade