Magic Leap One: Vive forAR

Video review

How I wrote this review

As an interested mixed reality technologist, I’ve been watching Magic Leap for a while.

I mean that in a few ways.

The first is that I went into my first Magic Leap dive after a couple hundred hours hacking the SDK (Javascript bindings). I smelled the code before I walked into it.

The second is that I don’t have eggs in any basket. I’m cheering for everyone in the space. I’m watching the race, but I don’t have a ticket on Rony’s Pony.

Finally, being an engineer, I don’t believe in magic. I believe in smart people working hard and making things at the intersection of technology and market. I feel read enough that I know what’s reasonable in T time and D dollars.

Context in reviews is important! Especially for something like Magic Leap.

That said, I think Magic Leap is on the right track. I feel like this week they shipped the Vive of AR.

AR vs VR

The AR landscape

I feel like “Augmented Reality” as a concept has until this point been plagued by the convenient specter of mobile devices. That’s most folk’s introduction to AR, and while it works great for filters, phones don’t have either the compute or the lenses to make it more than a gimmick. They must exist, but I don’t think I’ve ever met a user of ARKit.

On the flip side you have the Hololens, but that’s an enterprise device with an enterprise price tag and enterprise developer relations in an industry that’s built on experimentation and wow factor. Its current iteration is also a few years behind Magic Leap.

The VR landscape

VR is significantly more mature industry. We’re getting started on the adoption curve, but VR is “a thing” with a market definition and I’ve met my share of people who identify as VR gamers. Even lower-end mobile VR is a thing.

So what does this have to do with Magic Leap? Because I feel like Magic Leap is basically the AR of VR.

The experience

The best way I can describe Magic Leap is that it’s like someone took VR and injected the real world into it.

By this I mean:

- An multitasking operating system
- An app store
- A head mounted display
- 6DOF for everything
- A controller
- Requirements for a play area
- Modern GPU hardware
- A microphone
- A massive, well-implemented SDK
- Engine integrations

If this sounds like a desktop VR platform, that’s not a coincidence.

The difference with Magic Leap is that it also packs in inside-out tracking, meshing, object recognition, and hands tracking out of the box. But this all feels like an addendum to the solid rendering APIs we’ve grown to love with VR. That is, there is a play space, your application gets matrices and poses, and delivers a binocular render of whatever.

Whether you’re a user, using Magic Leap is basically like adding AR to your VR.

The magic

I probably just made it seem like this is a device and stack that adds not much that’s new. But adding the real world to VR changes a few things.

For one, your application is going to suck if it’s not doing mesh occlusion. AR is meant to interact with the real world and if you ignore this your app will be crossing people’s eyes as objects draw behind real walls, which is a physical impossibility the human visual system isn’t prepared to deal with.

Thankfully the Magic Leap SDK provides excellent API support for injesting meshes and depth, especially if you’re coming from Unity + Unreal.

But this is just the start. If your application is only occluding content it still is not going to offer much in the way of new experiences. For that, you want to get friendly with the mesh.

The mesh

The ML1’s secret sauce is its meshing system. It’s really good. Which is good because so much of the ML1 rides on it.

As look and walk around your space, the hardware is using AI, SLAM, and sensor fusion to piece together a voxel mesh of the real world. You can turn this on if you want to see it in action, but it's chugging along even if you don't.

What this means for applications is that they get a complete, and pretty accurate view of the real world in their world. If an app wants to let you throw a cube and have it bounce around the room, or shoot a rouchet out the window, or place a trex on the desk, or... that's as easy as adding a collider.

The persistence

One other important aspect of AR embraced by Magic Leap is that you're not teleporting around inside a cage or riding a roller coaster, like you do in VR.

You don't teleport at all. Instead your content can be experienced anywhere. You can leave it and come back and it'll still be there. This can work even when the environment changes slightly, or you power off the device.

This completely changes the way you design and experience content. AR encourages smaller pieces of content that interact with each other, not monolithic gigabyte games that take over your reality and get discarded when the next cool thing comes along.

This brings us to...

The operating system

Lumin OS is the Android-derived operating system that runs on the Magic Leap One hardware. As a user one hint at this is that the charging cables are the same ones that work with your Pixel, but otherwise the hardware feels nothing like Android. The Lightpack compute unit is ventilated, with a fan.

There are more tastes of Android remaining if you are a developer. If you're familiar with `apk`s, Magic Leap has its own `mpk` format, and there is a debug bridge called `mldb` based on `adb`.

But the star of the show is the application runtime, which is very much tailored for mixed reality. Meshing, suspend/resume, recording, and other things come standard for all applications. Some applications even get persistent multitasking. To understand that we'll have to talk runtimes, because Lumin OS has two.

The two runtimes

Magic Leap acutally comes with two separate ways of making content:

- Lumin SDK immersive apps, and
- Lumin Runtime apps

Immersive apps are like if you literally took VR and added AR. They take over your view of reality while they are active, and they are intended for things that reasonably fill the entire play area while being too complex to coexist with other apps. One example is Create, one of the first-party sandbox experiences from Magic Leap.

Lumin Runtime on the other hand is based around a scene graph, which is managed, blended, and user interfaced by the operating system. Lumin runtime allows you to for example run The Helio web browser alongside media players, mutiplayer avatars, and such.

I think both have their place, and this was the right design choice. AR is different in that it makes perfect sense to have streamlined micro-applications as a first class citizen in the operating system. Lumin OS provides a rich set of services to base your applications on -- much richer than OpenVR or OculusVR, which makes multitasking applications on a runtime actually make sense.

There's a couple of other things that just make sense with Magic Leap one, and they were a bit of a pleasant surprise to me.

Surprise one: audio

You might think that this is a graphics headset and audio is a solved problem, but the spatial audio on the Magic Leap One is not only great (without requiring headphones), but also plays a critically important role in the experience.

Since you are immersed in the messy, noisy real world while wearing a Magic Leap One (as opposed to contained in a carefully constructed reality), it can be hard to find things in the world. It's literally possible you'll lose something behind the fridge, or your trex walks to the other room while you're not looking (or cooking).

Surprise two: avatars

I was pleasantly surprised to find that there is a full social avatar system in place too, and it does eye tracking, voice tracking, samples hand gestures, and more.

I didn't get a chance to try out the social functionality yet, but it looks like the ingredients are there for a solid API layer for apps.

The lenses: FOV, resolution, opacity, near clip

Ah, the thing that everyone was talking about before, during, and after the launch of Magic Leap.

Let's get the easy one out of the way: the opacity of the display is a non-issue. The projectino is opaque enough that it's prefectly capable of fooling you there is a hole to another dimension in your wall. The only case where opacity was noticeable was when looking out through a brightly lit window.

Resolution is a bit more complicated. First off, there is no screen door effect like you see in VR. Secondly, you don't see pixels. It's very much a hologram, and hard to describe otherwise. I suspect the total resolution is less than that of VR devices, but it's concentrated in a smaller field of view. In short, I did not notice the resolution of the device to be able to tell you what it is -- which is not something I can say for any VR headset.

Finally, the field of view. You're not getting VR FOV, but it's better than the Hololens. It's a hard-clipped recangle straight ahead. And, you mostly don't notice, because in AR things tend to follow the laws of physics you're used to. Toss something over your shoulder, hear it land behind you, turn around, and it's there. This is partly why audio plays an important role here -- it does a lot to increase the immersion regardless of where you're looking.

One thing that played a much more important role in the experience for me was the somewhat far near plane clipping on every one of the default apps. I believe this is a technical store requirement as well. It feels like an aesthetic decision with regards to hand tracking and occluding the meshing rather than a graphical limitation. Which makes sense, but for the creator edition I might have relaxed this requirement so that you could really look at objects up close.

The grafix

The graphics are another non-issue. Geometry on the default apps is dense, animated, and fluid, and from the technical side you have full OpenGL and Vulkan.

You sometimes can tell that the CPU is loading and isn't feeding frames to the GPU -- something you see in VR too -- but this seems like a software optimization opportunity than anything in the hardware.

I also did notice some slowdown when filling the space up with multiple Lumin Runtime apps, which compete for the machne's cores. This was a pretty rare ocurrence, and did not happen at all in any of the full-scale immersive applications, which own their render loop.

The comfort

A question that I was surprised to get was how comfortable the unit is. The answer is that it's more comfortable than a VR headset, mostly because:

  • The fit is looser, since light bleed is not an issue
  • There are no cables to step around compared to wired VR
  • Magic Leap clearly spent a lot of time thinking about ergonomics

The unit can run tethered or untethered on a battery pack. The cables are USB-C, but a little on the short side for a mixed reality headset. I suspect you could.

The battery lasts about as long as you’d expect, and it depends on usage. You’ll get a couple hours out of it when doing heavy things like running a ton of apps and recording review video ;) The whole thing can, of course, run indefinitely when plugged in.

The thing I did notice is that when coming out of AR, reality had a screen door effect, as if reality had turned to VR. This wasn't too different from what you get when coming out of a full day of VR, but it was definitely curious. Not at all unpleasant, but interesting biological minutiae, like the afterlights you see when you come home on a sunny day. More science is needed to get to the bottom of this!

The unfinished

What excited me was that there are several unfinished things sprinkled throughout Lumin OS. Yes, excited -- because that means there's more in the works, and there's plenty of room to experiment with the platform. That's what the creator edition is for!

One part that feels unfinished is the social system. It seems to be fully fleshed out but hasn't been interwoven into existing experiences yet.

Another is the content on Magic Leap World (the ML1 app store). On launch day, there's not much — mostly just first-party applications. For a creator edition, this is expected; the idea is that the explorers buying the gear today are the ones that will fill up the store. That’s us.

The Helio web browser (which runs on Lumin Runtime) also feels like it's touching on something great, but right now the only thing that separates it from a regular web browser is the ability to load and pull out 3D models from the page. That's kind of neat, but the existing web is a gateway to everything.

Speaking of, I just got the go-ahead from Magic Leap to ship V8 to Magic Leap World, so I’m getting antsy to ship Emukit and other Javascript/WebXR experiences…

Should I get one

This question answered by a question: are you a creator? Then get yourself a Magic Leap One: Creator Edition. Otherwise, wait.

(Under creators I include professional Youtubers/streamers/user generated contenteers, modelers, and of course developers)

If the name didn't give it away, this is a product made for makers. The technology is here, but you will be starved for content if you're looking for two thousand dollars worth of enrichment and entertainment today.

Links to what I do

Exokit WebXR Engine for Magic Leap: https://github.com/webmixedreality/exokit

Twitter: https://twitter.com/webmixedreality

Hanging out in Discord: https://discord.gg/Apk6cZN