ARKit for Designers — Designing for AR-First

There are plenty “wow!…made with ARKit” articles floating around. So lets go a step beyond that and talk about “lessons learned so far” in Part Time Evil’s ARKit work.

Learning your Space

In most Tango experiences (or anything with an infrared camera), you get a pretty instantaneous idea of the space. Since ARKit and ARCore are using the accelerometer and cameras to approximate space via point cloud (interesting description in plain English here), you need to give it a few seconds to learn.

If your application is using real-world planes, and not just doing hit detection, this is even a bit longer. This is because ARKit will generally need you to move around some. To oversimplify it (this isn’t a tech article), this allows it to sense parallax and thus planes.

As designers, we have our hands full with new ARKit users:

First. Users need to be guided to help detect the space.

Most videos out on the internet are the “best case/best setup” of a particular AR experience. The process for finding surfaces and planes…those are not included in the videos. Also not shown…tutorials about how the AR tech may not work as designed in certain places.

Its easy to see the trend. 1) People will become really good at finding the right places to use their AR apps. (“come into the living room because the carpet has a pattern!” 2) The AR software/hardware combo will become faster all the time, so that your apps will become better at figuring out the real-world space.

Here is an example of plane finding on YouTube by user Mark Dawson.

Second. True AR-First experiences may have real-world spatial requirements.

A true AR experience may require certain real-world dimensions. For example, there are many videos floating around with giant dinosaurs walking down the street in AR. However, these won’t be so great in your living room with a 4 foot shrunken dinosaur, or a dinosaur sticking through the ceiling. And they also won’t look cool walking through a tree in your yard. These are both things that can easily happen without instructing users to find the right space.

As designers, we need to guide users to discover the space to make the app work the best, or to make the game the most fun. In one game we are making called Boom, we’ve tried to design plane-finding into the game as part of the fun. One trick to ARKit…the more the user moves around, the more useful planes you find!

Here is an example of that game. And if you notice, the user didn’t quite find a big enough plane!

So obviously we could resize the structure that we place in that video. To a point. OR, we could say that we don’t care about the plane below it. BUT, this game is designed as an AR-first experience. The planes are necessary for the objects to bounce from them. It’s what we think makes it cool.

So that gives the designer a few options.

  1. Give feedback when found planes are big enough. For example, turn the planes green when they are large enough to place the object. Do this in realtime as the user explores.
  2. Auto-place the objects (this is how the “Boom” game is going to work). However, you need to provide feedback in this case. “Keep looking around, we need to find a good surface.”
  3. Resize your objects to fit planes. This can happen to a point, to as long as size doesnt break the game logic. We think this can be combined with #2 as a “compromise solution.” Place objects automatically within a certain resizable threshold.

We all have a lot to learn

Even though we’ve been doing AR products and designs for a long time here at PTE, there is still no substitute for getting in there and using your app. A lot of our new designs have come from us making really crude ARKit prototypes to test our ideas. And quickly, we either learned a way to make it better, or a way in which we should totally change direction.

I know it sounds like common knowledge to say you should prototype rapidly, test, re-design, and repeat. However, this is even more true with AR. Here at Part Time Evil, we committed to make AR apps that weren’t just gimmicks, but were real AR-first designs. And the only way to do that is to get in the AR world as a user and think about what makes it different and fun.

Right now (Oct 2017), you will see two types of apps for ARKit; the ones that are mobile games and apps placed on a surface, and the ones that are designed natively for AR. This is analogous to the old days of mobile, when some apps were shoehorned websites on mobile, but the next generation were “mobile-first” (touch-friendly sized buttons, interfaces displayed well on small screens, etc).

Both have their place, but when working in AR-First experiences, you need to get into the AR space, play as much as you can, and design for the challenges (and new super powers!) of the medium.