I recently had the opportunity to work with 3 brilliant engineers to build an exciting app called “Festival” that incorporates Apple’s newly released ARKit.
Imagine you’re at Coachella. You paid $300 for a ticket and you’re ready to brag about it to all of your friends on Snapchat using the dog filter. But there’s tons of stuff going on around you and it can get pretty chaotic (or as your friend Jimmy calls it “lit”). You at Coachella might be thinking:
- Which darn stage is Frank Ocean playing at?
- Oh wait a sec, where in the world did Jimmy go?
- God I’m hangry now. Where is the closest In-N-Out?
Festival aims to solve these pain points by providing users with an immersive map that places virtual location pins that identifies artists on stage, friends nearby, and delicious food vendors.
Designing in AR
I’ve never designed in AR nor in VR prior to this. So I looked in to the methods on how to prototype Festival and eventually figured out that Unity and Unreal Engine are really the only 2 ways to do this. I wasn’t really familiar with both of them and my engineers wanted to build it in Xcode so I used Origami to prototype my ideas.
We imagined a simple concept where numerous circular pins surround the user indicating that some entity is located at that exact GPS coordinate. When in the focus view, the pin spins and displays a small information panel from the bottom. Tapping on this panel displays more information. The reason we incorporated a focused state was because we wanted to make it easier for users to scan through multiple pins in a music festival.
As a note, Framer works great too and you can use the native camera on your phone in either method. In any case, it is bit of a hack-y method since both Origami and Framer doesn’t support AR natively. I worked my prototype by mapping the rotation rate of my device to the XY-coordinate of the circular pin.
Next, I did some 3D modeling work using SketchUp (free) and Blender (free) to define the design/motion of the pins. I exported my 3D model as a DAE file and sent it over to my engineer so he can place them in the AR environment. Although building the 3D model in SketchUp was a breeze, actually animating it in Blender was difficult as this tool has one of the most complex interfaces man has ever seen. This is mainly due to how Blender is capable of video editing, 3D modeling, animation, photo editing, and pretty much everything you can imagine. Photoshop is truly a godsend compared to this UX of this tool.
I also did some high-fidelity 2D screen designs on Sketch to actually define what goes inside the information panel. Unlike most projects I worked on, this took the least amount of time as my main focus was prototyping the AR experience in Origami.
Over the course of few days, we kept pushing out improvements to our product and ended up with a solid demo that features Festival’s potential. Take a look below to see what it’s capable of.
Upon opening Festival, users will see virtual pins representing artists, friends, and food vendors that are close by. Each pin is mapped on to the floor and depending on how far the location is, the size of the pin adjusts accordingly. Like I mentioned earlier, Festival incorporates a focused state that enables users to scan through multiple pins in their environment. The spinning ring around the circular picture indicates that it’s in focus and the panel from below shows what it is and how far it is located. I’ve also color coded the rings: Green for Friends, Purple for Artists, and Yellow for Food.
Expanding the panel shows contextual information based on whether it’s a friend, an artist, or a food vendor. If it’s a friend you can message or call them, if it’s an artist you can see which stage they are performing on, and if it’s a food vendor you can check out the menu. I intentionally made the panels scalable since based on the entity, some could have less/more information.
The Navigate button (which is definitely the most prominent CTA) doesn’t work! We imagined that tapping that button will draw a line between you and that entity but we just didn’t have time to implement it. Definitely something I wanted to incorporate if our team had more time.
Check out the full demo video. Sorry, no audio!
Reflection & Next Steps
The amazing thing about Festival is that we can envision this kind of app be applied in many scenarios. For example, if I was just on the street and was looking for food, having virtual pins that point me to the direction of a restaurant would be super handy. Or if I wanted to see where certain items are in at my local grocery store, I can just look around in the app.
Designing in AR is quite an incredible experience. There’s something magical about seeing your designs move around in the environment around you. I was pretty intimated in designing for such a new medium but I’d say the barrier to entry is not that high. If you are interested in designing the future, AR is the right place to be at. Using tools like Origami or Framer could get you pretty far and it would be a great starting point.
I’d love to challenge myself to build more AR experiences and keep exploring what this new technology is capable of.
Thank you for sticking till the very end. Feel free to check out the rest of my work at www.YukiAsakura.com!