iOS Bites — An Intro to Building AR Experiences with Reality Composer

You’ve heard of deep dives? iOS Bites is a series of short dives into new concepts and approaches in iOS development

There are a handful of companies that are investing in creating tools for AR development: Mozilla has A-Frame, Microsoft has the Mixed Reality Toolkit, Amazon has Sumerian, Google has ARCore, etc. Apple’s entry in this list has been ARkit, but at this year’s WWDC Apple unveiled a suite of tech that would support AR development; ARKit 3, Reality Kit and Reality Composer.

In my opinion, Reality Composer is the coolest of the three (although people occlusion, and motion capture are big contenders) and I’d like to dig into it separately from the rest of the suite. What is Reality Composer? Reality Composer is an application that ships with Xcode 11 and lets anyone proto-type and/or build AR experiences. Think of Reality Composer as the equivalent to the A-Frame inspector, a visual GUI that helps you build scenes.

In this article I will go over the Reality Composer application and walk you through the biggest features that’ll help you make AR scenes. More specifically I’ll walk you through:

1. Scene creation and navigation
2. Instantiating and importing objects
3. Manipulating objects with behaviors
4. Adding Physics to your scene

To use Reality Composer you’ll need to have installed Xcode 11, after which you can search for RealityComposer in spotlight and get started building AR apps.

Scenes

A scene is composed of:

• Anchors
• Objects
• Behaviors
• Physics

When you create a new Reality Composer project (.rcproject) you’re prompted to choose an Anchor. This is how ARkit attaches your AR app into the world. There are four kinds:

1. Horizontal: Think table tops, ceilings, floors.
2. Vertical: This would be a wall.
3. Image: Any 2D media in the world, pictures for example.
4. Face: Self-explanatory; think Snapchat filters.

Choose an anchor (for this article I’ll be using horizontal anchors). You’ve made your first AR app!

It’s pretty basic though; just a cube in the center but hey, you started. From here you can do a number of things; but first things first, let’s go over how to navigate the world, which from here on out I’ll call the view port.

Navigation

This is actually pretty intuitive and easy. You have three gestures to navigate the view port with, and as an Apple user I’m sure these’ll sound familiar:

1. Two finger pan: Two fingers on the track pad you can move the camera in any direction.
2. Pinch to Zoom: You can zoom-in by bringing your fingers close together and zoom-out by separating them.
3. One finger click and hold: This lets you rotate the camera about the center of the scene.

Objects

Reality Composer ships with its own library of objects: basic prototypes, text, and a handful of objects under various categories. You can get there by clicking the plus button at the top of the screen. Comparatively the asset library does not have the same breadth of content as say Google Blocks, Google Poly, SketchUp, etc. You’ll get the most value out of importing your own content, you can import objects but they have to be in the file format USDZ. Luckily, Apple developed an application to do this exact conversion: Reality Converter. Import your .obj files (Reality Converter also supports .gltf and .usd) either by drag and drop or File > Import and then export by File > Export.

Once you have objects in the scene you can start building your composition. Selecting any object will materialize three cones and a colored ring around your object. We’ll call these widgets.

The cone widgets let you move your object in 3D space along the three axis’: green is the Y axis, blue is the Z axis, and red is the X axis. Tap and hold, you can now drag the object along that axis.

If you tap a cone, the ring widget will change its color to match the tapped cone. If you click and hold the ring and move your mouse along the circle you’ll rotate the object.

If you click and hold the ring and drag away from the object (or towards) you’ll scale the object.

If all you wanted was a virtual tableau of objects to drop into the world you’d be done! But you can still do so much more to make your scene more dynamic.

Behaviors

Pick an object in your scene and in the upper left corner of the editor is the Behaviors button. Selecting this opens the Behaviors pane. A behavior is defined as a trigger and an action and Reality Composer has a set of predefined behaviors for you. For example, the tap and flip behavior will flip an object (action) when it is tapped (trigger).

A trigger is any criteria that will launch an action, or more specifically an action sequence (complex actions created by combining actions together in groups or manipulating the execution flow with loops). You can obviously use any of the predefined behaviors at your disposal but you can also create brand new ones. When you add a new behavior the custom option is at the bottom of the list of predefined behaviors.

Physics

You can add even more realism to your scene if you add physics to your scene. You can manipulate physics in three ways:

1. Collisions: This is how objects interact with each other. Think bowling; a bowling ball object has to interact with nine pins.
2. Forces: There’s a force action that you can apply to objects to move them. (Additionally, there’s also gravity that your objects are subject to.) Think bowling again; you can have a behavior that when the user taps the bowling ball it applies a force to the ball.
3. Materials: This is the friction that is applied to your object, the opposing force that controls your objects movement. Think bowling again; a bowling ball moving on ice will travel farther than if it’s on a material like concrete or wood.

To manipulate your objects with physics you have to say the scene participates in physics and that the object participates in physics.

Object

For an object, select an object in the scene and in the upper right corner tap the properties button to open the properties pane. At the very bottom there is a physics section. Check “Participates” (if it’s not there, tap the triangle next to the words “physics” to drop down more options).

Now your object is eligible for physics experimentation!

Scene

For the scene, unselect all objects, open the Properties pane, and check that the scene participates.

Conclusion

Reality Composer is a fun new AR tool provided by Apple. Take what you’ve learned and try and see what you can build with it! Use the asset library or if you’re looking for more options checkout Sketchfab. If you want some inspiration some really cool AR apps are ARia’s Legacy — AR Escape Room, Angry Birds, Ikea, Night Sky and there’s a bunch more. This article only touches on a few of Reality Composer’s capabilities but I hope I was able to pique your interest.

P.S. Try to replicate that gif!

Related Articles

• iOS Bites — An Intro to Combine and the Reactive Pattern

DISCLOSURE STATEMENT: © 2020 Capital One. Opinions are those of the individual author. Unless noted otherwise in this post, Capital One is not affiliated with, nor endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are property of their respective owners.