To create your first RealityKit app with SwiftUI, you will need to have updated to MacOS Catalina. Neither of these new frameworks will work on MacOS Mojave or any earlier version. You should also make sure you have the latest version of Xcode by checking the Mac App Store and checking that there are no updates for Xcode that you have not yet installed. Having the latest version of Xcode and MacOS is essential to making sure this app works correctly.
It would help to have a working knowledge of the Swift programming language, although anyone can copy the code and read my explanation of what the code does.
You will also need an iPad, as AR can’t be emulated by the iOS Simulator.
To create this project in Xcode, go to File > New > Project and select the Augmented Reality App template.
You need what’s called a ‘personal team’ in order to install your app on a device. You will not be able to test on an iPad without having signing capabilities. If you don’t know what any of this means, it might help to find out about it on Apple’s Signing & Capabilities workflow website. You’ll need to give your project a name and save it somewhere on your Mac. I called mine ARSwiftUI, but it doesn’t matter what name you choose. The code will still work just as well.
The left panel in Xcode is called the Project Navigator. If it isn’t displayed, there’s a button in the top right corner of Xcode that has a square with a line on the left edge of it that will show the Project Navigator. Here you will see all the default files that were added to the project. Right at the top is a blue icon with the name of your project. This represents the specific app we’re making, otherwise known as a target. On the line below, there’s a folder of the same name. Right-click or control-click on the folder, select New File… and create a Swift file called DataModel. Note that this isn’t a SwiftUI file, for reasons which will become apparent in a moment.
Because SwiftUI only displays a user interface and can’t store data, we need a persistent data model. The @Published identifier allows SwiftUI to be notified of any changes to DataModel’s properties, and to update the user interface accordingly. SwiftUI will not be notified about changes to any property that does not have this identifier, so we have to use it for all of our properties.
This is an instance of the DataModel that will always exist, since it is a static instance of the class. As the class is marked as final, the static instance is the only instance that can be created. We don’t want multiple versions of our DataModel, each with different properties, so this is a good thing! We don’t need to add @Published to our static instance, and Xcode wouldn’t let us if we tried.
When you create an AR project in Xcode, you might notice that ContentView comes with a struct called ARViewContainer. The initialiser for the DataModel class has most of the same code as the makeUIView method of that ARViewContainer struct. Instead of creating the arView variable as a let constant inside makeUIView, we are storing it at global scope in our DataModel class. This way the AR View will always be accessible, as it is saved permanently.
Why would we want to disable AR? As the user interface gets more complicated, it may be helpful to be able to run the app in the iOS Simulator to see how everything looks. If you try and run it in the iOS Simulator with AR enabled it will crash, so it’s good to have it off by default and have a toggle to activate it when you want to.
xTranslation, yTranslation and zTranslation
These are unsurprisingly float values that store how far you want the box to move from its initial position. To make them easier to display as part of the user interface, we are storing these as a number in centimetres. In didSet we are calling translateBox, which then uses these values to move the box. This means that every time the value of these properties changes, DataModel changes the translation of the box accordingly.
This is a method that we will use to move our AR box around. It converts our X, Y and Z translation floats to metres, which RealityKit uses as its units. Then it combines these into a vector that can be used as the translation of the box.
To understand what this function does, it would be useful to understand what the Experience.rcproject file contains. This is a Reality File to which you can add as many scenes as you want. It comes with a single scene that’s simply called Box, and this scene contains an AR Entity object called steelBox. The scene has a horizontal anchor for the box, which means it finds a flat surface of a certain size and places the box right in the middle.
Hopefully this clarifies how I use the if let steelBox condition to acquire the scene, then the first (and only) anchor, then the steelBox Entity object.
In order to connect our DataModel class to our SwiftUI, it will need to be passed as an Environment Object. We do this in a method at the top of AppDelegate called application that takes a parameter of didFinishLaunchingWithOptions. All you do is add it after the contentView parameter passed as the rootView parameter of the UIHostingController constructor. This makes it available to our ContentView, and subsequently any of the other views that we want to create. Notice that we are passing it the shared static instance of DataModel, meaning this is the one that we want to use to store all of our data.
Right click the root folder at the top of the Project Navigator and create a new file. From this point on, we only want to make SwiftUI files, not normal Swift files. Call it ARDisplayView.
Cut the ARViewContainer struct that was provided in ContentView and paste it here. We don’t need most of the body of makeUIView, because the DataModel creates the AR View for us. Therefore we only need to return the AR View that we have saved.
Unsurprisingly this will be the user interface for our app. You’ll notice that our UI is displayed as a List, which means that all of our controls are stacked vertically. If the list gets too long for the screen, it will automatically become scrollable, which will help you if you use this tutorial as a starting point for building a more complex UI. Since all the controls we are using are relatively thin, the UI panel isn’t very wide. You can see this in the frame modifier that appears at the end of the List.
This is how we enable the AR View. At the start of the ARUIView struct, you’ll notice there’s a declaration for a variable called data. Without having to do anything else, the DataModel shared instance that we added in AppDelegate will automatically find its way here. This allows us to toggle the boolean for enabling AR directly.
Each Stepper has a label which displays which axis it controls and the current value in centimetres. They have a range of negative 100 to positive 100, so they can be used to move the box a maximum of 1 metre in any direction.
Now you can finally rewrite the ContentView file that was provided. This shows you the simplicity of building SwiftUI interfaces, once all of the functionality is separated into individual views. We use an HStack, which stacks the UI to the left and the ARDisplayView (if enabled in the DataModel’s static instance) to the right. If AR is disabled, there is a flexible Spacer on the right instead. Otherwise, the ARUIView would be in the centre of the screen until you enable AR, which doesn’t look very good.
This is what you should see when you install the app on an iPad. Once you have flicked the AR switch and the iPad has found a flat surface, the steel box should appear.
Now you should be able to use the controls to move the box around, 1 cm at a time.
This tutorial is just a way to get you started with RealityKit and SwiftUI. It’s easy to see how you could add sliders to move the box in the various directions, or possibly to rotate it. There isn’t time to explain it here, but it is possible to send messages to your Experience Reality file that trigger events there, such as animations or adding new models. You will probably need to make your ARUIView’s List frame wider to accommodate more controls, but you can rest assured that the List will scroll when you put too many controls in.
Just be careful, because a SwiftUI View can only contain 10 child elements, so it may be better to create separate VStack views for your movement controls and your other controls.
About Rob Sturgeon