ARKit Tutorial: Infest your room!

Hi and good day! This tutorial will teach you the ins and outs of using Apple’s ARKit, an investment and outcome that the company is really proud of.

Augmented Reality is the next big thing when it comes to mobile platforms, and here we’re gonna learn from scratch to master the most important features in creating engaging experiences.

We’re gonna try to “populate” the room we’re currently sitting on with cute little spiders, and will have to do so regardless of the type of plane that we’re going to detect: either horizontal or vertical.

Let’s start by creating an iOS project using the Augmented Reality template, using the SceneKit framework (for 3D experiences).

Xcode template for ARKit

Apple’s AR template covers a lot or AR use cases so it’s best to put your ideas on paper and clean up the code that doesn’t suit your needs.

Of course that those of you who do not have any experience in using it or, at least, haven’t already checked my article regarding ARKit “must knows” can let that code be until final revision, in which you could delete every method you will not eventually use.

Next, we will navigate to www.turbosquid.com, a website that hosts a large variety of 3D models made by professionals for professionals, and in the searcher, we will type “spider” because this is the bug that we want to infest our room with. For this tutorial purpose, we do not want to invest any money in downloading fancy 3D models, so in the menu drop-down for filtering the results, we will access the Price item and set the min and max to zero, then apply changes.

Spider 3D models on TurboSquid

Now, the filetype that we are looking for to integrate with our iOS app is called “Collada” and it comes with the extension .dae, so in the first item of the filter menu, named Formats, select extension .dae. The first one that comes from the web request is no other than the cute little spider that we are going to use in this project :).

One thing to consider is that to download any content for this site, be it free or premium, cannot be done without creating a TurboSquid account, so please do that when you’re prompted. After downloaded, drag the file into the newly created Xcode project and convert it to a .scn file. To do so, you just need to go to the Editor menu and click “Convert to scan file format”. A prompter will show up and you just have to hit “Convert”.

First, we’re going to look at the viewDidLoad() method to analyze the first boilerplate code:

override func viewDidLoad() {
super.viewDidLoad()
// Set the view’s delegate
sceneView.delegate = self
// Show statistics such as fps and timing information
sceneView.showsStatistics = true
// Create a new scene
let scene = SCNScene(named: “art.scnassets/ship.scn”)!
// Set the scene to the view
sceneView.scene = scene
}

The ship scene is not needed in this tutorial, taking into account that we are going to use spiders and this is the default model in every newly-created app, so you can safely remove the last two lines from the above snippet. Instead, insert these two lines:

sceneView.debugOptions = [ARSCNDebugOptions.showFeaturePoints]
sceneView.autoenablesDefaultLighting = true

What does the above code block do? Well, let’s start how do we want to implement this project: in the process of running a session and detecting horizontal/vertical planes, yellow dots called feature points will start to show up on the camera feed of the device, described as notable features with 3D world coordinates that are essential for the analysis that the library conducts in order to better track the device’s position, orientation, and movement. They can also contribute to contouring real-world objects in the camera view. In order to visually estimate the planes that are being detected, the presence of feature points can improve development a lot :).

In viewWillAppear() method, we can see that the session configuration is set and the session is run. In order to detect both horizontal and vertical planes, you just have to write this single line of code:

configuration.planeDetection = [.horizontal, .vertical]

Basically, the application flow will be the following:

• The user is searching through the camera-enabled from the application to detect some vertical or horizontal planes
• After the show up of feature points, place the camera to analyze an actual plane until a blue-lined grid will appear; that will be the plane you are looking for
• After a plane is detected, the user can tap on it to place spiders into the real world
• The placing of spiders should be done according to the type of plane that is detected (we have to rotate the model when on the vertical plane)

And that should be it! Now, after we succeeded to create our first ARKit app, let’s run it and see what we have achieved:

Demo horizontal and vertical plane detection

Perfect! The feature points are starting to show up but.. there’s no grid appearing yet! To do this, we have to call a ARSCNViewDelegate method, named renderer(_:didAdd:for:) (don’t forget to make the UIViewController class conform to this protocol). This method will be aware of every plane you will detect through the camera and will add an ARPlaneAnchor to it. Also, the main purpose of the method is to create a SCNNode and attach to this new ARAnchor, but, in order to properly add the grid to better visualize the plane, we should add a SCNPlane to the node first:

let plane = SCNPlane(width: CGFloat(planeAnchor.extent.x), height: CGFloat(planeAnchor.extent.z))

At first, the detected plane will be assigned a center with value (0, 0, 0). The extent property of planeAnchor is the estimated width and height of the detected plane. This can vary according to the time spent on scene analysis and while the process of detection continues (the user still wanders around that surface). Because of that, even the center of the current plane can change relative to the transform property value (matrix containing position, orientation, scale relative to world coordinate space).

As reflected in this case and also in the Apple’s official documentation regarding the properties of ARPlaneAnchor, always use the x and z directions to determine the width and height of the detected plane, because the y-component is always zero and only the two mentioned earlier are relative to the transform value.

Let’s add the grid.png (from my project) inside art.scnassets folder as material to the plane variable and, finally, to the geometry property of the SCNNode that will be attached to, with position relative to the x and z coordinates of the center of the planeAnchor (y is 0 here too):

let planeNode = SCNNode()
planeNode.geometry = plane

Now, add this node to the current parameter of this method and run the app:

Added grid to detected plane

Wait! This grid should not “stand” onto the plane! This is clearly wrong, but, luckily, I have the solution. Place this line of code before adding the plane node to the main one:

planeNode.transform = SCNMatrix4MakeRotation(-Float.pi/2, 1, 0, 0)

As seen above, the transform of the node should be changed in order to place it right onto the detected plane. The object above is altering the position and orientation of the node by rotating it around the x-axis with -pi/2 radians (clockwise).

When running it again, you should see the following results:

Correct grid addition

Great! Now the grid is correctly placed onto each type of plane!

Next, we should handle user taps onto the planes that are available in the current session. This can be done using the delegate method that detection or more touches from the current view/window, called touchesBegan(_:with:) or touchesEnded(_:with:). For this tutorial, I’m going to use the first one mentioned. This method will return a set with all current touches with the location attached to them. This is used in the hitTest(_:with:) method, and we are going to detect touches on a plane with types existingPlaneUsingExtent:

let results = sceneView.hitTest(firstTouchLocation, types: .existingPlaneUsingExtent)

For more types, you can check the implementation documentation on the Apple Developer site :). The hitTest method will also return a set of hitTestResult objects, so consider the first of them and then add the rootNode of the scene with the spider included, to attach the node to that point. It is also required to add a position to the node, therefore set the coordinates of the worldTransform property of the hitTestResult.

if let hitResult = results.first {
let diceScene = SCNScene(named: "art.scnassets/crsppdr.scn")!
if let diceNode = diceScene.rootNode.childNode(withName: "Sphere", recursively: true) {
  
            diceNode.position = SCNVector3(
                  x: hitResult.worldTransform.columns.3.x,
//according to boundingSphere of 3D model, we have to calculate y position such that the spider will realistically be placed on a surface
                  y: hitResult.worldTransform.columns.3.y,
                  z: hitResult.worldTransform.columns.3.z
            )
}

Consider the screenshot below:

We can easily see that the boundingBox of the model is pretty large because the height is… no smaller than 4.29m :). So all we need to do is to change the y position of the node containing the spider by adding the following code:

diceNode.boundingBox.max.y/400

Or you can use the boundingSphere property of the node:

diceNode.boundingSphere.radius/400

This will assure that the model is placed perfectly on the smooth line of the plane with a decent dimension.

Now run the app:

Incorrect placing of spiders on vertical planes

Oops! That was not the result we were looking for. Although, the dimension of the 3D models is now pretty accurate.

One last thing to consider is the following: How can I place the spider correctly on a wall because the current result is in the last screenshot above? Well, is pretty simple, we can add a rotation about x-axis with pi/2 radians (counterclockwise) to:

• modify the Euler angle of the node:

diceNode.eulerAngles.x = Float.pi/2

• modify the rotation property:

diceNode.rotation = SCNVector4(1, 0, 0, Float.pi/2)

• modify the orientation property, using quaternion multiplication processed by GLKit (OpenGL library):

let orientation = diceNode.orientation
var glQuaternion = GLKQuaternionMake(orientation.x, orientation.y, orientation.z, orientation.w) // initial orientation of node
let multiplier = GLKQuaternionMakeWithAngleAndAxis(Float.pi/2, 1, 0, 0) // the rotation we want to apply
glQuaternion = GLKQuaternionMultiply(glQuaternion, multiplier)
diceNode.orientation = SCNQuaternion(glQuaternion.x, glQuaternion.y, glQuaternion.z, glQuaternion.w)

I strongly recommend you to use this last version of node rotation because it can avoid losing a degree of freedom (gimbal lock) like in the case of Euler angles and are more efficient than matrix rotation. A phrase from Mother Wikipedia says that too: “Unit quaternions, also known as versors, provide a convenient mathematical notation for representing orientations and rotations of objects in three dimensions. Compared to Euler angles they are simpler to compose and avoid the problem of gimbal lock. Compared to rotation matrices they are more compact, more numerically stable, and more efficient”. It is also not recommended to directly use a SCNQuaternion because it is mandatory to know the initial orientation of the object in order to properly obtain the final orientation by applying the rotation. Furthermore, SceneKit doesn’t include in the library methods to perform operations on quaternions, hence GLKit should do the best job. Otherwise:

So, choosing the third version of rotation, we should obtain the following well-functioning application:

Of course that, because of the laptop plane being higher than the table plane, some spiders might be levitating :), but that should be it for now (I’ll leave the rest to your research purpose). So that was it! Thank you for your attention and, if you appreciated my tutorial and the hard effort I put on it, I’d suggest you hit the claps button!

You can find the entire code on GitHub right here. If you find something abnormal, you are free and even asked to report it to me (in the comments area below). Stay tuned because … the best is yet to come!

Zipper Studios is a group of passionate engineers helping startups and well-established companies build their mobile products. Our clients are leaders in the fields of health and fitness, AI, and Machine Learning. We love to talk to likeminded people who want to innovate in the world of mobile so drop us a line here.