Step by step Beginner’s guide: Markerless AR with Unity 3D and AR Foundation

Augmented Reality (AR) is revolutionizing the way we interact with the world around us, merging the physical and digital realms to create immersive experiences. Among the various techniques in AR, markerless AR stands out for its ability to seamlessly integrate virtual objects into real-world environments without the need for predefined markers. In this article, we’ll delve into the fascinating world of markerless AR using Unity 3D and AR Foundation, exploring how these powerful tools can help you build captivating AR applications that recognize and respond to the real world with a good accuracy. Whether you’re a seasoned developer or just starting out, this guide will provide you with the knowledge and resources to create your own first markerless AR experience.

Photo by stem.T4L on Unsplash

🧨 Installing and setting up Unity Hub

To proceed you would need to create a new Unity Hub account if not already created. And install the latest Unity Hub version from the internet which in my case is 3.8.0.

🎯 When opening Unity Hub for the first time, you will land on the below UI

🎯 Navigate to the Installs tab which is located on the vertical left menu

🎯 Choose the recommended LTS Unity version (2022.3.33f1 LTS in my case) and click Install.

🎯 Make sure to check the Android Build Support and its dependencies (OpenJDK and Android SDK & NDK Tools) and then click Continue

🎯 In the next screens accept the licence agreement terms and conditions and click Install. It will take around 10–20 mins for the installations to complete (depends on the network stability of course 😉)

🎯 Once above installation is completed you would see an UI like below

Congratulations on installing and setting up the required starters 🎰. You’re well on your way to creating truly immersive and innovative AR experience. Keep pushing forward, and let’s continue exploring the possibilities together!

🧨 Creating and setting up the AR markerless project using AR Foundation and AR Raycast Manager

🎯 Navigate to the Projects tab in the vertical left menu and on the top right click New project

🎯 Select 3D Built-in Render Pipeline from All templates and give your project a name in the right side and click Create project

🎯 Once the project is created you will land on the below default UI which has a Main Camera and a Directional Light in the project hierarchy

🎯 Navigate to Window > Package Manager

🎯 Click on Packages: In Project which will open a dropdown and select Unity Registry

🎯 Search for AR Foundation in the search bar on top right and click Install as shown below

🎯 Again search for Google ARCore XR Plugin in the search bar on top right and click Install as shown below

🎯 While installing Google ARCore XR Plugin, you might get the below pop-up and make sure choose Yes which will enable backend support for the project

🧨 Configuring Android Player Settings

🎯 Navigate to File > Build Settings and choose Android as the platform and click Player Settings

🎯 Choose Player in the left vertical menu and choose Android symbol tab as shown below

🎯 Then expand Other Settings and you might notice that Auto Graphics API is enabled by default. Make sure to disable it.

🎯 After disabling the Auto Graphics API, make sure to remove Vulkan from the Graphics APIs list

🎯 In the Identification section, set the Minimum API Level to API level 26 (Android 8.0 Oreo)

🎯 In the Configuration section, set the Scripting Backend to IL2CPP

🎯 In the Target Architectures, disable ARMv7 and enable ARM64

🎯 Select XR Plug-in Management in the left menu and choose the Android symbol tab. Enable Google ARCore and close the Project Settings windows

🎯 And finally choose Switch Platform after the Player Settings are configured properly as said above for Android

🎯 Click Add Open Scenes if the scene is not yet loaded as shown below and close the Build Settings window

🧨 The CORE of the project

🎯 Delete the Main Camera Game Object from the hierarchy and add XR Origin and AR Session by right clicking on the hierarchy and choosing XR > XR Origin and XR > AR Session as shown below

🎯 Choose AR Session Game Object in the hierarchy and in the Inspector window add two new components i.e AR Plane Manager and AR Raycast Manager

🎯 Add an AR Default Plane Game Object by right clicking on the hierarchy, and choosing XR > AR Default Plane. After creating it, drag and drop it into the project Assets to create a AR Default Plane prefab

🎯 Drag and drop the created AR Default Plane prefab from the Assets into the Plane Prefab field of AR Session’s AR Plane Manager component

🎯 Create a 3D Game Object (Cube in our case) and change its scale to 0.1 on all axes using Transform in Cube Inspector (so that the cube is not too big for mobile screen widths)

🎯 Drag and drop the 3D Cube Game Object from the project hierarchy into Assets to create a Cube prefab as shown below

You can delete the AR Default Plane and Cube Game Objects from the project’s hierarchy once their prefabs gets created

🎯 Create a C# script (CubeRaycastScript.cs in my case) in the Assets folder by right clicking in Assets and Create > C# Script

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;

public class CubeRaycastScript : MonoBehaviour
{
    public GameObject cubePrefab;
    GameObject cubeGameObj;
    bool objectPlaced;
    ARRaycastManager arrayman;
    List <ARRaycastHit> hits = new List<ARRaycastHit>();

    // Start is called before the first frame update
    void Start()
    {
        // Initialise fields
        objectPlaced = false;
        arrayman = GetComponent<ARRaycastManager>();
    }

    // Update is called once per frame
    void Update()
    {
        // If user touches any part of the screen, the below condition will be true
        if(Input.touchCount > 0)
        {
            // Performs a raycast from the touch position to detect if it intersects with any AR planes
            if(arrayman.Raycast(Input.GetTouch(0).position, hits, TrackableType.PlaneWithinPolygon))
            {
                // Retrieves the pose (position and rotation) of the first raycast hit
                var hitpose = hits[0].pose;
                if(!objectPlaced)
                {
                    // Instantiates or Creates the cube at the hit position and rotation. (this block will be executed only once for the first valid touch)
                    cubeGameObj = Instantiate(cubePrefab, hitpose.position, hitpose.rotation);
                    objectPlaced = true;
                }
                else
                {
                    // Updates the cube's position to the new hit position
                    cubeGameObj.transform.position = hitpose.position;
                }
            }
        }
    }
}

🎯 Attach the above created script to AR Session Gsme Object in the Inspector area by dragging and dropping the script file from Assets to Add Component button

🎯 Drag and drop the Cube prefab we created earlier from the Assets into the Cube Prefab field of Cube Raycast Script component

🎯 Open File > Build Settings and choose Android in the Platform and Switch Platform if you haven’t already did, and then finally choose Build

🎯 During the build process, you will be asked to choose the APK name and the location to save the APK to. Choose any APK name of your choice and save it. Once done you might get the below pop-up and make sure to choose Yes

🎯 Transfer this APK file to any desired Android mobile phone and install the app. Scan through your surroundings and planes will be detected and touching anywhere on the detected plane will place our Cube Game Object on the touch point (as shown below in my case)

Congratulations on reaching the end of this tutorial 👏👏👏! You’ve taken significant steps toward mastering markerless AR using Unity 3D and AR Foundation. By understanding how to utilize ARRaycastManager and effectively place and manipulate objects in an AR environment, you’re well on your way to creating immersive and interactive AR experiences.

To recap, we covered:

• The fundamentals of markerless AR.
• Setting up your Unity project with AR Foundation.
• Implementing touch-based interactions to place objects in the AR space.

I encourage you to keep experimenting and expanding on what you’ve learned. The potential of AR is vast, and your creativity is the only limit. Here are a few suggestions for further exploration:

• Instead of Cube, try to get a model from Unity Asset store and make the project work (Do not forget to drag and drop the new prefab into the Cube Prefab field of Cube Raycast Script component).
• Integrate AR with other Unity features like animations and physics.
• Explore more advanced AR Foundation features such as image tracking and face tracking.

If you have any questions, feedback, or want to share your AR projects, please leave a comment below. I’d love to hear about your experiences and see what you create! Additionally, don’t forget to check out the Unity and AR Foundation documentation for more in-depth knowledge and updates.

Thank you for following along, and happy developing 🐣!