ARCore: The Beginner’s Guide

Mindinventory

About AR-Core

• The term augmented reality was first invented in 1992 by two Boeing engineers Thomas Caudell and David Mizell.
• The very first project of Augmented reality named “Sword of Damocles” was built in 1968 by computer scientist and researcher Ivan Sutherland.
• His goal was to build the ultimate display that is capable of transforming the real world into your world space.
• After that, Google launched its first project named Tango to use it in smart glasses.
• In 2017, an upgraded version of Tango, ARCore was introduced.
• ARCore is an application programming interface i.e. API to build and develop mobile AR applications.
• Augmented reality is basically a direct or indirect live view of physical or real objects by computer-generated information.

Augmented Reality(AR) vs Virtual Reality(VR)

• The first and very important point in VR is that it’s handled by a highly configured computer machine or smartphone, but AR does not meet this requirement.
• Virtual Reality creates a simulated environment and places the user inside an experience, while Augmented Reality is a direct live view of a real-world environment.

These differences make Augmented reality better for day-to-day applications because users can continue to take the experience with their engaged world.

Nowadays AR is being used in multiple domains like, shopping and retail, educational, healthcare, gaming, social media etc.

Hardware that supports AR

• Accelerometer: Measures the velocity or acceleration of the device.
• Gyroscope: Measures/maintains the orientation and angular velocity of the device.
• Phone camera: A phone camera helps the AR system to create the surrounding environment by supplying live feeds.
• Magnetometer: Provides the orientation of the device relative to the north direction. So that your application can auto-rotate the digital maps according to your device orientation.

Currently latest smart phones have defined hardwares that supports AR.

Environmental understanding

• This is the process of the ARCore system for seeing, processing, and using information about the physical world around AR devices.
• The process begins with creating a plane that is used for motion tracking.
• In ARCore terms plane means the cluster of feature points that are being used to determine the current camera position.
• These planes help the ARCore system to properly place the digital assets in real-world space such as in mid-air or on the floor.
• Once you place the digital assets in real-world space, you can then hit test or ray-cast to check what plane the user is tapping on.

Tracking in AR

• In terms of Augmented Reality, tracking means recognizing the real-world space or objects using the brain of the hardware device like when you move around, the device tracks the tea table, flowers, plants, trees, etc, and then passes these objects to ARCore systems.
• There are two types of tracking, outside-in tracking, and inside-out tracking.

Outside-in tracking

• It means that it is maintained by the devices that are mounted on the wall or on the ceiling and then feed the live images to AR-supported devices.

Inside-out tracking

• It means that cameras and sensors are built into the devices.
• Smartphones are an example of this type of tracking.

Tracking: Mindinventory

Placing and positioning

• There are basic rules in AR to place the digital assets and the first rule is Place. When you place any digital asset, it sticks with that position even if you are in motion. This could be something like a concrete wall, floor, etc.
• If you look away and come back to that position the object would be there for you.

Placing and positioning: Mindinventory

Pose

• Pose means any object’s position and orientation relative to the camera position and the real world around it.
• ARCore uses an accelerometer and gyroscope to measure the velocity and orientation of the camera and then provide necessary data to the system to create a virtual environment.
• Once the ARCore system knows the pose of your camera, it knows where to set the digital assets to seem more realistic in your environment.
• As soon as you place any digital assets, they should be properly scaled with correct orientation as you walk around or towards them.

Anchors

• Anchor means the points in your environment where digital objects should be held. Anchors are also referred to as anchor points.
• In other words, when you place any object that point of the real world is considered as an Anchor point, which keeps the object for an infinite time even if the user moves around or leaves that place and comes back.

Light estimation

• Nowadays, smartphones have light sensors that help to dim or brighten depending on where you are standing.
• Current AR technology estimates the lighting, color, and temperature to make objects more real.
• The ARCore system fetches the light, color, and temperature by scanning pixels from the camera image, which helps to lighten/darken the AR object.

Light estimation: Mindinventory

Scaling

• Scaling is also an important part of ARCore.
• When you place an object at a specific distance, it should be properly scaled for real experience.
• Let’s say when to place an object. First, it looks with actual dimension, and then it gets smaller and smaller as you go away.

Scaling: Mindinventory

Occlusion

• Occlusion is what happens when one object is blocked by another one, in another word you can say hiding one virtual object behind the another or real ones.
• Same way AR objects have to behave to be seen as real.
• This means that the AR system has to calculate the distance and has to understand where the object is.

Congratulations!! You got a basic understanding of ARCore which will help you to kick start your first Augmented Reality application.

Like what you read? Don’t forget to share this post and clap.
Stay tuned for later updates. In the next blog, we will see how we can use and implement ARCore for Android mobile apps.
Thank you!