Getting Started: Augmented Reality Using Android ARCore

Pabi Moloi
Sep 5, 2019 · 4 min read
Image for post
Image for post
Photo by David Grandmougin on Unsplash

I had the opportunity to play around with ARCore to create an augmented reality experience for an Android app. I have attempted AR with Vuforia before, but I had to integrate the AR experience with the existing app. I wanted to create something simple enough to help me learn about ARCore and it’s capabilities.

The first thing that came to mind was to render a layout when an image is scanned, click on the layout and open a web link. Seems pretty simple enough, right?

There is a lot that one can do with ARCore, but in this blog post, we will focus on the use case mentioned above.

Brief Intro To ARCore

ARCore is a platform for building augmented reality experiences. It enables your phone to do the following:

  • Sense its environment.

How does ARCore integrate the real world and virtual content? It uses the following capabilities to do that:

  1. Motion Tracking- allows the phone to understand and track its position relative to the world. The phone’s camera is used to identify feature points. ARCore’s motion tracking technology then tracks how those features move over time. ARCore determines the pose(position and orientation) of the phone as it moves through space using the combination of the movement of features and readings from the phone’s inertial sensors.

The ARCore SDK is available for the following platforms:

  • Android

For this blog post, focus is on the Android Platform. We will build an AR app that responds to 2D images in the environment by making use of the Augmented Images API.

Augmented Images API

The Augmented Images API has the following capabilities:

  • Tracking and responding to images that are fixed in place. With ARCore 1.9, ARCore also has the capability of tracking moving images or an image on a flat object held by the user as they move their hands around.

To ensure that the API works as intended, the images that will be used should:

  • Fill at least 25% of the camera frame.

Now that we have the gist of things, let’s get to some code 👩🏾‍💻

Do check if your device is ARCore supported device. The list can be found here.

Enabling ARCore

To enable ARCore for your Android app, you can follow the guide here .

Setting Up An AugmentedImageDatabase

The database will be used to store a list of images to be identified and tracked by ARCore. There are two ways of creating an AugmentedImageDatabase:

  1. Load a saved image database- if you have an existing image database use:
    AugmentedImageDatabase.deserialize() to load it.

2. Create a new database

Loading an image to an existing database

You can load an image to an existing database as follows:

Best Practices for Reference Images

  • Augmented Images supports PNG and JPEG file formats.

Image Tracking

The ARCore session looks for images by matching feature points from the camera and compares them against those in the image database. To check if we have any matches, we loop through the images in the database.

When we have a match, the layout is converted into a render and is added in the ArSceneView.

The following code sample is how we set the image that is matched:

Setting the onClick listeners for images:

Method to open the web link:

The Layout

It was quite a challenge for me to get the hang of ARCore, I hope this post gives you a good head start.

So, What Next?

There are a couple of apps you can build using ARCore. Fancy a bit of Snapchat 🤳🏽 like I do? You can play around with Augmented Faces to create your own version. Augmented Faces allows you to detect different regions on faces and overlay assets on those regions.

What AR app do you want to build?

References

Get in touch.

DVT Software Engineering

Making an impact in Software Engineering

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store