Augmented reality in mobile applications

Recently our team was working on tourist app that enhances user experience of sightseeings. In some ways app was like all other tourist apps — it includes basic informations about attractions and activities that can be found near sightseeings,list of upcoming events and so on. Since building mediocre apps isn’t our thing, we were thinking on how to improve experience of sightseeing. We came up with idea of using augmented reality for showing informations about sightseeings. Before speaking about specifics of of this let’s first talk about what augmented reality is.

Introduction to Augmented reality (AR)

In basic words, augmented reality displays real world objects that are processed and altered by the computer. For example, we could create 3D objects and displayed it in specific place in real world by placing tracker object or using phone’s GPS. Image bellow shows one of the most popular AR app. (image frommacrumors.com)

Pokemon game in augmented reality

For more details on augmented reality check this wikipedia page.

How we built and Augmented reality app

Ok, and it’s cool stuff, but how does it work?

For augmented reality to work you need camera,accelerometer,rotation sensor and some math skills and GPS. Idea behind all of this is to use data from real word, data gathered by the sensors, and process it in the way computer can use.

From camera we get informations about view perspective — vertical and horizontal field of view, angles and direction in which phone is facing. For calculating where your object should be placed you can use some sort of detection objects like placeholders or rely on sensor data like GPS. In our app we wanted to use geolocation for displaying augmented objects so we went with using sensors instead of placeholders.

For displaying objects based on location GPS and accelerometer and rotation sensors are mandatory. Other sensors like gyroscope, magnetic field and pressure sensors can be used to suppress sensor jitter and make your positioning more accurate by removing redundant sensor values.

Finding proper object location

Most challenging part of making an AR is the math part. To be fair, it isn’t a rocket science but isn’t the elementary stuff ether. Most math work can be done with integrated math function, you just need to make sure that you provide right sensor data. Difficult part is deducing how to transform your sensor data into position on the screen. For that you should be familiar with how 3D projection works and what azimuth is. Let’s explain this briefly on object that is represented by a single dot. You need to calculate distance between your phone(GPS location) and object (your dot) and find a bearing to that location. After you know the bearing (azimuth) you can calculate location of object on the screen. To make location on screen accurate you should take into account height difference. That problem was tricky to solve because every solution we found had a problem. We tried using GPS, Google Api and barometer sensor. Problem with GPS was that it is very inaccurate. Problem with getting height from Google Api is that it’s only free for limited amount of requests, which was a problem for us because we didn’t know how much client base would we have and what pricing plan would be good for us. Last solution was getting data from barometer sensor which offered okey result but only few phones had barometer sensor. After a while we decided to implement GPS and barometer solution. If barometer sensor is available he will provide height, and if not, GPS will provide height.

Great, we finally have something to work with, an object location. But this “dot” doesn’t do much good for our sightseeing app. To make use of it we have created simple “Marker” that shows name of location and distance to location and enables user to open single detail page for that location.

Improvements

This was our first attempt to build and AR, and we think that we have laid a good foundations for the future versions. In future version we will work on more accurate height value by implementing Google Api paid solution or finding new solution. Furthermore, we are keen on improving user experience by finding new ways of displaying informations.