Augmented Reality App Development
This is a story of how we launched the PeakVisor app — an augmented reality app which overlays camera screen with peak names and height, and a rendered 360 degrees mountain panorama.
Augmented Reality App Idea
It all started several years ago when we were toying with a bunch of all available quite advanced sensor mobile phones while trying to figure out which value a smart device could provide for an outdoor activities app. The prototyping iterations were done under the umbrella of the Routes.Tips project that over the years has evolved into a service dedicated to helping people explore a variety of travelling options suited for a weekend getaway or for longer vacation activities.
The initial idea for prototyping was that maps and compasses aren’t that intuitive, in fact in some cases they turn into painstakingly cumbersome tools. Getting directions in an unknown city or on a hiking trail in the mountains might be very complex. While in the city taking a wrong turn might be easily fixable, the wild outdoors are not to be messed around. An error in your route calculation can lead to awful consequences.
You don’t need to be “geographically challenged” to experience map and compass complexities. Since school I have been participating in orienteering challenges and the results were not that bad. Yet, I go hiking in the Alps very often and by making some shortcuts or badly interpreting the signs in the local language I sometimes find myself very tired and miles away from the target. Yes, I do appreciate the saying of J.R.R. Tolkien “Not all those who wander are lost” and find great joy in being in the wild. However, there have been cases when I would have appreciated it even more if I were pondering over it in the comfort of a nice Alpine chalet.
Wouldn’t it be easier to see directions on the camera screen?! Wouldn’t it thrill you to see a walking path, attractions, sights, and useful information and to choose for yourself what challenge to take up, not relying your fate in the hands of Doom? At the end of the day, when we are in a new place, the most common question is “What is it? What’s the story behind this?” I think wasting energy on figuring out the relations between map markers and visible objects is also a total waste of time; it undermines your traveling experience.
Taken all that into account we came up with an idea of augmenting the camera screen with travel routes (and we have plenty of them on the Routes.Tips platform).
Hacking the Augmented Reality App
Little did we know back then! It took us more than a year to brainstorm and figure out the idea, adjust the maths, prototype in realistic environs, hack the precision, and launch the first version of something downright useful.
The initial assumption was that the hardest part would be maths coordinates. Basically, the app needs to figure out the phone (and its camera) 3D position, calculate the field of view depending on the phone camera optics, translate the map of the surrounding terrain (latitude, longitude) into a 3D scene. That means quite a lot of maths, and hence a plenty of places to have potential bugs. What’s more, debugging the algorithm is not exactly a piece of cake. With hindsight, I could tell that it really was.
One of our talented developers mastered all the maths pertaining to the app, so we could play with it. It became clear that we had to get much deeper into the hardware to do the job.
iPhone devices are equipped with an accelerometer, a gyroscope, and a magnetometer (simply, a compass). It turned out that depending on the weather conditions all the sensors could be inaccurate. Lower temperature, wind, motion, could lead not only to imprecise data values, but also even to the ongoing values drift. It looks as if every second the North slips several degrees (left or right).
Even slightly biased sensors could ruin the whole experience and lead you in a completely wrong direction. Thus, we had to dedicate a lot of time and effort in fighting even minor inaccuracies coming from the sensors. It was a great time! Hours of work outdoors, in the mountains, with epic views, testing the compass!
What that Mountain is?
While the primary idea was to augment camera with routes, we still decided to stick to presenting only mountain summits for starters. It simplifies the task a lot, the app renders only points (no trails and more sophisticated shapes). Also, we wanted to make the project very handy and solve one simple problem very well.
A lot of people when hiking or when visiting an observation viewpoint have a question:
What is that mountain?
Getting that information on the map is doable but in most cases very hard. In the countries where mountain tourism is a very important part of national budget, you might see instalments with photo panoramas where all the mountains on the skyline are properly labelled. Like this one in Switzerland on Monte Boglia’s summit (above the Lugano).
We consider these panoramas as an MVP, which confirms that there’s a problem that must be solved! Wouldn’t it be great to have such an augmented skyline wherever you go?
Find all the peaks with the PeakVisor app
Having done all the heavy-lifting for most of the mobile sensors, we started to envelope the technology over the app. We called it “PeakVisor” based on the online feedback we got from our friends’ network.
We removed most of the elements we had used to debug the sensors and algorithms but some of those tools were so liked by the team that we decided to leave them available through to the App Store. Here are our most appreciated elements:
3D Mountain Panorama
Finding all the peaks on the skyline required a lot of map checking, and having a high precision 3D mountain panorama rendered for the current location simplified this task a lot. So, we built a real-time terrain rendering engine into the app.
3D Compass and Altimeter
A fab cunning 3D compass and an altimeter appeared to be so useful in the mountains, that we preserved them too.
Sun Trail
There is a Sun trail and the current position of the Sun is pictured on top of the camera view. While preliminarily we used it to calibrate the compass when being far from mountains, the feature appeared to be very helpful to plan photos and journey itself. Oh, by the way, people from other parts of the world where it is not that sunny as in Italy could see the Sun in any moment!
If you hike or simply spend time in the mountains, then please let me know whether the app would be useful for you!
The PeakVisor app is available in the App Store but we already started thinking about the Android version as well. For now we have only the Alps in the app but the very next release (probably already at the time you are reading this) would contain the whole world. We have just needed some time to make an interface to manage the maps data and make everything work smoothly even without Internet connection.
Hopefully, I’ll soon have some time to write about our marketing efforts and results. That’s all for now! Happy and safe outdoors!
Did you like the story? What do you think of Augmented Reality apps?
Please check the PeakVisor app and let me know what you think of it!
