My internship at VR lab

Working on semantic-enhanced Augmented Reality app for Android

Ewa Śniecińska
Sep 3, 2018 · 3 min read

Intro

In 2017 I spent summer at Virtual Reality Lab at my university — Poznan University of Economics and Business. All this time I was closely working with engineer and Ph.D. student Dariusz Rumiński . During this time I learned a lot about designing and developing AR apps.

What did I work on?

The project I was working on is divided into two pieces. Unity plugin (CARL Modeler) and Android application (designed for Moverio BT-200). Unity environment is used to create contextual AR presentations. The presentation context includes location, beacon, device type and time. The plugin is integrated with the Vuforia library and use a semantic database to store context, AR scene (ImageTarget) and 3d objects included in the scene. Then Android app uses data stored in the database to show created AR scenes in a chosen context.

My tasks

  1. Make Unity plugin and Android app more user-friendly
  2. Find and/or fix bugs in Android app
  3. Refactoring
  4. Vuforia and jPCT-ae integration

Working on CARL Modeler (Unity plugin)

Before this internship, I had a small experience with Unity (for creating a 2D game). But I still spent a lot of time learning a tool and what most important for the project how unity plugin works.
As existing plugin was written in a rush my task was to make the code a little bit cleaner and make the whole experience of using CARL modeler more friendly.
One of the biggest issue with the plugin interface was a lack of feedback messages. So added appropriate popups. Now the user knows if there is any missing data or what when wrong/good. There were also many smaller issues like GPS location showed as longitude and latitude while most of the people use reverse order.

Fixing Android app designed for Moverio BT-200

During my time at university, I learned a Java (and XML), but Android development was something new for me. I was learning on the go while I was testing app. Then I started to fix founded issues. Most of the bugs were related to using voice commands inside the app.
There were also many issues (in design and code) related to different kind of edge cases like lack of internet connection, not working database server. no location permission etc.
Just like in case of Unity plugin app I added appropriate toasts and popups to let users know what when wrong or good. I also wanted to fix small UX issues like the color of AR pins of the map (it was the same color as location pin).

Adding support for .md2 files

The first version of the app uses only .obj 3d files it’s good, but we really wanted to add animation. To do that I started by creating Unity plugin for importing .md2 files to Unity and adding .md2 files support to CARL Modeler plugin.
After that, I had to find a way to render md2 models in an Android app based on Vuforia. To do that I used jPCT-AE 3D engine. My integration of Vuforia and jPCT is available on GitHub.

Reflection

Working on AR app is an incredible experience. At the beginning of the internship, I was really intimated, but it didn’t take my long get familiar with the new software, 3D models and Augmented Reality. I definitely learned a lot during this summer, and with ARCore and ARKit I think if you want to create future this is the place to be.

I’d love to challenge myself to build more AR experiences (maybe even VR) and keep exploring what this new technology is capable of.

Ewa Śniecińska

Written by

UX/UI Designer and Android developer

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade