GSoC ’17 Final Report

This summer, I was quite fortunate to work on the ViSP-Unity Augmented Reality Demonstration project with my amazing mentors Marc Poliquen and Fabien Spindler, as part of the Google Summer of Code Program. I worked on providing ViSP functionality in the Unity3D engine, and I'd like to take a moment now that the program is over to summarize what I've done, recall the many things learned over a plethora of successes and failures, and talk about future work to be done in the space.

This post aims at linking and listing out all the feature milestones developed during my project and the relevant commits, along with the goals missed and work to be done in the future.

Milestones Achieved

  1. WebCam plugin

Here are commits to show the work I have put in for phase I of the project, which was to write a plugin that passes webcam feed from Unity to ViSP C++ shared library: https://github.com/aryamihirsingh/visp_unity/commits/webcam_plugin

2. ViSP blob tracker plugin

The following link of commits, amount to the work I have put in for the phase II of the project that was to provide single blob tracking functionality: https://github.com/aryamihirsingh/visp_unity/commits/blob_tracker

3. ViSP four blob detection and the pose estimation.

Finally coming to the AR part (Phase III) of the project, for which the code is written with the intent of simulating pose estimation in Unity using four blobs. Here are the commits showing my work: https://github.com/aryamihirsingh/visp_unity/commits/four_blob_track

Some Links

Here are some links for Augmented Reality Demonstration provided by ViSP. Writing shared libraries were largely derived from the following code.

Here is the link to the Github repository for ViSP Unity:

Here is the like to my fork, dedicated to all my GSoC’17 work

Lessons Learned

While I learned about tools like Unity and various types of Augmented Reality applications in general, I also gained some more general advice from my mentors that I will be sure to carry on into the future.

Always ask for help if you need it

I was lucky to have easy access to my mentors through a slack channel, Google Hangouts conversations, and thus we frequently talked about the various tricks we were thinking about for the Augmented Reality application and other aspects of the code. Additionally, we had several video calls to discuss anything that was hard to convey by internet message. My progress was catalyzed immensely by their constant availability, and I’m extremely happy that I asked for help whenever I needed it.

Never assume something is easy

When writing my proposal I assumed that implementing Augmented Reality Application in Unity would be a straightforward affair. Indeed, writing a first few getting started versions were quite simple. However, later parts (blob tracker) were much more involved. As a result, I spent far more time on the final two phases of the project than initially planned, which limited the amount of time I had to work on other things originally outlined in the proposal.
Thus, it’s important to be flexible and be willing to change your work priorities when obstacles come up.

Milestones Missed / Future work

  1. One of the things that was originally planned in the proposal and could not be completed, was placing a GameObject in Unity at the center of the four blobs, depicting the correct orientation of the gameObj showing the pose of the blobs.
  2. Android app for Augmented Reality Demonstration. An android app that can be used by devices to provide augmented reality functionalities offered by ViSP.

I am very much going to continue contributing to ViSP and work towards bettering the functionalities of ViSP in Unity engine. :)

Thank you.