NCNY8 at HackPrinceton
HackPrinceton is a 36-hours hackathon that is held bi-annually in the fall and spring seasons jointly organized by Princeton University and the Major League Hacking. Passionate hackers from all over United States and Canada travel to Princeton University in New Jersey and form teams to hack unique projects with the hope of becoming the winning teams or simply for the spirit and experience the enjoyment in participating in a hackathon.
This Spring (Mar 30-Apr 1), four of our National University of Singapore (NUS) students consisting of Gwen Tan (Year 3 Information Systems), Albert Yeoh (Year 3 Computer Science), Chen Jie Yi (Year 3 Business) and Selina Wong (Year 3 Business Analytics) took on the challenge of hacking a product within 36 hours. A dynamic team of students with different skillsets and expertise, they produced DocentAR — an interactive ios Augmented Reality (AR) museum guide that shares the narrative of an artwork in an engaging and comprehensible manner.
The Inspiration Which Sparked The Idea
The four of them were seated in the convocation room during their brainstorming session. Despite narrowing down to wanting to build an AR project, they were limited by their expertise. The large portraits of notable figures and professors from Princeton University in the convocation then sparked an idea — what if paintings could come alive and narrate the story behind an art piece, such as in the way the fat lady guarding Gryffindor’s homeroom in Harry Potter?
Unfortunately, magic wasn’t real, so the team took a step back by revolving their idea around this concept but in a more feasible yet engaging way.
Building DocentAR
With little knowledge in building AR products, the team knew their limits and were aware of the steep learning curve in the beginning.
Within the team, Albert was the designated leader who proposed the four of them split into pairs to pick up the necessary skills in Blender, a 3D creation suite designed for modelling 3D graphics and Unity, a game development platform in C#, which leverages on Vuforia SDK to make deploying an AR platform possible.
The team started off on a smaller scale. They first tested the viability of the technology by setting a subway metro card as an image recognition target on their iPhones which allows a 3D model of a table to appear upon detection of the target. After witnessing the success of the test, they then proceeded on to a larger scale, by building a 3D model of Vincent Van Gogh. Each of Van Gogh’s individual body parts were painstakingly created in Blender, before combining them in Unity into an asset.
With a model that appeared in the AR platform, the team felt that the project was still lacking — a voice! They proceeded to record a voiceover for Van Gogh, and wrote C# scripts to integrate the voiceovers and animations to make Van Gogh’s model come alive in the AR.
Despite having close to no knowledge in programming with C# and Xcode, the team picked up the programming language on the spot with the help of various online tutorials and coding forums from the Internet. They also produced a prototype which displayed the app’s home screen and navigation page through Xcode.
DocentAR
The final presentation day came when the team had to present their project to the judges at the designated booth. Many of the judges came by and most of them were deeply fascinated by the project. One of them even suggested them to go on a few steps further to develop interactive models which could respond to speech! Think Google Home on an AR level. They were also given generous feedback on how to leverage on machine learning as well as tap into museums’ databases to generate narrative scripts for different paintings in the museums.
Learning Points
Although the team did not win, they felt satisfied with what they produced at the end of the day and the skills they’ve gained over the 36 hours of the project. It was a concerted teamwork effort with everyone on the team willing to put in their utmost effort to make building DocentAR possible, no matter how challenging it was to them. With a leader, things also felt smoother. Albert did a great job in breaking down a difficult task into smaller, bite-sized steps and delegated each of the mini-tasks to everyone in the team to build the app in an efficient manner. Not only that, everyone on the team was very supportive of one another, where each of them took turns to rest in order to boost their productivity and energy levels!
Apart from that, the team was very grateful to have been given the chance to step out of their comfort zones and acquire certain technical skills right from scratch. They were proud of creating a product which was capable of delivery an engaging experience to its audience.
Check out the team’s project submission to HackPrinceton Spring 2018 on Devpost!
