UGA NMI Capstone Journey: VolumeAR
Product Pitch:
If you have ever had to move a couch for someone you know that there are at least two major anxiety points in every such trip: how are you going to fit through the door going out, and then the same again going back in when you reach your destination. We may have vague understandings of how large something is or what its dimensions are, but for many people written numbers without a visual reference point is close to meaningless. Unless you always carry around a tape measure it can feel impossible to know if your hunch about something’s size is correct. What’s even worse is if you’re buying something online, sight unseen, at which point you have no visual reference. Isn’t it difficult to shop for furniture online, trying to pick something when you’re not sure if what you’re buying will fit through your door?
The solution to all this uncertainty is my app, VolumeAR. It is a tool designed to leverage Augmented Reality to help users better understand the world around them and solve problems just like these. We often have a vague understanding of what some arbitrary unit of measurement looks like, but unless someone is constantly carrying around specific tools with them wherever they go there was no way to be sure. This is no longer the case.
Using the power of AR, VolumeAR allows any user to instantly turn their iPhone or iPad into an on the go reference device. The app presents users with three basic shapes and the power to adjust them however they like, all while placing the objects in their space. Seeing really is believing, and now its understanding too.
The Problem:
That all being said, what am I really doing? VolumeAR seeks to solve a simple but challenging problems: how big is this thing, how much space will it take up, what would something that size even look like? Without physical access to something, it can be hard to visualize how much physical space an item might take up. Users can take precise measurements, but seeing something is often much to comprehend and process than taking raw measurements and imagining what they mean. Similarly it may be impractical or impossible to access an item you’re planning to fill space with prior to actually getting it. For example, it a user is planning to buy a tv, how much space would it take up on the wall, or would a refrigerator fit into their kitchen? Both items are large, bulky, and expensive, so it's not like a user could easily borrow one from a friend or acquaintance to see how it fits into their space. While it cannot help with stylistic fits, VolumeAR can help users to determine if the bulk of an object fits where they want it to go, or alternatively it can help them determine just how large or small an item they want to fill their space.
Product Features:
VolumeAR is an easy to use app designed around showing users the space occupied by an object in their space. Users select from three basic shapes (a cylinder, cube, or sphere) and manually adjust inputs (in either inches or centimeters) for height, length, and width to determine the exact size they are looking to visualize. A simple to navigate and operate user interface allows the user to focus on visualizing things in their space, and not get bogged down in how to operate the app.
Technologies Used:
In creating this app I used several different technologies and applications. The heart of the app is running on ARKit 3.0 from Apple. This is an Augmented Reality development platform that allows the creation and implementation of Augmented Reality models and features in iOS apps. The skeleton of the app was created using Swift in Xcode 11.3. Xcode is a coding application for mobile app development across Apple’s whole range of operating systems. Swift is a coding language that, while not as new as the more recent SwiftUI, is more fully featured when it comes to AR development. These two were chosen both because of prior experience using them and because of the uniformity of delivered experience iOS development allows for (Coding for landscape and Portrait is bad enough, never mind the plethora of configurations available on Android…and I say that as an avid Android user. Finally to create the actual AR models themselves I originally tried using Reality Composer from within Xcode. This is a powerful tool for creating 3D models. However, it proved too new and not as fully featured as the older SceneKit, specifically it only allowed scaling to be done uniformly across all three axes which meant it was not suitable for what I wanted to do. Ultimately I ended up using Scenekit to generate the shapes live in the app. They are a bit more crude in looks than the models I could make in Reality Composer, but they allow the app to function, which is the most important thing. One final addition I made was to include a single SwiftUI page, the Welcome page to be exact, embedded in my Swift storyboard.
Lessons Learned:
What did I learn from this project? Oh man, where to even begin!
To begin with I would say that one of the greatest lessons I gained from this assignment is how to manage a project, from start to finish, completing multiple different components with various deadlines and overlaping requirements. I know now that I was not nearly organized enough in the beginning. Looking back on it now vs where I am today it almost feels like I was winging (I wasn’t, but allow the comparison for dramatic effect!). This can be seen in some areas of my project, or at least would have been seen for those who were with me along the way, when things like the Style Guide were made after other documents … which needed to comply with the then unmade style guide. This created a whole headache of having to go back and fix things that could have been easily avoided if I had taken the time to plan things out from the beginning and recognize what relied on what rather than just aiming to take down the biggest project one at a time.
From a technical side I learned how to code for AR. I went from 0 to 100 (figuratively) in terms of experience with it these past 6 months. And while I was taking a class in iOS development at the same time, the fact that most things I relied on hadn’t been fully implemented in the updated SwiftUI I was learning meant I had to go back and also teach myself a lot of things in Swift 4.0, a language we touched on in class and then quickly moved past. As someone who had never properly coded before starting my NMI journey a little over a year ago, it has been a bit of a wild ride, but it’s also been an incredibly rewarding one!
Overview of my Emerging Media and Capstone journey:
But now that I've mentioned some of the lessons I have learned throughout the process, here’s how I went about learning them! I began my EMS journey just over a year ago in June 2019. I had always been interested in the internet, digital communities, and interesting application of technology, so the degree seemed like a perfect place for me to explore several of my interests all at once. I also learned pretty quickly that things in the NMI can move pretty quickly! From my first online courses over that summer to jumping right into learning in my first weeks on campus, I have thoroughly enjoyed the speed of my EM journey, and while I would absolutely love to spend more time with the amazing people of the NMI, and the UGA community in general, I’m also incredibly thankful that I have made it through to the other side! Its a hard process, especially when you try and get it all done in one year!
But this medium post is largely about my capstone journey more specifically, the journey within a journey so to speak. And before I begin speaking about it, I really just have to say: Google really hung me out to dry! Originally I was intending to make a kind of scheduling action for smart lights using Google assistant. However, Google’s introduction of the feature in the upcoming version of Android meant I had to pivot and quickly decide on an alternative. VolumeAR was the ultimate result of that pivot. In JRMC7011 we had to come up with 3 ideas for projects and explore them as our final project for that class. VolumeAR wasn’t actually one of my 3 as I felt my other two ideas (the scheduling app being the third of those 3) were either infeasible alone or too nebulous to attempt. So VolumeAR was my 4th bite of the apple and ultimately I’m happy I ended up where I did because I get to explore a very cool technology that is on the cusp of revolutionizing how we interact with digital technologies.
Furthermore I decided to undertake this challenge at a time when most of the technologies involved are being rapidly innovated on or left behind. This made for a very interesting (and difficult) learning curve, but as always Google was my friend. I spent countless hours scrounging StackOverflow and YouTube for tutorials and how-to’s, and ultimately I think I might have even learned a thing or two from all of them!
Ultimately the biggest challenge I faced was the question of scaling individually along an axis and letting the other two remain at their current values. Well, that and developing for iOS whilst not owning an iOS device. Combined with the Covid-19 pandemic meaning my friends were suddenly less likely to want me to come over and borrow their phone or iPad for some reason, meant I had to borrow an iPad that the NMI (very graciously) loaned me. However, I wasn’t able to collect it until close to April due to the University being shut down for two weeks following spring break, which meant that I effectively had about a month of coding where I could test as I went. I had written some things previously of course, but without being able to test (oh btw, you can’t run the simulator built into Xcode when you’re testing VR, fun fact that you probably didn’t know and I certainly didn’t) I was essentially coding hypothetically. It turned into a bit of a race, but I managed it in the end. This all being said, if I could do it over I would have planned things out much more thoroughly than I did, but more on that in the next section.
Once I had the issue of scaling sorted I still had to build the UI that let users interact with the function I’d made. Specifically I needed to find a way to allow the label I’d placed have an up to date display of the current value of each individual axis. I tried a several different methods, and was rather pigheaded about trying to make one thing work, until eventually I had a bit of an epiphany moment (in the shower obviously!) where I realised I was doing things the hard way for no reason, spending hours and days trying to solve a problem to save me time that I wasn’t actually saving. So before we get into the actual advice section here’s a bonus one: be flexible. Don’t keep running at the wall cause you think that’s the only way. The cool thing about tech is how we can innovate and find new and interesting ways to use it. Take a deep breath (or maybe a nice hot shower?) and find the better path.
Advice for Future Cohorts:
That being said, here's my more specific advice! The first one is to make sure you take the big boss project in 7011 seriously! Think through your ideas, don’t just pick them cause you need another one and you want to be done. Really think through how you would implement the project, what tools you’ll be using and what investment in time it will take to get comfortable using them. I ended up doing a project that wasn’t even one of my three and that sucked at first, so make sure and do your best to find three ideas you’d be happy pursuing and you’ll be in a better position than I was!
Another thing is if you can, start early! Start getting comfortable with the technology and/or services you’ll be using as soon as possible, it will pay off massively in the long term and is something I wish I had done. Also get used to browsing forums and year old walkthroughs, the life of self learning is rough but can let you find some cool and random stuff that you didn’t even know you were looking for! I never thought I’d be learning about AR, but because of the awesome way that the NMI has Emerging Media planned out you get to try a bunch of different things very quickly and once you find something you just gotta run with it! Especially if, like me, you're trying to do all of this in one year!
Another thing I’d add is that Google is going to be your best friend and best teacher. The faculty of the NMI are all incredibly, but during capstone you will be largely guiding your own learning and as a result you need to learn not only what you need to know but how you phrase your question to get the answer you need. I spent countless hours on forums, tutorials, and explanatory documents. The more comfortable you are using Google (or whatever your search engine of choice is) the better off you will be!
Finally I’d say, again, plan ahead as much as you can, space out your work and if you can work with others. Even having someone to talk to to make the time go by can be a great benefit to your morale as you make your way through the project. Its certainly a rewarding one, but its a long battle!
Logo:
My thought process behind my rather…clean, shall we say, Logo is that my app is all about creating objects. Making shapes that are the users own. They put in the values, they create the object in real time exactly how they want it. That being the case, why have an image or object they can’t manipulate. I thought I’d let the name speak for itself, in a nice clean text, and let the objects be those of the user’s making. I later added a geometric cube to the text as another version of the logo to serve as an inspiration, a starting point from which the user can go nearly anywhere.
Links!
Here’s the link to the section of my website which contains all the relevant bits and bobs for this project. Some neat stuff, should check it out!
http://millsplays.com/VolumeAR.html
Thanks!
Thank you for reading my NMI journal and checking out my work on VolumeAR. If you’re a future or just starting NMI student, I wish you all the good luck I can and know that you’ll do incredible things. To my my friends, teachers, and everyone else who helped me out along the way I have nothing but gratitude. This has been, despite 2020 just being generally awful, a truly wonderful time in my life and one that I am positive will not only stay with me, but will shape the journey of my life for years to come. Thank you. Danke. Ta much. Until next time :)