Design for the Future: Google Street View Adventure in VR
Meet our team: James Capone, Yash Mittal, Samantha Nolle, and Nigesh Prajapati
Introduction
The theme of this HCI sprint was Design the future, where our team was tasked with creating one possible application and an aspirational future of virtual reality (VR). When considering the aspirational future of VR, we were interested in the ways that VR could be used to provide new meaning and understanding to preexisting online resources, all while providing a fun and engaging user experience.
Introducing Google Street View Adventure!
Google Street View Adventure is an immersive, choose-your-own-adventure-style game that integrates an engaging world travel story line with realistic audio and 360 VR Google Street View visuals. Users explore the world by making choices in each location to which the game transports them, all while experiencing the sights and sounds of four unique cities. Using A-Frame for web VR development, we structured the user experience around the use of a virtual reality headset and noise cancelling headphones, and thus experience the game’s plot in a fully immersive environment.
Design Process
Learning the Technology
No team members had prior experience working with VR before this project. As a result, we started with looking at multiple, differing ways in which this technology had been previously implemented. We played with examples on A-Frame, which is an open-source web framework for building VR experiences. The experience helped us understand the benefits and limitations of this technology, and analyze what we might be able to achieve at the end of this project.
Brainstorming and Filtering Ideas
Our team was captivated by the ability of Google Street View in VR to fully immerse and transport a user to a specific location despite any barriers to travel, such as mobility, funding, or time constraints. While traversing the spaces for exploration contained within this resource, all while using a VR headset, we wondered how the capabilities of Google Street View might be expanded to provide a new, engaging, and meaningful experience for a user. We wondered how a design might expand beyond the visual experience of simply navigating different locations. We considered existing games that revolved around traveling through spaces, such as GeoGuessr and the Oregon Trail game. Our group discussed that while GeoGuessr provided meaning to the process of visiting virtual locations, the repetitiveness of its structure meant that it did not remain fun or engaging for a user for longer periods of time. Similarly, the structure of Oregon Trail was fun and engaging, as it provided a constantly evolving storyline that simulated the unpredictability of an adventure, but it fell short in providing the immersive and meaning-providing aspects of realistic visuals.
We brainstormed how a combination of the positive aspects of each of these existing designs might be combined, in order to create a new experience. We imagined a new type of game, which integrated the immersive visuals of 360 Google Street View in VR with the engaging structure of a choose-your-own-adventure style game. From this idea “Google Street View Adventure” was born. We imagined the use of imagery from real locations around the world captured on Google Street View, coupled with realistic sound effects, potentially captured from Youtube videos taken at the locations in question. Finally, we envisioned a plot line that allowed users to make choices regarding the direction of their journey, and that these choices would shape the game’s narrative as they continued to play.
Implementation — First Iteration
Our team decided to split into two working groups. Yash and Nigesh focused on coding the VR aspect of our project, while Sam and James focused on creating a storyline, and identifying street view visuals and audio to accompany the storyline. This allowed for our two main goals to be accomplished simultaneously. We were able to tackle the creation of the VR environment, while also drafting the storyline our game would follow. Yash and Nigesh were chosen for the coding aspect because they had familiarity with JavaScript, while Sam and James were chosen to create the storyline because they weren’t as familiar with JavaScript, but they had creative ideas for the direction of the project.
We began brainstorming the storyline with the goal of following a structure similar to the Oregon Trail game. We were inspired by Oregon Trail’s engrossing plot, as well as its ability to transport people to a different part of the world and educate them about it. One important aspect of the game was user control. The gameplay was always changing and unique because the user makes decisions that influence the plot. However, this game was lacking in overall immersion. While it was engaging and informative, it didn’t actually bring people into a realistic visual environment. We wanted to recreate the excitement of the oregon trail plot in VR to provide this much needed full immersion. We decided to change the setting of the Oregon Trail to locations around the world in order to be more fresh and current. This was partially because of logistical limitations. We needed 360 degree images for this game, but there would not be any real, historical images of the oregon trail during this period of history. We changed our setting to be a tour of the world, showing what different places across the globe are like. This story would allow for people to go to places they couldn’t visit in the real world, while keeping control in the user’s hands.
We began drafting the narrative, sketching out the decisions and consequences in a tree-like pattern. A user’s choices would ultimately affect the outcome of his or her journey, where each branch of the tree would either end in a negative consequence, or feed back into the main story.
We used Google Street View to begin exploring the cities we chose for our plot, and to begin to select the street view locations we’d eventually use in the game. About half way through this process, our group discovered that we would be unable to use navigable street view locations due to time constraints, and would instead need to 360 photo spheres uploaded to Google Street View by Google users. Thus, we began the process again, this time downloading photospheres for use in the game’s locations. In order to download the photos for use in the A-Frame code, we used a process similar to the steps discussed in this article. Finally, we selected ambient audio for use in the locations from the AudioBlocks website (i.e. wind sounds for desert locations, crowd sounds for urban locations).
A-Frame and Glitch: Coding the VR Experience
Getting a hang of A-Frame was an uphill battle. Even though some of us had familiarity with web development, we had difficulty in understanding the scope of VR development using A-Frame. Fortunately, we found this tutorial on How to create a Virtual Tour using A-Frame which was almost what we wanted, minus the sounds.
To better user experience, we wanted to ensure that there was a smooth transition between different scenes of our story. A key ingredient of getting this right was crossfading sounds. One of the main reasons we wanted to use sounds in our project was to make the VR experience more immersive, as discussed in the article New VR Tech Aims to Take Surround-Sound to the Next Level. Regarding why we wanted crossfading, we found a medium article Design Practices in Virtual Reality by Jonathan Ravasz, where he talks about how “a gentle introduction to the new environment can be achieved by fading-in the ambient soundscape of the place”. This lowers the shock factor by allowing to build a mental image of the environment via sound. We used the animation engine Tween.js to fade sounds in and out.
We also wanted to make sure the users can easily distinguish what objects are interactive in our design and what objects are not. One of the ways we achieved this was making a very distinct icon for an interactive object, that was different from rest of the environment; we referred to them as “portal”. We made sure to put the portals in locations that made sense and was easily visible to the user. In addition, Jonathan Ravasz talked in his article about how interactive objects should react differently when a user turns their attention towards them. We achieved this by having the reticle (the pointing circle in VR) change its size when the user pointed at an interactive object.
Resources Used
The following tools were used as resources for this project.
- Street View Download 360 to download equirectangular images of our locations
- Storyblocks Audio for the sounds and Beautiful Audio Editor to overlay audio clips and create a binaural recording effect
User Testing and Feedback
After our first iteration, we decided to test our design with a few users to get a sense of how our project is doing. We received both positive and negative feedback from our users.
Implementation — Second Iteration
After the initial iteration, we realized the scope of the storyline was too large. Having previous choices matter later on in the game became too difficult. This resulted in an abridged version of the storyline that cut out this initial choice. It also shortened some of the other aspects of the journey.
Based on our user feedback, we made some changes to our design as well. We pushed the text on the screen more backward so that it is easier for the users to read the storyline. We made sure to put the portals at positions that was easily viewable for the user and made the most sense. In addition,we added “The End” at the end of the story in order to show when the users have reached the end of the game.
We really wanted to implement a “Play Again” button so that we did not have to reload the entire game every time it was over, however, due to time constraints, we were unable to implement it.
Conclusion and Future Work
After two weeks of learning a new technology, our team was quite satisfied with the end result. If we had more time, we would have definitely like to implement the original storyline, with more twists and turns. Having more interactivity in one scene, such as picking an item or driving a car, would have been good additions too.
Designing an immersive experience in VR was as challenging as we had anticipated. We made up for the lack of interactivity in our game by using binaural audio and equirectangular 360 degree images. We are positive that once Google Street View is open-sourced in the VR community, our game can be extended to add more interactivity, with users being able to navigate within each scene.
