Hybrid MR Environments Experience
Project 2. Hybrid Pittsburgh Campus Tour | Christy & Jess
Overview
- Design and construct a hybrid campus tour for a particular site at CMU
Initial Process
On-site observation
There is plenty of room around the fence for visitors to freely interact with, but the chairs and the tent are distractors that take away attention and are visually unappealing. We are aiming to film when they’re gone!
As the visitors approach the fence, they can learn more about student organizations in categories, split into 10 sections: Service/Charity, Professional/Academic, Technology, Entertainment/Arts, Shared Identity, Sports, Greek Life, Social/Gaming, Publications, and Political/Activism. At about 5 feet away from the fence, the visitors will be able to hold an AR paint brush, which will basically be their mouse cursor for learning about student orgs. They can swipe the paintbrush to move back and forth between contents, up and down to scroll. To click on a particular organization and now more about them, they can dip the brush in the virtual paint bucket and splatter the paint onto the section to “click”. From the individual clubs, users will find a section where they can save information and interest forms to their phones for future reference and application.
Personas
We try to keep our personas diverse and inclusive, by creating personas with various background (international, local, etc.) and wit various interests! We also kept in mind to incorporate the accommodations that they need to later show that in the video. For example, there would be subtitles available for people like Allison, and club categories like “sport” would be what Ben is interested in.
Christy’s Self reflection: As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?
I believe it is the role of a designer to encourage healthy use of digital media; something that aids our daily experience like IoT does, but also something that isn’t too harmful for the human body, like addiction or health hazards. I think it is also really important that digital media like AR should clearly be distinguished from actual reality, so that people who are using the technology aren’t confused of what’s real and not. Even in computer games without augmented reality, I’ve read that addicted users would be confused of what’s real and what’s in the game and lead themselves to dangerous situations.
November 19
We took a campus tour today, since neither of us had done a CMU tour. We thought it might be helpful to see what they discussed on the tour as well as what questions real prospective students and parents had. I (Jess) definitely learned some things I didn’t know before, which was surprising.
November 22
We chose the Fence as our tour stop. We have two main interactions: a virtual paint-the-Fence activity, and an interactive student org directory.
When someone approaches the Fence for the first time, they’ll be shown a little bit of basic information about what the Fence is. We haven’t decided how that information will be conveyed yet, but we will soon.
After that first introductory message, the Fence interaction will change depending on how close the viewer is to the Fence. If they’re further than 30 feet away, they’ll see the Paint Your Own Fence activity. If they’re closer up to it, they’ll be able to explore the interactive student org directory.
Here’s Jess’ storyboard of the Paint the Fence interaction. (The person is closer than they would be in reality; I just had to show it that way so it would fit into the storyboard reasonably.) Some of this has changed since I made it — I decided to add a “Back” button on the ring, and colors show up on a spectrum so that visitors can use exact colors instead of being limited by a certain palette. Essentially, when in the Paint the Fence range, a ring will appear around your wrists. Different tools will be listed around the ring, and visitors can turn their hand like they’re turning a knob to rotate the ring around. They can tap on a tool to begin using it, and then certain elements will appear depending on what tool it is. Text would have a fonts menu, a color menu, and a keyboard so users can type text (they can also speak it aloud, if they’d rather). Base color has a color spectrum for viewers to choose from.
November 29
We decided to cut the “Painting the Fence” interaction from this project. While we would likely include it in a real-world client project, for time and simplicity we chose to focus on the interactive club directory at the Fence.
We also made some changes to our Fence interactions. Rather than a paintbrush cursor, visitors will simply use hand gestures to navigate between clubs. Upon stepping into a category area, they’ll see a few spheres with club logos on them. They can swipe to scroll through, and if they grab one, that club’s most recent Fence painting will appear on the Fence. They’ll also see a video of the club in action. If they want to learn more or save the club for later, they can place the orb into a virtual box that appears on their wrist, then access this info from their phone later.
November 29–December 8 Progress of Sketch Video (Christy)
Structure of video:
I thought that it would be a more familiar and smooth process if the tour guide program recognized the user by their names and information based on their registration info for the tour. This would not necessarily have to happen at the fence, but when they first begin the tour.
We divided the the function of the tour so that the first half could provide the user with brief information about the context of the fence (macro), and in the second half, they can enjoy a personalized directory of the club organization (micro).
Plan for technical executions
- Adobe aero using dimensions for assets that needs to be fixed in their locations
- Green screen dimensions for moving assets or ones that should not appear to be on top of human body
- Green screen filming of hands to show interactions.
- Photoshop to create digital images
Assets created for selecting the club categories in threshold 2:
Initially, all the assets were white because we wanted to keep the ui relatively simple and clean, but after getting feedback realized that the white assets on a colorful background makes it really hard to recognize the assets from a distance. We realized an important point that whichever color we choose, if the physical fence happens to be the same color as the assets, it would be a problem. We decided to program the assets so that they would always be programmed to be in the complementary color of the painted fence. So in this case, it would be green.
We went to take new footage.
Then, I proceeded to create the assets to represent each club after the user enters one section of the fence.
We were contemplating which 3d shape would best fit the idea of navigating through different clubs. We were leaning mostly towards a sphere, but since we are conveying the concept of a fence that is painted, a droplet shape could also well represent the idea. However, we ended up choosing the sphere because it seemed to best work with the idea of user holding it in one hand while they watch the interactive video.
For this video, the club we chose as an example to convey the idea was Lunar Gala, partly because it is one of the most well known CMU student org, and party because it had the most footage we could use as the 360 video portion.
We thought that our interactions was pretty simple and self-explanatory, which is why we decided to use small captions to explain each stage (image above) instead of using vocal narrations. We were careful not to make them in similar styles so that the audience would not be confused which asset is part of the tour and which asset is not.
Music:
As we don’t have a vocal narration, we thought it would be a good idea to convey the mood through music. We were looking for something that’s positive, rhythmical, and engaging!
During the final tweaks, the part that I focused mostly on was timing and speed. I realized that because I have been looking at the video in after effects for too long, I didn’t realize some parts would be too rushed for others to understand.
For example, when the users see the 360 hologram video of the club they selected, the part where the user clicks he button and reads the sample text was done too quickly, so I adjusted the time map to make it longer while keeping the same clip of video.
December 4
For our presentation, we split it into the following sections:
Problem Space
Personas
Software & Hardware
Flowchart
Sketch Video
Key Interactions
Value
Jess designed some icons to use in our presentation to keep text on screen to a minimum.
Icons for presentation created by Jess. (The bridge at the bottom is the logo for The Bridge, and was not created by Jess.)
Jess also created this systems map to explain how different elements work together to make the tour technically possible.
(A note from after the presentations: If Jess were to revise this map, it would also show the users’ personal technology and how the Hololens sends their tour data to their phones.)
Christy’s End of Semester Self reflection:
I think the second project was gave me an opportunity to extend my abilities to visualize user interactions as it was in video form. I began to utilize animations to demonstrate interactions in the first project, but I think I was able to consider the importance of speed, timing, size of text, angle of shooting, as I was able to create the footage and the interaction assets from scratch. I think the skills needed for the second project was more difficult for me to achieve because it was more challenging to convey my version of the 3d spatial experience into a design project for others to understand as well. One thing that was very interesting during the second project was that while creating the sketch video, I had to distinguish what is appropriate to display in first person or third person, and if the audience watching this video could understand what was going on. I think this aspect translates to the role of an environments designer. I think an environment’s designer’s role is to shape the future and narrate stories of how new technology and the virtual world can enhance our lives in the physical setting. At the same time, I think it could also be something as small as improving someone’s daily routine. I think there are no set limits to an e-designer’s role, but they solve problems in a physical context and suggest solutions that could potentially better our future as well.
link to Jess’ reflections: https://jaj2.medium.com/e-studio-discussions-88ba926d2549
