IxD Studio Project 3: Interactive Environments
Carnegie Natural History Museum | Team Michelle Cedeno, Hannah Koenig, Yiwei Huang, Amrita Khoshoo
10/28 Kick-Off Meeting
We had a quick kick-off meeting to discuss our working styles, project priorities, hopes, and dreams. Go team!
10/29: CNHM Impressions
We talked about our first impressions of the Carnegie Natural History Museum (CNHM).
A few key takeaways:
- Important topics include the Anthropocene/humans’ impact on the environment and coloniality.
- Successful experiences are participatory, allowing museumgoers to express themselves. They inspire conversation. And are developed around one key idea or narrative.
- Deep time is an important concept, but it’s difficult to teach and engage in. A good deep time example is the interactive museum timeline (get an example from a slide deck). Questions to consider: How can we visualize deep time or change over time?
- Engage audiences by considering two questions: Why do I care? What does it mean to me?
- Consider making the future natural (create a way for audiences to personally connect with the future).
- Complex design systems don’t work well (e.g., prescribed wayfinding systems are not usually successful).
- The future of museums is a more personalized, curated experience.
- Exhibit design has three main goals: affective, cognitive, experiential.
- Designing for families with children. How can we inspire cross-generational conversations?
- Interesting fact: 12 million specimens of which on 3% percent are visible.
One conversation we had was about the mineral room. We all really liked this space, but it felt decontextualized. There were hundreds of minerals on display in large cases, each accompanied by a small plaque noting its origins. The room conveyed the beauty of the minerals more than anything else. This may be intentional, but there could be an opportunity to redesign or augment this space through the lens of the Anthropocene.
This connected to an idea Becca mentioned about contextualizing artifacts. There is an effort to take artifacts that are not on display and situate them within the context of their ecosystem.
We also took the Morris Hargreaves Mcintyre culture segments test. Becca mentioned that the museum bases its user personas off data from this website. Yiwei and Michelle got Stimulation, Hannah Essence, and Amrita Affirmation.
Our next steps include:
- Revising the museum both as users and observers.
- Honing in on a concept and user.
- Starting preliminary research in prep for the research presentation and prototyping.
10/30: Tech!
We took a look at HoloLens, Oculus Go, Oculus Quest, and Reality Composer. As we consider creating projects for AR/VR, it was great exposure to emerging tech and prototyping for emerging tech.
11/3: Back to the Museum
1.Reflections:
problem space:
- Wayfinding — how to define Natural History Museum VS Art Museum?
- Outdated content/visual style/medium
- The Mineral Room: decontextualized, hard to understand( overwhelming scientific phrasing, not explained)
- The Jewelry Room: the birthstone exhibit relates To people, but are not very informative.
- The Anthropocene Living Room: experience is passive.
- The Bird Room: was the Most visually non-interactive. And the “skin display” is a bit horrifying information to kids.
Our sticky experience:
- Mystery room in the bird hall(information is well structured, not overwhelming, fun to have an interactive display)
- The night sky in the American Indian Hall (information well structured, nice place to be guided by sound )
- The temperature + lighting + sound effect in the Polar room. the Igloo in the Inuit room
- The weaving and color dye display in the Indian American Hall. contextual
User profiles:
Cross-generational Group
- Kids/Teenagers: How to make them understand bigger questions. How do we trigger their curiosity?
- Parents: How do we engage parents as educators for their kids in the experience?
- Grandparents: How do we trigger quality time for grandparents and their grandkids?
Adult group (we didn’t talked through this group)
Some Concepts:
- Mineral Hall: Recontextualization, to retell the story in the mineral hall, inputting deep time telling/ Anthropocene/ cross-generational communication into it.
- Bird hall: Immersive experience, to show how birds come from dinos(deep time telling), show human impact on birds(Anthropocene), show their extinction, and biodiversity in the ecosystem.
- System-level: Create channels of narrative stories on anthropocene/decolonization/ using forms as passports, scavenger hunt…
- Take back your virtual exhibit friend (animal/mineral/fossil), trigger going back to the museum, be educational of one specific exhibit.
In-class feedback:
- Think about the existence of the intervention: Does it happen everyday/ weekly ? or it is a temporary Event?
- Do affinity map, and generate possible user journies.
- Incorporate system thinking (eg. Bird + other species in the ecosystem)
Prior student’s work: (the ASPEN)
- Affinity map
- Primary Interview
- Key insights design principles
- Before and after the user journey
- Presented Multiple Concepts
- Usertesting
- Technical investigation
- Design touchpoints: Web UI/gift takeaway/ 3D printing objects/ Touchscreen installation
Some Inspirations:
- Leveraging the tech using WebAR + 5G ( / social component)
- Presenting using a T shape structure (systematic + one tailored immersive experience design)
- Build up the connection between design artifact
- A new way of indexing of information in AR/VR and how they project to reality
Technology Presentation (topic AR):
11/6: Affinity Mapping and Brainstorming
Takeaways:(to be edited)
- Design a system narrative focusing on the Anthropocene
- Land the narration and prototype in the Mineral room
Plans:
- Perform personal research on the following topics:
- Anthropocene (Michelle& Amrita) Participatory Museum (Hannah& Yiwei)
- We’ll revisit the CHNM the third time on Saturday and after which do the journey map.
11/9
This Saturday was a big day for us Curlers. We were ambitious and set out to do a lot. It was the first time we were back at the museum since we visited the first time with Becca. We set out to explore each exhibit in-depth to really narrow what room we wanted to focus our efforts on for the project.
Before exploring we enjoyed a nice breakfast together and shared our research on minerals and the Anthropocene
What is the anthropocene?
Geological time scale is known as the international chronostratigraphic chart: eons, eras, periods, epochs, ages. This is deep time. Geologists define chunks of time by looking at layers of rock for changes. The Anthropocene is considered an epoch.
There are 4,000–5,000 types of minerals, and 203 are man-made.
What is man-made?
Mineralogists say humans create synthetic minerals for technology, like silicon chips or foam or manufacturing like drills; because of mining or smelting or the movement of rocks — human impact on the environment — air and groundwater interact with new substances = new minerals.
Mineral characteristics
hardness, streak, color, luster, fluorescence, cleavage, specific gravity (density)
“Visitors will like it” is not a good enough reason for participatory activities.
Working with museum stakeholders
Institution: participatory experience must satisfy some aspect of the mission
Participants: participatory experience must provide a sense of personal fulfillment that matches with a visitor’s perceived self-identity.
Audience: you don’t paint murals for the sake of painting, but because others will enjoy them in the future.
How can we use this? What can visitors provide that staff can’t? How can they do some meaningful work that supports the institution overall?
Design principles:
- Participants thrive on constraints
- Entry points need to be personal, not social (engage individual users, then forge connections between them)
Museum as Visitor
After talking about our independent research we walked the Museum as visitors. Our Sticky experiences include:
“Stargazing” in the Native Americans hall
Changing the lighting and temperature in the Polar hall
The Stratavator in the Fossils room
Minerals are pretty
Museum trip as researchers
Observations
The jewelry room seems to stimulate more conversation among visitors than the mineral rooms. The birthstones display is popular.
On Saturday afternoon, there are lots of couples and adults in this hall, with fewer kids.
In the room with the mirrors, people pointed and brought friends over for minerals from Pennsylvania.
Conversations
We approached several different types of people to help us understand what people were getting out of the mineral exhibit. What we heard includes the following:
Eye candy: minerals are so beautiful! We loved looking at them.
It would be nice to know more about where they come from.
Because my companion knows so much about chemistry and mineralogy, I’m getting an ‘expert’ tour and learning so much more.
You don’t know what you don’t know
Ideation for storyboards
After exploring the museum for hours we went back to campus to discuss our insights. We then moved to a group ideation session to help pinpoint our intervention so we could better storyboard.
Findings
The problem space we identified in the mineral include the exhibit’s lack of
CONTEXT — LIFECYCLE — ENGAGEMENT — ACCESSIBILITY
To better understand where can we help and what are our objectives are we directed our efforts towards the project’s briefs Information Browsing and Experiential Learning aspects.
We landed on a Concept Framework for our storyboards and wanted to make our experience participatory, personalized, and accessible. We broke on Saturday after a long and productive day. WE continued until the late evening for a group bonding activity and enjoyed a delicious Hot Pot at Yiwei’s apartment.
11/11
Monday finally came. Peer Review was in the air, and us Curlers were ready to tackle the day. Not having spoken with any professors for a week we were eager to share our collective research and storyboards. We presented everything we had collected on our target audience, participatory research, mineral research, and mineral room observations.
We presented the framework of our experience shown below.
We also demonstrated how our experience will include AR Glass.
We also went through each of our storyboards including the storyboard for orientation, navigation, participatory experience, and systems-level data viz prototype.
We were pushed in several different directions. We saw both all professors and for our peers, we met with Ekta and Ranu and Eustina. All were helpful as we presented our findings to them as well. After class, we gather as a group to check in with how we felt with everything. We were confident that we would hammer out the details of our final concept by Wednesday.
11/13
Monday came and gone. Tuesday passed and Wednesday finally arrived. Having decided space was good for us better to come to an agreement on our final idea, we thought prototyping the navigational AR would be a great starting point for our working session for Wednesday.
It was a slow start and we ended up talking about all of the different information we wanted the museum to display. Dina and Matt came by and pushed us to think of how we can make our navigation aspect more social. From this conversation, we started generation more ideas. With Hannah’s great facilitation skills and Yiwei’s fascinating mind we developed a concept we were all proud of.
We thought in addition to the AR glasses we can have a mineral pal to help guide you through the exhibit. With recognition technology and a sticker on your palm, an ar projection of a personalized mineral will pop up in your hand. Entering each room, the mineral will change shape and create an interactive and experiential learning experience.
Because we shifted our concept so much on Wednesday we had very rough storyboards to get feedback on. Nevertheless, we were excited by our concept and pushed forward with our ideas. Daphne and Q encouraged us to make storyboards so we can better articulate and visualize our ideas so we can get better feedback. That was our next task.
11/14
We met super briefly to draw and write out our storyboards. We compiled them together and sent them along to our professors to get feedback. Visualizing them more got us more excited for prototyping!
11/16
Saturday was another big day! We started off the day by reviewing the comments from our storyboards. We then mapped out the journey map a bit further to nail down the granular.
We talked more about AR glass and technology. We talked about the placement of AR UI and how . it would be best interpreted. We talked about how we would talk about technology and radius. the glasses would cover. We talked about whether we were trying to make . navigation social or not. WE decided it was more for individual learning. WE also talked about how the rock would be personalized. WE talked about the system view and how the onboarding would look. We talked about it all. We really focused in on the detail. It was productive, yes but we soon realized we needed to go to the museum and divide and conquer some tasks to get the most out of the day and make sure we knew the mineral room’s floor plan well.
At the museum, we bought our own personalized mineral and traveled around the exhibit with our physical mineral pal. It was a fun exercise to really travel around with a mineral. we found many examples of how we can have users interact with their mineral around the exhibit. It was fun finding our mineral doppleganger as well. We found some constraints and rules and so we got even more excited by our concept. We filed the exhibit and made a detailed floor plan of the mineral room so we know spatially what each room entails and where our participatory experience will live.
We called it a day after a long day. After mapping out the rest of the semester we knew we were in over our heads and that we needed to start executing and prototyping fast!
Our to-dos for over the weekend and Monday are included below:
For Monday’s in-class working session:
- upload video and image footage from Saturday museum visit: Amrita
- complete floorplan sketch and upload: Yiwei
- gesture list and prototypes: Yiwei + Michelle
- info browsing prototypes (2D or 3D, as far as we can get): Hannah + Amrita
- user journey with tasks for each room: Michelle
Tasks for discussion on Monday:
- participatory mineral experience prototype
- branding guidelines
- systems-level user visit data visualization prototype
11/18: In-Class Working Session
In class on Monday, we met to compare notes on our progress for gestures and info browsing prototypes.
We developed a detailed journey flow that included specific tasks for each room in the mineral hall. This helped us to identify the key gestures we would need for the mineral AR in the palm. So far, we are working with opening and closing the hand to view and put away the mineral, pinch to scale or resize, and one finger rotation to rotate the mineral (all at the beginning of the exhibit). Later in the journey, we will introduce shifting the wrist side to side to see light refracting on the surface of the mineral (luster).
We looked at different options for how we could arrange information in the glasses UI. We identified some questions to answer as we continued working on UI:
- what is 2D and what is 3D?
- how is color used?
- are there cards?
- how is text positioned?
- what information is visualized at what level of proximity (the specimen, the case, the room)?
To answer the last question, we thought about giving each room a series of points along a route, like a numbered art museum audio guide. When the user gets closer to a case or specimen, that proximity allows for a deeper dive into specific information.
Our final discussion of the day was about our lenses. What should they be called, and what content should we include in each of them? How many should there be? We came up with three different options for organizing and naming the lenses.
Lastly, Yiwei gave us a demo for Cinema4D. This was so helpful to get us started in 3D modeling and to test out a workflow from C4D into AfterEffects.
11/20: Presenting to Becca
We met on Tuesday to prepare for an opportunity to show our progress and get feedback from Becca at the CMNH. This meeting focused on prototyping our gestures and discussing the participatory experience in the projector room at the end of the journey. We also thought about criteria we wanted in the minerals we choose to display in the palm in the concept video, and started to brainstorm in a matrix.
For the participatory experience, we thought about how visitors could give back their mineral and reflect on the relationship they had developed with it over the course of their journey through the mineral hall. We thought about asking them to fill in the blank on the phrase “Minerals are _______” and prototyped a few different ways it could be projected onto the wall by the entrance to the hall.
On Wednesday, we had the opportunity to show Becca our work to date. We were excited to get her feedback on the info browsing lenses, our participatory experience ideas, and any guidance she might have on branding. Her feedback included:
- When there’s a ton of info, people get overwhelmed and shut down a little. People want guidance as to what they should pay attention to in a museum environment.
- We could push on a storytelling option that is more dramatic, narrow, and story-driven. Rather than trying to be broad and give tons of options, just tell one beautiful, dramatic narrative, and make it great.
- Our idea of the floating gem to signify info browsing is good. Maybe it’s “my” personalized floating gem. Seeing other people’s gems would be great.
- Consider choosing a task for each space that’s limited in scope.
- We are on the right track with a human impact experience in the mineral hall. The museum wants to tell an Anthropocene story in the mineral hall, as it feels separate from everything else right now. Now the CMNH has the mineral collection — how do they build context and tell a natural history story?
We also got feedback from Q on our early UI. He urged us to consider the ratio of text to image, and advocated for selective use of text to avoid dizziness when reading. He gave advice that we should first decide what kinds of content we would want to display in the glasses UI, and allow that to guide whether we opt for flat cards, cards on 3D planes, a carousel, or something else.
We met on Thursday to regroup after getting feedback. We discussed what feedback to incorporate a few different options for how to do so. We agreed to think through an updated narrative for the experience and reconvene over the weekend.
11/23: Back to the Museum
On Saturday, we met for a full day of activity. First, we went through an updated, synthesized concept for the mineral experience. We then headed back to the museum to figure out exactly how each interaction would work.
With our details discussed, we headed back to studio to make a concept video storyboard to reflect the experience and present during peer review. We sketched it out on a wall first, and mapped out the related assets we would need for each scene in the video.
We met again on Sunday to push our storyboard and UI forward. Working from fresh high-resolution images from the museum thanks to Amrita and Michelle, we worked through UI considerations including cards, color, style, and icons. After brainstorming various ideas, we decided to call our system Ocular, and experimented with an initial brand.
We looked at examples of UI for AR headsets like Hololens to see what systems were already in place. We broke on Sunday evening to finish a version of UI, work on a video script, make AR prototypes, and make physical prototypes for our palm and case image markers.
11/25: Peer Reviews
On Monday, we presented our concept and video storyboard for peer review. We asked for feedback on which parts of the experience to highlight, if the interactions we described were understandable, and on our branding progress.
Feedback from our peers included:
- Questions about how AR experiences are triggered or initiated, and the role of physical markers. When you’re in front of a diorama, do your glasses automatically pick up a sticker, or do they have to trigger it by opening their palm?
- Ideas for alternative materials in the case of the palm marker. “The sticker on the hand… I know I’m going to lose it.” Do you need a sticker in your hand? What if it’s a glove? A temporary tattoo could be cool, or maybe a stamp.
- Positive responses to our set of gestures, and encouragement to incorporate more gestures throughout the experience. “Once I get into the AR glasses, I want to do more gestures. Maybe the info-browsing interface could prompt the user to do certain gestures, like panels for swiping.”
- A reminder to incorporate the moment of finding your AR palm specimen in a case somewhere in the mineral hall, which we discussed but forgot to include in the storyboard.
We also spoke with Dina and Matt. Dina’s main piece of feedback was to focus on our participatory experience finale and to give it a stronger ending to conclude the journey through the mineral hall.
We regrouped after class and had a team check-in to see how each other were doing and how we were working together. Then, we went back to the whiteboard to integrate feedback and update our experience flow, down to the micro-interactions. With a task list distributed, we were ready for Thanksgiving break.
11/26 -12/1 Thanksgiving break
During the break we really wanted to make sure our framework was tight and that the user interaction was not excessive or tedious. We wanted ocular to be influenced more heavily by recent AR/VR products. We looked closely at the UI and UX of Hololens and Oculus.
Decisions and territory covered today
We started the day with a lot of
Doing product research on what's already out there was inspiring and informative. It was during the holiday break that we learned about self haptic feedback and UI models of we went through a lot of iterations on the UI but having it informed by hololens and oculus was a great help.
Over the break, we also discussed what the participatory experience would be like. We came up with a few options. One including a data visualization of and one more mystical based on
A very fun and moving piece.
In addition to UI discussion, participatory discussion and gesture discussion, we also talked about the scripts for our video and audio guide.
We made several iterations on how the script would fully display our design interaction but we are happy with what we decided.
Sunday 12/1
We Curlers met for another blockbuster day on Sunday 12/1.
We started with a design review and discussed the brand, visual system, and UI/UX. In true Curler fashion, we were not afraid of making changes to the name of our system. While we were initially working with Ocular, we felt it was too similar to Oculus and the logo’s thin strokes were not translating well across scales. We liked how the Ocular logo made use of the sphere as the O, and so we tried to come up with ways to make the diamond work for an A name. We talked through many other ideas and did not rest until we found and user tested a name we were happy with. User testing revealed the A of our ideas was not translating as such, and so we separated the diamond from the name. Our winning name: Aurum, which means gold in Latin and conveyed some of the elegance we were going for in our concept.
Design Review: Logo/Name, visual system, UI/UX (figma link)
- Review script/plan out film story (script link 1, script link 2)
- Review footage (pics, pans, videos) / Amrita & Michelle share filming insights (footage folder)
- Talk about system level?
- Start asset creation
- Discuss workflow
- Unity, Cinema 4D
- Low fi narration
- Physical stamp, floor markers, case markers
- Put a video scene together / a few scenes together? — combo of live footage/pan + unity assets
- Discuss/create final presentation deck
- Monday deliverables:
- Script
- UI + Visual System + Logo
- Video Scene(s) — combo
Monday 12/2: In-Class Working Session
In class on Monday, we had a productive session to kick off the last week of classes for the semester.
First, we discussed AR and the toggle gesture prototype video and how to enhance signifiers to show when an object is selected. Next, we talked through our 3D workflow and what dimensions were appropriate for 3D assets. To determine this, we used physical objects to prototype and decide what scale felt right to have placed in your palm.
Next, we talked through our video script and what to show in the concept video. We went back and forth on whether we should highlight our concept as a marketing-style promotion video, or highlight the series of interactions in the user journey. We also talked about whether it made sense to have just one voice actor to be the audio guide for the glasses, or if we should have a second voice actor to provide narration for the concept as a whole.
After discussing our visual system and making notes for the next round of edits, we did some rapid concepting for the country room interaction on the one hand, and the participatory experience on the other. In 10 minutes, we came up with ideas and shared them with each other in order to nail down the final question marks in our experience flow. For the participatory experience, we also drafted a series of prompts to find just the right one to inspire visitors to reflect on their virtual mineral and express a hope or a wish about the future. We got feedback from Dina and Daphne on the idea and assigned tasks for the next meeting.
Wednesday 12/4
We headed into our last class of the semester with a long agenda. First, after looking at options for “AR glasses” based on our speculations about how they might look 10 years from now, we sent away for a few pairs and landed on the more lightweight option of the two.
With glasses in hand, we set about reviewing the list of final deliverables to make sure we weren’t missing anything for our final presentation next Tuesday. We reviewed a draft concept statement and had a discussion about tightening our focus to tell a more specific mineral story. We decided to move forward with a more specific human impact story about minerals that fund conflict in the world. This allowed us to reduce the number of options for information browsing we planned to give visitors, from three “lenses” or themes (natural science, human impact, and culture) to just two (natural science and human impact).
From there, we moved on to our visual brand and script. This allowed us to nail down our shot list and make a plan for filming, including casting, narration, equipment, props, and schedule. Next, we got feedback from Daphne and Dina about expectations for the final video and presentation. They shared that it will be important to talk about the research we did to contextualize where our concept came from. We closed out the day with a work session to move assets forward.
Thursday 12/5: Filming Part 1
We returned to the museum to start filming our final video. We covered a lot of ground within the mineral hall. Specifically, we captured first-person and third-person shots of our user Amy (Yiwei!) as she moved through the exhibit.
We returned to Studio to process our video and make a plan of action for Friday (filling in film gaps and moving forward with production).
One interesting learning — iPhones have better video quality than DSLR.
Friday 12/6: Filming Part 2, Recording Audio, and 3D Assets
Friday was production, production, production.
In the morning, we returned to the museum to film onboarding and fill in any gaps. We arrived with all the production tech possible — a DSLR, a GoPro, a stabilizer, 3 iPhones, and a magic arm. It was a full on production!
BTS shots:
After filming, we starting working on our final video assets. We built our UI and UI interactions in Cinema4D. Our plan was to then move them into After Effects and place them in our scenes/match them to live-action.
We also recorded and cut our final audio.
Saturday 12/7: 3D Assets! UI! Presentation!
On Saturday, we continued moving forward with Cinema 4D and After Effects. We also solidified our onboarding and exhbit UI.
We also started working on our 3D hand assets in Cinema 4D. Because we were focusing on conflict minerals, we chose to develop/animate Gold, Pyrite, and Diamond models.
Animating the video was an arduous, but fun process. We had to create/animate assets in Cinema 4D, then render or move these models into After Effects. In After Effects, we had to track the motion of our shots/map 3D assets to motion and cut out hands/people. It was a meticulous process, but overall worth it.
Participatory Experience C4D + UI modeling:
One video scene in the works was our Locations Room scene:
(add Gif)
Sunday 12/8: More 3D assets! More UI! More presentation!
We worked on solidifying our visual brand and logo.
We also worked on refining and simplifying the UI for our participatory experience.
Video production was still ongoing. We continued animating in C4D and After Effects. A few scenes in the works this day:
Monday 12/9: Putting it all together
On Monday, we stitching our assets, video, and presentation together.
Gesture animations (gestures inspired by Hololens/Oculus research):
Tuesday 12/10: Final Presentations and Reflections
Final visual brand guidelines:
Major learnings
Patterns, affordances, and signifiers for emerging AR and VR technology (haptic feedback, proximity, gesture control). We drew a lot of inspiration from Oculus’ emerging VR hand-tracking guidelines and Hololens gesture controls.
Designing interactions for and in 3D. For the majority of us, it was our first time designing for 3D virtual objects. We had to consider 3D digital objects and how they would interact in a 3D changing environment.
A LOT about prototyping for AR. We looked into ~20 different tools.
- AR prototyping: Aero, Reality Composer, Unity, Dimension
- 3D modeling: Cinema 4D, Sketchup, Sketchfab
- Hardware: Leap Motion, Oculus, Hololens
- UI/IXD: Figma, XD, Illustrator
- Video Production: Photoshop, After Effects, Premier Pro, Lightroom, Audition, Go Pro
Tools we ended up using for our final stuff included Cinema 4D, Sketchfab, Figma, XD, Illustrator, Photoshop, After Effects, Premier Pro, Lightroom, Audition, GoPro.
And that’s a wrap!
