04/03 Envisioning “Shared Reality”

Today we sketched the overall language learner’s journey and discussed what kind of prototypes we need for the next evaluative research workshops. Over the weekend we all agreed with designing the solution under the overarching theme of Shared Reality — experiencing the daily life of local people with multi-sensory stimuli—that goes beyond classic learning / translation.

Service Landscape

We also talked about designing a platform that connects learners and local people. Through the bodystorming, we learned that it is more valuable to enable learners to start building relationships with their new neighbors before landing into the new environment. We got the idea from existing pen pal social network platforms and sketched the landscape of our service.

The platform supports two types of mode; social networking mode and individual practice mode. Users might meet their local partner and ask him/her to take a tour with them or introduce local people. Or, users might get into the traditional virtual classroom and practice target language alone. For the detailed user scenario, we will mainly focus on social networking mode.

Before jumping into paper prototyping, we pulled up some key questions about this social networking mode as below:

• How does the system match learners and local partners?
• What would be the best way to introduce each other and build trust with each other?
• What is the most natural way of meeting each other in immersive space?
• How do local people interact with holographic image of learners?
• How does shared reality help language acquisition?

We were all on the same page that even if learners and local partners meet through the platform, the way they introduce each other and have a meeting should be as same as the real social gathering experience. When we hang out with our friends in the real world, we set time, place and the purpose of meeting. Even when learners and local people meet through the mixed reality experience, the solution should be able to provide real ‘hangout’ experience. For example, the learner might ask his/her partner to have a walking tour with him/her in Paris District 9 at 3pm CET.

Paper Prototyping

It was super difficult to explain the possible scenarios and technologies by word, so we quickly made simple paper prototypes and simulate the important moments of the social networking mode. The most challenging parts of envision were:

• How might the local partner introduce the 3rd person to the learner and how might these local people see the learner at distance?
• How might the local partner share the field of view with the learner?
• What kind of MR devices are needed for our platform?

Because of its look, it is hard to imagine that all learners/local partners/3rd people wear Hololens and interact with each other. In addition, Hololens has a huge limitation that it can’t scan the image of wearers themselves. Considering these questions, we brainstormed once about other possible MR devices that looks less socially awkward but effectively share the reality.

Ashlesha brought up the idea of a portable camera/beam projector that scans the surrounding local environment and also project the image of distant learners. This might solve the problem of scanning each participant and projecting holographic person in the local space at the same time.

Reference for Immersive Experience

As a reference, Julia shared the video of Jack-in-Head, a mixed reality device that shares the immersive experience between two people. This is exactly what we envisioned about the shared reality. While the local partner walks around the learner’s target neighborhood with the MR device, the learner can see , hear and feel the space from his/her partner’s perspective.

Julia mentioned that the ghost user might experience uncanny ‘decoupling’ between his/her body and the field of view because the ghost user doesn’t walk around but still be able to navigate the space. We are not really sure if the learners will like this decoupling experience. That’s the one thing we need to figure out over the next workshop.

However, she said that it was still interesting to feel how the body user sees the world. If the local partner is able to share not only his/her field of view but also the breeze, smell, sound of the surrounding space and the language he/she speaks, the learner must be able to experience more real-like local environment.