04/12 Stage 4: Evaluative Research
Presentation & Feedback
After developing our concept and conducting several evaluative design workshops, we synthesized our key findings and talked through what we’ve found so far and showed our concept in the form of storyboard and stills of of the concept to Irina and Kevin from Microsoft.
Recap of Target Context, Design Implications, Key Concepts
Last time we talked about our target users as language learners with intermediate to advanced proficiency who plan to go abroad for more than a year.
Through the whole design research process we kept in mind the 6 design implications — learn language and culture together, encourage to build meaningful relationships, build confidence, help express learners thought, encourage conscious critical thinking and finally embody language and culture.
We also introduced the direction which was learning through social interactions so users can internalize grammar and expression instead of memorizing them. Finally, we proposed that our solution would provide feedback on everyday conversation to our users so they can lower the emotional barrier and build confidence.
Next we talked about our evaluative phase and what we were up to in the past couple of weeks. We evaluated our concepts with various research methods including speed dating, body-storming, paper prototyping, user testing.
Speeddating
Since each key concept has different directions and targets different context, the first thing we had to do was narrowing down to a one single concept. In the meantime, we also brainstormed how might mixed reality can provide more value to language learners and created a total of 12 storyboards. We asked 6 potential users about which part resonates you the most and which part do we need to reconsider.
For passive exposure, we came up with the idea of converting surrounding signs and texts into a target language, giving learners more chance to get used to the new language in their current space.
For immersive virtual environment, we tried to bridge between learners and native speakers living in abroad and even their neighborhood too. We also came up with a couple of gamification ideas including brainstorming and hologram props.
For learning in contexts, we focused mainly on visual, textual prompts that can help learners speak the language with more confidence. Also, we also thought about “the etiquette guide” that helps embody the proper social manners.
We found that some critical questions were emerged repeatedly from our participants:
The first one is, they don’t think simple passive exposure would help that much. They think it’d be easier to use google translator or other smartphone apps. Rather than that, they wanted to maintain their interest in language learning by exposing themselves into more immersive environment where forcing them to speak the language such as conversation club, travel and study abroad.
When it comes to immersive language environment, they were curious about the sustainability of the solution in terms of business perspective. For example, How to recruit the native speaker? Do learners have to pay for hiring a tutor? And who is generating the content?
For this same reason, we should be able to provide strong motivation for both learners and tutors to keep participating in the immersive learning environment.
Despite of these critical questions, our interviewees thought that contextual conversation practice with native speakers is the most compelling idea among all scenarios.
So In the end, we finally selected an immersive social learning platform as the single main concept.
However, we also felt that the other 2 key concepts can be the features of the platform, so we bodystormed other scenarios related to passive exposures and expression aids to validate the idea.
Body-storming
We turned to bodystorming to flesh out this immersive social learning environment and explore some questions we’d need to answer moving forward.
To find pain points and opportunities, we bodystormed scenarios about the idea of language learners hosted by native speakers in their own realities. Embodying the experience helped us take a more critical look at our ideas as well as get an idea of what mixed reality could–and should–enable.
We came across questions like how do we move within realities and from reality to reality? How do different users interact with each other? How do they interact with third parties?
Within these scenarios, we experimented with adding a few features from our initial three concept areas as well, including captions, conversation prompts, vocabulary help etc..
The key takeaways were:
Put relationships first. From our previous research, we heard that relationships play a vital role in motivation and that interpersonal connections are where people learn best. We experienced that for ourselves through bodystorming. How could mixed reality help language learners become connected to a new community?
Second, visual cues like prompts and captions could be a barrier to building relationships. Its learning benefits weren’t worth that cost. Therefore, we want to create two separate modes: an immersive social interactive mode and an individual practice mode, to make use of helpful features while eliminating the barrier they might create between people.
Considering the social contexts that a language learner and a language partner might be embedded in, being able to interact with others in either space is a big value add for mixed reality.
Paper Prototypes
Bringing these takeaways forward, we moved to paper prototyping to further refine our concept of connecting people for learning languages in context.
First, we created paper figures and props to use in our discussions. Enacting scenarios using paper figures rather than our own bodies helped us to clarify whose reality we were talking about, who was a hologram, and who was “real”.
We also did some paper prototyping of the shared view and what features might be helpful where. We worked through this exercise ourselves and also brought these paper prototypes forward to our workshops, where we asked participants to play with them as well.
Evaluative Workshop
To test our concepts:
We used paper prototypes and sketches to walk our participants through the experience of using the system
We asked for feedback, noted their instant reactions to each scene, any hesitations they had about it, and probed on social boundaries
What we learnt was different learners have very different needs in terms of what helps them learn best. (Lydia one of our participants learnt through grammar tables, whereas Jon liked to practice in a real social context)
People have set expectations for giving and receiving feedback on their language use
People had varying levels of trust for their hosts/native buddy. They want to know what their hosts are getting out of it and if there’s any accountability–are they who they say they are?
People loved the social aspect and the ability to experience the environment — before arriving, people want to know what the neighborhood is like: walkable? Friendly? Amenities?
Something we observed was people want to be as engaged in the moment as possible with as little intervention from the system. Interventions and helping elements like grammar tables, caption etc.. might be helpful in an individual practice mode, but this would interrupt a conversation which isn’t good for building relationships.
Final Concept
We used what we gathered in our research to shape our vision of the future of language learning and came up with our final concept.
Scenario
Babl is a platform which allows learners to experience the native language and culture by connecting them to local/native people. The learner is exposed to accents, gestures, tones of various native speakers. This MR solution helps learners to adapt and absorb a new culture and language.
John gets a chance to work at an office in Berlin for a couple of years. He’s never been to Germany before, so he’s taking a language course at school to prepare. So far, he has a basic knowledge of German but he isn’t confident in conversation. Besides learning the language, he also needs to find a place to live and communicate with his future co-workers.
A friend from John’s German class tells him about Babl: an immersive language learning platform where you can meet local people and experience your new home. He signs up for the platform and tries to find a partner in Berlin who can help him. He provides some information about himself and answers a few questions about what he’s looking for in a language partner.
John is connected to Markus, a university student in Berlin. After chatting a little, they set up a time to meet. Markus knows downtown Berlin really well, so John wants to hang out with him and explore the area to feel what life in Berlin is like.
When it’s time for them to meet, John and Markus activate their MR devices. When Markus switches to “share mode”, John’s hologram appears in Berlin and he can experience Berlin from Markus’s perspective. As Markus walks down the street, they chat in German about day-to-day life in Berlin.
One stop on their walk is a famous cafe in Berlin. John wonders how to order coffee in German, so he asks Markus to grab a coffee. When he speaks with the barista, John “bookmarks” the scene and captures the moment. Later on, he can replay the bookmarked scene in his own individual practice time.
Markus spots his friend Melissa at the cafe. He goes over to say hi to her and he switches to “Meeting Mode. In this mode, John’s hologram appears in the cafe and he introduces himself to Melissa. After the three of them talk for a bit, they make plans to hang out all together once John arrives in Berlin.
After the meeting with Markus is over, John goes back to his individual practice mode. Here, he can access the scenes he has bookmarked and interact with them to help him solidify what he practiced with Markus. He can slow the scenes down, see captions, access information about grammar and vocabulary, and even interact with a chatbot to continue the conversation.
Mixed Reality = Shared Reality
We defined mixed reality as shared reality. This was something brought up by one of our participants during our research and it really inspired us.
We are moving towards our next steps and have started exploring the key moments and values in our MR solution. In mocking up these moments we are facing challenging questions.
- How to interact with the platform to use these different modes? How can we explore gestures and what do the interactions look like?
2. We are still grappling with how the hologram looks like, what social dynamics might come into play, how it’s controlled, and how it interacts?
3. How to best help the learner when they feel stuck? What is the optimum level of support that is required to help learners critically think and express themselves.
Next Steps
Feedback:
We got great feedback from the Microsoft team. They asked us detailed question and pushed us to think granular and flesh out a strong concept.
Some of the points were:
- When Jon and Marqus meet using this MR device, which one of them is outside or are they both outside?
2. Which of them is using VR and MR? Should either of them have VR and why?
3. They advised us to focus on the details of the interaction.
4. What is the motivation for Marqus to use this system? Show every person’s motivation, learner and native partner.
5. Should holograms be used at all? Does it fit into the scenario and is it necessary for the concept?
6. How does the hologram concept work? What can the hologram do? Can it manipulate the environment?
7. Could there be other mechanisms or gestures that could help to from the system?
8. Another suggestion was to have a time bank model. Where for every one hour of being a tour guide the native partner who is also a learner will get one hour of tour guide session from someone else. Build give and take to keep motivation.
