Week 14: Presenting Evaluative Research
Carnegie Mellon University
Graduate Interaction Design Studio 2
Spring 2018
Evaluative Research Presentation
The efforts of this week were largely focused around presenting our work from the evaluative research phase and then charting our path forward based on the feedback we received. In order to effectively convey the narrative arc of our design and research efforts over the course of the past few weeks we chose to divide the presentation into five key sections which can be seen outlined below.
Early Ideation
After a quick recap of our previous work, we began the evaluative research presentation with a series of slides that outlined the 15 concepts that resulted from our initial concept brainstorming session. Following a dry run through our presentation we thought it would be best to not walk the class through each scheme, but rather highlight the important themes that arose from this exercise. As a result of this feedback session, we made an effort in slowly build the slide such that the audience could see the themes that were associated with each concept.
Methods
Following the discussion about the early ideation phase we introduced the methods we employed to narrow down into the final series of three schemes. We began with the results from the speed dating exercise, where we walked our potential users from business, design and stem backgrounds through each concept. During these one on one sessions, we asked them to tell us wether or not they would be interested in using what was described in each scenario, and why. You can see the resulting yes, no and maybe responses represented here.
We then moved on to discuss the mapping exercises where we sorted each theme based on several criteria of what we saw as importance. An example being, ranking each scheme based on how well a concept leveraged AI based on our research findings. We then introduced a Y-axis representing feasibility. Which allowed us to see which schemes emerged as both an effective use of ai and feasible. Initially we set off to quickly flip through the results of each of these mapping studies, but following our dry-run we determined it would be most effective if we showed one in detail and the remaining three on a single slide. This allowed us to move through this content a bit quicker without sacrificing clarity.
The final component of the early method section was to highlight out career timeline, we outlined when the learning was happening within each concept. Introducing the class to some of the questions that arose during this exercise, such as: Is the concept a lifelong learning tool, something only appropriate for the workplace, is it geared to students or would it be appropriate for someone who is making the transition into the workforce?
Before presenting the final concept, we introduced the shortlisted three schemes that arose after employing these evaluative methods. Although we didn’t do into great deal of detail about each gave a quick description and discussed how we ultimately used a combination of speed dating, and interviews with the subject matter experts we spoke to during the exploratory research phase to build out final concept.
Final Concept Storyboard
In order to present our final concept we decided it would be most effective to tell the story using a combination of photos and mockups. This allowed us to introduce our personae but also walk the class through the primary touch-points of our mentorship platform. We began the storyboard by introducing our primary user, Taylor. Taylor is an engineering student about to graduate and start a full time job but is looking for some guidance as she makes her transition.
She signs up for our platform and inputs her career ambitions, learning goals and interests. With her profile built, Taylor can now find a mentor who can help her along the way.
Taylor is matched with a mentor, Brooklyn, based on shared interests, expertise and learning goals. The system provides information about each other to both of them so that they have an understanding of why they were matched. This also helps in giving context for starting the conversation.
Taylor and Brooklyn engage in short conversations through messages. The system helps supports these interactions with helpful suggestions, provide additional resources and plan longer face-face interactions.
Taylor and Brooklyn have longer discussion on her goals and together they set actionable milestones and activities that will help Taylor achieve those goals.
The virtual agent prepares notes from the conversation providing relevant links and resources.
The system prompts Taylor and Brooklyn to reflect on their progress regularly. If there are big differences in their reflections AI can flag so they can adjust accordingly
Taylor connects another mentee Gabriel, with a similar background who’s facing similar challenges and together they can practice for specific goals or benefit from knowing about each other’s experience
Leveraging AI
After introducing our concept storyboard we took a moment to situate our project within the theme of the studio by discussing how we intend to leverage AI to promote learning. Through a systems diagram we identified the opportunities for gathering data, and then how this data could be used enhance the core interactions of the platform.
Although this speaks to our overall strategy, we though it would be best to explicitly highlight the four key moments within our proposal where we intend to leverage AI. Following our conversations with subject matter experts during this phase, all have confirmed that these are feasible and effective uses of AI.
Testing
Following the overview of our concept and discussion surrounding the use of AI, we wanted to show the class the methods we employed to validate our design decisions so far. We achieved this by showing the two methods an animation of the resulting test as well as the insights that arose from these exercises.
Next Steps
To conclude the presentation we introduced out next steps as well as the open ended questions that we are looking to answer over the course of the remaining weeks.
Feedback
Following the presentation we took a moment to unpack the feedback we received from our peers and professors. The resulting comments largely fell into two overarching themes.
01. What are the incentives for mentors to join / engage with the platform?
02. How can you clarify the mentor selection process and avoid bias?
Over the next three weeks we hope to address these questions and bring further clarity to our platform.
Planning for the Coming Weeks
Researching Mentorship
The first and perhaps most important next step is to speak with existing mentors. Although we have engaged with many potential stakeholder throughout this process, the mentorship model emerged late in the course of the studio. As such, we are looking to have conversations with mentors early next week to get a better sense of what their incentive is to mentor, as well as better understand their needs.
Research Methods Presentation
In addition to the efforts above, we are preparing for our research methods presentation on this coming Tuesday (04/23). Where we intend to provide an overview on the methods we used over the course of the semester, and also reflect upon their successes and failures. We have developed a presentation template which serves as both documentation of the method but also a look into what worked and what we would change if we were to try it again.
