Intersect — Case Study Report
Learner Experience Design
Carnegie Mellon University
Stacie Rohrbach
Team | Gautam Yadav, Anna Boyle, Ting-Yun Ho, and Tomohiro Nagashima
Application Intersect
Connecting current students and alumni to uncover intersecting interests and guide personal and professional growth.
Intersect is an application designed to assist sophomore students (audience) in becoming better at evaluating and exploring their interests through engaging with and reflecting on alumni stories. Through engaging with Intersect, students will be able to expand their knowledge and practice lifelong learning skills (design approach).
The Design Team
Gautam Yadav
Gautam is a learning engineer who applies evidence-based research to create effective educational technologies and products. He is in the Masters of Educational Technology and Applied Learning Science (METALS) program at the Human-Computer Interaction Institute at Carnegie Mellon University. He is passionate about exploring and creating the best experience for the learner and designing meaningful interactions between learners, technology, and products. He conducted relevant learner research to support each design decision and content development expertise in this project.
Anna Boyle
Anna is a first-year Master of Design student in the School of Design at Carnegie Mellon University. Before starting Graduate School Anna worked as a Creative Manager and Lead Graphic Designer for a non-profit and university within higher education. As a designer with experience working in schools, she is passionate about accessibility and access to education through design practice and enjoys thinking about how design decisions and human-centered approaches have the ability to enhance everyday lives.
Anna’s role in this project included concept development, project management, design research, visual design, and UI/UX Prototyping.
Ting-Yun Ho (Carol)
Carol is a cross-disciplinary designer with a background in information systems. With her 4-year experience designing enterprise system as an application consultant, she enjoys making sense of the world with user flows, business processes, and technology. However, she found her world view limited her to leverage technology to aid humanity, therefore she decided to pursue her Master of Arts in Design Student at Carnegie Mellon University. By practicing the learning framework and iterating, she found that everything in life is connected to the experience of learning, and learned to embrace the uncertainty throughout the process of the learning experience, and the process of designing the learning experience itself.
Carol’s role in this project included concept development, design research, and UI/UX Prototyping.
Tomohiro Nagashima
Tomo is a PhD student in Human-Computer Interaction and Learning Sciences at Carnegie Mellon University. His research explores how to use interactive technologies to help people learn. He has worked on a variety of topics in multiple fields, including mathematics education, instructional design, learning analytics, online learning, open education, and teacher learning. Tomo’s role in the project included concept development and refinement, research and analysis, and ideation and prototype design.
The Problem
What learning problem(s) did you identify (learning gaps)?
One of the biggest challenges in today’s educational systems is to effectively help students become self-directed learners so that they can engage in lifelong learning throughout their lives in the rapidly-changing world. The skills taught in the formal education system do not necessarily prepare students to become good lifelong learners. Therefore, despite its importance, students do not have an opportunity to practice self-directed learning skills.
Who is affected by them? (stakeholders) In what way? When?
University undergraduate students are specifically affected by the problem. Although undergraduate years mark the point in life when people start to realistically think about their own short-term and their long-term goals, there is a lack of opportunities to help students prepare for their future.
In what context(s) are they affected?
Our initial survey study with undergraduate students revealed that students want to explore other interests. Despite the importance of the skill of exploring and evaluating interests, it is hard for students to find a service designed to help the skill both within and outside the university. Based on our user interviews with undergraduates and masters students, we learned that many students figure out their personal interests later in life. This insight revealed the design opportunity to aid students in self-reflection and evaluation in an effort to understand their interests earlier in their undergraduate program.
Your Audience + Context
Who is your audience? Why?
Our target audience is sophomore students who have a basic understanding of their current major but want to start exploring other possible directions through taking elective courses, and participating in internships, or other events. We believe that introducing the Intersect app to these students in their sophomore year could help them better plan for their future with a better understanding of themselves and their own interests.
We are focused on sophomore students because we believe it would be too early for a freshman to think deeply about their career. Through our interviews with experts in Academic Coaching and Career Services, we learned that many freshmen are already adjusting to various changes in their daily life and have limited choices over what courses they can take their first year, therefore, we didn’t want to add something “additional” to their load. Therefore we concluded that sophomore year students can start thinking about what their interests are so that by the end of the third semester, they are able to choose an internship and/or courses that align more with their interests. That being said, we do envision that this app could also be useful to people outside of the student demographic and aid in individuals discovering their interests and finding resources available to them.
In what context(s) does your work reside? Why?
For the context, sophomore students will be introduced to Intersect in a workshop by career counselors in their department. Students will then be given a brief introduction and onboarding lesson to how Intersect works and how it could help them develop and grow within and outside of school.
From the insights, we developed from the first survey we also decided to leveraged alumni stories as a way of helping students realize their own interests, as this feature was something students wanted as a career tool. This feature provides students the opportunity to reflect on how alumni stories can be connected to their own interests, as well as their own academic journey.
Significance of The Topic
Why/How is the topic you investigated important?
Self-evaluation/exploration skills is one of the core components of lifelong learning skills, it can be learned in various ways but currently, there is no service effectively assisting students in developing the skill. By developing the skill of self-evaluation/exploration, students can find interests and needs that are important to themselves, some of which they would not realize otherwise. Exploring interests plays an important role in the undergraduate years. It is a fundamental component of lifelong learning skills.
What may happen if it isn’t addressed?
Although some students may develop the self-evaluation/exploration skill without deliberate practices, our initial survey showed that out target students are eager to explore interests but they do not know how to do it. They also have a fixed mindset toward their knowledge and their career. If our topic is not addressed, these students would not have an opportunity to explore interests.
What were your goals for the project?
The goal of our project is 1) to help students develop the skill of self-evaluation/exploration that would transfer to other contexts (e.g., not just “interests” but also “needs”) while also 2) to give students a concrete outcome, which we give in the form of the interest map.
Design Process
What steps did you take? Why? What did you learn/achieve at each milestone?
We have applied every single learning theory/model that we learned to our problem. Each model made us realize important aspects of the problem which we would not have otherwise noticed. In most cases, these models helped us generate mode ideas or widen our perspective (DISCOVER and DEVELOP phases in the figure below).
What theories and approaches did you leverage? Why? What did they afford?
Structured Flow of Goals: our overarching mission is to assist students in gaining self-directed learning skills, which are broad and not something that can be learned in a short period of time. We want to use this model to show that we are focusing on one of the sub-skills (i.e., self-evaluation).
The Cycle of Practice and Feedback: it would be important that learners get timely, targeted feedback so that they can realize what they are learning and why they are learning it. We make sure to provide step-by-step activities and opportunities for continuous engagement (i.e., students can use the service again and again)
Ambrose’s Motivation Model: one big challenge is how to make our activity fun and engaging. Undergraduate students do not have time. The key to having our learners actively participate in our learning experience would be to design something that is fun and engaging. Moreover, we also wanted to make sure to keep learners engaged throughout the entire learning process but not just at the beginning. We leverage Ambrose’s model and its recommended strategies.
What challenges did you encounter throughout the process?
The hardest part was when we were trying to develop our prototype. As mentioned earlier, the models helped us generate a variety of ideas, but it was us who had to decide which one to move forward with (which was hard!). We had many discussions for each of such decision-making events.
It was also challenging to make a prototype — since the topic of lifelong learning is somewhat an abstract concept, developing a concrete experience centered around the abstract concept was really challenging.
Design Approach
What conceptual model have you created for your project (diagram of the framework)?
We combined the Cycle of Practice and Feedback model and Ambrose’s motivation model in an integrated way. The emphasis is put on the importance of continuous engagement.
What scenarios are you using to describe the structure of your proposal?
When they are introduced to the tool, students will first answer self-assessment questions so that they can get their first interest map. After that, students either will be recommended or will explore alumni stories that they can read. Through working on the prepared prompts when reading alumni stories, the interest map “grows” and students can find new interests.
After graduating, the student becomes alumni and can also come back to re-assess themselves and connect to the alumni with a similar career level, or the career level they want to advance on.
What are the facets/features of your prototype?
The main features of our prototype are: Self-assessment, Interest map, and Alumni stories. Engaging with self-assessment (first time only) and alumni stories (multiple times) helps the interest map “grow”. The flow of the interaction is intentionally kept simple and minimal so that busy undergraduate students can keep engaged.
Self Assessment
Interest Map and Student Interests
Evaluation Methods
What evaluative methods did you employ? Why? What was the structure of your evaluation?
At the final presentation sessions, we asked our participants to use the interactive version of our prototype and to answer several usability-related questions in the form of a survey. Because the sessions involved multiple participants and were conducted virtually, we initially thought about conducting think-aloud sessions or interviews. However, they proved to not be the most effective ways to get participants’ insights. To quickly get participants’ feelings, we decided to go with a survey. We asked the following 12 Likert-scale questions (1: strongly disagree — 5: strongly agree) and one open-ended question:
I enjoyed using the prototype.
The information, flow, and structure of the prototype were clear.
“I wish I had used this tool when I was an undergraduate student.”
The prototype looks engaging.
The tool would be useful to explore interests.
I would be able to keep using the tool even if I were busy.
I would reach out to students with similar interests as mine
I would recommend this tool to my friends.
If I were a user, I would be able to learn something by using the tool.
The tool gives an opportunity to engage in alumni stories.
I liked that the interest map “grew”.
It would also be useful for people other than undergraduate students
Study Outcomes
Research Phase 1
What information did you collect?
We collected information on student current interests and needs and how they approach developing their future goal as a student and tested our concepts with a variety of students ranging from undergraduate to Ph.D.
What did you glean from the analysis of your findings?
Based on our survey, we found that most students have understandings of what kind of career will their major leads to. However, the students still want to explore different possibilities within the campus, but lack the knowledge and skills on where to start. And this impacts their ability to pivot their career in the long run. We believe that by helping the students to develop self-directed skills can help them explore with the notion of knowing what they want, and can also cultivate the life-long learning skill in the long run.
What have you identified as logical next steps in the study?
Features we would like to develop in the future include making an online alumni community so they can connect with alumni having similar interests or reach out to alumni near their area. They can also access resources through their interest maps locally in the area they are living in and online to further develop their interests. This will help build a stronger alumni community and will fulfill our mission of lifelong learning even for alumni who are working full-time jobs.
What do you see as the implications of your work? How might it be applied broadly?
The skill of finding meaningful similarities and differences would help students evaluate not only their interests but also their needs, goals, and other important components of lifelong learning.
Research Phase 2
What information did you collect?
In the final survey, we collected answers to the 12 Likert-scale questions (1: “strongly disagree”, 5: “strongly agree”) and one open-ended question. Six participants completed the survey.
What did you glean from the analysis of your findings?
The overall average score was 4.17 (out of 5). Among the 12 questions, 11 of them scored over 4.00. The following item received the highest score (4.83):
I wish I had used this tool when I was an undergraduate student.
However, there was one item for which our participants expressed mixed feelings: I would be able to keep using the tool even if I were busy (average score: 2.67).
What have you identified as logical next steps in the study?
It is an interesting trend that we saw a low score only for the question above but not for others. Although it is not clear from the survey why our prototype made some of our participants feel that they would not be able to keep using it, here is our attempt to interpret the data:
Long-term vs. short-term learning/tasks: Intersect is designed for long-term use but not for short-term use. The tool and the experience are structured in a way that assumes that users would continuously use it. However, it might be that the design is not engaging/motivating enough to attract undergraduate users, who are always busy with short-term tasks. They work towards many deadlines, which typically are for short-term tasks, such as homework and papers. They rarely get an opportunity to engage in long-term thinking (about their future career…etc).
Although we wanted to address this problem and we did make the task minimal, which would require only about 10 minutes to complete each task, it might still not be motivating enough for continuous use. For example, the visual feedback we give in the form of the interest map is designed so that it keeps growing each time users complete the task, but the degree of “growth” might be too little to keep users’ motivation. It might also be that, as one participant mentioned in the survey response, the unclearness around “what to do next” made our participants feel that they wouldn’t use it again. We intentionally did not design a “smart system” that sends recommendations in order to encourage learners’ own interest exploration, but it may be worth trying to improve the system so that it guides users a bit more than it currently does.
What do you see as the implications of your work? How might it be applied broadly?
Self-reported feelings are not rigorous measures for learning, but our survey still gives us important insights into the perceived quality of the system. It is important to note that participants gave high scores on many of our survey items, which were generated based on our goals and hypotheses. We were pleased with the positive feedback that we received from students.
