Defining Value for Our Design Solutions: Benchmarking and Soliciting Feedback through Research
Executive Summary
In this sprint, our team focused on evaluating whether mentoring programs and TMP would value our ideas. During this evaluation, we encountered two primary challenges.
Firstly, testing 7 different ideas through both interviews and implementation trials on TMP’s website was impractical due to the costs of implementation and the complexities of recruiting participants. To address this challenge, we revised our approach by narrowing seven ideas into five, and then integrating them all into a single prototype. This allowed us to efficiently gather qualitative and quantitative feedback during the interviews, striking a balance between cost and efficiency.
Secondly, we faced the challenge of balancing practicality with experimentation. While some of our ideas were highly innovative, the potential costs of implementation were significant. Therefore, our team sought a better balance between innovative design and practical implementation constraints.
This sprint has deepened our understanding of how users value interactions with TMP resources. The insights gathered in this sprint are crucial for refining our design ideas and ensuring they are aligned not only with user needs but also with practical implementation.
What do TMP’s clients value about current ways of accessing TMP’s online resources?
To better understand how TMP clients are currently interacting with online resources, we conducted a website usability testing. We asked a mix of TMP-affiliated mentoring programs and external mentors and mentees to conduct tasks in a Think Aloud study with TMP’s website. Some of the tasks included finding specific resources on the website, initiating service customization, and hypothetical scenarios to determine where users expected to find certain information on the website and what steps they take to reach their desired destination.
We analyzed how long it took to find or use information and compared the steps people took versus what we expected them to do. This will help us understand how online resources support the needs and expectations of a diverse range of mentoring stakeholders. These will serve as quantitative and qualitative benchmarks and guidelines for the success of our final design solution. These benchmarks will allow us to determine how successful we are with creating easy-to-access and timely resources where mentoring stakeholders find and use what they need quickly.
We employed methods, such as thematic analysis or affinity mapping, to identify patterns in the data. From the themes that we defined from the data, we crafted models of factors that influenced people’s decisions when choosing a relevant resource, barriers to finding these resources, how clients view their relationships with TMP, and whether needs and expectations are fulfilled by online resources.
How does TMP value our design ideas?
Inspired by the principles of abductive reasoning, our team realized the importance of aligning the core values of our project with those of our clients. Therefore, we held another collaborative session with TMP to discuss the values that our design should deliver. This session deepened our understanding of TMP’s priorities and established a benchmark for evaluating and refining our prototypes.
Science Fair: Gather TMP’s Insights on Low-Fi Prototypes
In our collaborative session, the first activity was a “science fair,” where we presented our low-fidelity prototypes for usability testing to TMP staff members. This activity allowed us to collect feedback directly from clients, gaining insights about which functions they find valuable and assessing the feasibility associated with implementing these functions. The feedback also helped us understand TMP’s business strategy concerns, enhancing our perspective on what is feasible and desired by TMP.
Prioritization Activity: Align with TMP’s Core Values
The second activity of the collaborative focused on prioritization. We extracted key concepts from our current lo-fi prototype and presented these to our clients. They categorized these concepts into ‘must have’, ‘should have’, ‘could have’, and ‘will not have’. This activity was crucial for us to identify TMP’s core values, such as supporting mentoring programs in choosing resources, reducing barriers to applying knowledge, and providing 24/7 assistance with TMP’s services. Understanding these priorities allows our team to refine our design focus to align with TMP’s most critical needs.
Will clients value our new ideas?
Prior to the collaborative session, we integrated our five ideas into a single web platform separate from TMP’s website to test our ideas. In this sprint, we are using a web platform format as a vehicle for testing the value and desirability of our ideas prior to settling on a format. We are also using our prototype as a way to elicit feedback about the channel, modality, or format of our ideas to transform and further narrow down our ideas in the next stage of parallel prototyping.
The five concepts we are seeking to test within an integrated prototype this sprint are:
- Chatbot
- Institutional Profile/Account
- Online Resource Customization
- Reviews/Comments on the Website
- Mentoring Program Hub + Co-working Session
We created a mid-fidelity and interactive prototype through Figma and are currently testing our designs with mentoring programs. We are excited to learn and hear client perspectives on value, format, clarity, organization, and preferences around our concepts!
How can we revise our design ideas?
Based on the feedback from our current website usability tests and TMP’s staff’s feedback, our team started a SCAMPER brainstorming session to refine our design ideas. Through this activity, we combined and transformed existing ideas to explore new ideas that address both TMP and mentoring programs’ concerns and needs. This activity encouraged us to think innovatively during the converging phase, ensuring that our design is both creative and grounded in the users’ needs.
Upcoming Work/Milestones
- This week, our team will be presenting our work from the spring and the current prototypes to TMP’s board. We are looking forward to generating interest in this work, recommendations for partners in this work, and soliciting the expertise and feedback from board members for the project.
- We will also be doing another round of Pecha Kucha, a short but exciting presentation style, for HCI practitioners to receive feedback on the progress of our project.
- To evaluate the financial and technical feasibility of our concepts, our team will be reaching out and having conversations with technical experts to provide TMP with cost estimates for the end-of-project, long-term implementation plan.
- Internally, our team will continue activities such as SCAMPER to explore our ideas in different combinations and modalities for future iterations of the prototypes.
Lastly, we will be starting our first round of prototype testing for the summer, following the website usability testing last week. Our parallel streams of research and design will enable us to deliver an evidence-based, valuable solution for TMP by the end of the summer.
