Contextual Research, Human-AI Teaming, and Prototyping: Data-informed Exploration for Converging on a Solution
Executive Summary
In this sprint, our team has been synthesizing the data collected throughout the semester to generate actionable recommendations for TMP. Through analysis of our research activities so far, we identified two key challenges: refining the feedback collection process and enhancing data utilization and collection. From this, we’ve outlined three potential solutions: a client portal to boost transparency between TMP and its clients, a data quality control pipeline to elevate data management, and an AI to streamline TMP’s routine tasks. Moving forward, we’ll deepen our analysis of interview and survey data, incorporate feedback from TMP on challenge and solution prioritization, and refine our prototypes. For the next sprint, we aim to develop practical solutions that align with TMP’s organizational mission and goals.
Intro
As we enter the final month of our spring semester, our team is transitioning to converging on recommendations and solutions. While we continue to collect valuable data through interviews and surveys to enhance our understanding of the mentoring ecosystem, we have started to generate insights and pinpoint potential challenges we can address. At the same time, we’re beginning to develop actionable recommendations for TMP, aiming to create a meaningful impact through our research.
Ongoing Data Collection
During this sprint, we focused on understanding the depth, impact, and context around the challenges we have identified for TMP. One of the research methods we used was contextual research, where we traveled to OneValley to sit in on a monthly staff meeting.
We initially set out to:
- Observe the tools used by TMP
- Understand details of one of the main environments in which they conduct their work as they are a hybrid team
- What activities does TMP collaborate on
- How they interact with one another to collaborate
- Generally learn more about their organization’s partnerships, initiatives, goals, and backstage processes.
It was insightful for us to hear about some of their future initiatives such as building out a resource library and how they are planning to align their organizational resources with DEI efforts such as increasing language accessibility considering that much of our conversations with TMP have centered around data. Our contextual research resulted in uncovering and validating challenges with consistency in the quality of data collection and considerations for where our solution may fit in with their other organizational improvement efforts.
In addition to our contextual research, we have continued to interview and survey program leadership, mentors, and mentees who have used services, tools, and resources from TMP. This effort has largely been to understand details of the needs and challenges manifest for each stakeholder in the TMP mentoring ecosystem. Currently, we have seen trends in programs leveraging their networks to acquire the training, resources, and volunteers that they need to provide mentorship. We’ve also heard feedback about increasing resource accessibility for TMP clients and plan to target understanding this further as we continue to interview. We will seek to integrate the data gathered from our parallel research activities into powerful recommendations for TMP.
Identified Challenges and Insights
Our journey through a sea of data that we have collected has led us to identify the key goals TMP strives to reach and potential opportunities for its strategic initiatives. To determine the criticality and impact of these challenges, we conducted a detailed prioritization exercise by scoring each challenge based on its alignment with our objectives, the impact on TMP’s decision-making, the urgency of resolution, and the feasibility of overcoming it. By employing a quantitative scoring method, we ensured our assessment of the challenges was not only thorough but also systematic, eliminating the risk of subjective judgments.
Some of the opportunities we analyzed include:
- Evaluating the impactful qualitative enhancements in mentoring programs after providing tools and resources
- Gathering detailed and robust feedback from mentoring programs and mentors to enhance TMP’s services
- Scalable and streamlined framework for conducting needs assessments
- Seamless storage, analysis, and strategic application of feedback data
Solution Exploration
After determining our guiding challenges and insights for solution exploration, we began the process of realizing solutions to high-priority challenges. We started out by listing out solutions that we knew would not work or be desirable, and this helped us identify some initial properties of solutions that we believe should be incorporated into a solution.
Some of these properties were:
- Enable collaborative knowledge sharing and analysis of data and information gathered between staff members and between TMP and mentoring programs
- Gathering information from staff members in a non-invasive, ethical manner (e.g., clearly stating what data would be gathered from their work, avoid them feeling like a system is monitoring them)
- Creating more consistency with data entry and reducing the manual burden on staff to do repetitive tasks relevant to data management
- Leverage more data that is currently gathered such as needs assessment note-taking and informal engagement monitoring of participants in training sessions
We then generated several ideas through rapid idea generation methods such as Crazy 8’s, allowing us to explore broadly what interventions could be used, the context of use for solutions, and features to include. It was exciting for our team to put pen to paper on ideas we had been sitting on during our initial exploration and research activities earlier in the project. We considered both physical and non-physical ideas (i.e., feedback lunches), eventually narrowing down to 2 prototypes for our first round of parallel prototyping.
Current prototypes under consideration:
- Client-TMP Communication Portal: We envision a shared workspace for TMP and a program leader to allow the programs that TMP serves to see a visual timeline of the service delivery process. This platform would facilitate continued communication between TMP and programs throughout their relationship, providing visibility into the steps TMP is taking and the impact they are generating for the program.
- Data Quality Control AI-Assistant + Platform: We are exploring using AI to reduce manual data entry, data cleaning, and providing suggestions on how to analyze or use the data gathered from multiple data and feedback streams.
In addition to the two prototypes mentioned above, our team is exploring the realm of human-AI interaction. We began by conceptualizing the potential dynamics between various stakeholders and AI, similar to a complex equation where we mapped out all kinds of relationships involving three key elements: TMP, their clients, and AI.
With a clearer understanding of these relationship dynamics, we analyzed the routine tasks of TMP staff and brainstormed different types of AI that can streamline repetitive and manual tasks for both staff and clients. Our goal was to pinpoint AI interventions that are accessible and practical, focusing on solutions that don’t require substantial financial investment or advanced technological resources. We aimed for “low-hanging fruit” solutions that are straightforward yet effective in addressing immediate needs.
Following this, we categorized possible AI interventions to investigate recurring themes and patterns, ensuring that our proposed solutions are not only feasible but also aligned with the core needs and functions within TMP. As we continue to explore solutions and opportunities for automation, we will use the five dimensions framework as a guide to determine the appropriate scope, functionality, and aesthetics of the proposed solution for the end of spring.
Next Steps
As we started the final month of the semester, our team has begun to reflect on our entire Capstone journey so far. We synthesized our experiences to identify key challenges, and insights for potential solutions, opportunities, and future directions. We plan on integrating this information into a research report, project website, and an upcoming spring presentation to showcase our work in digestible ways.
Other steps our team plans to take in the next sprint:
- Incorporate Feedback on Prototypes: We received feedback from peers and faculty on the feasibility of these prototypes, the importance of prioritization, and our ambitiousness with functionality to refine prototypes by the end of spring. These are important issues for us to consider as we need to budget for development and concept validation before diving deeper into the design process in the summer.
- Wrap-up Data Collection: We plan on wrapping up interviews and closing our survey by the end of next week to allow for analysis and synthesis of participant data prior to further prototype development.
- TMP Alignment on Prioritized Challenges and Solution Directions: We plan on eliciting feedback from our TMP contacts to ensure we are closely collaborating on solution exploration and addressing their high-priority challenges.
Thank you for reading!
Appendix:
- Contextual Research AEIOU Notes framework
- Challenge Prioritization Framework Inspiration
- Worst Ideas Brainstorming
- Crazy 8’s Brainstorming
- What is Parallel Prototyping
