Getting Down with Data: Research Synthesis

Driving our scope to a target audience and learning from our first week of interviews

Recap from the last post: Our team built out a (living) plan for the coming semester to determine our key research questions and methods. Read on to see what we came away with in our first week of research!

The hardest question of any design project is not, What should we design? Instead the question is, Are we designing for the right problem?

Given the enormous domain of career development for K-12 students, this week we agreed on these areas to explore first:

• Who: Early high-schoolers (9th & 10th graders)
• What: Future high-demand jobs, Industry/professional exposure, Accessibility to career resources
• Where: Southwest Pennsylvania region (Greater Pittsburgh Area)

Our future research may inform us on different areas, but this was our first step into the foray.

We used four main activities to inform our areas of exploration: secondary research, primary research, a synthesis exercise, and then scope-narrowing.

MRW people ask about our project

Just Googlin’ Stuff: Secondary Research

We split up the team to cover several topics for secondary research to help us inform our scoping decision, and we re-grouped at the end of the week to share our insights.

• Pennsylvania Education Policy: Chapter 339 is a statewide policy that requires school districts to create a counseling plan focused on career readiness and education for students within the district. While the plan has very ambitious intentions, we found that many schools can find loopholes to comply with the policy without making a real impact on students’ future careers.
• Age and Educational Psychology: There are many phases in career development for students. During the growth phase for ages 4–14, students develop a conceptual model of their world, flawed or not, that dictates how they view themselves and societal structures. Starting in early high school, students begin the exploration phase and develop their career interests. From 11th grade, they start to “get serious” about the logistics of their future plans, which are eventually solidified after high school during the crystallization phase.
• Socioeconomic / Multi-cultural Factors: For students with low socioeconomic status, they may face less involvement from parents and teachers, less exposure to types of jobs, and a limited professional network. Additionally, minorities may face racial discrimination and perceived disadvantages from stereotype threat.
• Demographics in Our Region: We focused on 10 counties in the southwestern Pennsylvania region. Allegheny County, which includes the Pittsburgh metropolitan area, is the most densely populated and ethnically/socioeconomically diverse region. The highest rates of food stamp recipients are in the Pittsburgh urban areas, which are also predominantly Black neighborhoods. The surrounding rural counties have generally lower population density, fewer schools, and less diversity.

The Interviews: Primary Research

For our initial primary research, we wanted to get a broad idea of the domain and different perspectives from the many stakeholders on career development for high schoolers.

No pressure, though!

At this early stage, we talked to anyone who had any expertise in education, even if they were from outside the southwest PA region. With our recruitment chair Alexis hard at work over the weekend, we conducted a total of 13 interviews in our first week.

Who we talked to:

• 2 students to get first-hand perspective on the experience of the student
• 2 parents to understand parental influence on students’ career decisions
• 4 guidance counselors to get educator perspective and priorities
• 1 high-level education administrator to get the top-sight perspective on the education system and governing policies
• 4 outreach programs to understand how external organizations are involved with students’ interests and career goals

Tips for Recruiting

• Start recruitment early — This can’t be stressed enough. At best, it takes minimum 3 days to set up an interview from first contact to finally scheduling a confirmed time.
• Keep track of your contacts — Our team developed a system in Google Sheets for keeping track of who we’re reaching out to and scheduling interviews. We included their background and contact information as well as the dates of our communication with them.
• Reach out to your personal network — Many of our first interviews came from our personal networks, including fellow MHCI classmates that worked in education before the program or close relationships to guidance counselors in our hometowns.
• Get creative — Post everywhere! Think about Facebook, NextDoor, Craigslist, public libraries, telephone poles, local cafes, etc.

Tips for Interviews

• Be prepared — Write a script ahead of time on what you’re going to talk about in the interview. Start with an intro about yourself and the project, and be sure to include any necessary consent permissions. Create a list of guiding questions and include specific questions for that interview.
• Don’t be afraid to ask follow-up questions — For interviews where education is not their job (students, parents, etc.), we found it helpful to keep probing with follow-up questions to get interviewees to elaborate on their answers.
• Take good notes — Like really good notes. Ideally, interviews should be done in pairs with one person as a dedicated note-taker. If that’s not possible (we’re all crazy busy grad students), be sure to ask the interviewee for consent and record the interview. It sounds like extra work, but it’s actually easier to take good notes from the recording rather than frantically trying to get everything down during the interview.

I Have an Affinity for You: Synthesis Exercise

This is where the interview notes came in. Because we had so much rich qualitative data, an affinity diagram was the appropriate synthesis exercise to understand our individual interviews.

“Hmm…where does this note belong?” — Everyone

With quotes pulled from each interview, we clustered similar ideas and compared across stakeholders. This process exposed which sentiments were common from many interviews and which perspectives were not as universal.

Our first iteration of the affinity diagram

Our 5 biggest insights (in no particular order):

1. Start engaging students with career interests in middle school so that they have more time to consider what pathway is best for them.
2. Students’ self-initiative affects their engagement with and pursuit of career goals.
3. There’s too much focus on 4-year college degrees while there is high-demand for well-paying trade skills.
4. Students choose from limited career options due to a lack of exposure to different kinds of career possibilities.
5. Accessibility to resources is a barrier for students.

To manage our affinity diagram in the future, we digitized it into a Google Sheets matrix, which was able to count how many of our interviews agreed with each sentiment. We also noted strong quotes that represented each sentiment and retained the note id number for traceability back to the original interview documents.

Tips for the Affinity Diagram:

• Make sure to take breaks: This is a marathon, not a sprint. What was effective for our group was to set a 10-minute timer for shifts. Half the group would cluster notes while the other half can take a break to organize their notes, check email, etc.
• Don’t be afraid to keep moving notes around: Notes don’t have to be static in one place, which is why we use post-its and blue painter’s tape. If a note seems like it should be in another group, move it!
• Write additional clarification on your notes: Sometimes when a note is out of context from the interview, it doesn’t make sense to other team members. We found it helpful to write any additional context or clarification on the note to help other people understand the insight.

Check out this article from the Nielsen Norman Group to learn more about affinity diagrams.

Pun credit: Nathan LeBlanc

A New Scope: Narrowing the problem

Finally, we had the information we needed to begin narrowing our scope. We used the whiteboard to write out two categories: Who and What.

Brainstorming narrowed scope options

We thought about Who across several dimensions. Should we look at a specific age range, or a specific career interest across grade levels? How do we factor in other variables such as socioeconomic status or minorities?

We agreed that early high school students are a good target audience to explore since it is the most impressionable time in a student’s journey where they are still exploring postsecondary options before they must decide on a path by late high school.

For the What, we discussed several areas to explore that would best benefit our target audience.

From our brainstorm, we settled on high-demand jobs in the southwest Pennsylvania region, industry/professional exposure, and accessibility to resources as our high-priority subjects moving forward with our project.

Whew, that was a long week! We conducted our first round of interviews and got to a narrower scope in our project. This was just the beginning of getting familiar with our domain and taking a first stab at what our end goal might look like.

If you’re still reading, thanks for sticking with us. Check out what we did next in our following blog post.