Q-Interview: Robert Roach and Natalie Rose share their insight on Lumina Foundation’s Community Partnership for Attainment
Welcome to The Q — an interview series where we invite the Equal Measure team, clients, and colleagues from the field, to share their insights on evaluation, philanthropic services, emerging trends in the social sector, and more. In this interview, we sat down with Robert Roach and Natalie Rose, Consultants at Equal Measure. Robert has worked on several projects related to postsecondary access and completion including the implementation of two U.S. Department of Labor-funded Trade Adjustment Assistance Community College and Career Training grants. Natalie currently works on multiple evaluation projects, including Lumina Foundation’s Community Partnership for Attainment initiative, and the BUILD Health Challenge, among others.
Robert and Natalie, can you describe the aims of Lumina Foundation’s Community Partnership for Attainment (CPA) initiative, and share why diversity, equity, and inclusion (DEI) are such integral aspects of it?
Robert: The goal of Lumina Foundation’s CPA initiative is to foster and encourage relationships between K-12 schools, postsecondary institutions, businesses, non-profit organizations, and other sectors to improve postsecondary access and completion for students in 75 U.S. communities. I believe that DEI is crucial in any collective impact initiative, because it is important for a diverse set of individuals to contribute to — and implement — a partnership’s strategies.
In the case of CPA, this involvement will help the partnership’s strategies resonate with students in different communities. For example, partnerships often will decide to pursue a certain strategy to help more students get access to college. But, that strategy may either be the wrong approach, or it may not reflect what students experience. The best way to discover what would be most helpful to students, and most relevant given challenges they may face in college access and completion, is to ask them, or better yet, to include them in developing these strategies.
Natalie: I agree, and in the context of postsecondary completion rates, we must look at how those numbers are affected by inequities or differential access to college. Student success is not just about one’s ability to perform academically. Rather, we need to consider factors that influence students’ ability to learn, such as availability and amount of financial aid, opportunities for internships that round out an academic experience, and exposure to academic support programs.
It’s also important to recognize that when we think about students in college, we should consider them as products of their environment. So, if students enter college requiring developmental coursework, because they weren’t adequately prepared to enter college, then they may become deterred from staying in college. Thus, in postsecondary completion efforts, the student’s environment is a leverage point to consider when developing strategies to help students succeed.
You both had an opportunity to conduct site visits in a number of CPA communities. What were some of the issues you dealt with while trying to apply a culturally responsive evaluation lens during those visits?
Natalie: I think we needed to continually remind ourselves that we were coming into these communities as outsiders, and that we were just there for a snapshot. For instance, after collecting data on the site visit, we shared what we learned with the interviewees and asked, ‘does this ring true to you?’ We wanted to ensure that we didn’t misinterpret what we heard.
We also needed to acknowledge that our own experiences might affect how we interpreted what we heard and saw. When visiting a community, it’s very easy, and comfortable, to rely on what we already know or believe. But, if we learn or experience something that we’re not as familiar with, or that challenges our deep-seated knowledge and beliefs, we must accept that. We must understand that the realities in those communities may be completely different from our initial assumptions.
Robert: I’d add that there are inherent power dynamics when working as an evaluator in communities. And those dynamics can play out in how people interact with you. In any site visit, it is important to build rapport with the interviewee, and to conduct the interview as an open-ended conversation. However, in some cases, I found it difficult to build that rapport.
Robert: Well, I think part of it was because we were perceived as representing the funder. The communities believed that what they shared with us could have implications for their work moving forward. I also think some challenges in building rapport had to do with our personal identities — not just what we look like or how we identified ourselves, but their perception of us as outsiders in their communities because we didn’t live there.
Let’s talk more about power dynamics, which is one of the topics we discuss in the Issue Brief. Can you share what you learned about power dynamics from your work with the communities?
Robert: I think power dynamics can come into play during site visit interviews. For instance, it is important to consider who else is in the room during the interview. On several site visits, we interviewed the same person in different contexts. Sometimes that person was alone. Sometimes that individual was joined by several people with whom they were comfortable, like their peers. And sometimes, that person was with a group in which everyone didn’t exactly view each other as peers. During the site visit interviews, I noticed that some individuals were more open when they spoke to us alone, or when they were with their peers.
But, when those individuals were with a group that they did not consider as peers, I sometimes observed that the interviewees would use non-verbal cues, as if to say, ‘when I spoke with you alone, or with my peers, I said what I really meant, but now I have to couch my responses more politically.’ It is important to observe those interactions, because so much of communications is non-verbal.
Natalie: We can’t necessarily moderate the entire conversation, or predict its direction, but we can try to bring in all the voices. We can try to facilitate a more equitable discussion, in which everyone at the table is comfortable speaking up and feels safe to do so.
How can we ensure that a discussion is more equitable?
Natalie: We can do several things. For instance, we can ask someone who appears more reticent to share their thoughts; we can shift the discussion away from a more dominant participant; and we can follow up with someone who may have looked like they wanted to share, but did not in a group conversation. In these conversations, it is very important to listen, as what we hear will add depth to our understanding of interviewees’ experiences, and enable them to share information in a more comfortable manner.
Along with who is in the room, are there other situations or contexts that affect power dynamics?
Robert: Yes. I think you also should consider where and when the interview is taking place. If you interview someone outside of their element — in a location that might be comfortable for us, but not for the interviewee — then that might bring out a side that isn’t really them. They may become nervous and uncomfortable, and as a result, we may not glean a full or true understanding of their work within the partnership or how they are helping shape strategies in the community.
You both touched on the importance of creating space for individuals to talk openly about issues related to DEI. How do you do that?
Natalie: To start, I think that as evaluators, we try to be open about our biases, and control for them as much as possible. During the site visits for CPA, when we asked the interviewees about how they define diversity, equity, and inclusion, we let them drive the direction of the conversation. We did not want to come in with specific definitions, and impose those on a community. It could end up being a messier process, but also a more authentic process.
Robert: I think that asking questions about context — such as, what are some political, social, economic, or other factors in your region that may be affecting your work? and how are those factors driving success or creating barriers? — can be effective in spurring interviewees to have an open conversation about issues and practices related to DEI.
Natalie: And through asking those questions, we can acknowledge to the interviewees that we are not from that community, and that there are nuances that we should learn more about.
What have you learned from responses to those open questions about context?
Robert: One thing we’ve learned is that the partnerships have different ways of communicating their postsecondary completion strategies to the community. Much of that communication has to do with the history of what has worked and what hasn’t worked in that community. But issues of DEI are also major factors in how the partnerships communicate about these strategies.
Natalie: They may not have talked about concepts in the way we would have, but they live in these communities. The impact of what they’re doing is something they’re going to experience every day, after we leave.
As you reflect on this initiative, are there any lessons that advance your own thinking about how to integrate a focus on DEI into your work?
Natalie: I think that this initiative reinforced my belief that it is essential to focus on issues of diversity, equity, and inclusion in any community in which we work. It is imperative that we ask questions to allow us to get at the root of how communities think about DEI. That’s an important first step for communities to begin breaking down structural and racial inequities. In addition, I was able to extrapolate what I learned from the CPA initiative and apply it to my other client projects.
Robert: I agree, and to add to what Natalie mentioned, focusing on DEI should be a guiding value that is a critical part of your everyday work as evaluators. With the CPA initiative, we incorporated a DEI lens into all parts of our evaluation framework. In our site visit interviews, we didn’t just ask questions specifically about DEI. Rather, we asked questions to learn about how those principles are incorporated into all facets of the partnerships’ work, and how the partnership members interact with each other.