Ensure Data Efforts Drive Toward Equity and Inclusion
Interested in data for scale? You probably want to talk about fancy dashboards or advanced analytics. But before we get there, our interviewees emphasized how critical it is to first lay a foundation of equitable and inclusive data practices on which to build.
“Data has even more power than we might recognize for poverty eradication, provided it is shared back with communities. Let’s transform an extractive exercise into an interactive one. Let’s restore to people their fundamental right to own, understand, and control their data and information. Such a matter of moral right is universal and never contingent on economic status…. To be ethically defensible, data must be gathered through transparent social understanding and relationships.” Ann Cotton, Founder, CAMFED (in Data as a Democratic Practice)
An equitable and inclusive data approach will help answer questions, such as, what are the best practices for listening to and collecting data from clients? If this is done well, you will be able to deliver more value to your clients and make your business model stronger and more sustainable in return. Or, how can we most effectively give voice to the diverse stakeholders who are experts in their own lives? If successful, better solutions will surface and, through data, you can empower and uplift the communities your mission seeks to serve.
What do we mean by Equity and Inclusion?
While many definitions exist, for the purposes of this paper, we are focusing on the following:Equity: the “fair treatment, access, opportunity, and advancement for all people, while at the same time striving to identify and eliminate barriers that have prevented the full participation of some groups.” (Kapila et al, Why Diversity, Equity, and Inclusion Matter)
Inclusion: an environment which supports, serves, and values all individuals such that they can fully participate and that their perspectives are used to inform action.
This article focuses on what U.S. Monitor describes as “two important strands that are interconnected but not identical: a focus on [client]¹ voice and an emphasis on diversity, equity, and inclusion.” Like U.S. Monitor, we see these two “strands” support and reinforce each other in the context of data to drive impact at scale, especially with the populations with which social enterprises work, and thus include them both in the points below.
While embedding an equity lens in your data is important at all points throughout your development, scale introduces a new pace of growth and reach that has the potential to exacerbate bias and demands a close consideration of equity. Many organizations, although they are faithfully serving their missions, may not have interrogated their data to ensure equity and inclusion. This foundation will help you do just that and will serve as a critical lens through which the remainder of the strategies in this paper can be viewed.
The organizations we interviewed shared three key questions they ask themselves to illuminate possible bias and identify opportunities for improvement in the collection and analysis of data:
1. Who determines what gets measured?
2. How is it measured?
3. Who gets to analyze and interpret the data?
To make it useful for your organization, below we break down each of these questions and provide advice, tactics, and examples of how organizations have approached them.
Question 1: Who determines what gets measured?
Too often, social enterprises collect data about their clients without consulting them in the process or even truly understanding what is important to them.
Top tips & tactics include:
A. Ensure your data is keeping you accountable to your clients. Take stock. Go through your existing metrics and ensure that the things you are measuring — and thus that are driving decisions — are holding you accountable not just to yourself and your funders, but also to your clients.
Harambee Youth Employment Accelerator (Harambee) recognized the tendency to create metrics that merely hold the organization accountable to itself and its funders for activities delivered. Thus, instead of creating measures such as “have had x conversations with a young person on phone,” which center on the organization, it collects data that holds itself accountable to the clients that the mission is intended to serve: youth that are currently unemployed. Data centers the young person, even being written in their voice, such as “I [young person] am supported in the network” and “I [young person] secure a work opportunity.”
B. Provide opportunities for your clients to help define what data points matter. Bring M&E closer to the front-line, ensuring closer ties to staff or stakeholders who can help to amplify the voice, concerns, and interests of the ultimate client and who may be best positioned to identify ways to directly engage clients for their input.
“Poor people best understand their own lives. Let’s listen hard and learn from them as to how, together, we can solve the colossal and growing problems facing us all,” says Cotton of CAMFED. For example, Health Leads has leaned into a listening approach in recent years. Initially it collected data on the wrap-around resources that its patients accessed (i.e., food pantry) — but, in digging deeper, recognized that, from the patients’ point of view, “wellness” is not so narrowly defined. Health Leads has begun to experiment with measurement approaches that are embedded in a more holistic inquiry into whether patients have the essential resources they need to be healthy, thus holding itself accountable to wellness as defined by clients.
Question 2: How is it measured?
The methods employed to collect data can easily perpetuate inequity and bias if not thoroughly interrogated.
Top tips & tactics include:
A. Identify biases in data collection. Test data questions with different constituent groups to ensure questions account for different life experiences, interpretations, and cultural nuances.
Chicago Beyond’s Equity Series shares examples of how inequity can be built into metrics, such as “measuring housing ‘overcrowding’ for participants from a culture that values extended family,” or “measuring ‘progress’ on self- actualization for participants from a culture that prizes interdependent families over independent individuals.” In another example, one interviewee reported discovering that a question in a key pre-program survey referenced a male- dominated activity, and thus the results favored men over women.
B. Ensure that all voices are captured. Use multiple collection methods (quantitative and qualitative) to capture different voices. Make sure these methods do not leave anyone out by asking whether some groups will have difficulty accessing technology, feel less comfortable with format, or are unavailable during data collection.
Harambee uses multiple methods to gain insights from the youth it serves, acknowledging that while some youth may be available in person for a focus group, others will be available only by phone at certain times of day or certain days of the week. Health Leads has an individual with expertise in community-based participatory research and qualitative methods on staff, which is core to its ability to advocate for alternatives to more traditional quantitative research and to challenge existing narratives. As we discuss in more detail in the systems change section, qualitative data methods (e.g., outcome harvesting, comparative case studies, etc.) are as important and rigorous as quantitative and offer alternative means of capturing stories of different stakeholders.
C. Dismantle power dynamics that can bias data collection. Interrogate your data collection methods to identify power imbalances: Who is collecting the data? How could that be perceived by those from whom data is being collected?
For Harambee’s on-the-phone experience (HOPE) calls — one of its primary data-gathering tools and key touch points with youth — interviews are conducted by young work-seekers themselves. Data about youth are gathered by youth, placing youth at the center of data collection and product design, and decreasing the likelihood that those being interviewed will feel uncomfortable with the interviewee.
D. Think about what biases your data collection processes might be hiding from view. Work with the program and data teams to articulate the stakeholders that you are seeking to serve and ensure that they are being included in data collection and/or create a plan to address that.
When Harambee began to use machine learning to suggest work opportunities to youth, it used parameters that included cost/time/distance to work location (shorter distance to work was shown to correlate with retention). However, Harambee realized that youth living further away from economic centers would be left out using this data, as they were further from potential jobs; therefore, different interventions — that made use of work-seeker location and locally relevant market intelligence — were required. Learn more about machine learning and how it can perpetuate equity here.
Question 3: Who gets to analyze and interpret the data?
The individuals and groups who are the subjects of data collection should have the opportunity to actively use and interpret the data, and their voice and experience should be brought to bear in decision-making stemming from that data. Top tips and tactics include:
A. Empower stakeholders by providing access to data. Work with stakeholder groups to understand what data they would like to have, when, and in what format. Communicate and share the data in ways that are relevant and easily understood by impacted communities.
The Family Independence Initiative, an organization that supports families to move out of poverty, not only collects data from families to identify trends and make relevant resources available to those families and communities, but also compiles and analyzes that data to share back with the families. This sharing of data enables families to track their own progress and better understand the trends in their communities. CAMFED finds that sharing data creates opportunities to celebrate progress and reinvigorate community efforts. Theresia Moyo, Head of Education at CAMFED Tanzania, reflects, “By sharing the results of community action at local level — for example through graphs on the increased recruitment of qualified female teachers as a result of a community initiative to build teacher housing — young women help communities celebrate their achievements, and aim higher.”
B. Give diverse stakeholders a voice in interpreting the data. Slow down to engage people at all levels of your organization and of the impacted community to develop ideas about what the data could mean and to bring these stakeholders’ unique perspectives to the analysis.
To ensure its clients have a voice in interpreting the data, Harambee regularly convenes focus groups of members of its target demographic and leverages its regular communications with youth through call center conversations to illuminate additional insights from the data.
C. Commit to taking action on client insights. At an organization level, measure and track the integration of client input and insight into decision-making to hold the organization accountable for not only collecting the data but using it. (For more context, see Evans et al, Re-Imagining Measurement: A Better Future for Monitoring, Evaluation, and Learning in the Social Sector.)
Damon Francis, Chief Medical Officer of Health Leads, warns that merely providing an opportunity for and tracking client insight is insufficient if it does not lead to shared decision-making power. “People get very tired of giving advice that isn’t followed. A pathway to increased authority via participation in governance is really important.” For example, Federally Qualified Health Centers have long been required to have governing boards in which patients constitute a majority of members, and this design ensures that insights from those most affected by inequities are closely connected to decisions about budgets and organizational strategy.
D. Present data in ways that are accessible for all. Find out what familiarity different groups have with various presentations of data, work with them to allow unbiased, productive access, and test formats and channels with target audiences.
Chicago Beyond shares considerations for those making data more accessible, such as ensuring the language used is easy to understand (and testing that in early report drafts), working with those who will be consuming the data to determine the best format for its presentation, and determining which forums are best for sharing data (e.g., social media, community discussion).
Notes:
- “Client” refers to the individuals that social enterprises aim to serve whether they be paying customers, product or service users.
Read next: Using Data to Power Scale, Create a Learning Culture as you Scale, or return to see all articles in Data for Scale.
Watch next: SOCAP2020 Session: “Who Determines What Gets Measured & How?” with CASE’s Erin Worsham, Living Cities’ JaNay Queen Nazaire, Harambee’s Sharmi Surianarain, Chicago Beyond’s Shruti Jayaraman, and Pratham’s Siddhesh Mhatre.
This article was written by Erin Worsham, Kimberly Langsam, and Ellen Martin, and released in June 2020.
