It’s Time to Rethink ‘Best Practices’
Our reflections on ‘best practices’ and ‘bias’ in data analysis.
This blog post accompanies our demographic and indicator data release for our Global Mapping Survey. Read our data report here.
When learning about research methodologies, we are often taught to remove all bias from our research and that the only data that is valid is “neutral data” with “standardized procedures”. The idea of neutrality implies that the inquiry is completely free from the researcher’s perspective, positionality, and lived experience. But this is never the case. As human beings, the questions we ask are based on our own interests and framed by our perspectives.
Complete “objectivity” in research is a myth.
The team at the Racial Equity Index (REIndex) is conscious that the history and origins of these values and methods associated with “objectivity” are constructed and reproduced by predominantly white researchers, informed by ideologies of heteropatriarchy and imperialism, and as a consequence, have produced ‘data’ and research procedures that are informed by such lenses. These methods of knowledge production have created a type of status quo and cemented differentials of power that persists to this day, with numerous harmful consequences.
These consequences are reflected in the preference for quantitative over qualitative research methodology and the assumption that the randomized controlled trial (RCT) is the standard of objectivity and any other type of research will undergo significant scrutiny. This then feeds into how topics are studied, who is conducting the research, and who is the participant — and ultimately who holds power within a research study and the knowledge produced from its findings.
The role of research in upholding the current global racial hierarchy has historically received little attention, however, more recently these reflections on how racism shows up in the research process are finally getting some recognition across the sector from how it continues to show up in funding for research, the actual research methodology, both qualitative and quantitative studies, and what research is then published.
With transparency and collaboration as core values for the REIndex, in November 2020, we released a Global Mapping Survey to crowdsource key indicators to measure racial equity within global development organizations and institutions. The survey also asked respondents if they had themselves experienced and/or witnessed racism in the global development sector along with a number of key demographic questions to get a sense of the survey respondents’ background including, self-identified racial identity, gender identity, age group, current country of residence, capacity in which they are working and/or participating in the global development sector, and tenure in the sector.
As part of the data analysis, our team is tasked with reviewing and coding over 200 open text responses that discuss experiences and witnessing of racism in the sector. We had not accounted for the emotional burden and toll that this would take on our all-volunteer BIPOC team (all of whom have experienced racism in the global development sector). To read numerous accounts of discrimination and harm is exhausting and saddening. Throughout this process of coding open-text responses to racism and violence in the sector, we have required time, breaks, and space to engage with these charged topics.
Recognizing as a BIPOC collective that our lived experience directly ties into the research we were undertaking, we engaged in reflexive practices as a team to see how that was informing this work. Our lived experiences and the context around the origin of REIndex are not something that we shy away from. We are engaging with topics of racial inequity in the global development sector and a reality that impacts both us as researchers and as part of the communities who have contributed to and are engaged in this work. With this Global Mapping Survey, we did not set out to ‘prove’ anything — it is already a well-established fact that the development sector is founded on and rife with white supremacy and heteropatriarchy. The purpose of this Global Mapping Survey was to assess trends of racial (in)equity in the sector across geographies and populations.
As Toni Morrison said: “The function, the very serious function of racism is distraction. It keeps you from doing your work.” It was not a surprise that our team (all BIPOC women) have received accusations of biased research practices in our Global Mapping Survey process.
We anticipated our work to face criticisms and attempts at invalidation. But we are using our lived experience to push back on the status quo of the development sector. We are also prioritizing the insights and input from BIPOC populations, as this group is most impacted by institutional and systemic racism.
At REIndex, we continue to ask ourselves important questions in our data collection and analysis processes, such as data by whom? For whom? With what values and interests in mind?
Throughout each stage of our global mapping survey process, from survey design, creation of indicator definitions, data mining and analysis, we have pushed against the status quo and reflected on these “standardized” procedures, while constantly thinking about the implications of the work we were undertaking and our role in it.
But one thing we will not do is apologize for being driven by our lived experiences of racism in the global development sector and our passion to hold this sector accountable. This passion has led us through our work to this point and will continue to do so, wherever our research in the build of this index for racial equity will lead us.
Established in June 2020, the Racial Equity Index Group is a collective of BIPOC (Black, Indigenous and People of Color) who currently work or have spent part of their career working in international development and are dedicated to holding the sector accountable through the creation of a global racial equity index.
Follow our work at TheRacialEquityIndex.Org
