Unlearning and Neutrality
What I learned in graduate school about evaluation could be summed up pretty briefly: Michael Quinn Patton’s Utilization-Focused Evaluation and Donald Campbell’s classic work on threats to validity. It’s now been more than 20 years since graduating, but I’ve had plenty of conversations over the years that tie back to these when talking about rigor and quality in our shared work.
Happily, the conversation around evaluation seems to be changing again, increasingly recognizing that in a post-fact, complex world full of inequities, we, too, need to change some of what we do to respond to the world around us.
Part of what I’ve been increasingly wrestling with is not just how to do new things but identifying what it is I need to let go of. I’ve typically seen knowledge gain as a cumulative accretion of bits of information, adding to an ever growing mountain from which we have a useful perch. But what if these deposits, these ways of being, have actually been blinding me from seeing another transformed world, where things could be meaningfully different and more equitable than they are today? What might I un-learn to gain a whole new vista?
But, it’s so hard to un-learn! I’ve been doing this work for more than 20 years. Most of my training has come from one dominant perspective — it’s going to take active effort to expand and change those years of professional learning and experience. It feels a bit like suggesting I need to re-consider how I walk, something I do nearly instinctually and certainly habitually.
But I might have landed on one of those things I need to root out of my standard practice: the idea of neutrality.
Most of my peers in evaluation were likely trained like me: that mitigating threats to validity and reliability were core to our work as neutral purveyors of new knowledge and information. While any good social scientist would know and tell you that neutrality isn’t possible. Everything is socially constructed. Everyone has biases. Yet we tend to operate in many, many ways as though things are values neutral.
For example, a core part of my evaluation practice is identifying purposes and audiences, the better with which to develop questions, data and products that will meet those needs. I’ve typically thought of those purposes, such as a focus on accountability or learning, for example, as neutral choices. It’s good to be clear what you need to accomplish, but I’ve not interrogated the values and impacts of that choice. Learning for whom? Accountability to whom? Who is left out? Who is harmed or disadvantaged?
When thinking of collecting data and protecting human subjects, I think of the principle to “do no harm.” But that presumes that the status quo is harmless. What if instead, as exhorted by Dr. Margaret Beale Spencer, our standard was to do the most good possible?
Given my training and focus on utilization-focused evaluation approach, I’ve typically felt that if my clients have used the insights they’ve gained from evaluation to improve their work, work that is always grounded in a social mission, that’s success. I have slept well at night and felt professionally satisfied. But what if that use doesn’t actually advance their mission? What if their use of the evaluation in fact only reifies power imbalances or historic inequities such that the use makes no positive social difference at all? What if it does harm in unintended ways that this blind spot has kept me from acknowledging?
I’m beginning to better see that this neutral stance continues to center whiteness by acting as though race and other power dynamics are not at play.
What if, instead, I routinely interrogated who benefits and who is harmed in the many aspects of my work that I have previously thought of as important choices, but choices that have been equity neutral? Could that help with some of the undoing of other practices that I don’t even know to question yet?
While I’ve long since worked in spaces that are more acknowledging of complexity and systems and have different standards of evidence and excellence, I haven’t previously spent time really interrogating how I do my work and how those strategies, approaches, frameworks, and muscle-memory way of doing things could potentially be unproductive, upholding systems of oppression and inequities. This seems like a good place to start.
Thanks to many academics and practitioners whose work and thoughts have contributed to my thinking, many, though not all, at this spring’s Culturally Responsive Evaluation and Assessment (CREA) Conference. Of particular reference, listed in no particular order, are: Eric Jolly; Dr. Margaret Beale Spencer, Jane McAlevey via Dominica McBride; Dominica McBride; Dr. James D. Anderson; Dr. Tom Schwandt; Jara Dean Coffey; Dr. Nicole Bowman. Thanks also to colleagues who reviewed an earlier version of this piece: Leonor Robles, Julia Coffman, and Audrey Jordan.
