Meaningful Data Evaluation and How Evaluators Can Change the World

(Kind of.)

One of the most important aspects of any data evaluation endeavor (especially when it comes to organizational or institutional data) is buy-in from stakeholders.

What does what I just said even mean?

Well, first: data. Many of us first learn that data is numerical, and therefore complex and boring, easy to tune out. If you go on to study some form of research in college, you learn that data can be quantitative (numerical, statistical, etc.) or qualitative (survey analysis, interview, thematic coding.) But, still, “data” makes us think that we must have some sort of training to understand its complexities. In reality? Data is the world around us. Data comes from the way in which we study that world and make meaning out of it. It’s rooted in factual analysis, sure, and is sometimes numerical which can turn most people off. But when I think about data, I think about it as the stuff of the world around us. Data is anything we can see, feel, observe, quantify, touch, taste: all depending on the context in which it’s used.

In some contexts it is enough to simply collect data and leave it at that. All of us are collecting data, every minute of every day: What you ate for dinner? Data. The shampoo you use in the shower? Data. The feeling you had when you saw the sun this morning? Data. But, importantly, all data for which there is usually no further context in which it will be utilized. Now take this example: on my commute home I noticed a man taking a selfie on the seat in front of me with his friend. That noticing is itself a piece of data that I don’t necessarily need to do anything with. But what if I were writing a piece about the performative qualities of male:male friendship? In that case, I would be able to use that simple observation in a more meaningful way. Another context in which data must often be used in a meaningful way is within an organization, for programmatic evaluation. In a case such as this, various stakeholders are interested in the meaning of your data. To determine why stakeholders are interested, it’s important to understand the audience of your data and it can vary greatly. Stakeholders are people who have contact with or a vested interest in the programs at an organization: program staff, upper management, board members, funders, donors, etc. This is where evaluation is key.

Data evaluation is the process by which the things that an organization does are audited (i.e. inspected by an independent body such as a research department.) Armed with the power of perceived objectivity (because, let’s face it, does true objectivity really exist for any human anywhere?) observations, trends, outcomes, and processes are reflected and reported upon. If necessary, an organization can change what it is doing to improve the results of the data evaluation procedure.

Data evaluation really does not matter if the independent body (evaluators) do not have sufficient buy-in from stakeholders (those who are interested in the results of the evaluation.) Evaluations become more rich, fruitful, and meaningful when buy-in is increased. What is buy-in, you ask? Well, it’s the degree to which your stakeholders are interested, passionate about, engaged, otherwise on board with your data evaluation procedures or results. It is fundamentally important for the research team/evaluators to make sure that stakeholders are invested in being evaluated. There are a bunch of ways to ensure that this happens, but what I’m beginning to understand is that one of the most important aspects is ensuring that all stakeholders feel they have a voice in the process and that their voice is being heard. This is where change can be fostered and take flight.

Think about effective leaders. Politicians are the first that come to mind. Barack Obama was particularly successful as President because he was able to thoughtfully engage (most) demographic groups in the U.S. Even if he didn’t always pass policies that made life easy for every single group in the country, what he did do was make most Americans feel important, heard, understood. Hitler did that too: he recruited an army of mostly young boys by making them feel like they were understood and heard. And, very reluctantly, I’ll mention Trump here, a dude who got elected largely because of his promise to hear the voices of poor White Americans, coal miners and other skilled laborers. Again I say: regardless of what these people actually did, what I mean by effective leadership is the idea that these leaders were able to get people on board with their ideologies based on their talent in making people feel listened to and understood.

Effective leadership, then, can spark real social change. It starts with listening, engaging with stakeholders (or constituents), understanding the problems which they face and identifying with them: creating buy-in, just like an effective evaluator does. Now, our politicians definitely need to get better at actually following through on the promises which they’ve used to garner buy-in. But if the right person comes along who can harness public engagement for good and use it to help people understand the -isms (racism, sexism, classism, etc.), then systemic social change is more than possible.