Funders are from Mars, Evaluation Consultants are from Venus

Julia Coffman, Center for Evaluation Innovation, October 2019

If we want evaluation to add real value to social change efforts, both funders and evaluation consultants have to re-examine their roles and how they relate.

This entry is part of a series of evaluator responses to critiques of evaluation and our visions for a better future. Read more about the reason for this blog and the other responses here.

“Funders are from Mars, Consultants are from Venus” is a comment made by one of the attendees at our most recent Evaluation Roundtable convening. The Evaluation Roundtable is an informal network of foundation leaders in evaluation that aims to improve evaluation and learning practice in philanthropy. We at the Center for Evaluation Innovation coordinate the network and convene it periodically.

While our Roundtable convenings normally include only the foundation representatives who participate in the network, this time we tried something different — we included a group of evaluation consultants who work with foundations. We wanted to hear what both sides of this relationship wanted the other to do differently to tackle the common criticism that for all of the money and effort spent, evaluation in philanthropy is not returning enough value for social change efforts.

To kick off the funder-consultant conversation, we handed out sticky notes and asked each participant to finish this sentence:

We collected the results, and then themed them to identify where we converged and diverged.

The attendee making the Mars-Venus comment was of course referring to the bestselling 1992 book about relationships. In it, author John Gray concluded that many of the relationship problems between men and women stem from their different norms, customs, and ways of communicating. On the sticky notes we collected, we found evidence of this same phenomenon between those who commission evaluations (foundations) and the consultants who conduct them.

There’s a lot to dig into in the feedback. Some of our key takeaways follow.

Foundation evaluators want consultants to push back and to push harder.

Evaluation consultants often refer to themselves as “critical friends” to their foundation clients. But foundation leaders were clear that they don’t think evaluation consultants are being critical enough.

Foundations want consultants to offer more honest and candid critiques. Specifically, they want consultants to ask more hard-hitting questions about the foundation strategies they are evaluating. They want evaluators to unpack, test, and challenge the assumptions inherent in them. [Note: We heard this from evaluation leaders in foundations. This desire for more push back is not necessarily shared by the program staff who craft these strategies.]

Foundation leaders also want evaluators to push back more on what funders request of them on evaluation. If the terms of reference do not seem appropriate, they want consultants to say so. If the time frame and budget are too short and and too low for the scope of work, they want evaluators to be candid. If the foundation is asking for a developmental evaluation but setting up the contract to be something else, they want consultants to question the RFP rather than just respond to it as written.

To push harder, foundations and evaluators need a different kind of relationship.

Evaluation consultants say that a lot has to change about their relationships with foundations for consultants to feel safe in this kind of “pushing back.” This is especially true for evaluators of color, where implicit biases can lead to different reactions when they push back compared to their white colleagues.

The problem is that foundation-consultant interactions primarily are transactional instead of relational. They are based on traditional contracts that feature methodological work plans and a set of deliverables and schedules. Our agreements do not focus on how the foundation and evaluation will relate to one another, communicate, or build trust. Instead, they reinforce the uneven power dynamics that are inherent in fee-for-service relationships.

So what does it mean to move from being transactional to being more relational? The table below offers examples of how they differ.

Both foundation leaders and consultants at our Roundtable convening expressed a strong desire to be more relational. Foundations want consultants to understand them better so that the work can be better aligned with foundation culture, dynamics, and context. Consultants want foundations to help them acquire that understanding so that they can better navigate foundation systems and their complex interpersonal dynamics. As one consultant put it: “I wish foundations would proactively and regularly share inside contextual information to support relevance and responsiveness in our role, processes, and products.”

We need to change the mental models that govern our relationships.

How we approach foundation-consultant relationships now is not helping us to understand each other, build trust, challenge each other’s thinking, and ultimately get more relevant and useful evaluation work and products. We need a different approach that shifts us away from the limiting mental models that get triggered when we think about evaluation consultants.

As if the mental models and negative associations triggered by evaluators wasn’t bad enough (triggering thoughts of fear, judgments, defensiveness)….

…they get even worse when the term consultant is added (triggering thoughts of for-profit motives in a nonprofit world).

There is some precedent for changing these models in how foundations have worked on their relationships with grantees.

Many foundations have tried to improve their relationships and more equitably distribute power by referring to and treating their grantees as partners. Use of this term helps to trigger different mental models to put foundations and grantees on more equal footing, and give a sense that this is a relationship in which both sides have agency.

Because a different label alone does nothing to actually change the real and uneven power dynamics between funders and grantees, norms and behaviors that go along with this re-framing must reinforce it. For example, grant agreements might include specific promises that grantees can expect from funders, such as:

· Quality interactions — You will be treated with respect and candor.

· Clear and consistent communications — You will know when and who will make the decision on your grant, and you will be provided clear communications on the foundation’s strategy, grant process, and the amount of time and assistance you could expect to receive from the foundation once your grant is awarded.

· Feedback — You will have opportunities to provide feedback–and we will use that feedback to make continuous improvements.

When grantees are treated as partners, their points of view are considered and their time is compensated (including for convenings and other “extra” activities). They are trusted with more flexible funding and given the freedom to adapt their plans as needed. Their relationships with foundations are personal, making them deeper and longer-lasting.

Changing our norms and practices is doable.

Men Are from Mars, Women Are from Venus was wildly successful because it helped millions of couples diagnose what was getting in the way of satisfying relationships (i.e., different ways of communicating), and then helped to improve them by suggesting manageable behavior shifts (e.g., The woman gives the man three options of what she wants to do on a date a week in advance, the man gets to decide but doesn’t tell the woman until the date. This allows the woman to anticipate the date, while the man gets to feel as though he made a decision while still receiving clear communication on what the woman wants.)

What will it take to shift our relationships so that evaluation can add more value to social change? Just changing the labels and calling ourselves partners won’t do it. We have to do things that help to shift how we think about our relationships so that we treat each other as partners in the work.

Our Roundtable convening participants offered some specific ideas about what it would mean behaviorally to be better partners. While the legal need to write contracts can’t be avoided, consultants suggested some changes in how foundations approach them. For example, they suggested:

· On RFPs: Engage us as thought partners (as opposed to/in addition to) evidence partners. e.g., have retainer agreements or open-design RFPs; Provide the same level of flexibility and compassion to evaluators as they (often) do for their grantees.

· On timing/timeframes: Integrate evaluators, grantees, and foundation staff at the beginning of the funding process (at the same table from the beginning); Allow ample time for relationship building in the evaluation, learning, strategy process; Work with consultants for longer strategy cycles/arcs to lead to deeper analysis, insights, and relationships.

Several efforts within the sector are focused on adding more specific ideas on how to improve relationships between foundations and evaluation consultants and there are many ways to get involved for those who are interested in digging in. The Funder and Evaluator Affinity Network (FEAN) is considering various ways in which funders and evaluators can work together differently to deepen the impact of evaluation and learning on philanthropic practice. The Equitable Evaluation Initiative is challenging all of the orthodoxies around how evaluation consultants and foundations relate.

Looking at the foundation and consultant feedback side by side, it is clear that we are not actually from different planets. We both value and want many of the same things — better strategies, better relationships, more equitable and useful evaluations. Just like couples and the Mars-Venus book, understanding each other better and working on the specifics of our relationships and how we communicate will help us to get there.

Julia Coffman is founder and director of the Center for Evaluation Innovation and co-director of the Evaluation Roundtable. Twitter: @julia_coffman @Eval_Innovation.