Evaluation — A Beautiful Practice for our time
“All the diversity, all the charm, and all the beauty of life are made up of light and shade.” — Leo Tolstoy in Anna Karenina
Among three methods to learn wisdom, Confucius said, reflection is the noblest.
In his recent book Elon Musk, one of the smartest, most successful innovators of our time, notes his single best piece of advice: “Constantly think about how you could be doing things better and questioning yourself”. He also seeks feedback from others; he hires the best people in any field who can provide “consistent and truthful”,” preferably negative” feedback.
A few days ago, all 194 member countries of the World Health Organisation (WHO) committed to an “independent and comprehensive evaluation of the global response to COVID-19”. This was a highly significant moment, given the boiling-point belligerence with which the US has been pointing fingers at China about the origin and management of the virus.
All three examples relate to the meta-discipline of evaluation — a systematic practice based either on (often assisted) self-reflection or on assessment by others. It collects and integrates evidence to help inform or improve ‘something’, and get greater, better impact. ‘Something’ can be actions, events, organisations, projects, policies, strategies, systems, societies (and more). Evaluation determines whether these have been successful (or are on the way to success). Or credible, valuable, useful, significant, or worthwhile …. Or have been contributing to change — including to the transformations humanity needs now to get us out of the crises in which we find ourselves.
Evaluative actions have been part of humanity’s lives for millennia. The first recorded example refers to personnel evaluation in China around 2200 BC. Its modern version was established fewer than 100 years ago; it now has tens of thousands of practitioners around the world.
Evaluation is a beautiful practice.
Of course, I base this conviction on my own sense of beauty. The nature of ‘beauty’ has been the focus of many philosophical discourses over millennia, so you may want to challenge me. Use another description if you wish — perhaps “useful”, or “noble”. Or even the opposite — useless, or destructive, if your experience with it has been negative. But for me, it is in its deepest essence beautiful.
I cherish beauty wherever I find it, and not only in what we can see. Poetry can be beautiful. Or music, a mind, the truth (sort of) — even an idea. It is important that we nurture beauty, and strive to create it, while remembering that some well-known philosophers have argued that things are beautiful only in relation to the uses for which they are intended, or when properly applied.
In his delightful book, A Beautiful Question, Frank Wilczek, winner of the Nobel Prize in physics, meditates in 430 pages on whether the world is a successful and beautiful work of art. He answers his central question “Does the world embody beautiful ideas?” with a resounding “yes”. He argues that the innate human appreciation for beauty lies in the deep design of nature itself. He finds beauty in symmetry and geometry; in conceptual purity, mathematical order and harmony; in “All Things are Number”; in complementarity; and in the truths, possibilities and almost-truths of “deep reality”.
Does evaluation contribute to the beauty in the world? Or to the beauty of the world? Can it do so?
My response is that evaluation as practice is intrinsically beautiful — but how it is practiced, frequently not.
The beauty in evaluation
That which is striking and beautiful is not always good, but that which is good is always beautiful. — Ninon de L’Enclos
Why do I consider evaluation as something “beautiful”?
One, it is rooted in something that comes naturally, and that has been there for millennia.
In a thought-provoking analysis Ernest House draws on Daniel Kahneman’s famous book, Thinking, Fast and Slow, to conclude that both the complementary systems 1 and 2 thought processes are fundamentally evaluative; that evaluative thinking constitutes the core of our thought processes; and that professional evaluation can be considered “an institutional embodiment of our thinking processes”.
Evaluating is thus a natural thought process, and a primary strategy by which humans adjust to their world.
As noted by Michael Scriven in his provocative analysis of roadblocks to its recognition and revolution, Evaluation may well have existed for several million years in the form of “a simple commonsense-level practice, a proto-applied science”, long before the emergence of the species Homo sapiens.
Two, it is in its very essence aimed at doing good in the world.
Evaluation is not done to report or to understand (only). It is done to trigger positive action. We use it to make incremental advances or, where possible, to help transform the lives of others (often those most disadvantaged and vulnerable), heal ecosystems, and pull the planet out of distress. We do this by supporting the agency of people and societies to help themselves and the world around them. This helps both them and us to live skillfully and mindfully in the modern world.
In his book, Evaluation Foundations Revisited, Thomas Schwandt provides a fine list of some of the most valuable purposes of evaluation: It determines programme effectiveness. It demonstrates return on investment. It guides resource allocation. It improves ongoing programme operations. It assists in creating a climate of critical reflection. It informs the citizenry. It demonstrates agency, accountability and transparency. It provides information needed by decision-makers.
We can add that it can also direct planning and strategy, support advocacy, and contribute to our pool of useful knowledge about issues of importance.
Perhaps most importantly, it has the charge to strive to “speak truth to power” whenever feasible and desirable.
Such a profession can add a lot of value to the world. It carries great responsibility. It also gives life meaning.
And it has to be done well.
Three, in the long term, it will be too fundamental to ignore.
As Schwandt notes, evaluation is “a professional practice committed to promoting the public good through judging the value of human activity.” It is also still a highly contested arena, with great diversity, inherent tensions, and challenges by other disciplines.
Scriven argues that evaluation is a busy, constantly evolving trans-discipline and, despite persistent denial of this fact by other disciplines, it is actually deeply involved or embedded in all disciplines. It is an essential tool in the toolbox of every other discipline. It just still has to be recognised as such.
Four, it compels us to have multiple perspectives on a matter, and to integrate — acknowledging the inherent complexity yet underlying orderliness of the world.
We are moving from almost 500 years of dominance of Cartesian-Newtonian science to an era of holism and a systems perspective on life. Although much has been lost over past centuries, much has been gained; perhaps one had to precede the other to bring us to this important juncture.
But now evaluation professionals have to be systems thinkers. A good evaluator will view matters from multiple perspectives, respect the ecological interdependence of things, and tell a story that connects what is being evaluated with the wider picture.
Good evaluation searches for good practices and patterns rather than “best practices”, and integrates wherever this makes sense. It analyses and judges in context, and with respect for context. It connects the “micro” with the “macro”, and recognises order within the apparent chaos.
Five, it forces us to try to understand and deal with the “real world”.
Evaluators are immersed in the real world. Real lives are at stake, and when societies and ecosystems are vulnerable or fragile, the stakes are high. Evaluators have to understand past and current realities, and for good advice, do their best to predict what reality may look like in future.
Plato mused that everyday life is a shadow of reality. Even though I do not interpret this as Plato has done, I like to believe that evaluation is one aspect of the “adventures of the mind” that can bring us a step closer to the essence of reality, which is “clearer and more beautiful than its shadow”.
Six, it recognises the value of complementarity and balance.
Wilczek embraces the “wisdom of complementarity”. In the East, this wisdom has been reflected for thousands of years in the fundamental principle of yin and yang. In many respects, good evaluation honours this wisdom. Sometimes we observe opposing or contradictory forces in an organisation or system as “tensions”. Sometimes we should rather view them as matters of balance and complementarity.
Good evaluation considers positive and negative influences on interventions. It looks out for positive and negative impacts. It strives towards objectivity while recognising subjectivity. It uses and integrates quantitative and qualitative designs and methods. It values independence as well as engagement.
Evaluators also strive for balance — in their judgments, and in what defines or directs their assessments. They balance the need for precision and rigour with what is practical and feasible. They have been steered by phases of reductionism, and are now being enriched by integration and holism.
Even these are complementary, and beautiful when seen in balance.
And finally, evaluation professionals have to engage with some of the most positive aspects of life, and with some of the most distressing.
Seven, evaluation engages some of our deepest, most valued capabilities as humans.
Evaluation is an art and a science. Schwandt points out that evaluators — like historians, archaeologists, investigative journalists, scientists and detectives — have to display respect for multiple types of evidence, be persistent in seeking it out, and take care in weighing it.
Evaluation requires technical acumen. It also demands critical thinking and reasoned judgment which need human interpretation that is based not only on evidence, but on intuition, deep understanding, lived experience (perhaps transferred over centuries), consideration of values, engagement, responsiveness, a certain measure of empathy, and more.
Studies argue that there is a good chance that complex problem solving, social and systems skills will be far more in demand in future than physical abilities and content skills. Futurists estimate that there will be major job losses over the next two decades, driven by disruptive technologies and increasing automation. Who knows what will happen? But despite Google’s DeepMind AlphaGo, it remains unlikely that the evaluation of complex matters will be done by machines — at least in the near to medium term future.
In the end, all that matters is that we should strive to display, and enhance, the promise of beauty that evaluation holds — for the benefit of humankind, and of everything around us on our beautiful planet.
What is just, also always agrees with truth, knowledge, and — in the deepest understanding of the term — it is beautiful. — Plato