Getting started with principles focused evaluation
In yesterday’s post we talked about the WHY of using principles focused evaluation (PFE) in our work as a platform and community of doers who are passionate about Dudley becoming a more welcoming, kinder town based on caring connections between its people, the places they love and the planet we all rely upon.
In this post we wanted to work out loud about the steps we have taken in our journey of developing and testing out this new approach. The depth of reflections and thinking during this development stage is key to the utility of PFE in improving the value of the work we do as a team.
Our journey with PFE has started with open and heartfelt team conversations about our personal values. We asked ourselves:
“What truly matters to you in this work?”
We then went on to refine our own principles by synthesising and sense checking them with those expressed in the voices and actions of doers & encouragers. We revisited what 2 years of research and experiments told us about:
“What is it that our doers and encouragers tell is important to them in this work?”
We have reflected upon these values and findings, and then explored how they could come together to form guiding principles in our design and practice as a platform. In clarifying these principles we took time to identify overlaps and counter principles — this step really helped us all with clarifying what is really understood by each principle, and the values we share upon which they are rooted.
A further refinement step involved reflecting upon the overlaps between our own principles and those of fellow travelers whose ways of working inspire us (Ethical Redevelopment, Impact Hub Birmingham, Participatory City, Radical Help, Permaculture).
Prior to exploring our principles in this way our design process had involved thinking through the range of goals we want to prioritise and make possible through this work. The goals were identified by extracting pivotal outcomes from our practice based research from 2 years of experiments, and also from the initial systems mapping of Dudley High Street we had undertaken as a team in 2017–18. We then mapped those goals against the principles that had emerged so we can be more intentional in the way the principles guide us towards those goals.
Putting our principles to work
Our next step in terms of ‘operationalising’ these principles in our work was to make certain they are genuinely effective & actionable to use in practice. In his recent book Michael Quinn Patton advises ensuring they are meaningful by checking they fulfill his G.U.I.D.E criteria.
By GUIDE Patton means: they help with Guidance (priority setting), they have Utility (i.e. actionable), that they are Inspiring and grounded in our ethics, that they are Developmental (i.e. they are applicable to a range of contexts) and Evaluable (i.e. you can document and judge the results).
After testing our 8 principles against this GUIDE criteria we spent time developing questions that will help us evaluate firstly, if /how we are sticking to the principles; and secondly, the consequences for people, the townscape and the natural world of animating them in our work.
As part of this process we undertook an exercise as a team where we theoretically re-ran experiments from the last 2 years that hadn’t worked as well as we had hoped at the time. We wanted to know would the principles have affected our decision making, or the nature of the failed experiments? The answer was a resounding yes and in deeply constructive and critical ways. This brought home the potential value of using the principles in this way to inform lab practice.
In our next steps with PFE we have begun to experiment with the logistics and processes of the evidence capture around:
1. Adhering to the principles in our day to day practice and decision making;
2. Surfacing and testing the patterns between the practice of these principles and our platform goals.
In our post tomorrow we will be sharing our 8 draft principles and the layers of evidence capture planned to support this approach in our work in 2019.
