Where are our charity partners in their data journeys?
By Antonio Campello, Chapter Lead of the Scoping and Impact Committee at DataKind UK
This is part two in a series of blogs about measuring impact at DataKind UK. Read our first article on how our data maturity framework works and what levels of maturity we saw in our charity partners last year.
A couple of weeks ago, we shared our Journey to Impact report on how DataKind UK scopes, analyses and records the impact of projects, with a companion blog summarising some of the findings of the report, and describing how far typical charities we’ve partnered with are in their data maturity journey.
In this post we share more insights from the impact assessments, and some examples of how engagement with DataKind UK’s programmes — such as DataDives — can positively influence the data maturity of the organisations we work with.
Our approach to project scoping and impact assessment
To understand how we assess the impact of our programmes, it is useful to have a high-level view of each step of the scoping process.
DataDives are six- to eight-week projects culminating in a weekend event, but the preparation starts much earlier, from the time we receive an application from an organisation. They might apply to one of DataKind UK’s programmes via the website; be referred after going to an Office Hours appointment; or be encouraged to apply after discussions with DataKind UK staff.
Prior to a project, the Scoping Committee conducts a data maturity call (described in our first post) and collectively refines the project before passing it on to Data Ambassadors, a team of 3–4 data experts/project managers who will work with the charity to deliver their project.
Three months after the DataDive weekend, one or more representatives of the SCO fill in an impact assessment survey and have a call with the Scoping Committee. The committee wants to capture any organisational changes since the project, for example any immediate impact that the outcomes of the DataDive might have had. However, our main focus is to try and understand whether there is the potential for longer-term impact for our partner organisations, or in other words, whether we are influencing their data maturity. Below is a table of the steps we take for a DataDive project.
How can we influence data maturity?
In 2020, the Scoping Committee reviewed impact assessment notes from previous years to search for evidence of data maturity change. Because the process described above is relatively new and has evolved over time, we found that historically we did not gather enough data for a quantitative analysis of data maturity scores before and after projects. To search for evidence of data maturity change in previous years, we had to take a more qualitative view.
For example, two of our main findings from an analysis of 2018–2019’s project notes were:
1. (Sustainable) data maturity change takes longer than the time window we use for impact assessment
Our final formal impact assessment touchpoint with an SCO takes place 3–4 months after a project. This gives enough time for organisations to digest the findings and start acting on them. However, effective and sustainable improvements to data maturity might only be seen much longer after the impact assessment call. For example, as we keep in touch with our charity partners in the longer term, we will see improvements like recruiting their first data analyst, or improving their data collection tools — but these might be a year or more down the line.
2. Every data maturity journey is unique
Despite the difficulty of assessing this, we saw evidence of improvements in many areas, with each organisation seemingly following distinct paths for data maturity. We give some examples below.
Real-life examples
Here are a few examples of improvements in data maturity that we’ve seen in organisations after their projects with DataKind UK:
Data use: After a DataDive, one of our partner organisations started regularly using a combination of internal and external data to forecast demand for one of their services, which they had never tried to do before.
Data quality: Recommendations about data quality are very common outputs of projects, although on many occasions the organisation is not able to implement them. But when this was possible, it led to significant data quality improvements. For instance, during a DataDive one charity found that the way they record their services on their database was overcomplicated and didn’t support the type of analysis they wanted to do. They have since re-designed their data warehouse using a simplified, analysis-friendly taxonomy.
Culture: A positive change in data culture gets mentioned by almost all charities as an outcome of a DataKind project. Some began presenting data to their broader organisation more often; some began sharing their findings externally; and two charities planned to hold regular “mini-DataDive” sessions internally with their own staff.
Tools: Most organisations are introduced to new tools during a project, however sustainable use means training staff to operate them. Two of the organisations we assessed started using automated dashboarding tools to generate regular reports, and one introduced more sophisticated statistical software (R/Python) to their daily toolbox.
Skills: One organisation was so inspired by a DataDive that they hired two data-related roles, using their learning from the project to specify the skills of the new recruits.
What’s next?
These examples are just a taste of how we can influence data maturity for the better in social change organisations. As we continue to record impact assessment systematically, we hope to be able to share many more examples in the future!
This article was based on the Journey to Impact 2020 report. Read the full document here.
