How far are our charity partners in their data journey?
An overview of DataKind UK’s projects with charities in 2020
By Antonio Campello, Chapter Lead of the Scoping and Impact Committee at DataKind UK
This is the first of two blogs about how we assess our charity partners’ data maturity. You can now read the second one, Where are our charity partners in their data journeys?, which includes examples of how engagement with DataKind UK’s programmes can positively influence the data maturity of the organisations we work with.
DataKind UK engages with social change organisations (SCOs) to transform their impact through the use of data and data science. DataKind UK’s Scoping and Impact Committee (one of its five volunteer-led committees) has been working to systematically measure the impact of these programmes. In particular, over the last two years, we have reformulated the way we assess projects to give us quantitative insights into the data maturity of our charity partners before and after their work with us. Today we’re sharing the result, our Journey to Impact 2020 report, describing the way we scope, analyse, and record the impact of projects.
This is the first of two posts where we will share some insights on the types of organisations that engage with DataKind UK and summarise some of the findings in the Journey to Impact report.
The data maturity framework
To measure whether our programmes (DataDive and DataCorps) help charities to make better use of data, we need to have a way of quantifying how far along a charity is in terms of their data journey. Thankfully, the Data Maturity Framework, developed by DataKind UK in partnership with Data Orchard in 2017, fills this gap. It allows us to classify charities into a five-point scale of data maturity (0–1: Unaware, 1–2: Emerging, 2–3: Learning, 3–4: Developing and 4–5: Mastering), on seven key themes:
- Uses: Purposes for collecting and analysing data, range, and benefits of data usage.
- Data: Quality, collection, sources, and data assets.
- Analysis: Types of analyses, techniques used, quality of reports, and means of communication.
- Leadership: Attitude, investment, and plans for data development.
- Culture: Team engagement, data sharing, and governance.
- Tools: Data infrastructure, storage, and quality of tools.
- Skills: Internal capacity, skills, training, and access to knowledge.
Based on the framework, we developed a methodology to estimate the data maturity journey of organisations before and after working with us. This is achieved through a call between two Scoping Committee members and the SCO’s representatives. Each committee member asks questions that cover each theme, noting key points and eventually coming to a quantitative score. Having two assessors is crucial given the subjectivity of the topic. The committee members then debrief the rest of the committee, and discuss any areas of particular interest or concern. Although this method is not error-free, it typically allows us to have a good understanding of the SCO’s data maturity.
It is worth noting that we do not use numerical scores to decide whether a charity is suitable for a DataDive or a DataCorps. Instead, we use qualitative insights to inform the way we scope and advise further actions for the projects. Numerical scores are used solely to gain an aggregated insight of the sector and programmes.
Here is what we found by applying our methodology in 2020:
Most of the charities assessed in 2020 are developing their data maturity
Most charities that engaged with DataKind UK’s core programmes in 2020 fell into a ‘developing’ level of data maturity (between 3–4 on a 0–5 scale). It is perhaps not surprising that organisations that seek data science support already have a head start in comparison to what we believe to be the average, lower data maturity for the sector. According to the Data Evolution Report from 2017, “for those (organisations) at a more advanced ‘developing’ stage, they need help accessing and developing advanced skills and embedding good data practice across the whole organisation”, which would be an ideal outcome of a DataDive.
Most of the charities are in the Developing stage of data maturity. Each point of the graph represents an organisation, and error bars represent the difference in the score given by the two assessors.
There is a skills gap among the charities we assessed
All of the organisations assessed in 2020 had Skills as one of their lowest scores. No organisation assessed had an in-house, full-time data scientist, even the ones that scored the highest in this theme. Training is often not widely available to staff, or restricted to on-boarding, induction, or compliance. It is not uncommon for many data functions to be the responsibility of a single employee, or delegated to service providers or volunteers.
Tools and uses are common strengths
On the other side of the spectrum, the charities that applied for DataKind UK programmes in 2020 were usually developing a good set of tools for data storage and maintenance. Most of them employ a centralised CRM system with reporting capabilities and use third-party tools to collect data online. More sophisticated software (e.g. Python, GIS, R) were much less common. All organisations fell, at least, into the emerging stage of Uses, recording data about all of their clients and starting to collect appropriate data in order to measure their service’s quality.
What else?
Clearly, assessing how far organisations are in their data journey is just the first step to understanding how we can generate impact. We have also started collecting data on how we can influence data maturity improvement through our programmes. More information on that can be found in the full Journey to Impact 2020 report, and will be summarised in a forthcoming blog post.
