So what are we going to do about it?!
Week 3 at Parkinson’s UK: getting stuck in my data maturity survey, and carving my way out again.
Last week I wrote about using the concept of equipoise in research: if you know the answer, don’t ask the question. *Looks sternly at people using spurious surveys as PR fodder*.
(As a re-cap, dear reader, my survey is about Parkinson’s UK data maturity: how data are understood, used, and analysed; the infrastructure and tools that are used; and the organisational culture and leadership vision for data. Fit that in a nutshell, on indeed in a survey!)
The theme of this week has been ‘and so what?! What are you going to do about it?’ I had — blissfully — somehow forgotten how long surveys take, once you start piloting and gathering multiple rounds of feedback. So the past week has been the week of the survey quagmire and survey carving.
The scalpels I have taken to the survey looked something like this:
What will I do with the findings?
Whilst also a theme of week two, this has been a renewed theme: if I won’t use a survey question’s findings to either understand overall data maturity, or to filter for specific teams or groups of people… then I just don’t ask it.
Questions that have fallen to the wayside have included some rather lovely questions that ask staff where they think data has the most potential to help Parkinson’s UK i.e. in finding a cure, or in raising awareness, or in raising extra income.
While really interesting, the survey findings were unlikely to influence where the data maturity strategy focuses its energies: that will be my call, and defined with senior leadership to align with each directorates strategic objectives. I won’t focus on, say, income generation just because it’s the most referenced theme in the survey.
Do I know what good looks like?
A key question when interpreting survey findings is what we should expect a finding to be. Often you’ll be told that 30% of x group do a thing… and it’s hard to know if that is a good or a bad thing.
This has been a strong question especially around skills: in an organisation as large and varied as Parkinson’s UK, what should I be aiming for in terms of individual’s data maturity and skills. If I don’t know the answer to that question and if the measure isn’t a useful baseline for the next time we do this survey, then the question needs to go!
Is this a survey question or an interview question?
This has been fun thing to decide, and has been the area where I have been able to slash and burn the most questions.
The Data Evolution survey that was my starting point was based on being on the outside peering in, and some questions are very top-line. However, now that I am on the inside of the organisation, I can just ask people, meaning I can focus more on what people can speak to, given their position and role. For example, a local adviser is unlikely to be able to answer a question as to whether data sharing is encouraged across the organisation. They will, however, be able to answer questions around how they perceive the culture, and their personal skills and data needs.
I’ve been having special fun with questions that will be both a interview question for senior leaders(e.g. has data been used to increase income) and attitudinal questions in the survey (e.g. are staff aware that data has been used to increase income?).
Partial scalpel: should I be asking everyone or a few people this question?
Parkinson’s UK staff who don’t have particular data, insights or analytics responsibilities, and who are not senior managers or above, will be happy to know that the survey is shorter for them. For the rest, yup, there will be questions about A/B testing and predictive analytics. Sorry.
Is there an element of behaviour change here?
In the pilot survey I asked people to define data before giving them my definition. Receiving 15-ish slightly different answers was enough to convince me that I have no interest in qualitatively coding 500 answers to that question. Or would even know how. Instead, I have opted for a pub quiz multiple choice warm-up question that will maybe get people thinking… and maybe even modifying their behaviour.
Initially my scale for this questions was from ‘data breach!’ to ‘it’s fine!’. But giving that too many answers were, really, it depends, AND to hammer the behaviour change idea home, I’ve now opted for this unusual scale:
- No-one does that, right!?
- I’ve never done this
- Right… that’s not ok, is it?!
- Is this a problem?!
- This is ok!
Be in touch!
The focus for this year’s data strategy will be to work with individual directors to understand what a 12 month plan and 2020 target should be, and where there are now. The ‘where we are now’ scoring will be part survey results, and part in-depth interviews with each of them.
Is anyone else doing this kind of work in house? Might be good to swap tips (and questionnaires, once my results are in!)