What does debt advice achieve? A look at our latest report, ‘Measuring Client Outcomes’
By Josie Warner, Research Insight and Planning Officer
Today we’ve released our latest report ‘Measuring Client Outcomes’. I’ve been working closely on this pilot for the past two years (to the day!), and the importance for running this project still resonates as strongly today as it did on day one.
This is something completely new for StepChange Debt Charity. The project is essentially a study that looks at what happens to our clients up to 15 months after their first debt advice session.
‘Measuring Client Outcomes’ is focused on what happens to clients after debt advice across several areas of their financial situation, taking into account circumstances such as wellbeing and further susceptibility to income shocks.
So why does this matter?
Ultimately, this helps us gain a better understanding of our clients. We can analyse these results across many different types of client information which we gather at the advice stage, such as demographics, types of debt and recommended solutions. For this first stage we’ve focused on ‘vulnerability’ and ‘client budget’.
By gaining a more rounded understanding of our clients, we’ll be able to have a better idea of what actually happens to them and why and how we as a charity can more effectively support them both through our service and our campaigning and influencing work.
We obviously want to better support all clients towards becoming less vulnerable to debt in the future, and we particularly want to find out more about those who don’t progress on to managed solutions such as DMPs.
What we have done with this project?
We set this project up completely from scratch. With support from researchers at the University of Bristol, we developed an analysis framework and questionnaires through which we’d be able to gain and analyse results about how clients are progressing across a number of measures such as wellbeing, financial resilience and progress with their debts.
To gather the data, we sent an email invitation to a random sample of clients, asking them to take part in an online survey. We initially emailed clients at three months after advice.
Those who completed the first survey were invited to take part in the nine month survey and the process was repeated for the 15 month survey.
This isn’t to say the methodology didn’t have its challenges: for example, we typically find older and more engaged clients were more likely to take part.
However, it proved to be the most accessible and effective way for a large enough number of our clients to take part. This in turn provided us with a reasonably reliable set of results to analyse.
Over time this data will continue to build to create a clearer picture, though we still have work to do to ensure that we’re able to survey some of our harder to reach client groups, either through this methodology or supplementing this research with other methodology types.
Ultimately, it’s important for us to know about the outcomes of many different client types.
Here’s what we found
The results in the report are based on the results of 719 clients who completed at least the first three month survey. Some of these clients also went on to complete the nine and 15 month surveys, but the samples for these surveys at the moment is small due to usual drop off in survey completion rates, so we’ve chosen to focus so far on the three month findings.
As mentioned previously, we’ll be able to conduct more meaningful analysis when we’ve conducted more surveys gaining data from greater volumes of longer term clients.
1. Clients with positive budgets do better
This won’t be news to those who’ve read the report, but positive budget clients (who have a monthly income already exceeding monthly expenditure) have better outcomes overall than negative budget clients (who have more money going out than coming in).
This was expected, however it’s an incredibly strong marker in terms of whether clients are doing better or worse.
We’re very hard pressed to find a measure where negative budget clients are doing better than positive budget clients.
2. Vulnerable clients aren’t always doing worse
I know I’m probably not alone in thinking that ‘vulnerable’ clients (with an additional vulnerability to their financial situation) do worse financially, and for good reason. There’s a lot of evidence even in our own data sets which supports this notion.
Last year, our report ‘Breaking the Link’ highlighted that our vulnerable clients are often more likely to be behind on household bills, have stretched budgets, and are often simply at the sharpest end of financial difficulty.
However, the data from the three month survey suggests that this isn’t always the case.
For example, at three months after debt advice, vulnerable clients were more likely than clients with no recorded vulnerability to report progress on their debt problems. This is particularly the case if they had a positive budget, with 28% of this group saying their debt problem was completely sorted out just three months after debt advice.
While this is positive, it does raise more questions than answers for vulnerability and starts to highlight the complexities that lie underneath, particularly when cross-tabulated against budget. This is something we’ll be able to understand more about in the next stages of analysis.
3. Better progress with debts is linked to better wellbeing
At three months, we found that clients who reported better progress with debts also reported higher wellbeing scores, often in line with UK Office for National Statistics (ONS) averages.
What next?
We’re continuing to run these surveys at regular intervals to build a more detailed picture of what happens to different groups of clients after debt advice.
This will enable us to better understand the needs of our clients and continuously improve our service, support and solutions for those in problem debt.
We’ll also use this research externally to open a dialogue with creditors, charities and other organisations about what these results mean in a wider context of public policy. And we’ll use this to support our influencing work to ensure those in or at risk of financial difficulty are properly supported.
In terms of the project itself, the next steps will be to conduct more in-depth analysis from questions raised in this report.
The ability to track, compare and cross-tabulate different measures has already provided us with a huge amount of insight across this project. There’s a lot to do, but the exciting part is that our analysis so far has only scratched the surface.
You can download our full findings and reflections for the Measuring Client Outcomes pilot project from our website.
