Analysing and interpreting data for UK Poverty 2023
This week, JRF is publishing its annual UK Poverty report. This is designed to offer a wide range of insights into levels and experiences of poverty across the UK as we enter 2023. As in previous years, the latest official statistics on poverty are at the heart of this report. These are from the Department of Work and Pensions’ Family Resource Survey (FRS), and the Households Below Average Income (HBAI) dataset generated from this. The most recent FRS data was collected between April 2020 and March 2021.
This means that the latest FRS and HBAI data available cover the first year of the Covid-19 pandemic which impacted data collection and what we can and can’t say from the data. The DWP has highlighted several concerns with the data and published a more limited set of statistics than in previous years. This, and the change in the social and economic environment since data was collected, has added new challenges to how we analyse and report findings in this year’s report.
Nonetheless, the HBAI data is the best source of information on low-income families’ experiences during the first year of the pandemic. It also offers insights into their prospects for the years ahead. Although results from HBAI analysis need to be treated with more caution than in previous years, we believe there is value in reporting these — with additional caveats where necessary — along with findings from other relevant data.
Data collection faced new challenges in the Covid-19 pandemic
When Covid-19 hit the UK, FRS face-to-face interviews — like all in person activities — had to be suspended. These were rapidly replaced with telephone interviews to continue data collection at a time of much economic and social disruption. it is a great credit to the Government Statistical Service that data collection continued during this period, but the speed of this change meant that it was not possible to test how this — or the difficult situations faced by households during the pandemic — might affect the quality of the data collected.
The DWP has set out in detail their assessment of the impact of the pandemic on the HBAI data. They outline how the achieved sample for the FRS (the basis for the HBAI statistics) was smaller and less representatives than in previous years. FRS surveys before the Covid-19 pandemic collected data from 19,000–20,000 households. This fell to just over 10,000 households in 2020/21. Response rates were particularly low in the first half of the year, so different approaches were introduced in the second half of the year in some nations. This included ‘Knock to Nudge’, with interviewers following up in person (when COVID restrictions allowed) for sampled households where their telephone number was not known to try to arrange a telephone interview. Although these appear to have increased response rates, it is not clear what (if any) additional biases these introduced to the data.
We do know that the 2020/21 achieved sample was older, more affluent and families were less likely to have children than in previous years. Working-age respondents were more likely to hold a university degree than the wider population, and the DWP believes that changes in the estimates of the number of disabled people were driven, at least in part, by sample bias (rather than purely changes in the population as a whole). Although the undercounting of benefits is a known problem in survey research, the DWP also found inconsistent undercounting of the number of households receiving Universal Credit across the survey year.
Because of their concerns, the DWP did not publish additional breakdowns from HBAI data. They also advise that results, particularly for people with specific characteristics, should be interpreted with caution. This includes results from analysis of some of the groups most likely to experience poverty (including specific groups of children and lone parent families). Because of concerns about national and regional sub-samples, the devolved nations did not use HBAI data in their own poverty statistics for 2018–2020, nor did JRF’s Poverty in Scotland 2022 report.
Both the FRS and HBAI datasets are available for analysis through the UK Data Service. They include survey weights based on an updated grossing regime that weights responses by interview month (to balance the sample across the year) and by respondents’ level education (along with their age, gender and region as in previous years). This helps to make the sample more representative, but it does not overcome all data quality concerns, particularly when carrying out analysis of sub-groups of respondents.
The economic situation matters; average incomes fell in the first year of the Covid-19 pandemic, and the UK has faced many additional challenges since then
The broader economic context also has implications for the way we interpret and talk about findings in this report. Average incomes fell in the pandemic, meaning that the relative poverty line used to identify households in poverty in this report also fell. Therefore, it is possible that a household is classed as being in poverty in 2019/20 but not in 2020/21 not because their income increased but because the relative poverty line decreased.
Although the first year of the pandemic was a year of great economic turmoil, much has happened since then. The HBAI data therefore only offers a snapshot of poverty during one particular year when the UK faced many different challenges than it does today. Although much of the economy shut down during the initial Covid lockdowns, additional government support was introduced to try to mitigate the worst economic impacts. The £20 uplift to Universal Credit that offered a lifeline to many low-income families in the pandemic has since been removed. But as inflation has increased to record highs, and wages and benefits have failed to keep up with spiralling prices for energy, food and other essentials, low-income families face new financial pressures without this.
This is why, where appropriate, statistics from the HBAI data have been supplemented with findings from more recent data, as well as corroborating data from the same time period. This includes findings from JRF’s own Cost of living tracker that collected data from low-income households in three surveys fielded at 6-month intervals since October 2021. Although the data from this tracker is not directly comparable to that collected in the FRS, it offers additional insights into the increasing financial difficulties faced by low-income families.
