Ministry of Justice Data Lab
The Justice Data Lab is an analytical service run by the Ministry of Justice which uses UK reoffending administrative data to conduct impact evaluations for organizations that provider offender rehabilitation services.
Location: England, United Kingdom
Summary
The Justice Data Lab (JDL) was set up in 2013 by the Ministry of Justice (MoJ) and gives organizations working with offenders access to re-offending data. The information provided helps organizations to assess the impact of their work on reducing re-offending; provides evidence to funders and clients; and develops the evidence base of effective rehabilitation. The case for the JDL was presented to the MoJ in 2012 by New Philanthropy Capital (NPC) who surveyed criminal justice charities to understand the difficulties in evaluating reoffending. The vast majority found the process of accessing offending data difficult. NPC proposed the creation of a JDL to improve access to re-offending data, particularly for charities, and have worked with the MoJ and an expert panel since to make this a reality. The Justice Data Lab was launched as a pilot in 2013, and confirmed as a permanent service in 2015.
The JDL makes valuable administrative data available to those outside government who do not have the skills, nor capacity to transform the data into information. This was initially considered as an area of need for the charitable sector, however, public and private sector organizations have equally made use of the service. This data is highly personal and sensitive, but the JDL model ensures that personally identifiable data is only queried through internal JDL staff to ensure that privacy is safeguarded. By leveraging the administrative data available in the Police National Computer (PNC) and other linked datasets, the JDL evaluates the impact of a service using up to fourteen measures of reoffending, dependent on the nature and sample size of the intervention. Core measures provided for every analysis are reoffending rates, frequency of re-offenses and length of time to a re-offense.
To use the JDL, an organization needs to submit information on a minimum of 60 individuals who were provided their service, and details of their program. A team of statisticians calculate the reoffending statistics of this group, and use statistical matching techniques to create a comparison group who share similar characteristics. The JDL staff then perform comparative analysis of the reoffending rates of the two groups which is written into a ‘plain English’ report that is published on the MoJ’s website.
The JDL started with two full time statisticians, moving to four in 2014. Between April 2013 and October 2017, 167 analyses for social enterprises, public, and private sector organizations in England and Wales were produced.
Website: www.gov.uk/government/publications/justice-data-lab
Sector: Criminal Justice
Target Audience: NGOs and nonprofits providing offender services in the UK. Utilised by all organisations providing offender rehabilitation services.
Services offered: Impact Evaluation
Data Sources: The Police National Computer- the administrative data system used by all police forces in England and Wales. Employment and benefit history from the Department for Works and Pension (DWP) and HM Revenue and Customs. Offender risk and needs data from Offender Assessment System (OASys)
Who Houses the Data: UK Ministry of Justice
Addressing Data Quality Issues: The JDL team continually work to improve the methodology and quality of data by regularly inviting peer review¹. At the piloting stage of the JDL, services targeted at high risk offenders such as individuals with mental health issues, or drug/alcohol addiction could not be evaluated as the reoffending data did not flag individuals which had these conditions. If evaluated the calculated comparison groups would likely underestimate reoffending as they would be drawn from the general offending population rather than a high risk population. Over the years, in response to an NPC co-ordinated user feedback and expert panel, MoJ have integrated additional datasets to the service, meaning that high risk groups can now be evaluated. However not all offenders can be evaluated, young offenders under 14 are not evaluated due to the population size being small, and sexual offenders are not evaluated due to having a different offending pattern. The fact that the JDL team have actively engaged with users and potential users of the service has enabled the JDL to develop in response to customer need.
Addressing Data Governance and Oversight Mechanisms: In order to match the organization’s cohort of individuals to administrative data sets, the JDL requires access to individual names, dates of birth, gender and start/end date of intervention, at the very least. Since this raises many legitimate questions of privacy-related risks, the JDL was required by law to undergo a privacy impact assessment to show that it was in compliance with the UK’s Data Protection Act and the European Convention of Human Rights. The report’s objective was to identify any data protection risks and to propose solutions to mitigate them. It details the procedures and security mechanisms in place to handle data requests and transfers of data to MoJ.
The solution the JDL adopted to ensure data privacy and confidentiality is to ensure that personal data does not leave the MoJ. The JDL team are MoJ staff who receive requests from NGOs or other JDL customers to carry out an evaluation, but do not share personal data with the customers.
Analytical talent: 4 full time MoJ data analysts.
Methodology: The JDL caters to a specific set of customers- organizations who provide offender rehabilitation services and want to demonstrate impact in terms of re-offending. NGO/charities are the biggest user of the service, but just over half of evaluated interventions were drawn from public and private sector organizations.
The customer organization submits offender and intervention data about a cohort of individuals who received their services using the JDL’s data upload template. A minimum of 60 individuals must be submitted to enable statistical testing to be conducted. The JDL analysts use this information to calculate reoffending statistics of the treated group and use Propensity Score Matching to create a statistically matched comparison group. The re-offending rates of the two groups are compared and a report of the analysis is provided to the customer. The methodology is described in detail in a publicly available report.
Disseminating Findings: The results from the analysis are freely published on the MoJ’s website. No personal data is disclosed. To aid interpretation of the results, the JDL recommends that the report should be used in addition to other evaluations the customer carries out and it can not be used to compare with either the national average for reoffending rates or with that of other organizations due to different offender profiles that are evaluated.
Recently, the MoJ has geographically mapped impact reports. Alongside individual reports, and aggregated sector results, researchers and organizations have a variety of formats in which to make use of the data.
Successes/ Impact: The JDL has evaluated nearly 170 impact evaluations in 4 years. It has won high profile awards² due to its customer focused service and use of statistics. Due to the success of the model, the original pilot was extended and finally established as a permanent service in 2015.
Sample Project:
The Prisoner’s Education Trust (PET) is an organization that provides grants to offenders in prisons to pursue long distance education programs or to purchase materials for arts projects. In January 2014, PET approached the JDL with a request to evaluate the impact of their services on 195 individuals. The JDL staff was able to find matches for 90% of the eligible individuals to the police national computer and approximately half of them had identifiable custodial sentences in the period that PET group received the intervention. When they compared the reoffending rates for the two groups, the data lab found that 21% of the individuals who received PET’s grant reoffended while that number was 32% for the matched control group. The difference in the rates, as per the analysis, was statistically significant and helped prove PET’s grant made a quantifiable difference to the group who received the intervention. PET used this data to secure additional funding, and used the JDL in 2015 to undertake further sub-group analysis.
Funding resources: The service is funded by the UK Ministry of Justice
Link to website: www.gov.uk/government/publications/justice-data-lab
Contact details: MoJ: justice.datalab@justice.gsi.gov.uk or contact NPC to discuss their work in developing the JDL and other impact data labs in employment, education and health. Tracey.Gyateng@thinknpc.org
We’d like to thank Tris Lumley, Director of Innovation and Development, New Philanthropy Capital and Tracey Gyateng, Data Labs Project Manager, New Philanthropy Capital for their assistance.
Footnotes:
¹ See “A peer review of existing methodology- response”for latest peer review [as of November 2017]. Published by UK Ministry of Justice, February 2016.
² For example, The Royal Statistical Society’s Statistical Excellence in Official Statistics
Additional Resource: Another case study on the Justice Data Lab can be found here.
