The Policy Labs We Urgently Need
When it comes to evidence-based policymaking, states are out ahead of the feds. But these efforts to turn data into insights should be expanded.
By Beth Simone Noveck and Anirudh Dinesh
Cross-posted from Governing, December 13, 2017. Original article here.
Following a bipartisan call to “improve the evidence available for making decisions about government programs and policies,” Congress is poised to pass legislation to establish secure infrastructure to accelerate statistical uses of federal administrative data while also ensuring stringent privacy protections for that data. But some states are already out ahead, establishing offices known as data labs or policy labs to enable them to partner with academia and make use of their administrative data to evaluate and improve programs and policies.
By providing the technical infrastructure and governance mechanisms to help governments gain access to much-needed analytical talent, these data labs (along with those being established by local governments) are helping to convert data into insights and driving more evidence-based policymaking and service delivery. Urgently needed, however, are more such labs with the capacity to turn the data that states collect into improvements in policies and services. And existing labs need to be expanded to enable nonprofits to evaluate their own offerings using state data.
These labs are small groups of data analysts working inside or in tandem with government agencies to make administrative data more usable for policy evaluation while safeguarding personal privacy. And while these initiatives vary widely in their implementation, some states — including California, Illinois, Pennsylvania and Rhode Island — have achieved measurable successes by facilitating collaborations between government and academia.
The California Policy Lab (CPL), for example, is a university-government partnership, created this year and based at UCLA and the University of California at Berkeley, that aims to help cities, counties and the state improve public programs through empirical research, program evaluations and technical assistance provided by the universities. In a recent study, CPL evaluated California’s Enhanced Drug and Contraband Interdiction Program (EDCIP) and found that while the most intensive version of the program had led to a 23 percent decline in random-drug-test failure rates, it also showed a notable increase in inmate misconduct driven by drug-related rule violations. The findings led to Gov. Jerry Brown scrapping the $15 million program in favor of an increased number of sniffer dogs to uncover drugs in prisons.
In Chicago, the Center for Child Welfare Data helps multiple state governments create secure, longitudinal databases to make better use of their own data to help children. For example, the center helped Tennessee’s Department of Children’s Services build the infrastructure needed to monitor the experience of children who were found to be victims of child abuse or neglect and placed in foster care after the state was sued for its management of the foster-care program.
Some of these labs have a long history. Washington’s legislature set up the Washington State Institute for Public Policy back in 1983 to help policymakers make more evidence-based decisions. The institute’s cost-benefit evaluation tool to analyze public programs became so popular that it has been replicated in 25 other states as part of the Pew-MacArthur Results First Initiative.
Yet another interesting model of a data lab comes from the United Kingdom, where the Justice Data Lab (JDL), a facility in the Ministry of Justice staffed by only four full-time employees, was established to allow external social-services providers (NGOs and nonprofits) to evaluate the impact of their interventions by leveraging the administrative data held by the ministry. The Justice Data Lab model works by allowing only JDL staff to access and analyze personally identified data, sharing only the results of the evaluations, and not the data, with those outside the ministry.
In every case, the goal of a data lab is identical: to improve public services by facilitating analysis of programs. But they vary in how they operate. In collaboration with U.K.-based New Philanthropy Capital, the Governance Lab at New York University has developed a series of case studies to better our understanding of the existing landscape of data labs, to learn from their successes and drawbacks, and to help inform those who might be interested in setting up their own labs. In particular, the case studies look at the governance processes employed to share or access data and how these labs solicit expertise to enable program evaluation. The goal of the case studies is to provide data owners, particularly those that hold personally identifiable data, with a blueprint to securely, responsibly and ethically share that information.
The case studies show that several factors need to be taken into consideration when standing up a data lab. These include defining what services the lab will offer (such as evaluation, data cleaning, integration and establishment of knowledge-sharing networks); deciding whether a lab should be established within government or outside it; and determining whether it will serve only government or allow social-sector nonprofits and NGOs to avail themselves of its services. The studies also show that although technology is vital, the real linchpin for a successful lab is its ability to mobilize the human capital and talent needed to generate insights from the data.
“Often the people most interested in research do not have the relevant data,” said Evan White, executive director at CPL-Berkeley, “and the people charged with stewarding the data do not have the resources to pursue research.” The data labs play an important role in lowering the barrier to entry for individuals who want to be engaged in using administrative data for improving public programs and tackling difficult problems. In an era of limited resources and public skepticism, we can’t have too much of that.
