#8cantwait is Based on Faulty Data Science
Public Letter: #8cantwait is Based on Faulty Data Science
By Cherrell Brown and Philip V. McHarris
The founders of Campaign Zero released a set of policy demands this week titled, #8cantwait, based on a study the group conducted four years ago, in 2016. The policies put forth by 8cantwait.org are as followed: Ban chokeholds and strangleholds; Require de-escalation; Require warning before shooting; Exhaust all other means before shooting; Duty to intervene and stop excessive force by other officers; Ban shooting at moving vehicles; Require use-of-force continuum; Require comprehensive reporting each time an officer uses forces or threatens to do so.
In this piece, we first overview the irreconcilable issues to the data and study design which render all of the statistics and claims baseless. We then discuss cases of people who were killed by police in ways that were already banned by department policies that are consistent with the #8cantwait policies.
The campaign argues that if 8 policy shifts are made at the city level, it can reduce police killings by 72%. But the data and study design do not support that staggering statistic put forth in the least bit. Given the implications of policy surrounding police violence can mean the difference between life and death, the implications of using faulty data science and statistical analysis to make claims as large as these are irresponsible and may serve as an out for leaders and politicians looking for alternatives to more transformative demands.
The #8cantwait campaign seeks to further legitimize police at a time where the nation is witnessing the fundamental illegitimacy of policing in this country. One of the most harmful effects of the campaign is that it re-legitimizes the police by implying that these reforms can drastically reduce police killings and reform departments when people are pushing for transformative change that will truly end police violence.
We request #8cantwait be recalled. If the 8cantwait.org campaign decides to keep their campaign up, we are requesting:
1) The 72% claim and specific statistics surrounding the potential effects of individual policies that accompany the bar charts are removed.
2) A shift in language to specify along the lines of “These demands may have the possibility to reduce police violence, but the extent to which is largely unknown.
3) A public letter and addition to the site communicating that in the spirit of integrity the statistics have been removed as the data and models cannot verify the claims.
4) An addition to the platform that the campaign stands with the calls of organizers around the country to defund police and invest in community infrastructure and alternative emergency response models.
Campaign Zero has updated the #8cantwait platform site to reflect the 4 requests since the publishing of this article.
Irreconcilable Issues to Data and Study Design
There’s a number of issues with the data used and overall study design. The 72% number is baked into how the model is designed. Given that very few cities have implemented all of the policies, the model is extrapolating beyond the actual data in creating stylized estimates. The study makes a massive claim — that the policies will reduce police killings by 72% — despite only being based on 18 months of data and only 91 police departments. That is not enough of a time frame to even approach any real semblance of causality.
There are also many factors that can contribute to variation within and across cities, which can result in skewed data based on selection effects.
The study design only includes 7 basic control variables (control variables attempt to ensure that one variable is in fact correlated to another, and not actually a result of other variables), when studies with grave implications on life require a much more rigorous approach and design. Moreover, given policies such as these are often shaped by state-level factors, not including state-level fixed effects in the model (which try to account for patterns at the state level) also leads to the faultiness of the study’s design and baseless statistics.
Then there’s the issue of confidence intervals, which refer to the certainty that a claim being made from a study is true and not just by chance. The study uses a 95% confidence interval, but when it surrounds life and death, that is a careless and unacceptable threshold. 99% is the confidence interval often used in medical sciences, and at that level of confidence, the policies studied by #8cantwait would fail to be statistically significant at reducing police killings.
The most reliable predictors of police killings in the study were race and arrest rates, which were significant at the 99% and 99.9% confidence interval, respectively. Shifting away from the focus of policing towards alternatives, as well as mass decriminalization, will avoid encounters — and opportunities — for police violence altogether.
The data timeframe — 2015–2016 — was also after protest ignited in Ferguson — and spread across the country — which could have been an exogenous factor that affected different municipalities and in a variety of ways.
The study only includes data on police killings, which does not take into account the range of other kinds of police violence, yet is framed as a “solution” to police violence more broadly. Especially given that routine ‘less lethal’ force goes unreported, there is not enough evidence to suggest that the police killings track neatly to other use of force, and may vary given how departments implement use of force documentation.
Many departments have already implemented these policies and it has not reduced the likelihood of police violence. The Minneapolis Police Department implemented at least half of these reforms, and George Floyd was still murdered. San Francisco adopted every single one of the policies put forth by the #8cantwait, but received an F in their related Police Scorecard. Herein lies the issue: police violence does not match neatly to statistics in the ways that campaign zero puts forth in these 8 demands.
The use of statistics is largely a matter of interpretation. When people invoke data and statistics it can serve as a veneer of empirical proof that renders something difficult to critique. Police also use statistics and interpret them in a way to justify their actions.
If framed as potential policies that may potentially have an *undetermined* impact on police killings specifically, there may not be as much harm. But to frame it as the urgent demands that can reduce police killings by up to 72% is not only based on a faulty model of analysis, but is harmful and obstructs in moving closer to a world free of police violence.
The demands are framed essentially as a solution to police violence, and gives leaders an easy way out of implementing transformative demands that shift power and resources away from the police.
The Human Impact of Failed Reforms
Police departments across the country have already implemented many of the demands in the ‘8 can’t wait’ platform. LAPD has 5 out of 8, Chicago has 6 out of 8, Cleveland has 7 out of 8, Philadelphia has 7 out of 8, Washington D.C. has 7 out of 8, and San Francisco has 8 out of 8.
Minneapolis has a ‘duty to intervene policy’, meaning officers are required to intervene when other police officers are using excessive force. Derek Chauvin knelt on George Floyd’s neck for almost 9 minutes while two other officers further restrained Floyd and a fourth officer prevented bystanders from intervening.
On March 13 of this year, Louisville police, executing a no-knock warrant, broke into the apartment of unarmed Breonna Taylor using a battering ram, and fired several shots, striking her at least eight times. Louisville has a ‘warning before shooting’ policy, and is required to exhaust all means before shooting. Breonna died on the scene.
The NYPD banned chokeholds back in 1993 after a number of people died while being apprehended or in police custody. Eric Garner was murdered in 2014, after being placed in a chokehold by Officer Daniel Panteleo.
Oklahoma currently requires 2 hours of de-escalation training a year, and is number 3 out of 50 states for police involved killings. Civil unrest erupted in Baltimore in 2015 when several officers were found complicit in the death of Freddie Gray. Like Minneapolis, they had a ‘duty to intervene policy’ in place.
Charlotte has a ‘warn before shooting’ policy, but what of victims of police violence who are hearing impaired? A 29-year-old deaf man in Charlotte, North Carolina, was shot and killed by a state trooper in 2016. This man, Daniel Harris, reportedly was caught speeding but did not hear the sirens of the police car requesting he pullover.
Tamir Rice was killed in 2014 by Cleveland Police officer Timothy Loehmann. Cleveland PD already had a ‘warning before shooting’ in order, and no such warning was given. Officer Loehmann was fired 2 years later, but note solely because of the shooting.
We could list a dozen more examples of police killing Black people, despite having reforms in place.
Now isn’t the time to bet on reforms that haven’t protected Black people. Power concedes nothing without a demand, and we can go bigger. Defunding the police isn’t a pipe dream.
Los Angeles announced that it’s a proposed budget increase away from LAPD and into Black communities. Minneapolis organizers have pushed the Minneapolis public school system and their Parks & Recreation division to end contracts with the Minneapolis Police Department. This week Minneapolis city council began talks to dismantle the police department. We can not risk losing momentum with ineffectual demands that are left to the whims of police to actually enforce — especially since we’ve already tried many of them already.
Organizations like the Movement 4 Black Lives, Dignity & Power Now, Black Youth Project 100, and Black Lives Matter network are echoing the demand to ‘defund the police’- even climate organizations like 350.org have joined in the chorus pushing towards more transformative demands.
We request #8cantwait be recalled. At a bare minimum, if the campaign decides to keep 8cantwait.org up we request that the campaign acknowledges the irreconcilable issues with the data and study design and redact the claim that these policies will reduce police killings by 72% — as well as all the statistics that appear with the specific policies and bar charts on the #8cantwait site as the data and study design do not support the claim.
Sign on here
