PART 4: Predictive Policing, and Algorithmic Transparency as Anti-Discrimination
The black box and the broken window: influencing law enforcement mindsets
The likelihood that predictive policing algorithms implicitly reinforce bad habits — rather than correct them — should thus be cause for concern. PredPol’s implementation in Oakland has seen the production of disparate impact with regards to arrests on the count of race. Researchers note that the algorithm’s generated hotspots may validate the biases of police officers, who are encouraged to replicate their mistakes in over-policing black neighborhoods and frequenting the same places they believe to be hot-spots: regardless of the algorithmic filtering[45].
Again, citizens must ask themselves: what data is being synthesized by these algorithms? How is it being used against us as a citizen?
Using opaque and broad streams of data in predictive policing harks back to the “broken window” theory in policing: the idea that enforcing small laws will dissuade citizens from breaking larger laws[46]. The analogy should be understood as such: expanding the scope for the categories of predictions to be made, as well as the data to be transformed, will decrease the potential for crime and create forecasts with higher accuracy.
Given the failures of “broken window” policing — in 5 million stop-and-frisk interrogations since 2002, nine out of ten New Yorkers have been completely innocent, and many of those stopped were minorities — law enforcement should move away from judging individuals in broader contexts. As Cathy O’Neil argues, stop-and-frisk is “built upon a simple and destructive calculation” — a near-repeat effect where stopping large populations of people in select neighborhoods will uncover a few major suspects and many smaller ones. This may satisfy the results-based mindset, primed on the actionable forecasts by the model, but additionally results in a vicious cycle for these select populations — often minorities — who are disproportionately targeted based on bias. This results-based mindset, searching for operative outcomes, continued to conjure problems in areas that didn’t need that much attention, eroding probable cause in favor of satisfying a prediction based on group characteristic[47].
Predictions become self-fulfilling prophecies in this way — by priming officers and judges to look for problems where there were none, based on historically biased data and historically biased attitudes. Officers feel the need to act quickly to ensure maximum safety[48], and when they face the threat of violence, they often respond with violence[49]. These expectations can be heightened by being assigned to an area designated as a hotspot. This results in escalation, not de-escalation. This seduces and reveals subconscious bias; it doesn’t eliminate it.
Transparency, Efficiency versus Fairness, and Rawlsian Distributive Justice
To counter these implicit biases that create controversy in our algorithm, and design a “fairer” algorithm, Cathy O’Neil argues that we should visualize an “efficiency-fairness” spectrum — a choice we are often faced with. Our legal institutions, O’Neil argues, reflect the large efficiency loss incurred to bring about fairness, while predictive policing algorithms favor efficiency and objectivity. The question we must consider is whether or not, in the interest of fairness, we sacrifice some efficiency[50].
But how are we even defining fairness? Is O’Neil’s conception really fair?
Consider the possibility that it isn’t. Richard Berk would probably argue that the dichotomy between optimizing accuracy and fairness that O’Neil has established is a false one. “When it comes to crime,” he says: “sometimes the best answers aren’t the most statistically precise ones. [..] Court systems want technology that intentionally over-predicts the risk that any individual is a crime risk[51].” This is an argument that accuracy is fairness — that society benefits from a larger reliance on the algorithm. Its mystique matters little when the tool improves the sentencing overall over a law-enforcement and judicial environment without the tool. Similarly, the Nozickean libertarian would argue against the necessity of positive rights in this case. “The government’s duty is solely to protect against the infringement of negative rights!” he would argue. “The government has no duty to make these proprietary algorithms more ‘fair’ or more ‘transparent’, because it is not required to provide citizens with ‘opportunity’. Forcing algorithms to be more ‘fair’ per some metric is an intrusion on the right of the citizen to be free and left alone, and impedes its own efforts towards achieving more effective law enforcement.”
This normative claim, in conjunction with Berk’s argument, presents an inverse Blackstone’s formulation in effect — that it is okay to incarcerate Luke Skywalker as a consequence of the immense public good reaped by imprisoning Darth Vader. Bentham would additionally agree with this utilitarian assessment, warning against the sentimentality of the Blackstone formulation in its originality:
“[T]hose sentimental exaggerations […] tend to give crime impunity, under the pretext of insuring the safety of innocence. Public applause has been, so to speak, set up to auction. At first it was said to be better to save several guilty men, than to condemn a single innocent man; others, to make the maxim more striking, fix the number ten; a third made this ten a hundred, and a fourth made it a thousand. All these candidates for the prize of humanity have been outstripped by I know not how many writers, who hold, that, in no case, ought an accused person to be condemned, unless evidence amount to mathematical or absolute certainty. According to this maxim, nobody ought to be punished, lest an innocent man be punished”[52] (198).
Persuasive though this reasoning may be, it falters under the conclusions established about the fallibility of data and algorithms. In a perfect world, such a system might be optimally utilized. We ideally would not fear the possibility of disparate impact, nor would its influence be large. Such a world would allow for the pure ends-oriented objectivity of algorithms and data because no variables that designers use would proxy with historical patterns of discrimination against race and gender. This Nozickean worldview would rightfully promote efficiency as the main objective of an algorithm because there would be no other important factors.
But in an environment like the United States, where the threat of racial discrimination and shadow of disparate impact still looms over us, distributive justice is necessary. A purely efficiency-based algorithm is susceptible to the “results-based” mindset detailed above — inflating the “success” of data collected through biased methodology. Additionally, it is important to consider algorithmic transparency — something that does not merit discussion in the Nozickean worldview. Transparency establishes accountability and allows for a dialogue between users, subjects, and designers on the desired and fairest values to regulate the algorithm.
References
[45] Smith, Jack, IV. “Crime-prediction tool PredPol amplifies racially biased policing, study shows.” Mic. Mic Network Inc., 9 Oct. 2016. <https://mic.com/articles/156286/crime-prediction-tool-pred-pol-only-amplifies-racially-biased-policing-study-shows#.fXQWMae4q>.
[46]Peters, Justin. “Broken Windows Policing Doesn’t Work.” Slate. The Slate Group LLC, 3 Dec. 2014. <http://www.slate.com/articles/news_and_politics/crime/2014/12/broken_windows_policing_doesn_t_work_it_also_may_have_killed_eric_garner.html>.
[47] O’Neil, Cathy. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown Publishing Group (NY), 2016. p. 93–94.
[48] Associated Press. “When are police justified in using deadly force?” Los Angeles Times. Tronc, Inc., 13 July 2016. <http://www.latimes.com/nation/nationnow/la-na-police-deadly-force-20160711-snap-story.html>.
[49] Angwin, Julia, Jeff Larson, Surya Mattu, and Lauren Kirchner. “Machine Bias.” ProPublica. Pro Publica Inc., 23 May 2016. <https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing>.
[50] O’Neil, Cathy. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown Publishing Group (NY), 2016. p. 94–95.
[51] Brustein, Joshua. “This Guy Trains Computers to Find Future Criminals.” Bloomberg Technology. Bloomberg L.P., 18 July 2016. <https://www.bloomberg.com/features/2016-richard-berk-future-crime/>.
[52] Bentham, Jeremy, 1748–1832. A Treatise On Judicial Evidence: Extracted From the Manuscripts of Jeremy Bentham. London: J.W. Paget, 1825. <https://catalog.hathitrust.org/Record/100634192>.
