Should secret code help convict?
Silicon Valley venture capitalist Marc Andreessen famously quipped that “software is eating the world,” taking over functions we used to leave to humans. The criminal justice system is no exception. Predictive policing systems can assign individuals “threat scores,” facial recognition software can cross-reference a suspect’s face against enormous databases, and algorithms can help judges and parole officers perform risk assessments when setting bail, parole, or probation.
Software is also changing evidence. But history offers us a cautionary tale for what happens when new evidentiary techniques — like tool mark, fingerprint, and bite mark analysis — aren’t vigorously tested and validated. Computer code is yet another vector for error. And, in at least some cases, defendants should be able to examine it.
Code that convicts
TrueAllele Casework is a proprietary computer program that parses DNA mixtures — samples that include genetic code from more than one person. While analyzing DNA from one person is relatively easy to interpret, these mixtures — often found on a piece clothing, or in a sexual assault kit — can be incredibly difficult for most of today’s labs to scrutinize.
TrueAllele looks to pick up where most forensic labs would leave off. “After DNA evidence is collected, [forensic] analysts convert it into signals that appear as peaks and valleys,” writes Joe Palazzolo in the Wall Street Journal. “Most labs discard signals that don’t rise above certain thresholds, but TrueAllele uses all the data to tease apart a DNA mixture, separating the genetic types of each person in the sample so they can be compared with the DNA of suspects.”
From there, the TrueAllele software implements a form of statistical analysis called probabilistic genotyping to interpret the DNA data. Essentially, algorithms compare the actual DNA data to different statistical models, weighing the probability that the data matches the model. (It does this by examining 100,000 different combinations of possible variables and how well each proposed variable might explain the DNA data.) After all is said and done, if the program determines that the data supports a match between the evidence and the suspect’s DNA, the program helps an analyst to calculate a “likelihood ratio” — that’s when a forensic scientist can say a match between the suspect and the evidence is a trillion times more probable than pure chance or coincidence.
Of course, TrueAllele is pitched to help law enforcement solve previously unsolvable crimes, ranging from homicide, to sexual assault, to burglary. And the public could be assured that the crimes weren’t just solved on speculation, but on thorough, unbiased statistical modeling. That, according to Dr. Mark Perlin — the computer scientist who developed the program — means TrueAllele is “making the world a safer place.” That TrueAllele has been used “in over a hundred cases” in the U.S. and guilty pleas are a common outcome may be evidence of that claim.
The imperfect track record of new forensic techniques
But where new technologies are used to convict, they need to be proven to work. Unfortunately, this has proven difficult. For example, earlier this year, the Justice Department and FBI acknowledged that faulty hair analysis led to flawed testimony in hundreds of trials. And just last month, as Spencer Hsu reported in the Washington Post, a D.C. judge ordered the District government “to pay $13.2 million to Santae A. Tribble, who was jailed for 28 years after being wrongfully convicted … at trial through exaggerated claims about the reliability of FBI forensic hair matches.” A 2009 National Research Council report evaluated the state of forensic science bluntly:
In a number of forensic science disciplines, forensic science professionals have yet to establish either the validity of their approach or the accuracy of their conclusions, and the courts have been utterly ineffective in addressing this problem.
So how, and when, should new evidentiary technologies like TrueAllele be used in the criminal justice system?
The answer to that question hinges admissibility standards of scientific evidence as well as how the technologies actually work.
Take a deep breath. Here we go.
Two major standards govern how evidence from forensic technologies gets into the courtroom — one is from Frye. v. United States, one from Daubert v. Merrell Dow Pharmaceuticals, Inc. For decades, judges relied on a “general acceptance” standard advanced in Frye. Essentially, if a theory or forensic technique was generally accepted amongst scientific experts, then the evidence should be admissible. That changed in 1993 with the Supreme Court’s ruling in Daubert. The Daubert standard provides courts with five factors — testing, peer review and publication, controlling standards, error rate, and general acceptance — to examine in determining the admissibility of evidence. (Daubert is used in federal courts, though some states still rely on Frye.) The common thread between the two standards is general acceptance within the scientific community. But under Daubert, a new evidentiary technique would ideally satisfy multiple, if not most, of the factors.
What do these standards mean for TrueAllele? Well, in many respects the TrueAllele DNA analysis does not differ from standard DNA analysis. It does not alter how the DNA is collected, it doesn’t change the manner in which the DNA is extracted from collected samples, it doesn’t alter the amplification of the extracted DNA, and it doesn’t alter the typing of the amplified DNA. But, as I mentioned, where TrueAllele does differ is what’s done with the genetic DNA: it examines all the data — even if it doesn’t rise above a certain analytical threshold — and its software runs advanced statistical models to interpret all the data.
Each of TrueAllele’s layers — the DNA collection and analysis fundamentals, the statistical modeling, and the implementing software — work in concert to produce the evidence. And we have fairly high confidence in the reliability of two of those layers: DNA analysis and statistics. But we’re far less sure about one layer: the software.
Think of it this way: if you have an orchestra full of credentialed musicians, that orchestra still needs a conductor. But you probably don’t just want any conductor. In order for the orchestra to play an error-free symphony, you’d probably also want a well-trained, credentialed conductor. TrueAllele’s conductor is its software. And in my view, defendants should be able to see that software’s source code.
However, to date, Cybergenetics — the company behind TrueAllele — has refused to release its source code.
And that’s not for lack of trying: defendants have requested access to TrueAllele’s source code in California, Ohio, Pennsylvania, and New York, just to name a few states. The refusal to turn over the source code for examination, as Palazzolo has reported in the Wall Street Journal, leaves defendants, and their attorneys, exasperated: Though the “Sixth Amendment guarantees defendants the right to question witnesses against them . . . they can’t confront TrueAllele . . . if they don’t know the assumptions the software makes about [DNA] sample degradation, for example, or how it isolates real signals from mere noise.” Just last month, a Pennsylvania judge blocked defense lawyers for a man charged with murder from obtaining the source code of TrueAllele. One defense attorney expressed his frustration saying: “It’s like being given a description of what an automobile does without being able to open up the hood and poke around.”
Cybergenetics offers three main arguments for why TrueAllele’s source code shouldn’t be open to inspection by defense attorneys: First, disclosure of TrueAllele’s source code would hurt Cybergenetics’ business prospects because there’s a highly competitive commercial environment for DNA software. Second, where TrueAllele is used in a case against a criminal defendant, Cybergenetics provides “opposing experts the opportunity to review the TrueAllele process, examine results, [ask] questions” and discloses TrueAllele’s “underlying mathematical model.” (Cybergenetics even posts videos on YouTube explaining how the TrueAllele process works!) Third, according to Dr. Perin, “[s]ource code is not used to assess forensic software reliability … Computer accuracy is relevant; software text is not.” Instead, we should rely on the seven peer-reviewed validation studies, proving the empirical fortitude of the underlying techniques. In other words, the proof is in the pudding.
Opening the black box
In my view, these arguments are not entirely persuasive. TrueAllele is a relatively novel, patented way of analyzing DNA mixtures. Yes, it has several internal validation studies. But it has not yet been subjected to a vigorous, thorough, and independent peer review process. Because of that, it should be held to a higher standard than simpler, well-understood techniques.
First, though the commercial market for DNA software might be highly competitive, Cybergenetics’ concerns about hurting its business prospects are overblown. Practically speaking, there are ways for the software to only be disclosed to attorneys and experts working on the case in a discreet and secure fashion. Further, as Noel Erinjeri explained at Fault Lines:
No attorney is ever going to read TrueAllele’s 170,000 lines of source code into a public record. The only person who is going to review that code and make sense of it is the defense expert. Any testimony relating to it would have to be at a level that a jury (or a judge) could understand, which won’t be enough for a competitor to reverse engineer it. A judge could order that a copy be provided to the defense’s expert, and not be disclosed to anyone else.
Second, software is not immune from error. Take the example of STRMix in Australia — another forensic software program that promises to “resolve previously unresolvable mixed DNA profiles,” like TrueAllele. As the Electronic Privacy Information Center notes, an “error in the STRMix code … produced incorrect results in 60 criminal cases” in Australia. To be sure, that error has been downplayed by STRMix as a “minor miscode in an early version … [which required] a particular set of circumstances to ‘fire’ and so occurred very rarely.” Nevertheless, the error shows that software text is relevant in determining a program’s reliability. As Rebecca Wexler observes, coding errors have impacted other forensic technologies, too: “When defense experts identified a bug in breathalyzer software, the Minnesota Supreme Court barred the affected test from evidence in all future trials. Three of the state’s highest justices argued to admit evidence of additional alleged code defects so that defendants could challenge the credibility of future tests.”
Third, while validation studies may help prove an evidentiary technique’s reliability, they should not always be sufficient. That’s especially true for TrueAllele, where nearly half of the handful of peer-reviewed validation studies cited have Dr. Perlin, the program’s main developer, as an author. Experts have testified as much in cases regarding the admissibility of evidence derived from TrueAllele. For example, in a 2014 case — Ohio v. Shaw — experts for the defense noted that TrueAllele is not generally accepted in the scientific community and has not been subjected to a rigorous peer-review process. As one of those experts later told the Times Union, “[m]ost scientists who use [TrueAllele] probably have no idea how it reaches its conclusions.”
The “crucible of cross-examination”
Imagine that a 23-year-old criminal defendant faces murder charges and a potentially lengthy prison sentence. There aren’t witnesses to the crime, so his case mostly turns on a DNA mixture analysis, processed through a proprietary computer program like TrueAllele or STRMix. The computer program says the degraded DNA mixture on a piece of the victim’s clothing matches the defendant’s DNA profile. According to the prosecution, this evidence means he’s guilty beyond a reasonable doubt.
Before he’s convicted and spends decades in prison, is it fair for him, his legal counsel, and their forensic experts to inspect the source code behind the program that says he’s guilty?
That seems fair to me. As Erin Murphy, a professor at New York University School of Law, argues in her recent book Inside the Cell: The Dark Side of Forensic DNA:
Even if the [TrueAllele] software is operated by a trained expert, the source code still should be open and proven able to withstand challenge from disinterested, and experienced, reviewers. That is the only way to ensure that the results produced by a program that, on its face, seems to outperform other software packages in making use of the available information is doing so in a way that does not falsely inculpate suspects … we should be wary of vesting all faith in the accuracy of DNA interpretation in a single individual, or company, with a both a strong financial and personal stake in its own success and a refusal to open it scode to scrutiny even under a protective court order.
William Thompson and Simon Ford similarly argue that when an evidentiary technology company asserts that their procedures involve trade secrets, that company is shielding itself “from scrutiny by the scientific community at large.” In those cases, courts should tell companies developing the evidentiary technologies that “they cannot have it both ways. If [the companies] wish to assert that their procedures are accepted, they must open themselves to scrutiny by the scientific community so that their assertion can be put to the test.”
As the late Justice Scalia wrote, the Confrontation Clause in the Sixth Amendment “commands, not that evidence be reliable, but that reliability be assessed in a particular manner: by testing in the crucible of cross-examination.” More broadly, as Wexler observes, “[s]hort-circuiting defendants’ ability to cross-examine forensic evidence is not only unjust — it paves the way for bad science.”
My main concern is that we might be letting software off the hook here. But allowing defendants, and their experts, to inspect the source code is just one way of ensuring that software is not let off the hook. There are other ways, too. For example, TrueAllele could be subjected to a more rigorous, independently verifiable peer-review process. Its results could be found to be truly reproducible. And for the purposes of judicial expediency and efficiency, that method is probably preferable to granting defendants access to the source code on a case-by-case basis.
However, in the absence of such thorough validation, refusing to reveal the software’s source code presents a problem. We’ve already seen how software that isn’t thoroughly tested can cheat vehicle emissions testing. We shouldn’t let it cheat criminal defendants out of a fair trial.
