Prosecutor’s Fallacy

In my Data Science immersive course at Flatiron School, we recently learned about Bayes’ Theorem and its application to predict the probability of an event, based on prior knowledge of conditions that might be related to that event.

The rule is usually read as:

Lady Justice and her best weapon: Bayes’ Theorem

Regrettably, in the criminal justice world, Bayes’ theorem is sometimes exchanged for the Prosecutor’s Fallacy and, in the process, leads to miscarriages of justice. Below, I will discuss a case that has become synonymous with the Prosecutor’s Fallacy.

In 1999, a British woman named Sally Clark was convicted of the murder of her first two children.

Three years earlier, Clark’s first child, Christopher, had died at the age of three months. The child’s death had originally been attributed to Sudden Infant Death Syndrome (SIDS). Then Clark’s second child, Harry, died just over a year later at the age of two months. Harry’s death was deemed suspicious by the forensic pathologist who performed the autopsy. The pathologist then revisited the first child’s death and determined that it too was suspicious. Sally Clark was arrested some weeks later and eventually convicted of the two deaths.

Based on SIDS rates in the UK, the prosecution’s witness argued that the probability of there being two such deaths in the same family was roughly 1 in 73 million.

“You have to multiply 1 in 8,543 times 1 in 8,543…approximately a chance of 1 in 73 million which would happen ‘by chance’ once every 100 years.”
— Prosecution Testimony of Sir Roy Meadow, Pediatrician

There were two mistakes with this thinking:

1. Assumption of Independence

Professor Meadow assumed that the deaths of the two Clark boys were independent. However, risk factors for SIDS are likely to be common to multiple children within a family and one SIDS death is actually known to raise the probability of another in the same family. In Meadow’s assessment, there was no consideration of a possible unknown, hidden cause existing which might make a particular family more vulnerable to SIDS.

Unfortunately, when this mistake was drawn to the attention of the judges at the first Court of Appeal hearing, they dismissed it as “[in]capable of affecting the safety of the convictions.”

The British Medical Journal published an article after the trial referencing other studies which suggested that SIDS deaths are not random events, and that “recurrence” of SIDS in the same family would be much more frequent than 1 in 73 million live births. Recurrence might even occur in England about once every 18 months, rather than once every 100 years.

2. The Prosecutor’s Fallacy

The second mistake arising from Professor Meadow’s evidence as to probabilities is known as the Prosecutor’s Fallacy.

This consists of showing that the “innocent” explanation for certain facts is highly improbable – and then deducing that the “guilty” explanation is therefore the correct one.

So if H is Sally Clark is Guilty and E is the Evidence of her two sons’ deaths, the prosecutor’s fallacy hastily concludes that:

P(H|E) = P(E|H)

While Bayes’ Theorem states:

By equating P(H|E) with P(E|H), the prosecution completely ignored the prior (the probability of Sally’s guilt). But, we need the prior probability in order to nail down the posterior probability. So we need to ask: How often do mothers murder their first two children within their first year of life?

Unfortunately for Sally Clark, this was not taken into account and she was sentenced to two life sentences.

In 2002, a Mathematics professor attempted to accurately compare the chances of the two possible explanations: Two successive deaths in the same family, both by SIDS; or a double homicide. He concluded that successive accidents are between 4.5 and 9 times more likely than are successive murders, so it was statistically more rare for a mother to kill both her children.

In 2003, evidence emerged that the forensic pathologist who had examined both babies had incompetently failed to disclose microbiological reports suggesting the second son had died of natural causes. Clark’s conviction was subsequently overturned. She was released from prison having served more than three years of her sentence. Clark died a couple of years later.