Review: The Art of Statistics, David Spiegelhalter

Page numbers cited are from this version. I have used Amazon links for convenience, not for commission.

In brief

An excellent guide for gaining a better intuition for the statistical concepts in use in the real world today, with great concrete examples. Big emphasis on process and where things go wrong in practice. An essential read for practitioners to help us better articulate how and why we use statistics. Should feel readable to a general interest or beginner audience but a better place to start might be Naked Statistics which has a more pop science feel.

Extended ramblings

In my work in business analytics I find myself constantly having to come back to topics like the problem of HARKing or the difference between statistical significance and statistical power. They are concepts that enable the work I do but when it comes to explaining these simply, I would often fumble for the right words and question my own understanding of them. I would revisit the material and feel confident again for a while, but they are tricky to articulate well without regular rehearsal. What makes statistical decision-making work is unintuitive, but it does work, and that is what makes it so rewarding to learn and teach.

The Art of Statistics is a general primer on statistics, starting with mean, mode, medium and how distributions work. It is written for the general reader with no prior knowledge of statistics and its emphasis is on the practice of statistics in the real world. It covers a lot of ground, and touches on concepts such as the Bonferroni correction and ROC curves which seem ambitious for a general audience but are explained very well in simple language. As a practitioner with a constant need to educate and motivate non-specialists in my workplace (and myself!), I devoured this and found it a very useful reference for good explanations and examples.

I had a few a-ha moments. The first was around the idea of ‘population’. It has never sat easy with me that even in situations when we seem to have all the data, we would still consider that data to be a sample of a population. But the ‘population’ doesn’t need to actually exist in the world for it to be applied in statistics and he explains this beautifully, acknowledging that this is a more difficult idea than textbooks typically make out.

To use the example in the book, consider the survival rates of all children across a number of hospitals; even though we have collected all the data from all the operations performed, statistically we would still treat these as samples when determining whether there are significant differences in survival rates between the hospitals.

But what is the study population? We have data on all the children and all the hospitals, and so there is no larger group from which they have been sampled. Although the idea of a population is introduced rather casually into statistics courses, this example shows it’s a tricky and sophisticated idea that is worth exploring in some detail. — p.91

He goes on to list three kinds of populations and in particular the concept of a metaphorical population (in contrast to a literal population), which he describes as an imaginary space of possibilities that survival rates (or murder rates in a city or test results in a classroom) are drawn from. Essentially, it is what our dataset would look like if we had an infinity of data points.

Personally, I rather like acting as if all that occurs around us is the result of some random pick from all the possible things that could happen. It is up to us whether we choose to believe it is truly chance, whether it is the will of a god or gods, or any other theory of causation: it makes no difference to the mathematics. This is just one of the mind-stretching requirements for learning from data. — p.93

To touch on some other personal highlights:

• I found I got a better intuition for the prosecutor’s fallacy (the difference between ‘if the accused is innocent, there is only a 1 in a billion chance that they would match the DNA found’ and ‘given the DNA evidence, there is only a 1 in a billion chance that the accused is innocent’ — p.216)
• This had the best real-life example and explanation of a poisson distribution I’ve come across (frequency of daily recorded homicide incidents in the UK — p.223–226). It’s amazing really what the poisson distribution predicts with so few inputs and he goes onto reflect on the astonishing predictability of overall patterns made up of individually unpredictable events.
• I found super compelling the distinction between aleatory and epistemic uncertainty and their philosophical links to frequentist and Bayesian statistics respectively — p.240).

Again, the book’s focus is on statistics in practice and it was engaging to unpack questions such as ‘How many trees are there in the world?’, or ‘Is prayer effective?’ with the use of statistical techniques. Each of the chapters are supported by questions like these to guide the thinking and it was clarifying to not only see the suggestions on approach but also where it can go wrong in practice (how relatable too).

One difficulty with the practice of statistics is that there are many subjective choices to be made, which leads him to the admission that ‘the formal basis for learning from data is a bit of a mess[…] it is no wonder that mathematicians tend to dislike teaching statistics. (p.305)’. P-value thresholds are an easy example of this but more fundamentally the very definition of ‘probability’ is still debated. But the differences in these opinions tend not to matter where the final conclusions are concerned:

My personal view is that, while they may well disagree about the fundamentals of their subject, reasonable statisticians will generally come to similar conclusions. The problems that arise in statistical science do not generally come from the philosophy underlying the precise methods that are used. Instead, they are more likely to be due to inadequate design, biased data, inappropriate assumptions and, perhaps most important, poor scientific practice. — p.338

And indeed the last two chapters are dedicated to ‘How things go wrong’ and ‘How we can do statistics better’. There is a lot of discussion generally right now about the replication crisis, scientific malpractice and media misrepresentation of research and these closing sections double down on the usefulness of statistics and analyse how the different systems involved in statistical research could be improved, including a practical list of ten questions to ask when confronted by a claim backed by statistical evidence.

Finally, a delicious little footnote about research findings that are communicated in the popular press:

I sometimes follow what could be called the ‘Groucho principle’, after Groucho Marx’s paradoxical claim that he would never join a club that would have him as a member. Because the stories have gone through so many filters that encourage distortion and selection, the very fact that I am hearing a claim based on statistics is reason to disbelieve it. — p.358

Further reading

• A better place to start for non-practitioners with a general interest in statistics might be Naked Statistics. It’s an entertaining read with great stories.
• A better entry point for actually starting to learn how to apply statistics would be Statistics Without Tears. Or pick a course online.
• There is a lot of discussion right now on scientific malpractice and miscommunication. For more on this subject, try Calling Bullshit, a publicly available lecture series which was recently turned into a book. The first lecture starts with: ‘There is SO. MUCH. BULLSHIT. So much bullshit. So much bullshit everywhere. We’re drowning in it.’
• The author has made all the data from Art of Statistics with accompanying R code available in this public repository.