5 Minutes -

The Reverend Thomas Bayes’ An Essay towards solving a Problem in the Doctrine of Chances was published several years after his death. While this essay contained the components and inspiration for what is now known as Bayes Theorem, it was arguably not the intent or, at least, the emphasis of Bayes’ essay.

While Richard Price was responsible for publishing Bayes’ work, it was Pierre-Simon Laplace whose efforts eventually created the theorem that bears Bayes’ name. Most historians believe that Laplace’s work was quite independent of Bayes’ essay. Some even dispute that Bayes was the first.

So what does the theorem state?

Bayes Theorem measures inductive reasoning. Assuming you know (or can estimate) the probability of a specific event, Bayes theorem allows you to understand the impact of new knowledge on that probability.

The probability of any event A given the new information obtained from event B — p(A|B) is equal to the original probility of event A — p(A) multiplied by the probability of B assuming A is true p(B|A) divided by the probability that B would be true p(B).

For example, suppose that you had four coins in a can; a penny, a nickel, a dime, and a quarter. Suppose now that a random coin is taken from the can and hidden from your view. What are the odds that it is the quarter? Answer — 25%. This is event A.

Consider now that another coin is randomly drawn from the can. It is the penny. It is also event B. So what are the new odds that the first coin was a quarter? Simple math tells us 33%.

Bayes tells us to calculate this by taking the original odds for A — 25% and multiplying them by the odds of B assuming that A is true (the quarter). It is important to understand that event B is not B is a penny. Event B is B is not the quarter. Therefore p(B|A) is 100%. Finally, we divide by the probability of B (that B was not the quarter) which is 75%. So p(A|B) = 25% * 100% / 75% or as noted earlier 33%.

If another coin was drawn from the can, this being both event C and a dime, we would could quickly determine p(A) now 33%, p(B|A) once again 100%, and p(B) not 67% to learn that p(A|B) is now 50%. Very intuitive when dealing with a problem so simple, far less so when dealing with more complex scenarios.

Usage and Implications

Bayes theorem is applicable to all manner of inductive reasoning and probability determination. It is used in determining odds during games of chance like poker. It is used to build things like spam filters for your email. IT is applicable to nearly any matter of uncertainty where new facts are gained in an iterative fashion.

Bayesian Reasoning is crucial to testing and medical diagnosis. It is used in forensic studies and law enforcement. It is even used in the evaluation of major league baseball players, election forecasting, and meterology. This makes it one of the most critical fields of study in the area of probability and statistics.

If you have more time to devote to this topic, consider some of the following options.

5 Hours -

Khan Academy offers roughly five hours of online videos dedicated to probability and statistics. While Bayes is the focus of several, this resource will provide you a wealth of background and material that touches all of the basics and many more complex subjects as well.

5 Days -

Two excellent books that detail many of the uses and implications of Bayes Theorem can be found via the following links. The Theory That Would Not Die tells a broader history of Bayes, the theorem, and its impact on history. While The Signal and the Noise focuses more on the science of forecasting and how Bayes Reasoning impacts the world of prediction.

5 Weeks-

If you are looking for more and have the time to spare, consider enrolling in one of a myriad of courses dedicated to Bayesian Statistics. Courses are available online and from universities like MIT, Stanford, the University of Texas, and more…

Good luck and good learning! If you found this article helpful, please recommend it ♥.