A Measure Theoretic Approach to Probability (Part 1)
With applications ranging as far as economics, quantum mechanics, biology, and even politics, probability is arguably one of the most important branches of mathematics. However, the generally taught, and widely accepted, version of probability misses out on some incredibly satisfying intuition. In this article, we tap into this intuition. And in order to do that, we turn to measure theory. For the novice reader, measure theory, simply put, is the study of sizes.
Before we begin, it is important to define a few things.
σ-algebras
At the core of measure theory lies the concept of σ-algebras. And this concept is incredibly useful in abstracting the idea of ‘size’. But what do we mean by ‘size’ in the mathematical sense? Naturally, we know that size must obey the following intuitive property: if we break an object apart, the sizes of the pieces must sum up to the size of the object itself.
Now, before we discuss the sizes of objects and their parts, we need a way of representing these objects mathematically. As is very convenient, we represent objects in the form of sets. For instance, the set P could describe a ball of clay.
Now, we need a way of describing how the object can be broken into pieces. This notion of ‘breaking an object into pieces’ is described by a σ-algebra over P.
A formal definition of a σ-algebra is the following:
Given a set P and a collection of subsets P’, the collection P’ is a σ-algebra over P if the following conditions are satisfied:
Intuitively, each element of the σ-algebra represents a ‘piece’ of the object. The first criterion in the definition above establishes a ‘null piece’ (a piece of zero size) can be considered a piece of the object. The second criterion establishes the fact that if we break off a piece, A, of the object, then the remaining object (the complement of A) is also a valid piece. Finally, the third criterion establishes the fact that if we glue a set of pieces together, the resultant piece is also a piece of the object.
In fact, with these three axioms, it also follows that σ-algebras are not only closed under set unions but also under set intersections, set differences and symmetric differences. Mathematically, this means:
Another way of intuitively understanding σ-algebras is to think of a set F as a plate that is broken. The σ-algebra, ℱ, describes all the ways that the plate can fall apart along its cracks. The figure below illustrates this:
In this figure, the large oval at the top is a ceramic plate with cracks along its surface. Let the points on the plate be the set F. Above, we illustrate every subset B ⊆ F ⊆ ℱ. You can think of each B as representing the plate with some combination of the chunks removed. We are only allowed to remove a chunk if it is surrounded by cracks. The entire set of configurations of ways we can remove chunks of the plate is the σ-algebra ℱ over F.
Measure spaces
Now that we’ve satisfied the intuitive property of breaking objects down, we are in a position to quantify the ‘size’. And for this, we turn to measure spaces. We first define a measure in the following way:
Given a set P and a σ-algebra over P, denoted P’, the function
is a measure if the following axioms are satisfied:
And here, a measure is synonymous with size. So, the first criterion essentially says that measure cannot be negative. Second criterion says that the measure of nothing is 0. And finally, the measure of two different parts are simply the sums of their measures.
We can illustrate this by extending the example of the ceramic plate from above. Here, F is the object and ℱ is the σ-algebra over F.
Now, we combine everything we’ve covered to define a new idea: measure space. A measure space is simply a combination of a set (the object), a σ-algebra (the parts of the objects), and a measure (the ‘sizes’ of the parts of the object). A rigorous definition follows.
Lastly, we define a measurable space as the following:
A measurable space is a pair consisting of a set and a σ-algebra. The word ‘measurable’ in measurable space alludes to the fact that it is capable of being equipped with a measure. Once equipped with a measure, it forms a complete measure space.
Probability space
How does this relate to probability? Well, we can adapt the definition of a measure space to a probability space. A probability space is defined in the following way:
After having understood the fundamentals of measure theory, it seems that our new definition of probability has a far more intuitive way of being defined. The set Ω is the sample space, or the set of all possible outcomes. The σ-algebra over Ω is a way of partitioning all possible outcomes into events, a familiar concept. And the probability measure P is simply a way of assigning a value to the probability of an event occurring — another familiar concept. And, since some event in the sample space is certain to occur, P(Ω)=1.
With this, we have built the foundation for more advanced probabilistic concepts being developed into measure-theoretic concepts. In part two, we will delve into discrete and continuous probability distributions and how measure theory can be used to unify the two concepts.
Thank you for reading and I hope you have a great day!
