Chapter 2 — Probability Theory

This chapter assumes that you can handle basic combinatorics questions. If you need to brush up your knowledge on Combinatorics, then please refer to previous chapter https://medium.com/@sausingh0110/chapter-1-combinatorics-2f92270b19d6

Overview

This chapter will provide basic introduction to theory of probability needed to get started. We will look at some imaginary but realistic (what an irony !!!) problems to develop our understading of the world of probability.

Probability terminology is not alien to most of us. There are so many examples from real life where we use this term, for example:

1. When we check our weather app, it tells us the probability of rain or sunny weather during different hours of the day.
2. During black friday, you might wonder as what’s the probability that a given store would run out of door buster deals within an hour of store open.
3. During political elections, we wonder what’s the probability of a certain candiate to win the elections

Essentially, calculating probability of an event to happen is like calculating the chances of that event to happen. In real world, every event (like tossing coin, rolling dice, playing online card games etc.) has an element of randomness associated with it. Theory of probability empowers us to create mathematical models to predict the chances of event happening (or not happening) under different constraints.

Sample Space, Events and Experiment

Do not get worried by the terms if you are not familiar with them. These terms are related to each other and are actually quite simple to understand. Understanding these three terms conceptually will also help in creating bedrock foundation for theory of probability.

Tossing a coin, rolling a dice, guessing the price of a house, guessing the date for new season of your favorite show on Netflix or predicting the ranking of a document on Google search are all nothing but Experiments in the world of probability theory. There is randomness attached with outcome of each of these experiments. However in most of the cases it is easy to determine seat of all possible outcomes on experiment. For example:

• Tossing coin: Possible outcomes are {H, T}. Here H is for head and T is for tail
• Rolling a dice: Possible outcomes are {1,2,3,4,5,6}
• Price of a house: Possible outcomes are all positive numbers {0 to ∞}
• Date for next season of your favorite show on Netflix: Possible outcomes are all dates from today to ∞.
• Ranking of a document in a search result: Possible outcomes are 0 to N, where N is total number of documents indexed by Google.

All these sets of possible outcomes is called Sample Space in the world of probability theory.

Now let’s pick dice roll as an experiment for consideration here to dig deeper. Let’s say we have two players (A and B) competing against each other. A wins if dice rolls to an even number (2,4 or 6) else B wins. Now getting the even number on dice roll is considered as an Event. So an event can represent a part of sample space or an entire sample space depending on how we define an event. We could also define a new even by combining two or more different events using union (∪) or intersection (∩) or a combination of both operations. For example if event E is about getting an even number of dice roll and even F is the getting a prime number of dice roll then E∩F will give us {2}. And E∪F gives us {2,3,4,5,6}.

For the majority of rest of this post we will be trying to understand theory of probability using events and sample space as the basic building blocks.

Probability of Events

Before we begin let’s take a quick look at the notations used ahead

• S: A sample space would be represented as letter S
• P(E): Represents the probability of E to happen.

Let’s take a simple example of tossing a coin to understand this.

Sample space for coin toss event is defined as {H, T}. So when a coin is tossed (Event E) the possible outcomes are H or T.

Hence,

P(H) = no_of_arrangements_for_head_on_toss/size_of_sample_space = 1/2

P(T) = no_of_arrangements_for_tail_on_toss/size_of_sample_space = 1/2

Also, probability of an event E defined as outcome of the toss as head or tail (basically any outcome belonging to sample space) will be,

no_of_arrangements_for_a_head_or_tail_on_toss/size_of_sample_space

= 2/2 = 1

We can also generalize this observation and conclude following:

1. 0≤P(E)≤1
2. P(S) = 1

Following handwritten note dives deep in to how to derive probability of more than one event for an expirement.

Conditional Probability

Conditional probability helps in finding the answers to the questions like, “If event B has occurred then whats the probability that event A will occur?”. Typically A and B are related. However, if A and B are not related then probability of A can (should) be computed independently of occurrence of B. Conditional probability of A given B has occurred is represented as P(A|B).

Let’s take a concrete example to understand this better. Consider we are rolling a dice twice.

B: Event that first roll results in 3

A: Event that sum of two dice rolls is even.

If the first event (B) happens then A can be defined as : {(3,1), (3,3), (3,5)}

Given B the sample space is : {(3,1), (3,2), (3,3), (3,4), (3,5), (3,6)}

Given this, P(A|B) is 3/6. As can be seen above, we are actually looking at fraction of sub-sample space when computing conditional probability. In a formal manner the conditional probability can be defined as

P(A|B) = P(AB)/P(B). Here P(AB) is the probability of event thats sits at intersection of events A and B.

Bayes’ Law

Bayes’ law is best understood when you have a good understanding of conditional property (previous section). Bayes’ law describe the probability of an event in context of certains event conditions. Below is a handwritten note to explain Bayes’ law along with a related example.

What’s Next?

In the next article we will be covering Random Variables :) ?

Prev