Probability Basics - Explained
Introduction to Probability
Breaking into Data Science without a degree in a quantitative discipline can be intimidating at times. Finance, Marketing, and Management classes I’ve taken as part of the BBA program in Finance and Investment help me understand another essential component of the Data Science task s — the business logic. However, the curriculum of similar programs often lacks programming, math, and statistics classes, all of which are necessary to take a Data Science project from starting to the ending point. The article focuses on statistics and, specifically, attempts to explain the basics of one of its core components — probability.
The logical question to start with is “What is probability?”. Simply put — probability tells us how likely is something to happen after many trials, that is, in a long run.
Coin tossing is a typical example to explain the concept of probability. There are two sides to most coins — heads and tails. Therefore, if you toss a coin there are high chances that it will land either on heads or tails. To be fair, there is a minor chance it could land on its edge but we’ll omit that possibility for the sake of simplicity.
Heads and tails in this example are called the outcomes. Additionally, the set of all the possible outcomes is the sample space. In the coin-tossing example, the sample space is, you know it, heads and tails. If you were to look at a set of possible outcomes, but not all of them, you would be looking at an event. Since we established that the sample space of the coin-tossing example has only two outcomes, it is not the best example to explain the event concept. However, you can think of it as looking at the probability of getting heads, for example, while putting all the other possible outcomes in another bucket, i.e. not getting heads. These outcomes are mutually exclusive, meaning that only one of the possible outcomes can occur at any one trial. That is, if you toss a coin you can get either heads or tails, but not both at the same time.
As mentioned previously, probability tells us how likely is a specific outcome to happen after many attempts. Since we can get either heads or tails when tossing a coin, the probability of getting either one is roughly 50%. However, that doesn’t mean that when you toss a coin you will get heads and the next time it will be tails. Although that is a possibility, what probability tells you is that if you toss a coin 1,000, 100,000, 1,000,000, or more times, you will get heads approximately 50% of the time and tails approximately 50% of the time.
We can run a simple simulation to test this idea. Moreover, we don’t need any fancy software as a simple Excel spreadsheet with a couple of basic formula do just fine. In this simulation, we will toss our virtual coin 1,000 times and observe what percentage of the tosses land on heads and tails.
To start, we create a table for our 1,000 tosses. Then, we say that if the outcome of a toss is 1 we will count that as heads. Otherwise, it’s tails. Excel’s RANDBETWEEN formula helps us here as we can use it to simulate the outcome of a coin toss by placing the =RANDBETWEEN(0,1) formula in cell C5 and copying it down all the way to cell C1004. Next, we count the number of heads out of the 1,000 tosses in cell F5 by using the following formula: =SUMIF(C5:C1003,1)/COUNT(C5:C1003). Tails frequency is, then, simply =1-F5. As we can see below, our hypothesis that out of 1,000 trials, we are likely to get heads roughly 50% of the time and tails 50% of the time proves correct.
The exact number of heads or tails we get out of 1,000 tosses is unknown. Hence, this number is also called a random variable. Random variables come in two flavors — discrete random variables and continuous random variables. The sample space of the former can only be specified values. For example, each time we tossed that coin we could either get heads or tails. In our simulation, the values were 1 and 0. These are discrete random variables. The latter can take on a continuum of values. It is natural for me to name a stock price as an example since they can take on an indefinite amount of numerical values.
The last concept this introductory article will touch on is a probability distribution. As is the case with random variables, we look at the probability distributions of discrete and continuous variables separately. In a nutshell, the probability distribution lists all the possible outcomes and the probability of each of them happening over many trials. The probabilities of all the outcomes sum to 1 (or 100%).
We’ve previously established that coin tossing has only heads or tails as its sample space. Each outcome will occur roughly half of the time if we were to toss a fair coin many times. The probability distribution below depicts just that. Furthermore, it sums to 1.
But what if we wanted to toss two coins and are curious about the probability of getting both heads at least once? Or that the first coin lands on heads and the second one on tails? To tackle these types of questions we look into a cumulative probability distribution. Also known as a C.D.F, cumulative probability distribution tells us the probability that the random variable is less than or equal to a particular value.
Going back to our questions, when tossing two coins our sample space consists of four possible outcomes:
- Both coins land on heads
- First coin lands on heads — Second coin lands on tails
- First coin lands on tails — Second coin lands on heads
- Both coins land on tails
The probability of each of these outcomes happening individually over a large number of trials is about 0.25, or 25%. As previously stated, the sum of these probabilities is 1, or 100%. Therefore, if we wanted to know the probability of getting either both heads or first heads and then tails from our coin toss, we would add the probabilities of these individual outcomes. Hence, the probability of getting both heads or first heads followed by tails is 0.25 + 0.25 = 0.5 or 50%. We can expect this outcome to happen roughly 50% of the time when tossing a coin many times. Since C.D.F is the sum of the probabilities of the individual outcomes, it adds up to 1 as well.
Finally, the probability distribution of a continuous random variable shows how likely the variable is to be less than or equal to a specific value. To visualize this, we go back to our imaginary stock. Since stock prices can take theoretically an infinite number of values, we pick a few and observe the probability of a stock trading at or below the specified value.
Let’s say that our stock has been trading over the last few weeks in the $105-$115 range and has dropped to $100 only once. Based on this information we assign arbitrary probabilities of 0.2 and 0.7 that the stock will be trading at $100 and $110, respectively, one week from today. Our example further reveals that there is a 0.5 probability of our imaginary stock trading between $100 and $100 a week from today.
Whether you are just getting started in Data Science, are preparing for an interview, or studying for a Statistics exam, the article serves as a basic probability refresher or a brief introduction to the topic. If you’d like to explore it further, Introduction to Econometrics by James H. Stock and Mark W. Watson is a great resource.
I hope you found the article helpful and if so, stick around for the upcoming ones covering similar topics.
Works Cited:
Edition, B. (2017). Introduction to Econometrics 3rd custom edition for Baruch. In Introduction to Econometrics, Third Custom Edition for Baruch College (3rd ed., pp. 14–19). Pearson Education.
