When statistics don’t tell the whole story!
What exactly is the Simpson Paradox?
It’s the weekend, and you’ve decided to go to a restaurant with a friend for dinner. You don’t know the quality of the restaurants in your area, so you go to one of the restaurant review sites or apps and compare the ratings to decide where to eat supper. Finally, you choose one of the two restaurants listed below. The table below shows the results of these restaurants’ customer satisfaction surveys:
After reading this review, you will most likely select the Blue Restaurant because its satisfaction rate was higher. (Each round of voting had the same number of participants.)
Let’s move away from the restaurant and the weekend dinner to investigate the matter from the perspective of a data scientist. In the first step, we split the survey participants into two categories, “woman” and “men,” and analyze the issue. The following table is now available:
Something interesting is going on! The red restaurant got a higher percentage of satisfaction from both men and women but had lower total satisfaction than the blue restaurant. It’s also worth noting that the total number of respondents polled at each restaurant is 400.
In mathematics, this situation is known as the Simpson paradox. This issue is critical in statistics-mathematics because you have made a mistake just where you believe that you have moved data-driven and selected a choice based on the conclusions gained.
If we take the percentage of satisfaction as a metric of restaurant excellence in the previous example, the red restaurant has a more significant proportion among both men and women; nonetheless, its total satisfaction percentage is lower than the blue restaurant.
When does the Simpson Paradox take place?
The Simpson paradox happens when we split two occurrences into numerous categories, and the weights of the related classes are not the same. This Paradox also occurs when there is a hidden variable in the event that we did not consider when analyzing it mathematically. Simpson’s Paradox demonstrates that simply using data is insufficient and that story behind the facts must be understood in order to make the appropriate conclusion. To put it another way, if we want to be data-driven, we need to have a comprehensive picture of the available data, including how it is acquired, factors influencing the data, hidden variables, and so on. Otherwise, by examining incomplete data, we risk making a conclusion that may lead us wrong.
In the following sections, we will look at some additional real-world cases of the Simpson paradox’s impact in various sectors and become acquainted with its significance. Finally, we take a deeper look at the debate from a mathematical standpoint.
Which is the most effective treatment?
Assume there are two treatments for kidney stones, A and B. We look at these two cures as products and compare their efficiency to determine which one to utilize. For that purpose, we’ve requested our assistant to compile a study comparing the efficacy of these two treatments. The assistant reports the success rate of these two treatments in 350 kidney surgeries.
Is this enough information to make a judgment on which treatment to use? Certainly not! This information is terribly skewed and deceptive. We know that kidney stone surgeries are classified into two types: tiny stones and big stones. It should be mentioned that persons with tiny stones have a greater likelihood of success and are in better overall health. On the other hand, large stones are more complicated and dangerous to surgery on, and persons who have them are often in worse condition. We will now look at the essential information in a classified manner:
The results are amazing! Treatment A, which seemed to be less effective than treatment B at first, proved to be more effective in both small and big stone surgeries. The question now is why method B has a better cumulative success rate than method A. The solution is the same as we indicated at the beginning of the article: the hidden variable and cause that we have not addressed.
Because treatment A is more aggressive than treatment B, it is used for most big stone surgeries. People with massive kidney stones, on the other hand, are in a critical medical state that diminishes their chances of a successful operation regardless of treatment. According to the data table, 234 of the 289 effective treatments in Method B were conducted on persons with tiny kidney stones. As a result, treatment B appears to be a more appropriate treatment approach than treatment A when computing the cumulative percentage.
Now for just a product-marketing question: Considering the success of treatment method A in tiny stone surgery versus treatment method B, which treatment method do you think should be used in similar operations?
Of course, considering the current state of knowledge, this question cannot be answered precisely; however, assuming that treatment B is more expensive and limited and that the physical condition of people with large kidney stones is much worse than that of people with small kidney stones, it is preferable to use treatment B for small kidney stones; because, while the success rate is lower than method A, the chances of success in such operations are high in general, and if the surgeries are not successful, the chances of death are low. As a result, we became acquainted with the Simpson paradox’s impact on medical projects.
An investment of 100,000 Dollars!
During the first phase, your firm has secured a new investor and raised $100,000. Everything appears to be in line until one day a stockholder enters your room and says, sadly:
After investing $100,000, the average purchase of site users has dropped from $200 to $ 180, which is a shame.
You put on a happy front and convince him that everything is OK. Request that the board member calls a public meeting with the other board members to explain why. Because he is unfamiliar with Simpson Paradox, the board member is taken aback by your patience. Instead of just skimming the cover, divide the data table and be ready for the midday session. The prepared data table is as follows:
With these two tables, you enter the meeting. Based on the data gathered, it is apparent that the average purchase of new and existing consumers has grown. Purchases, in general, have also increased dramatically. As a result, this investment was a success. Everyone is happy, and there is no longer any concern for the board members.
In this case, too, we are faced with the Simpson paradox. Before investing, the company’s customers were fewer, and the so-called “company had more shares of a small cake”; however, after the investment, the company’s customers have increased significantly, and the company now “has a smaller share of a big cake,” so the average cumulative purchase per customer has decreased.
It should be noted that, in many cases, when there is a significant increase in customers, the average purchase of shipments may decrease, which is not always a bad thing. This result is because your audience’s personality may change and may not behave the same way as previous customers. Other parameters must be evaluated in this instance.
Digital marketing and click advertising
As a digital marketing manager, you are prepared to launch a marketing campaign. This campaign may be run in two ways:
The user clicks on the advertisement banner in the first approach (one-click method) and is taken to the website page. In the second approach (two-click method), the user sees the banner and clicks on it, first to a middle page with various relevant terms to the ad, and if he clicks on one of the keywords, he is moved to the desired website’s web page.
Considering that the audience has to go through more steps to enter the main page of the website in the two-click method, the initial prediction is that we will have a lower conversion rate; that is, the ratio of those who are redirected to the final website to those who see the ad is reduced; however, we expect that better quality users will be transferred to the website page, and thus the sales per user will increase. As a consequence, the campaign is performed using the A / B test approach, yielding the following results:
As predicted, the early findings show that the two-click approach has more significant sales per user than the one-click method. Is the problem, however, resolved? To better comprehend the issue, we distinguish between users and divide them into two groups: users inside the United States and users outside the United States. As a result, the data table will look like this:
Something incredible is taking place! In one click method, Customers’ average purchase in the United States and overseas is more significant than when using the two-click method. A digital marketing manager must understand this.
The hidden variable in this problem is the variation in user behavior between areas. Americans, on average, have more considerable per capita earnings, which explains why consumerism and shopping are more prevalent among them. This, together with the large disparity between the average purchase of American and non-American users, has resulted in a more significant average purchase of users using the two-click approach when we consider the issue as a whole.
This is also an important topic for product and marketing managers to discuss. In general, if you’re going to pay a fee for each visitor that accesses your website via an advertising banner, it’s evident that the amount will be higher if you employ the two-click technique. For example, if the firm you’re cooperating with for advertisement charges $ 1 for each person accessing the website via a one-click banner, the price will be at least $ 1.5 in a two-click strategy. Using a two-click strategy will not only cost you more potential revenue, but it will also cost you more money.
Discrimination Against Women at UC Berkeley
In 1973, a group of women’s rights activists launched a lawsuit against UC Berkeley, one of the world’s top ten universities. They claimed that the institution discriminated against men and women when admitting graduate students, with a larger percentage of males accepted. To back up their argument, they presented the following table:
At first look, there appeared to be a 9% sex discrimination between male and female applicants at Berkeley University; but, when the researchers disaggregated these statistics and evaluated them, faculty by faculty, they discovered some surprising outcomes. Six of the university’s 85 faculties have a substantial anti-male bias. Only four colleges, however, exhibit prejudice against women. We shall now exhibit and study the facts of this university’s six major faculties separately:
As is well known, four of these six prominent colleges accepted a larger percentage of women than males. Even in one example (School A), there is a 20% discrepancy in admission percentages for men and women, indicating that admission of males is likely discriminatory. As a result, the protest of women’s rights activists was not included. If it had been protested, the male community might have alleged unfairness and discrimination based on these findings.
Let us now look at what is causing this dilemma. What, in your opinion, is the reason that, even though the rate of admission of men is greater than that of males in most universities, the percentage of admission of men is around 10% higher than that of women?
The different kind of applicants is the cause of the Simpson paradox in this case. Many women apply to institutions with fewer admissions and more competitiveness (for example, English). In contrast, most males apply to colleges with higher acceptance rates and, of course, less rivalry to enter that college (for example, technical college). The data also reveals that there were 825 male candidates for College A, a big college, but only 108 women applied.
Let’s do some math!
After looking at numerous different Simpson paradox situations and models, it is not a bad idea to look at this subject analytically and describe the necessary condition for the occurrence of the “Simpson paradox” in mathematical language.
Assume we denote the probability of the occurrence of set A as P(A). We now partition A into n categories and assign each P a chance of success (Ai). Assume that:
Similarly, let us name the probability of the occurrence of set B as P(B). We now divide B into n categories and assign each P a chance of success (Bi). The Simpsons paradox happens if and only if we have:
In simpler terms, this statement refers to situations in which one event is more likely to occur in the general case than the other; but, when we separate that event and study the separated categories together, the probability of the categories occurring does not follow the original state.
As you can see, the Simpsons paradox has an extensive scope. As a result, in order to make data-driven decisions, it is vital to be aware of all elements of data gathering as well as the cause-and-effect links behind that occurrence.
