Descriptive Statistics Final Project — Python

Overview

Welcome to the Descriptive Statistics Final Project! In this project, you will demonstrate what you have learned in this course by conducting an experiment dealing with drawing from a deck of playing cards and creating a writeup containing your findings.

Be sure to check through the project rubric to self-assess and share with others who will give you feedback.

Questions for Investigation

This experiment will require the use of a standard deck of playing cards. This is a deck of fifty-two cards divided into four suits (spades (♠), hearts (♥), diamonds (♦), and clubs (♣)), each suit containing thirteen cards (Ace, numbers 2–10, and face cards Jack, Queen, and King). You can use either a physical deck of cards for this experiment or you may use a virtual deck of cards such as that found on random.org (http://www.random.org/playing-cards/).

For the purposes of this task, assign each card a value: The Ace takes a value of 1, numbered cards take the value printed on the card, and the Jack, Queen, and King each take a value of 10.

1. First, create a histogram depicting the relative frequencies of the card values.

2. Now, we will get samples for a new distribution. To obtain a single sample, shuffle your deck of cards and draw three cards from it. (You will be sampling from the deck without replacement.) Record the cards that you have drawn and the sum of the three cards’ values. Replace the drawn cards back into the deck and repeat this sampling procedure a total of at least thirty times.

3. Let’s take a look at the distribution of the card sums. Report descriptive statistics for the samples you have drawn. Include at least two measures of central tendency and two measures of variability.

4. Create a histogram of the sampled card sums you have recorded. Compare its shape to that of the original distribution. How are they different, and can you explain why this is the case?

5. Make some estimates about values you will get on future draws. Within what range will you expect approximately 90% of your draw values to fall? What is the approximate probability that you will get a draw value of at least 20? Make sure you justify how you obtained your values.

Investigation

1.Histogram depicting the relative frequencies of the card values.

Deck_Cards.csv contains deck of fifty-two cards divided into four suits, along with assigning each card a value: The Ace takes a value of 1, numbered cards take the value printed on the card, and the Jack, Queen, and King each take a value of 10)

Below is the format:

cards,suits,value
A,S,1
2,S,2
3,S,3
4,S,4
5,S,5
6,S,6
7,S,7
8,S,8
9,S,9
10,S,10
J,S,10
Q,S,10
K,S,10
A,H,1
2,H,2
3,H,3
4,H,4
5,H,5
6,H,6
7,H,7
8,H,8
9,H,9
10,H,10
J,H,10
Q,H,10
K,H,10
A,D,1
2,D,2
3,D,3
4,D,4
5,D,5
6,D,6
7,D,7
8,D,8
9,D,9
10,D,10
J,D,10
Q,D,10
K,D,10
A,C,1
2,C,2
3,C,3
4,C,4
5,C,5
6,C,6
7,C,7
8,C,8
9,C,9
10,C,10
J,C,10
Q,C,10
K,C,10

Load Data

import pandas as pd
import numpy as np
df = pd.read_csv("/Users/....../Documents/Deck_Cards.csv",header='infer')
print df.head(3)
print df.iloc[:,2].describe()
Output:
  cards suits value
0 A S 1
1 2 S 2
2 3 S 3
count   52.000000
mean     6.538462
std      3.183669
min      1.000000
25%      4.000000
50%      7.000000
75%     10.000000
max     10.000000
Name: value, dtype: float64

Plot Histogram

import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(211)
ax.hist(df['value'],bins = 10,range=[0.5, 10.5],facecolor='g', align='mid')
ax.xaxis.set_ticks(np.arange(0, 12, 1))
plt.title('Relative Frequencies')
plt.xlabel('Values')
plt.ylabel('Count')
plt.grid(True)
ay = fig.add_subplot(212)
ay.boxplot(df['value'])
plt.show()
Output:

Show Frequency Table

freq_table = df.groupby(['value'])
print freq_table.size()
Output:
value
1 4
2 4
3 4
4 4
5 4
6 4
7 4
8 4
9 4
10 16
dtype: int64

2.Sampling and Sampling distribution for sum and average of the three cards’ values

from random import sample
c1=[]
c2=[]
c3=[]
sum_s=[]
average_s=[]
for i in xrange(1000):
 rindex = np.array(sample(xrange(len(df)),3))
 dfr = df.ix[rindex]
 c1.append(dfr.iloc[0,2])
 c2.append(dfr.iloc[1,2])
 c3.append(dfr.iloc[2,2])
 sum_s.append(dfr.iloc[0,2]+dfr.iloc[1,2]+dfr.iloc[2,2])
 average_s.append((dfr.iloc[0,2]+dfr.iloc[1,2]+dfr.iloc[2,2])/(3.0))
sampling_df = pd.DataFrame({'card1':c1,'card2':c2,'card3':c3,'sum_col':sum_s,'average_col':average_s})
print sampling_df.head(3)
Output:
average_col card1     card2    card3     sum_col
0 7.000000     7         10       4         21
1 5.333333     4         10       2         16
2 7.000000     5          7       9         21
3.Distribution of the card sums. Descriptive statistics for the samples drawn.(At least two measures of central tendency and two measures of variability.)
print sampling_df['sum_col'].describe()
Output:
count 1000.000000
mean 19.646000
std 5.328703
min 5.000000
25% 16.000000
50% 20.000000
75% 23.000000
max 30.000000
Name: sum_col, dtype: float64
4.Histogram of the sampled card sums and comparison of its shape to original distribution
fig = plt.figure()
ay = fig.add_subplot(211)
ay.hist(sampling_df['sum_col'],bins = 25,range=[4, 34],facecolor='g', align='mid')
#ay.xaxis.set_ticks(np.arange(0, 40, 5))
plt.title('Sampling Distribution - SUM')
plt.xlabel('Values(SUM)')
plt.ylabel('Count')
ay = fig.add_subplot(212)
ay.boxplot(sampling_df['sum_col'])
plt.show()
Output:

The shape of the sampled card sums is same as that of normal distribution and this is in accordance with Central Limit Theorem.
5.Descriptive statistics and histogram of Sampling distribution for Average of the three cards’ values
print sampling_df['average_col'].describe()
Output:
count 1000.000000
mean 6.548667
std 1.776234
min 1.666667
25% 5.333333
50% 6.666667
75% 7.666667
max 10.000000
Name: average_col, dtype: float64
fig = plt.figure()
ay = fig.add_subplot(211)
ay.hist(sampling_df['average_col'],bins = 40,range=[4, 34],facecolor='g', align='mid')
#ay.xaxis.set_ticks(np.arange(0, 40, 5))
plt.title('Sampling Distribution - AVG')
plt.xlabel('Values(AVG)')
plt.ylabel('Count')
ay = fig.add_subplot(212)
ay.boxplot(sampling_df['average_col'])
plt.show()
Output:

The shape of theSampling distribution for Average of the three cards’ values is same as that of normal distribution and this is in accordance with Central Limit Theorem.
Also as perCentral Limit Theorem for Sampling distribution for Average of the three cards’ values,
Population SD/SD of Sampling distribution for Average of the three cards’ values is square root of sample size
3.183669/1.776234 = 1.7923702620262871 ~√ (3 )
6.Within what range will you expect approximately 90% of your draw values to fall? What is the approximate probability that you will get a draw value of at least 20? (For sampled card sums distribution)
90% of values will fall between the points corresponding to 5% and 95%.
import scipy.stats as sp
mean_sum_distribution = 19.481000
variance_sum_distribution = 5.435364
p1=sp.norm.ppf(0.05)*variance_sum_distribution+mean_sum_distribution
p2=sp.norm.ppf(0.95)*variance_sum_distribution+mean_sum_distribution
print 'It is expected approximately 90% of the draw values will fall between ',p1,' and ',p2
Output:
It is expected approximately 90% of the draw values will fall between  10.5406218108  and  28.4213781892
Manual Calculation by referring Z Table (For sampled card sums distribution):
95% is 1.64
5%  is -1.65
90% is between 95% and 5%
Mean = 19.481
Variance = 5.435364
Converting Z values to X
1.64*5.435364+19.481 = 28.39499696~28.4
-1.65*5.435364+19.481 = 10.512649400000003~10.51
The approximate probability that you will get a draw value of at most 20
p_20_atmost = sp.norm.cdf(z_20)
print z_20
print 'The approximate probability that we get a draw value of at least 20 is',(1-p_20_atmost)
Output:
0.0954857853126
The approximate probability that we get a draw value of at least 20 is 0.461964490173
Manual Calculation by referring Z Table (For sampled card sums distribution):
Z-score = (X-Mu)/Sigma
Z-score for 20 = (20-19.481)/5.435364 = 0.09548578531263009, left of 20
To get more than 20 i.e. atleast 20
1 - Probability(Z-score for 20) i.e 1 - 0.5359 = 0.46409999999999996~0.464
References:
Descriptive Statistics Final Project -https://docs.google.com/document/d/1059JMJ9C5dn7vKUrmfWYle57Ai3Uk9PzxPQBGj5drjE/pub?embedded=true
Python For Data Science Cheat Sheet - https://s3.amazonaws.com/assets.datacamp.com/blog_assets/PythonForDataScience.pdf
R-bloggers - https://www.r-bloggers.com/descriptive-statistics-final-project-with-python-r/
Cheat Sheet for Exploratory Data Analysis in Python - https://www.analyticsvidhya.com/blog/2015/06/infographic-cheat-sheet-data-exploration-python/
Z Table - https://s3.amazonaws.com/udacity-hosted-downloads/ZTable.jpg
Cheat sheet: Data Visualisation in Python - https://www.analyticsvidhya.com/blog/2015/06/data-visualization-in-python-cheat-sheet/