# Box-Cox Transformation.

The real-world data is not always distributed the way we want, that is Normal-Distribution It is always distributed in some distribution which we have no idea about some time it is skewed towards the right other time it has a long tail this leads us to miss normal distribution, Why we miss normal distribution you ask?

The normal distribution is the most important probability distribution in statistics because it fits many natural phenomena. It is symmetric distribution where most of the observations cluster around the central peak and the probabilities for values further away from the mean taper off equally in both directions.

The Box-Cox transformation is a particularly useful family of transformations. It is defined as:

where y^λ is the response variable and λ is the transformation parameter, For λ = 0, the natural log of the data is taken instead of using the above formula, here λ is a hyperparameter which has to be tuned according to the dataset

Let’s see box-cox in action

Import the necessary libraries

`from scipy import statsimport pandas as pdimport numpy as npimport pylab import matplotlib.pyplot as pltimport seaborn as snsfrom scipy import stats%matplotlib inline`

Let’s create a skewed distribution

`skewed_dist = stats.loggamma.rvs(5, size=10000) + 5`

Now let’s create a normal distribution

`normal_dist = np.random.normal(0, 1, 10000)`

Now let’s calculate the skewness of the normal distribution and look at the plot of the distributions

`sns.distplot(s)print("Skewness for the normal distribution:",skew(normal_dist))sns.distplot(x)print("Skewness for the skewed distribution:",skew(skewed_dist))`

Here you can see how the second distribution is left-skewed

skewness = 0 : normally distributed.
skewness > 0 : more weight in the left tail of the distribution.
skewness < 0 : more weight in the right tail of the distribution.

# Pearson’s Coefficient of Skewness

where X = mean of the distribution

Me = median of the distribution and

σ = standard deviation

The probability plot is a graphical technique for assessing whether or not a data set follows a normal distribution

`stats.probplot(normal_dist, dist=”norm”, plot=pylab)pylab.show()stats.probplot(skewed_dist, dist=”norm”, plot=pylab)pylab.show()`

In the above diagram for the skewed plot, we can see that the data is not normally distributed since the point don’t align with the red line the plot is not normally distributed

Now we apply our box-cox Transformation and plot it

`skewed_box_cox, lmda = stats.boxcox(skwed_dist)sns.distplot(skewed_box_cox)`

Let’s check the Probability-Plot and see whether the data is normally distributed or not and get the appropriate lambda value.

`stats.probplot(skewed_box_cox, dist=”norm”, plot=pylab)pylab.show()print ("lambda parameter for Box-Cox Transformation is:",lmda)`