Let’s first think of an interesting example. What if X1 and X2 are highly correlated in a regression? We know it is a multicollinearity problem and (X^TX) will be non-invertible. Usually we would solve it by ridge regression. But how would R solve this problem? The coefficient of b is zero! Why is that? We all know the closed-form formula for…

Linearity The first assumption to be made in linear models is that the relation is linear. The well-known Gauss-Markov Model assumes that where

K-means clustering is a simple algorithm for cluster data points in unsupervised learning. Well, it can be argued to be seen as semi-supervised learning because there exists a latent variable or a hyperparameter: n, the number of clusters. This is the same case in Gaussian Mixture Models. The k-means algorithm…

Data Visualization In the first stage, it would be nice to start with a data visualization and data summary. We can check the data distribution and get a rough idea of what features might be completely useless. …

A/B testing is a powerful decision-making tool. However, if social network spill-over effect exists, then the average treatment effect will be underestimated. This is because the control group will also be exposed to some condition of treatment. The question is: how to solve it? I have seen two ways of…

If we run linear regression in R, then we will see a term in the report “degrees of freedom” and that value equals n-p where n is the number of observations (data points) and p is the number of parameters (including intercept). But what does it mean? How is it useful? Short Answer: the degree of freedom represents the number of "free" or independent $N(0,1)^2$ in the residuals from linear regression. The following three parts will explain this further.