Explaining p-value — We have all used this is in our stats classes: the null hypothesis is rejected if p<0.05. This short blog is about an explanation of p-value, and how it is connected to the confidence interval and power of a test. p-value definition (from stats 101/Wikipedia) p-value is a probability of…

The idea behind AIC — All of us have used AIC for model selection. This blog is about the idea behind AIC, what is it and why is it used for model selection. While we have been told how to calculate AIC, at least I was never taught the logic behind why are we doing…

The intuition behind ML interpretation models — In machine learning, there has been a trade-off between model complexity and model performance. Complex machine learning models e.g. deep learning (that perform better than interpretable models e.g. linear regression) have been treated as black boxes. Research paper by Ribiero et al (2016) titled “Why Should I Trust You” aptly…

We all know how XGBoost dominates in Kaggle competitions due to its performance and speed. This blog is about understanding how XGBoost works (try to explain the research paper). This blog is not about how to code/ implement XGboost or how to tune its hyperparameters. XGBoost stands for eXtreme Gradient…

Cross-entropy is a widely used loss function in machine learning for classification problems. Information theory is used widely but the rationale behind using it is not explained in classes. In this blog, we go through an intuitive understanding of the information theory and finally connecting it with the cross-entropy function. …

Examples from the literature dealing with Type II endogeneity — With an introduction and dealing with a ‘simpler’ form of endogeneity (Type I), this part explores the more difficult problem of endogeneity by simultaneity. Simultaneity arises when Y causes X and X causes Y (rich gets richer). The problem is difficult as: 1. addition of Instrumental Variables (IVs) may not help 2…

Using ice-cream vendor example to explain Type I endogeneity — Here we discussed the meaning of endogeneity through examples, few possible sources of endogeneity and why is it important. This part focusses on ice cream vendor example to dive deeper into why we should be careful about endogeneity. The linear regression model is given by: Sales_i = alpha+beta Price_i+error_i (…

Examples describing different types of endogeneity — An ice cream vendor sells ice cream on a beach. He collects data for total sales (Y) and selling price (X) for 2 years. He gives the data to a data scientist asking him to find the optimal selling price. The data scientist fits a linear regression model, Y =…