StandardScaler vs. MinMaxScaler vs. RobustScaler: Which one to use for your next ML project?
Data scaling is a method for reducing the effect of data bias on predictions which is highly used in pre-processing step in any Machine Learning project. It can be applied to any type of prediction model, including neural networks and regularized linear models. In this article, I will discuss how to choose between different types of data scaling techniques: StandardScaler, MinMaxScaler or RobustScaler.
StandardScaler
StandardScaler is a fast and specialized algorithm for scaling data. It calculates the mean and standard deviation of the data set and normalizes it by subtracting the mean and dividing by standard deviation. Using StandardScaler is a common practice in ML projects if the data set follows a normal distribution. For more information, check out this link.
MinMaxScaler
MinMaxScaler is a simple and effective linear scaling function. It scales the data set between 0 and 1. In other words, the minimum and maximum values in the scaled data set are 0 and 1 respectively. MinMax Scaler is often used as an alternative to Standard Scaler if zero mean and unit variance want to be avoided. You can find more details here.
RobustScaler
RobustScaler is a technique that uses median and quartiles to tackle the biases rooting from outliers. Instead of removing mean, RobustScaler removes median and scales the data according to the quantile range aka IQR: Interquartile Range. By default settings, the IQR is the range between the 1st quartile (25th quantile) and the 3rd quartile (75th quantile) but you can modify it according to your need. Check out the details here.
Pros & Cons of Scaling Methods
Every data set has its own idiosyncrasy. Therefore, it will not be feasible to say that one scaling technique is better than another one in any situation. Below, I am listing the strengths and weaknesses of each of them to give you an in-depth understanding.
StandardScaler Strengths
- Easy to implement and works fast
- Applies regularization
StandardScaler Weaknesses
- Assumes the data set follows a normal distribution (Gaussian) which may not be the case
- Has scalability issues especially with large volumes of data
- Sensitive to outliers
MinMaxScaler Strengths
- Easy to implement and works fast
- Alternative of StandardScaler if zero mean and unit variance should be avoided
MinMaxScaler Weaknesses
- Sensitive to outliers
- Compressing the data set in a small range ([0, 1]) might result in loss of significant information
RobustScaler Strengths
- Handles outliers and skewness
RobustScaler Weaknesses
- Does not take into account mean and median
Conclusion
If you’re just starting out with data scaling in your ML project, you might find it helpful to consider reviewing your Exploratory Data Analysis (EDA) step and double check how your data set looks like. Then, you may find out which technique will work best for your application. For more information about various data scaling techniques, you can check out sklearn’s page here
