Feature Engineering in Machine Learning (Part 1)

Handling Numeric Data with Binning

Intro

In machine learning, data comes in different formats/forms. For proper insight and efficiency in our machine learning model, it has to go through some tweaks so we can have the best out of it.

Feature engineering sits right between “data” and “modeling” in the machine learning pipeline for making sense of data. Most data scientists and machine learning engineers agree that data cleaning and feature engineering takes most of the time in machine learning workflow.

Though this process is very important only few people emphasize it in. There a couple of deep principles involved in it and they will be explained in this series.

Binning

Binning is the process of converting numeric data into categorical data. It is one of the methods used in feature engineering. Binning comes in very handy for numeric features, especially when it is one with wide range. A number of ways binning can be done include

• Fixed-Width Binning
• Quantile Binning
• Binning by Instinct

Fixed-Width Binning

In fixed-width binning, each bin contains a specific numeric range. For example, we can group a person’s age into decades: 0–9 years old will be in bin 1, 10–19 years fall will be in bin 2. When the numbers span multiple magnitudes, it may be better to group by powers of 10 (or powers of any constant): 0–9, 10–99, 100–999, 1000–9999.

>>> import numpy as np
### Generate 20 random integers uniformly between 0 and 99
>>> small_counts = np.random.randint(0, 100, 20)
>>> small_counts
array([30, 64, 49, 26, 69, 23, 56, 7, 69, 67, 87, 14, 67, 33, 88, 77, 75,
47, 44, 93])
### Map to evenly spaced bins 0–9 by division
>>> np.floor_divide(small_counts, 10)
array([3, 6, 4, 2, 6, 2, 5, 0, 6, 6, 8, 1, 6, 3, 8, 7, 7, 4, 4, 9], dtype=int32)

Quantile Binning

Using fixed-width will not be effective if there are large gaps for the range of the numerical feature, then there will be many empty bins with no data. This problem can be solved by positioning the bins based on.

This can be done using the quantiles of the distribution. Quantiles are values that divide the data into equal portions. For example, the median divides the data in halves; half the data are smaller, and half larger than the median. The quartiles divide the data into quarters, the deciles into tenths,
etc.

The example below demonstrates how to compute the deciles of a numerical feature:

>>> deciles = data['column'].quantile([.1, .2, .3, .4, .5, .6, .7, .8, .9])
>>> deciles

To compute the quantiles and map data into quantile bins, we can use the Pandas library. pandas.DataFrame.quantile and pandas.Series.quantile compute the quantiles. pandas.qcut maps data into a desired number of quantiles.

### An array of counts that span several magnitudes
>>> large_counts = [296, 8286, 64011, 80, 3, 725, 867, 2215, 7689, 11495, 91897, 44, 28, 7971, 926, 12]
>>> import pandas as pd
### Map the counts to quartiles
>>> pd.qcut(large_counts, 4, labels=False)
array([1, 2, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 2, 1, 0, 3], dtype=int64)
### Compute the quantiles themselves
>>> large_counts_series = pd.Series(large_counts)
>>> large_counts_series.quantile([0.25, 0.5, 0.75])

Binning by Instinct

This actually involves a manual process of binning manually based on your own personal insight of the data and setting ranges we would like to bin our data into. This should not be a constant practice but is mot times useful when our data points spans irregularly or unevenly.

>>>data['column_bin']=0
>>>data.loc[(data['column']<=1000) ,'column_bin']= 0
>>>data.loc[(data['column']>1000) & (data['column']>=5000) ,'column_bin']=1
>>>data.loc[(data['column']>5000) & (data['column']>=50000, 'column_bin']=2
>>>data.loc[(data['column']>50000) & (data['column']>=100000) ,'column_bin']= 3

column_bin stands for the column that will be for the bins.

This can be adjusted by you to suit whatever range our data falls into.

Conclusion

Binning saves us a lot of stress from getting insights to preparation of our data for modeling. Binning may sound simple but that doesn’t reduce its strong effect in feature engineering.

Thanks for reading 🤗