What is Non-Linearity?
In regression modeling you’ll often hear terms like “linear” and “non-linear” relationships. What this refers to are the relationships between the data you have (independent variables) and what you are trying to predict (target variable).
As an example, if you were trying to predict the closing price of a stock (target variable), you would likely try to use the daily volume traded, high price, and low price (independent variables) to guide your prediction.
In the context of linear regression, the assumption is that the relationship between the independent variables and the target are linear, meaning that a change in your predictor variables (either positive or negative) will result in a constant change in the target.
In the real-world however, the relationship between the independent variables and the target are often non-linear. Non-linear relationships can exhibit various patterns, such as curves, exponential growth, or diminishing returns.
Linear Examples:
Driving across country — the time it takes to drive across the country is dependent on your speed. The more you speed (and risk a ticket), has a constant effect at reducing the amount of time required to meet your destination.
Salary and Hours Worked — If you are paid hourly and not eligible for overtime pay, the number of hours you work has a constant effect on your overall pay.
Non-Linear Examples:
Infectious disease — as a disease begins to spread through a population the number of infected often increases overtime. Take for example COVID-19. The number of cases was relatively low when the disease was first detected, but as the virus spread, the number of confirmed cases grew exponentially.
Income vs Happiness — As individuals’ income rises from very low levels, the increase tends to have a significant impact on their happiness. The extra income allows them to improve their basic living conditions. However, at higher income levels, the marginal increase in happiness diminishes, and the impact of additional income on overall well-being becomes less significant.
To handle non-linearity, non-linear regression models, such as polynomial regression, spline regression, or generalized additive models (GAMs) are often used. These models can handle non-linear relationships by including higher-order terms or using flexible functions to capture the non-linear patterns.
Another option is to use machine learning algorithms, like decision trees, random forests, or neural networks, which can capture non-linear relationships due to their flexible and complex structures. These models are capable of learning and representing non-linear patterns without explicitly transforming the data.
In summary, non-linearity in regression modeling refers to situations where the changes in your independent variables doesn’t have a constant effect on what you are trying to predict.
