An Overview of Different Time Series Models for Forecasting

When one wants to start predicting the future, it must come as no surprise that one must deal with time. This is the essence of time series modeling. A time series model can be thought of as a function F(t) = y that takes a given point in time (t) and produces some output (y).

The difficult arises when we want to estimate what y will be during a point in time that hasn’t occurred yet. The model then changes to F(t+h) = y, where h is some amount of time into the future.

There are several time series models that exist and today I will give an overview of the most common.

Seasonal Decomposition:

Oftentimes, time series data can demonstrate clear seasonality, ie certain patterns or trends that emerge over consistent periods of time and repeat themselves (these could be daily, weekly, monthly, yearly, etc). A seasonal decomposition breaks the time series model into three parts:

F(t) = S(t) + T(t) + R(t)

where S(t) is a seasonal component, T(t) is a trend component, and R(t) is the remainder component. During a seasonal decomposition, the trend will be able to explain a certain proportion of the observations, the seasonality another portion, and then the remainder of the observations are by caused by some unknown factor or series of factors (“the remainder” R(t)). Below is an example of a seasonal decomposition with the original data on the top, and the trend, seasonal, and remainder components below:

Exponential Smoothing

Exponential smoothing is a simple model that proposes that the model’s output at time t is a weighted average of the values of observations at t-1, t-2, all the way to t=1, with the weights assigned to each observation reducing exponentially the further back in time they are. Put more simply, t-1 will have the highest weight in the average, t-2 the second highest weight, and so on.

As time proceeds, the model values the information provided by an observation at a single point in time less and less.

The weights are calculated by some defining some beginning weight for the most recent observation (alpha), and the remaining weight are derived from this alpha as such:

Below is a table of examples demonstrating how weights assigned to each observation vary depending on the initial alpha value:

Note that for any value 0 < alpha <1, the sum of the weights will approximately sum to 1.

AutoRegressive Integrated Moving Average (ARIMA)

Combined with exponential smoothing, the ARIMA group of models is one of the most common forecasting techniques used by practitioners.

An ARIMA models combines two approaches to explaining future observations. The first is the autoregressive portion (AR) which postulates that a portion of the observation is some linear combination of the past observations. The second is the moving average portion (MA) which postulates that a portion of the observation is some linear combination of the past errors in the forecast.

These models require the time series to be stationary (ie there is no trend in the data), thus differencing or integrating (I) is a necessary step in most cases as a trend usually does exist in time series data.

The autoregressive portion of the model is expressed as such:

With p being the number of lags that the model is assuming influences the outcome (this is also known as the order of the AR portion).

The moving average portion of the model is expressed as such:

With q being the number of error term lags that the model is assuming influences the outcome (this is also known as the order of the MA portion).

Together, the expression for the ARIMA model becomes:

In order to determine which orders or p and q are needed ACF and PACF plots are required. ACF plots shows the autocorrelations between yt and all observations from yt-1 to yt-k. PACF plots shows the partial autocorrelations which is the correlation between yt and yt-k that isn’t explained by their mutual correlations with another specified set of variables.

In a future post I will demonstrate how to create all three of these models in python and how to use them to perform simple forecasts.