Linear Forecasting Models for Univariate Time Series Prediction

In my previous articles, I mentioned what are the basic terminologies associated with Time Series & why Time series have to be stationary. Now moving on, we will explore some commonly heard but difficult-to-understand time series, and forecasting models.

Let us consider a series with values X0,X1,X2,X3,X4,X5…….

• AR: It refers to AutoRegression. We know what regression looks like i.e.

Y=a+b*X

Auto here means with the past values of the Time Series. Example-if order is 1 then,

X3=a+c*X2+e

If an order is 2 then,

X3=a+b*X1+c*X2+e

where a,b, and c are constants

and so on. That is the dependence is not on other features but on past values of the same feature depending on the parameter set and this is what ‘Auto’ signifies!!

Here a=constant, e=white noise drawn from a normal distribution

• MA: It refers to Moving Average.But not the moving average we know!!! the equation for prediction goes like this order 1

X3= C+ a*E2

for order 2

X3=C +a*E2+b*E1

where a, and b are constants

where E1, E2, and E3 error terms on past values.For further details regarding how these white noise terms are calculated refer here.

• ARMA: It refers to AutoRegression Moving Average which is a combination of AR+MA equation for prediction i.e.

if Order for AR=1, MA=2 in ARMA than

X3=a+b*X2+c*E2+d*E1

If an order is AR=2, MA=1 than,

X3=a+b*X2+b1*X1+d*E1

where a,b,b1,c are constants

ARIMA: It is AutoRegression Integrated MovingAverage. Here Integrated term refers to differencing i.e., for example, calculating the 5th term with order 3, it would be 4th + (4th-3rd)+(3rd-2nd) terms.

If Order is AR=1,I=1,MA=1 than equation is,

X3=a+b*X2+X2+c*E2

Where

• AR terms are a+b*X1
• MA term is c*E1
• The difference term is X1

If order is AR=1,I=2,MA=1

X3=a+b*X2+X2+(X2-X1)+c*E2

SARIMA: It refers to Seasonal ARIMA. It is basically for series with seasonality components in them that haven’t been handled in any way. Here we need to provide 7 parameters 3 for ARIMA(AR, I, MA) and 3 for Seasonal ARIMA(Season AR, Season I, Season MA), and one for seasonality duration(12 months,6 months according to the data)

Hence for SARIMA(1,1,1)(2,2,2)12,(here 12 represents yearly seasonality)

X24=ARIMA(1,1,1)+(x*X12+x1*X0)+(X12+X12-X0)+(y*E12+y1*E0)

where x,x1,y,y1 are constants. The term to be forecasted is the 24th.

You can find codes to implement ARIMA here

The major drawback that we face with these methods is that when we have multiple features determining the time series, this cannot be done using any of the AR or MA models. It is not only the past values but some other factors as well that affect the forecasting many times.

I would be resuming on how to deal with such situations and some other models in my next article.