Vector Auto Regression for Multivariate Time Series Forecasting

Time is one of the most critical factor that decides whether a company will rise or not. That is why we see sales in stores or e-commerce platforms aligning with festivals. How can we analyze this kind of dataset?

Time series data is a set of collection of amounts that put together at equal time slots and ordered. Therefore, it is a sequence of discrete-time data. For example, a monthly number of sold fruits…

Figure Example for time series

A multivariate time series has more than one time dependent variable(time series). Each variable depends not only on its past values but also has some relation to other attributes. This dependency is useful for predicting the future values. Which model can be used for this aim?

Figure Univariate and multivariate graphs for time series

Vector Autoregression is one of the multivariate forecasting algorithm. It uses two or more time series influence each other. In this model, each attribute is a linear combination of the past values of itself and the past values of all the other variables. These past values of the series are used to forecast the future.

Let’s see the structure of the VAR model.

The typical AR(p) model is as below:

And a simple VAR(p) model can be written as

And VAR(2) process of the form:

For this model, we have multiple time series that influence each other, it is modeled as a system of equations with one equation per time series. If we increase the number of time series in the model, the system of equations will be larger automatically. For example, VAR(3) will be as below:

To build a model, we have to check a few properties of our dataset. For determining whether one time series can be used for predicting another, Granger’s Causality Test is a good choice. It’s a relationship shifted in time. It provides information about how the two series move together. If the probability value is less than the significance level (of 0.05), then the hypothesis would be rejected at that level.

Figure Examples show how the two series move together

To show the existence of a statistically significant connection between two or more time series Cointegration test is helpful. It is a statistical property of time series variables. All the series we have must be integrated of order d (number of differencing required to make a non-stationary time series stationary). If a linear combination of this collection is less than d, then the collection is said to be co-integrated and it means they have a statistically significant relationship.

After checking these features, we split the dataset into training and test data. Stationary is another important issue to build the model. How can we determine if our dataset is stationary or non- stationary?

A time series is said to be stationary if its mean, variance remain constant over time, not a function. Most of the time series models work on the assumption that the time series is stationary. There are three important check points:

1. Constant mean

2. Constant variance

3. An autovariance that does not depend on time

Figure Different graphs shows different cases about stationary

It is almost impossible to make a series %100 stationary, but we have to try to take it as close as possible. There are a few reasons behind non-stationarity of a time series. Two common reasons:

· Trend

· Seasonality

Figure Different graphs which show the different non-stationary situations

We can check stationary using the following unit root tests:

· Augmented Dickey-Fuller Test (ADF Test)

· KPSS test

· Plotting Rolling Statistics

If a series is found to be non-stationary, you make it stationary by differencing the series once and repeat the test again until it becomes stationary. Differencing handles both trend and seasonality.

VAR Order Selection

Before we can estimate a multivariate VAR model we need to specify the order p. The most common approach for model order selection involves choosing a model order that minimizes one or more information criteria (AIC, BIC, HQC) evaluated over a range of model orders. These criteria are:

· Akaike Information Criterion (AIC)

· Bayesian Information Criterion (BIC)

· Hannan-Quinn Criterion (HQC)

Usually, the AIC is the best criteria due to its favorable small sample forecasting features. The BIC and HQ, however, work well in larger datasets and have the advantage of being a consistent estimator.

Let’s see some examples with tables include criteria with lag information.

We choose p=1, because the lowest values seem here.

We choose p=3.

To choose the right order of the VAR(p), we fit increasing orders of VAR model iteratively and select the order that gives a model with the least AIC. We can also check other best fit comparison estimates of BIC and HQIC.

The other testing issue is serial correlation of residuals. It is used to check if there is any leftover pattern in the errors. If there is any correlation left in the residuals, then, there is some pattern in the time series that is still left to be explained by the model. In that case, there are a few classical actions.

· Increase the order of the model — p

· Persuade more predictors into the system

· Search a different algorithm for modeling the problem

Durbin Watson’s Statistic is a typical approach of testing for serial correlation of residuals.

The value of DW can change between 0 and 4.

· Positive serial correlation if it is closer to 0

· No significant serial correlation if it is closer to 2

· Negative serial correlation if it is closer to 4

After doing these operations and determining the parameters like p, d, etc., the model will be ready to apply. Then forecasts will be generated, but it is on the scale of the training data will be used by the model. Hence, to bring it back up to its original scale and de-difference it as many times we had differenced the original input data.

References

https://onlinelibrary.wiley.com/doi/10.1002/9781118625590.ch7

https://en.wikipedia.org/wiki/Cointegration

https://www.machinelearningplus.com/time-series/vector-autoregression-examples-python/

https://www.r-econometrics.com/timeseries/varintro/

https://en.wikipedia.org/wiki/Vector_autoregression#:~:text=Vector%20autoregression%20(VAR)%20is%20a,allowing%20for%20multivariate%20time%20series.

Lecture notes of Umberto Triacca — Lesson 18: Building a Vector Autoregressive Model