Time Series Model(s) — ARCH and GARCH
Student at Praxis Business School
What is this article about?
This article provides an overview of two time-series model(s) — ARCH and GARCH. These model(s) are also called volatility model(s). These models are exclusively used in the finance industry as many asset prices are conditional heteroskedastic.
ARCH — Autoregressive Conditional Heteroskedasticity
GARCH — Generalized Autoregressive Conditional Heteroskedasticity
- These models relate to economic forecasting and measuring volatility.
- Some of the techniques adopted in the finance sector — ARCH, ARCH-M, GARCH, GARCH-M, TGARCH, and EGARCH.
- ARCH model is concerned about modeling volatility of the variance of the series.
- These model(s) deals with stationary (time-invariant mean) and nonstationary (time-varying mean) variable(s).
Some of the real-time examples where ARCH model(s) applied: Stock prices, oil prices, bond prices, inflation rates, GDP, unemployment rates, etc.,
What is the ARCH model?
ARCH is an Autoregressive model with Conditional Heteroskedasticity .
“The ARCH process introduced by Engle (1982) explicitly recognizes the difference between the unconditional and the conditional variance allowing the latter to change over time as a function of past errors.”
Autoregressive: The current value can be expressed as a function of the previous values i.e. they are correlated.
Conditional: This informs that the variance is based on past errors.
Heteroskedasticity: This implies the series displays unusual variance (varying variance).
Why an ARCH model?
- Autoregressive models can be developed for univariate time-series data that is stationary (AR), has a trend (ARIMA), and has a seasonal component (SARIMA). But, these Autoregressive models do not model is a change in the variance over time.
- The error terms in the stochastic processes generating the time series were homoscedastic, i.e. with constant variance.
- There are some time series where the variance changes consistently over time. In the context of a time series in the financial domain, this would be called increasing and decreasing volatility.
- Volatility in Finance: Degree of variation price series over time as measured by the standard deviation of the series. Suppose that Si is the value of a variable on a day ‘i’. The volatility per day is the standard deviation of ln(Si /Si-1).
- In time series where the variance is increasing in a systematic way, such as an increasing trend, this property of the series is called heteroskedasticity. This means changing or unequal variance across the series.
If this change in the variance can be correlated over time, then it can be modeled using an autoregressive process, such as ARCH.
When to apply an ARCH Model?
In practice, this can be used to model the expected variance on the residuals after another autoregressive model has been used, such as an ARMA or similar.
Since we can only tell whether the ARCH model is appropriate or not by squaring the residuals and examining the correlogram, we also need to ensure that the mean of the residuals is zero.
Crucially, ARCH should only ever be applied to series that do not have any trends or seasonal effects, i.e. that have no (evident) serially correlation. ARIMA is often applied to such a series (or even Seasonal ARIMA), at which point ARCH may be a good fit.
ARCH Model of Order Unity:
ARCH(p) model is simply an AR(p) model applied to the variance of a time series.
ARCH(1):
A time-series {ϵ(t)} is given at each instance by ϵ(t) = w(t)*σ(t)
where w(t) is the white noise with zero mean and unit variance.
Var(x(t)) = σ²(t) = ⍺0+⍺1 * σ²(t-1)
where ⍺0, ⍺1 are parameters of the model and ⍺0 > 0, ⍺1 ≥ 0 to ensure that the conditional variance is positive. σ²(t-1) is lagged square error.
We say that ϵ(t) is an autoregressive conditional heteroskedastic model of order unity, denoted by ARCH(1).
ϵ(t) = w(t)* σ(t) = w(t)* ⎷(⍺0 + ⍺1 *ϵ²(t-1))
similarly ARCH(2):
ϵ(t) = w(t)* σ(t) = w(t)* ⎷(⍺0 + ⍺1 * ϵ²(t-1) + ⍺2 * ϵ²(t-2))
similarly ARCH(p):
ϵ(t) = w(t) * ⎷(⍺0 + ⍺(p) * ∑ ϵ²(t-i)
where:
- p is the number of lag squared residual errors to include in the ARCH model.
- i = (1,2,3,-,-,-, -, p) tells us the number of logged periods of the square error.
Interpretation:
- If the error is high during the period (t-1), it is more likely that the value of error at the period (t) is also higher.
- vice versa — If the error is low during the period (t-1) then the value inside sqrt will be low which results in a decreased error in (t).
- Remember, ⍺1 ≥ 0 for the positive variance.
- For the stability condition to hold, ⍺1 < 1, otherwise ϵ(t) will be explosive (continue to increase over time).
Note: As mentioned earlier ARCH(1) should only ever be applied to a series that has already had an appropriate model fitted sufficient to leave the residuals looking like discrete white noise.
Why does this model volatility?
From variance formula, we can derive the below equation:
Var(ϵ(t)) = ⍺0 + ⍺1 * Var(ϵ(t-1))
We can say that the variance of the series is simply a linear combination of the variance of the prior element of the series.
What is a GARCH model?
Generalized Autoregressive Conditional Heteroskedasticity, or GARCH, is an extension of the ARCH model that incorporates a moving average component together with the autoregressive component.
Bollerslev (1986, Journal of Econometrics) generalized Engle’s ARCH model and introduced the GARCH model.
Introduction of moving average component allows the model:
- To model the conditional change in variance over time.
- Changes in the time-dependent variance.
Examples include conditional increases and decreases in the variance.
Thus GARCH is the “ARMA equivalent” of ARCH, which only has an autoregressive component. GARCH models permit a wider range of behavior more persistent volatility.
GACH Model of Order p, q — GARH(p,q):
GARCH(1,1):
Here we are going to consider a single autoregressive lag and a single “moving average” lag. The model is given by the following:
ϵ(t) = w(t) * σ(t)
ϵ(t) =w(t) * ⎷(⍺0 + ⍺1 *ϵ²(t-1)) + β1 * σ²(t−1)
Similarly GARH(p,q):
A time-series {ϵ(t)} is given at each instance by ϵ(t) = w(t)*σ(t)
and σ²(t) is given by:
where α(i) and β(j) are parameters of the model.
⍺0 > 0, ⍺i ≥ 0, i =1, … q, β≥ 0, j = 1, … p imposed to ensure that the conditional variances are positive.
Here we are adding moving average term, that is the value of σ² at t, σ²(t), is dependent upon previous σ²(t-j) values.
Interpretation:
- The large value of β1 causes σ(t) to be highly correlated with σ²(t−1) and gives the conditional standard deviation process a relatively long-term persistence, at least compared to its behavior under an ARCH model.
- For p = 0 the process reduces to the ARCH(q) process.
- For p = q = 0, ϵ(t) is simply white noise.
How to configure the ARCH and GARCH Model(s):
- The configuration for an ARCH model is best understood in the context of ACF and PACF plots of the variance of the time series.
- This can be achieved by subtracting the mean from each observation in the series and squaring the result, or just squaring the observation if you’re already working with white noise residuals from another model.
- The ACF and PACF plots can then be interpreted to estimate values for p and q, in a similar way as is done for the ARMA model.
more details: ACF and PACF
ARCH and GARCH models Python code:
We look at the generalized python code using the above formula:
source for the below code: http://www.blackarbs.com/blog/time-series-analysis-in-python-linear-models-to-garch/11/1/2016
- Import required libraries
## Import required librariesimport os
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
from arch import arch_model
import warnings
warnings.filterwarnings("ignore")
2. ARCH(1) model:
np.random.seed(13)# a0 and a1 are constant and should be greater than 0
a0 = 2
a1 = .5# Random number generator with size 1000:
y = w = np.random.normal(size=1000)
Y = np.empty_like(y)for t in range(len(y)):
Y[t] = w[t] * np.sqrt((a0 + a1*y[t-1]**2)) #Formula of ARCH(1)# simulated ARCH(1) series, looks like white noise
tsplot(Y, lags=30)
output graph:
3. ARCH(1) Squared model:
np.random.seed(13)a0 = 2
a1 = .5y = w = np.random.normal(size=1000)
Y = np.empty_like(y)for t in range(len(y)):
Y[t] = w[t] * np.sqrt((a0 + a1*y[t-1]**2))# simulated ARCH(1) series, looks like white noise
tsplot(np.square(Y), lags=30) #squared Y
output graph:
Observation: ACF and PACF seem to show significance at lag 1 indicating an AR(1) model for the variance may be appropriate.
4. GARCH(1,1) model:
np.random.seed(1)# Constants
a0 = 0.2
a1 = 0.5
b1 = 0.3w = np.random.normal(size=10000)
eps = np.zeros_like(w)
sigsq = np.zeros_like(w)for i in range(1, n):
sigsq[i] = a0 + a1*(eps[i-1]**2) + b1*sigsq[i-1]
eps[i] = w[i] * np.sqrt(sigsq[i]) #GARCH Formula_ = tsplot(eps, lags=30)
output graph:
Observation: We have noticed that overall this process closely resembles white noise, however, take a look when we view the squared eps series.
5. GARCH(1,1) Squared model:
np.random.seed(2)a0 = 0.2
a1 = 0.5
b1 = 0.3n = 10000
w = np.random.normal(size=n)
eps = np.zeros_like(w)
sigsq = np.zeros_like(w)for i in range(1, n):
sigsq[i] = a0 + a1*(eps[i-1]**2) + b1*sigsq[i-1]
eps[i] = (w[i] * np.sqrt(sigsq[i]))_ = tsplot(np.square(eps), lags=30) #squared eps
output graph:
Observation: we can observe clearly autocorrelation present and the significance of the lags in both the ACF and PACF indicates we need both AR and MA components for our model.
6. Fitting GARCH(1,1) model to our simulated EPS Series:
# Fit a GARCH(1, 1) model to our simulated EPS series
# We use the arch_model function from the ARCH packagefrom arch import arch_modelam = arch_model(eps)
res = am.fit(update_freq=5)
print(res.summary())
Now, let's run the above model through an example using “SPY returns”,
- Iterate through combinations of ARIMA(p, d, q) models to best fit our time series.
- Pick the GARCH model orders according to the ARIMA model with the lowest AIC.
- Fit the GARCH(p, q) model to our time series.
- Examine the model residuals and squared residuals for autocorrelation.
end = ‘2015–01–01’
start = ‘2007–01–01’
get_px = lambda x: web.DataReader(x, ‘yahoo’, start=start, end=end)[‘Adj Close’]symbols = [‘SPY’,’TLT’,’MSFT’]# raw adjusted close prices
data = pd.DataFrame({sym:get_px(sym) for sym in symbols})# log returns
lrets = np.log(data/data.shift(1)).dropna()def _get_best_model(TS):
best_aic = np.inf
best_order = None
best_mdl = Nonepq_rng = range(5) # [0,1,2,3,4]
d_rng = range(2) # [0,1]
for i in pq_rng:
for d in d_rng:
for j in pq_rng:
try:
tmp_mdl = smt.ARIMA(TS, order=(i,d,j)).fit(
method=’mle’, trend=’nc’
)
tmp_aic = tmp_mdl.aic
if tmp_aic < best_aic:
best_aic = tmp_aic
best_order = (i, d, j)
best_mdl = tmp_mdl
except: continue
p(‘aic: {:6.5f} | order: {}’.format(best_aic, best_order))
return best_aic, best_order, best_mdl# Notice I’ve selected a specific time period to run this analysis
TS = lrets.SPY.ix[‘2012’:’2015']
res_tup = _get_best_model(TS)
output: aic — -5255.56734 | order: (3, 0, 2)
We can say ARIMA(3,0,2) best fits our time series.
Try to fit a GARCH(3,2) model using the best fit arima model parametersand plot the ACF and PACF of the squared residuals to observe the autocorrelation of the squared residuals.
Notes:
- Financial time-series have tails that are heavier than implied by a GARCH process with Gaussian {ϵ(t)}. To handle such data, one can assume that, instead of being Gaussian white noise, {ϵ(t)} is i.i.d. white noise process with a heavy-tailed distribution.
- In fact, GARCH processes exhibit heavy tails even if {ϵ(t)} is Gaussian. Therefore, when we use GARCH models, we can model both the conditional heteroskedasticity and the heavy-tailed distributions of financial markets data.
Summary:
In this article, you discovered the ARCH and GARCH models for predicting the variance of a time series.
- The problem with variance in a time series and the need for ARCH and GARCH models.
- ARCH and GARCH model(s) in brief.
- How to configure the ARCH and GARCH model(s).
- How to implement the ARCH and GARCH model(s) in Python.
References:
https://www.youtube.com/watch?v=JgJ_2HWMDFI&list=TLPQMTMwMTIwMjBKoNlY_gpw5A&index=1
https://faculty.washington.edu/ezivot/econ589/ch18-garch.pdf
