A Complete Introduction To Time Series Analysis (with R):: ARMA processes (Part II)
In the last article, we saw that a general ARMA(p,q) process can be written, with the help of the autoregressive and moving-average operators as
. We also discussed three important properties of an ARMA process:
- Stationarity
- Causality (current observation-only depends on the past)
- Invertibility (can correctly represent current noise as a function of the observations from the past)
We illustrated this with the ARMA(1,1) process, and we saw that we had to put some restrictions in order to ensure that the properties above are satisfied. This time, we will generalize these ideas to the ARMA(p,q). For this, we will make use of the basics of complex numbers, so if you need a review on the essentials, you can take a look at this article.
The Unit Disk
Let’s first recall the unit disk defined on complex numbers, given by the set
,that is, the set of all complex numbers with modulo less or equal to 1. Graphically, this is
The unit circle is the same set, but holding with strict equality
Now, recall the ARMA(1,1) process, given by
From the previous article, it follows that in order to ensure stationarity, we need
How can we generalize this to the ARMA(p,q)? In simple terms, we would like to have no roots of the autoregressive Phi polynomial on the boundary of the unit circle, that is, no roots on
Before morving forward, we will need the following proposition:
In simple words, this proposition is saying that if the Phi polynomial is not zero, then for all complex numbers in the unit circle, we can find coefficients with an arbitrarily close distance such that the power series of the inverse Phi polynomial is convergent. The proof of this relies on analysis, so we won’t go into that here. Armed with this, we are now ready to give formal definitions/characterizations of stationarity and causality in ARMA(p,q) processes.
Roots in the unit disk
We generally visualize the inverse roots as they are easier to visualize inside the unit circle than outside. We will use the following graph as reference.
Stationarity of ARMA processes
A stationary solution to an ARMA(p,q) process {X_{t}} exists and is also unique iff
that is if there are no roots of the autoregressive Phi polynomial on the unit circle. In the unit disk graph, and assuming we are plotting the inverse roots, only point B would violate the condition.
Causality of ARMA processes
Equivalently, if there are no roots of the autoregressive polynomial Phi on or inside the unit circle, that is
If we plot the inverse roots on the complex plane like in the graph we presented before, points A and B would violate this condition. That is, we would like to have the inverse roots strictly inside the unit disk!
Finding the coefficients of a causal ARMA process
Now that we know how to determine whether an ARMA(p,q) process is causal, how can we find the corresponding coefficients of the causal representation?
where the psi coefficients are determined by the recursive equation
where
Proof
Recall that if {X_{t}} is causal, we have that
where
Note that this implies that
We can make use of this fact to recursively solve for coefficients of the Psi polynomial. Written more explicitly, we have
Note that
Then, we can simply start comparing coefficients on both sides of the equation yields
Invertibility of ARMA(p,q) processes
or equivalently, if the are no roots of the moving average Theta polynomial on or inside the unit circle, that is
Visually, we would like to have the inverse roots strictly inside the unit disk:
, so that in the graph above, only the inverse root A would violate invertibility conditions.
Finding the coefficients of an invertible ARMA process
Similar to before, we can also find corresponding coefficients whenever the process is invertible.
where the pi polynomial coefficients are determined by the recursive equation
satisfying
Proof
We can show this in a similar way as we did before. First, recall that if {X_{t}} is invertible, we have that
This implies that
Once again, we can make use of this to obtain the pi coefficients, by noticing that
and
Then, comparing coefficients we obtain
Concrete Example
Let’s now see a concrete example using specific values for an ARMA(1,1) process. In particular, consider {X_{t}} ~ ARMA(1,1), given by
This implies that
We see that the corresponding AR(1) polynomial is
From which we see it has a zero at 1.25. Clearly, this falls outside the unit circle, which indicates that there exists an ARMA(1,1) process that is stationary and causal. Now, to find the causal representation of the process, we use the recursive formula
. Writing out the first couple of terms:
Which we can then, in turn, plug into
Similarly, the corresponding MA(1) polynomial is
which has a root at z=-5. This also falls outside the unit circle, which ensures that our process is invertible. Now, in order to find the inverse representation, we use the formula
, yielding
Which we can then plug into
One last brief example
Let’s consider the process X_{t} ~ ARMA(2,1) given by
Then, we have that
We can use the quadratic formula
to find the roots, yielding
We can then calculate the modulus for each:
Therefore we have no roots inside the unit circle, ensuring that our process is stationary and causal. However, notice that for the AR(1) polynomial
has a zero at -0.8, which indicates that the process is not invertible!
How to R
Now, we are going to simulate a couple of examples of plotting and examining polynomial roots for ARMA(p,q) processes in R. First, we import the required libraries:
Next, we are going to simulate an ARMA(2,1) process. . First, we simply create two vectors which will represent our models’ coefficients,
various ARMA models:
Next, we can numerically obtain the roots of both the AR(2) and MA(1) corresponding polynomials using the polyroot function as follows:
Note that we prepend an extra 1 at the beginning for a valid polynomial. We can now visually inspect these roots using the forecast::plot.armaroots package as below:
From these, using the conditions given before, we can see that since the inverse roots lie within the unit circle for both polynomials, we have a process that is stationary, causal, and invertible. If, for instance, one of our AR roots didn’t appear in the plot above inside the circle, then our process would no longer be causal or stationary.
Next time
That’s it for now! In the next article, we will take a look back at both the ACF and PACF, applied to ARMA(p,q) processes. Stay tuned, and see you next time!
