What Is Stationarity?
Stationarity plays a very important role in time series analysis. When we have a number of observations of a certain parameter at different times, we naturally want to make use of all of them with predictive or classification models, etc. But quite often our data is not (yet) ready for analysis as it is considered to be not stationary.
But what exactly does that mean? Well, stationarity can be defined mathematically as a measure of some stochastic (random) process where the cumulative distribution function describing this process doesn’t depend on a time parameter. Speaking more precisely, the process is considered strictly stationary, strongly stationary or strict-sense stationary when a partial derivative of the process cumulative distribution function by time is always equal to zero.
Sounds quite complicated, indeed. Not surprisingly, it also turns out that this condition is quite rarely met in real life. Strict stationarity requires all distribution parameters (often called moments) — such as variances, expected values, etc .- to never depend on time. There are additional types of stationarity (call these stationarity extensions), such as:
- First-order stationarity where a time series has a constant mean, while other statistics like variance can change over time
- Second-degree stationarity (often called weak stationarity) where the mean, variance, and auto covariance do not change over time. This is probably the most common situation with real-life data
So if stationarity rarely occurs in nature, how do we determine if our data is stationary or not?Quite often, it is hard to tell whether data fits into stationarity, and we need to use specific tests to determine if it is, like the ones below:
- Unit root tests (Augmented Dickey-Fuller test, Zivot-Andrews test)
- Prestley-Subba-Rao test
- Wavelet-Based test
- Run-sequence plot
All of the above are rare, except of the Augmented Dickey-Fuller (or ADF) test which is a best practice example of A/B testing. By attempting such a test, we are trying to determine if the process is stationary with a high degree of confidence. In other words, our time series data is stationary if it is free of any trends (linear or non-linear) and any kind of seasonality.
Given the argument that stationarity rarely occurs in nature, a classic example of a stationary process is white noise. It fits the definition since its mean doesn’t change over time, it is totally random (meaning no trend), and its variance is finite.
Looking at its plot, one can see only a set of values oscillating around a constant mean over time, within a given interval of finite variance. While this might seem arbitrary, white noise; however, does play an important role in signal processing theory, as well as having important applications in music, electronics engineering, and computing (as a random generator). It even help us fall asleep!!!!!
If our data isn’t stationary, and it is important that is it, how do we achieve this? We can impose stationarity onto our data using algorithms like those listed below:
- Differencing the data — the most common way of achieving stationarity with non-stationary data. By differencing we technically create a new data set containing the differences between the observations on a given and a previous moment of time. Normally one differencing is enough, even though sometimes we have to difference our data twice to achieve stationarity. Each differencing reduces our dataset by one observation, which is negligible with larger datasets.
- Fitting a trend — the method is used when we can detect a linear or curved trend within our data. We effectively fit the trend to our data and work with the residuals that are often stationary.
- Smoothing the data (informal term) — applying a square root or a natural logarithmic transformation to our data in order to achieve stationarity (adding some constant to the data in case of negative values)
But be warned, by assuming stationarity about our data we introduce some bias into our thought and analysis process. Indeed, stationarity permits us to use a number of algorithms and methods heavily relying on it, but we can never 100% conclude that our data is stationary. However, the importance of stationarity outweighs the risks for given data because stationary data are much easier to analyze and interpret, as well as to make inferences based on such analysis. The intuition behind stationarity is that for stationary data we analyze its distribution instead of analyzing individual values and their variance, and in order to be able to do so effectively, we need our distributions to not change over time.
