Multivariate time series forecasting and analysis of the US unemployment rate — Part 2

Photo by Anne Nygård on Unsplash

In the previous post (linked here: Part 1) we discussed the importance of unemployment rate forecasting. We also framed how we approach the analysis and time series modeling. So far we covered the problem definition and the data sources used.

In this post, we’ll look at few relevant python code, charts and data snippets.

And importantly, we will cover the data preparation phase and look at the exploratory data analysis which has unique set of analytical tools to understanding the time series data.

So, continuing on …

Here are the libraries used for this analysis. Statsmodels is a very useful package for working with classical time series methods.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from statsmodels.tsa.seasonal import seasonal_decomposepu
from statsmodels.tsa.stattools import adfuller
from statsmodels.graphics.tsaplots import plot_acf
from statsmodels.graphics.tsaplots import acf
from statsmodels.graphics.tsaplots import plot_pacf
from statsmodels.tsa.stattools import grangercausalitytests
from statsmodels.tsa.ar_model import AutoReg
import statsmodels.api as sm 

Here is a snapshot of a few sample records from the multivariate time series.

Multivariate time series data for Unemployment Rate forecasting

Using seaborn for univariate seasonal plots:

plt.subplots(figsize=(20,5))
sns.boxplot(x=ue_seasonal_plotdf['month'],y=ue_seasonal_plotdf['UnemploymentRate'])py

This gives us the seasonal plot by month, followed by seasonal plot by quarter. Leads us to believe nothing quite extraordinarily different across months or quarters.

Unemployment Rate Seasonal Plot by Month

Unemployment Rate Seasonal Plot by Quarter

3. Data Preparation Phase — Time series data has a unique characteristic in that there is a temporal dependence of variables. A variable at a current time has dependence on the variable at one or more than one periods.

A time series has three “systematic” components that can be described and modeled. These are ’base level’, ’trend’ and ’seasonality’, plus one “non-systematic” component called ’noise’. The base level is the mean value in the series. The time series is said to have a trend when the time series values increase or decrease over time (positive or negative slope). When there is a pattern repeated at regular intervals, the time series is said to have seasonality. Any random variations is the noise. Every time series data is a combination of these four components. While, base level and noise always exist in a time series data, trend and seasonality can optionally exist.

There are several factors that need to be considered due to the nature of time series data which includes preprocessing and preparation of data for the analysis of such data. There are several transformations that are applied to make the data work with the methods used in the analysis and forecasting of the time series data. Additional feature engineering is done using these transformations.

a. Box Cox / Log Transformation — This removes the changing variance over time. This is done using np.log() function using numpy library. This is applied to variables MoneySupply, ProducerPriceIndex, CandILoans and CRELoans for training and forecasting with the neural network models.

b. Differencing — This is applied to de-trend the data and to make the data stationary. Differencing is used to attain stationary time series which is a model assumption for working with Vector Autoregressive (VAR) models. The differencing involves using the difference in values over two consecutive periods of time. This is done using diff() function from the pandas library to calculate a differenced time series value.

c. Normalizing — Normalization of data makes value of each time series data point to be between 0 and 1. This data transformation is required to work with neural networks. This is done using MinMaxScaler() function from the sklearn.preprocessing library.

d. Standardizing — Standardization of data normalizes value of each time series data point on a z-scale with a mean of 0 and standard deviation of 1. This was done using StandardScaler() function from the sklearn.preprocessing library. This data transformation is required to work with neural networks.

For final forecasting of future (test) data, the normalization and standardization are reversed to compute the error. Both normalization and standardization is applied as part of data transformation to train and forecast using neural networks.

4. Preliminary (Exploratory Data) Analysis: The datasets for this analysis were sourced from the Federal Reserve database that did not have issues with missing data. The analysis is done separately for univariate setting, followed by multivariate setting.

Univariate Analysis

As shown in Figure 1 the unemployment rate is stationary (constant mean) with varying volatility over the range of period shown here.

Figure 1: Time series plot of US unemployment

Figure 2 shows the distribution of unemployment rate for this period, it ranges from 3.5% to over 10.5% with the distribution skewed to the right. This data doesn’t quite have a normal distribution.

Figure 2: Distribution of Unemployment Rate

Figure 3 shows a comparative view of the unemployment rate and its first order difference (1D) computed by calculating the difference in consecutive values of the unemployment rate data).

Figure 3: Distribution of first order difference of unemployment rate compared to unemployment rate

Figure 4 shows the distribution of the first order differenced unemployment rate. This data is centered around zero.

Figure 4: Frequency distribution of first order difference in unemployment rate

Time Series Decomposition

To improve forecasting accuracy time series data often requires to be analyzed by decomposing it to understand the underlying patterns.

There are three components: trend, seasonality and residuals. Additive decomposition can be expressed as:

Unemployment Rate = Trend + Seasonality + Residuals

In Figure 5 below, the top chart shows the original time series (same as Figure 1 above). This is decomposed into the three components trend, seasonality and the residuals, which is whatever remains after the trend and seasonality have been separated from the time series. This time series does not have a trend, but it does have a strong seasonality to it. The residuals are dispersed around the mean of zero.

Figure 5: Classical additive unemployment rate time series decomposition

Figure 6 shows a histogram of residuals which shows that the residuals are normally distributed across the mean zero.

Figure 6: Unemployment rate time series decomposition

Autocorrelation and Partial Autocorrelation

Autocorrelation shows the extent of linear relationship between the value of a variable in a time series at time t and the value of variable in that time series at a time t+h or t-h. A positive correlation indicates the value of the variables move in the same direction, whereas a negative correlation indicates the value of the variables move in opposite directions. When autocorrelation is zero, the temporal dependency is difficult to establish. Autocorrelation plots help understand this temporal relationship. In Figure 7 shown below, the x axis shows number of lags, and the y axis shows the autocorrelation which is normalized between the values of 1 and -1. The orange band shows the 95% confidence interval. This chart shows that unemployment rate autocorrelation can be seen until 24 lags (24 months).

Not only is there linear relationship between xt and xt+h or xt-h there is linear relationship between xt and the lags leading upto xt+h or xt-h. In other words, xt also has dependency on intermediate lags, therefore autocorrelation is not the correct measure of mutual correlation between xt and xt+h or xt-h. This requires removing dependency of the intermediate lags. This is done using partial autocorrelation function. Figure 8 shows a partial autocorrelation chart which shows temporal dependence of unemployment rate at a time t with that of t-1 or lag=1 with some weak dependency shown between lags 2 to 5.

Figure 7: Autocorrelation plot for unemployment rate

Figure 8: Partial Autocorrelation plot for unemployment rate

Test for Stationarity

One important assumption that is made for inferential statistics to be reliable is that the population from which a sample is drawn does not undergo any fundamental change over the individual samples or over the time during which the samples are collected. This assumption is important for the inferential statistics to be reliably representing samples not previously seen before. This assumption also applies to time series data (Pal & Prakash, 2017). This assumption in time series is called stationarity, and it implies that the mean, variance, and autocorrelation are constant, and so these can be used for future occurrences of the time series.

Augmented Dickey-Fuller (ADF) tests are used for detecting stationarity in time series data. On applying ADF tests on Unemployment Rate the p-value of the ADF test was found to be 0.033. And, since the p-value for Augmented Dickey-Fuller test is < 0.05, the null hypothesis that the time series is non-stationery is rejected, concluding that the unemployment time series is stationery. Detailed test results of ADF test on all the variables is reproduced in table below. ADF Tests (H0: time series is non-stationary) are assessed at a 5% significance level on original data after differencing (1d) used in unemployment rate analysis and forecasting are shown below. The p-values are obtained for 4 significant digits.

Results of Augmented Dickey-Fuller test for stationarity

Multivariate Data Analysis

Figure 9 shows all of the original macroeconomic indicators that are used for forecasting unemployment rate. Each of these indicators have been plotted for the same historic period this analysis was done for. A brief discussion of interesting observations for each of the variable follows.

The fed funds rate which is the interest rate that banks charge each other for overnight borrowings was near zero during the financial crisis of 2008 and following that for few years. Producer Price Index (Supplier price inflation), Money Supply (Amount of liquidity provided by Treasury) and Commercial Loan Activities (Commercial & Industrial loans, and CRE loans), are not stationary as determined by ADF tests. These four indicators were differenced to make them stationary.

Figure 9: Macroeconomic indicators for forecasting unemployment rate

Additionally, there was differencing applied not only across a single month, but also over a three month period as shown in Figure 10. This was done so as to understand the pattern in the data consistent with methods used for financial data analysis. Financial data is not only evaluated month over month, but also over key recurring periods for financial reporting such as quarter over quarter change, year over year change. This analysis was done using only the three month rate change for the purpose of exploratory analysis. It was not extended for use with any models in this analysis so as to keep the scope of variables to a limited size.

Figure 10: Macroeconomic indicators with 1 and 3 month rate changes

This concludes the exploratory data analysis.

In part 3 we will cover autoregressive methods for both univariate and multivariate analysis.

If this was useful, then here is a resource you might find of value even more where I publish more frequently: www.thefractal.co

On this site, you will find fresh ways of analytical thinking, applications of machine learning, and, more broadly — artificial intelligence as well as analysis and insights from emerging research and development on these topics.

Until next time.

References:

Cook, T., & Hall, A. S. (2017). Macroeconomic Indicator Forecasting with Deep Neural Networks. Retrieved from dx.doi.org: https://dx.doi.org/10.18651/RWP2017-11

Pal, A., & Prakash, P. (2017). Practical Time Series Analysis. Birmingham: Packt.

Hyndman, R., & Athanasopoulos, G. (2021). https://otexts.com/fpp3/. OTexts.