A Complete Introduction To Time Series Analysis (with R):: Prediction 1 → Best Predictors I

We’ve come a long way: from studying models to study time series, stationary processes such as the MA(1) and AR(1), then the Classical Decomposition Model, to Differencing and tests for stationarity. But how do we actually make predictions?? Well, as my Statistics professor said “starting linear is always a good idea”. So that’s what we will do! Be aware that in this section, we will be using some calculus and probability, so if you need a refresher on probability, check out this concept refresher that I wrote, or this excellent CS229 Probability Review document. Let’s jump into it!

Best Predictor of X_{n+h}

Recall the problem of obtaining the best linear predictor for some random variable, say Y. One could choose to solve the optimization problem

That is, we want to minimize the MSE (Mean Square Error)How can we find such a value of c? Well, from your Calculus class, you might remember that at the local minima and maxima of a function, the slope is zero, so one way is to take derivatives, set to 0, and solve for the resultant value. Furthermore, functions like the quadratic function are said to be “convex”, guaranteeing that the local minimum/maximum is also the global optimum. If some of these words don’t make much sense to you, don’t worry about it ( — Andrew Ng), as all you need to understand for now is that you can obtain the best value of c the way I described above and that that…