Multi-step bitcoin time series forecasting with different historical periods using AWS DeepAR

How far should you look back to predict the next 5 minutes?

In this post I will talk about multi step forecasting for two reasons. First, in my previous post one of the things that I did not like was that the sequence model did not capture a pivotal moment/trend change. Second, most of the time when people talk about time series forecasting, they mean one step prediction.

A picture might help put things in perspective; and if you have enough patience by the end of the article you will see some actual results. I will be brief.

Let’s consider the following 30 minutes timeline. Past is represented by 1–25 range and future by 25–30 interval. We are at minute 25 and we want to predict some value at minute 30. There are many strategies: direct, recursive, hybrid, multi-output.

30 minutes timeline

But I am not that interested in taxonomy or technical analysis (or at least not in the trend-lines definitions — not that I do not agree, I just have a different focus).

What practical options do we have and what differences do we note?

We will use a model that provides multi-output. This sequence model can predict in one shot a set of values: for 26, 27, 28, 29, 30. Plus you can pick how far back you will feed data into the model: last 5 minutes, last 25, etc. And this is the part that we are going to experiment with.

Using bitcoin data from my previous post I will shed some light on this matter. And this time I will be using AWS rather than GCP. I like AWS DeepAR model for multiple reasons:

• Transparency (not a black box model) — paper is available
• Flexibility for how far back and ahead you can go. For example, you can feed the last 30 minutes or the last 5 minutes to predict the next 5 minutes or the next 10 minutes
• Feeding data into a pre-built model allows us to quickly look at the results
• Low cost for experimenting

To be consistent we will have the same 6 features for all models (based on trades and market orders). In addition, data will be re-sampled at 1 minute intervals. The instance/machine used for training is ml.m5.24xlarge for all models.

While AWS does all the heavy lifting we still need to put the data in the requested format (might not be intuitive first, but with practice it will make sense). Will train all models for 20 epochs. All training jobs took similar amount of time.

AWS SageMaker training jobs duration

Here is the ranking of some of the attempted models based on the RMSE on the test data:

1. 25 minutes history — 104.52
2. 60 minutes history — 113.89
3. 5 minutes history — 254.95
4. 4 minutes history — 311.08

Now, let’s take a closer look at July 18 2019 because this was quite a volatile day in the bitcoin world.

Because the chart will get messy I will only display results from 3 models.

This is the actual price trend.

Bitcoin actual price for July 18 2019

Next, let’s add one by one each layer.

Will start with the 25 minutes history model.

Bitcoin actual price and predicted (5 minutes ahead) value using a model with 25 minutes of history for July 18 2019

Next, we are adding the 5 minutes history model.

Bitcoin actual price and predicted (5 minutes ahead) value using a model with 25 and 5 minutes of history for July 18 2019

Lastly, let’s add the 4 minutes history model.

Bitcoin actual price and predicted (5 minutes ahead) value using a model with 25, 5 and 4 minutes of history for July 18 2019

One of the questions that we wanted to answer was around change in trend: can we predict it? Let’s zoom into the biggest uptrend and downtrend.

The downtrend of the actual price is not that radical, but all models over-react.

All models over-react to the downtrend

The uptrend is a bit more obvious.

Predicted values for all models get out of line a few minutes ahead before the actual price spike

Given that we are looking at one day, it is not that simple to draw the conclusion if this is more noise or more signal. And this is not trading advice.

Finally a few observations are in order:

• A build in model is a lot easier to try, but I still feel that without hyper parameter tuning (which while available I did not try) the output is pretty erratic. In addition, pre-processing data, especially for inference, is a bit challenging (if you use dynamic_feat you need to provide more data: length(target) + prediction_length)
• I was impressed with Sagemaker — it gives you all that you need to get started
• Given our intervals, 25 minutes of history will get you the best RMSE when trying to predict 5 minutes ahead

Bottom line, July 18 2019 was a volatile day, but I still feel that this multi step/output model does not capture fully the change in trend.

I do have one more idea that I want to try, if I have time. Stay tuned.