USD-MNT Exchange Rate Follow Up
It’s always good to remember the basics
In December 2018 I wrote an article forecasting the USD-MNT exchange rate using machine learning. The subtitle was “When a good model doesn’t work”, so keep this in mind as you read below. The model I developed had an average error of about 28 MNT when forecasting 3, 6, and 12 months ahead. This sounds pretty good, but the end of 2018 saw a large depreciation of the tugrik and our model forecast appeared to be well under the trend. So how did the model do? Let’s take a look!
Look at the difference! That’s a pretty bad forecast. It’s not much better than drawing a horizontal line from the forecast date. So what went wrong?
Back to Basics
A month or two after I wrote Forecasting USD-MNT Exchange Rate — Part 2: Machine Learning I realized I made one of the biggest mistakes possible in time series forecasting. I gave the model data it shouldn’t have had.
When I performed the data split into training and testing sets, I did so randomly. What I should have done was take a certain amount of my data at the end of the dataset and save that as my test set.
What I was doing was overfitting my model by inadvertently giving it the information it needed to guess the test set.
As embarrassing as this is, I think it is important to note this as it’s a valuable learning experience.
Data Availability Lag
Another issue in our model is most definitely data lag. Balance of payment data is available in a 1 to 2 month lag via MongolBank. This means even if our model is good at forecasting exchange rates a month in advance, it isn’t useful unless we get the data on time.
This can be alleviated running simulations based on likely possible values, but it adds more uncertainty to our model.
Hopefully, this article serves as a reminder to never forget the fundamentals and stay vigilant in testing forecasts. In the coming months, I hope to recreate this model in a more reliable way.
I’m still not sure if it will be possible to forecast the Mongolian Tugrik accurately. According to a European Central Bank economist, forecasting exchange rates in developed countries is so simple it can be done on the back of a napkin. Of course, Mongolia isn’t a developed country, and at least to me, much of what is actually happening with exchange rates is still a mystery to me.