Hidden Markov Models for Time Series in R studio [Stock Market Data]
In this blog, you can expect to get an intuitive idea on Hidden Markov models and their application on Time series data. Further, I have also mentioned R packages and R code for the Hidden Markov model for reference.
This work has been done during my time at Praxis Business School(as a student). Now, Let’s get started…
Hidden Markov Models was developed by mathematician L. E. Baum and coworkers. Hidden Markov Models are a ubiquitous tool for modeling time series data. They are used in almost all current speech recognition systems and other areas of artificial intelligence and pattern recognition.
A Hidden Markov model is a tool for representing probability distribution over a sequence of observations. It assumes that the observation at time t was generated by some process whose state is hidden from the observer.
Let us take an example:
Suppose there are two friends named Alice and Bob. So Bob has a mood that changes based on the weather so if sunny then Bob is mostly happy and now he tells Alice on the phone that he is happy(Observations) so she infers from that information that it is sunny(hidden state) and if it’s rainy(hidden state) then Bob is mostly grumpy(Observation). Notice that here the weather at Bob’s place is hidden to Alice(Observer) and the only information she has is coming from Bob(i.e happy or grumpy). Let’s say we have observed that if it’s sunny then the probability of the next day being sunny is 0.8 and then the probability of the next day being Rainy is 0.2. And if today is rainy then the probability of the next day being Rainy is 0.6, so the probability of the next day being Sunny becomes 0.4. These Probabilities are called Transition Probabilities.
In the above dataset, we can see that the probability of being Happy given the day is Sunny is 0.8 and the probability of being Grumpy on a Sunny day is 0.2. Similarly, the probability of Bob Being Grumpy on Rainy day is 0.6 and the probability of being Happy on a Rainy day is 0.4. These probabilities are called Emission Probabilities.
We calculate the prior probabilities. P(S)=0.67 and P(R)=0.33.
Now, let’s say for three days Bob is Happy, Grumpy, Happy then what was the weather on these three days in other words what was the sequence of weather(hidden state) for these three days. We know that for each day we have two possible scenarios(Sunny and Rainy), so for three days let’s say Wednesday, Thursday, Friday, we have a total of 8 possible sequences(2*2*2=8). We will take each one of them and calculate what is the probability given these weathers(sequence of hidden state) that Bob was happy then grumpy and then happy and pick the one which gave us the highest probability. This is called Maximum A Posteriori Estimation.
For Simplicity(demonstration) let’s take 2 days(Wednesday, Thursday). The HMM model will look like the below picture. We have prior probability P(S)=0.67, Transition probability 0.2, Probability of Bob being Happy given the weather is Sunny P(H|S)=0.8 and Probability of Bob being Grumpy given the weather is Rainy P(G|R)=0.2. Note here the observed sequence is Happy →Grumpy and one of the four possible sequences of the hidden state is Sunny →Rainy. The probability of observing the sequence is the product of all these probabilities as according to Markov Property give a state at time t, the observation at the time t is independent of the states and observations at all other time indices.
P(hidden_state_sequence(sunny →rainy) , observed_sequence(happy →grumpy)) =P(prior(sunny))*Transition_probability(sunny →rainy)*P(happy|sunny)*P(grumpy|rainy)= 0.67*0.2*0.8*0.6=0.064
If you calculate this for all 4 possible scenarios, you will find that for hidden state sequence Sunny →Sunny, the joint probability is maximum. So, This model makes Happy →Grumpy most likely to happen for Sunny →Sunny sequence.
There are 3 types of problem that can be solved using HMM:
- Evaluation → Finding the probabilities of an observed sequence given an HMM model. (Forward/Backward Algorithm)
- Decoding → Finding the sequence of hidden states that most probably generated an observed sequence. (just what we did in the above example).(Viterbi Algorithm)
- Learning → Generating a Hidden Markov Model given a sequence of Observations. (Bauch-Welch Estimation)
Now, let us see how we can implement a Hidden Markov Model in R using sample data.
Package depmixS4 can be used to implement HMM in R studio(my version 3.6). I have taken a sample example from a blog where the data represents a physician’s prescription values with respect to time.
Here we are Learning the parameters of the Hidden Markov Model(transition probabilities and emission probabilities). Initialising the number of states as 2 in the above model.
In the above figure, we can see the transition matrix and the values of each state. From the response parameters, we can see two hidden states, high and low.High(114.73–27.23,114.73+27.23) and low(40.92–20.96,40.92+20.96). Let us look at the posterior probabilities..
We can see in the figure the hidden state labeled in the data. And that is the hidden sequence in our data.
Hidden Markov Model On Stock Market Data:
The stock market prediction has been one of the more active research areas in the past, given the obvious interest of a lot of major companies. Historically, various machine learning algorithms have been applied with varying degrees of success. However, stock forecasting is still severely limited due to its non-stationary, seasonal, and unpredictable nature. Predicting forecasts from just the previous stock data is an even more challenging task since it ignores several outlying factors.
HMMs are capable of modeling hidden state transitions from the sequential observed data. The problem of stock prediction can also be thought of as following the same pattern. The price of the stock depends upon a multitude of factors, which generally remain invisible to the investor (hidden variables). The transition between the underlying factors change based on company policy and decisions, its financial conditions, and management decisions, and these affect the price of the stock (observed data). So, HMMs are a natural fit for the problem of price prediction. The dataset has been scrapped off the WWW using the quantmod library in R and contains the information related to a particular stock’s performance. The goal is to find Hidde states and parameters of an HMM model using this data.
We extract information related to TWII, the dataset contains information related to opening and close stock price for a given day, the high and low price, and stock volume. We are interested in modeling with our HMM is the difference between the close value and the opening value of the current day.I have subsetted only 2 years of data.
After finding an HMM model for 5 hidden states, we get the above transition matrix.
In the above figure, we can see how our hidden states are mapped to the stock market graph. This gives us an idea about the different sequences of hidden states in the data and can infer a few insights from the hidden states obtained if we sufficient amount of domain knowledge.
Conclusion:
We have seen what Hidden Markov models are and applied to different datasets, got an intuitive idea on Hidden Markov Models and build an HMM on stock market data.
In the next blog, I will write about continuous-time Hidden Markov Models and algorithms in HMM for reducing the complex computation tasks. More Learning posts will come in the future so to check them out follow me on Medium, and stay tuned!!
Additional Resources and References:
- https://www.youtube.com/watch?v=kqSzLo9fenk → Must watch, the very nice explanation most of the content for the blog has been picked up from here.
- https://rubikscode.net/2018/10/29/stock-price-prediction-using-hidden-markov-model/ → Blog.
- https://blog.revolutionanalytics.com/2014/03/r-and-hidden-markov-models.html
- http://mlg.eng.cam.ac.uk/zoubin/papers/ijprai.pdf → This is an amazing paper to read.
- https://www.youtube.com/watch?v=j3r9a75zOvM → I have referenced this youtube video for my blog.
