How to Build a Market Simulator Using Markov Chains and Python

What is Markov chain

Markov chain is a mathematical system in which transitions happen from one state to another based on probability rules. As per the Markov system, the transition from one state to next is only dependent on the current state and not previous state, the states are discrete and can be represented in a matrix form.

Here is an example –

Let us say there is a brand X that sells dosas. There are many more brands that sell dosas and those are all competitors of X. As per current market analysis, X has around 20% share in the market, thus 80% of the market is captured by the other brands. Now, X wants to expand its business through advertising and campaigns. Let’s say, after the campaign –

• 90% of those already consuming X still remain with X
• 60% of those not consuming X will switch over to X

The two brands X and let’s say X’ (not X) represent the two states –

Let’s show the above conclusions using a transition diagram –

The diagram represents the probability of something staying in the same state or moving to another state. Let’s represent the same using a probability matrix, P –

Where XX = 0.9 represents those who stay with X, XX’ (0.1) represents those who use X’, X’X (0.6) represents those who switch to X and X’X’ (0.4) represents those who eat dosas of other brands. The above is called as a transition probability matrix and shows the transition from current state to the next state.

These states come as a result of the advertising/campaign for the dosa shop. How about the starting state?

Let’s represent the initial state as Si.

If you remember, the dosa share initially was 20% for the brand X.

This is called the initial state distribution matrix. Based on this information, we can predict the probability of people consuming brand X in the following week. Let’s say Si was the state this week. So, next week the state will be S1. To find this, we have to multiply the initial state with the probability matrix –

S1 = [0.2 0.8] 0.9 0.1 0.6 0.4

= [0.2*0.9 + 0.8*0.6 0.2*0.1 + 0.8*0.4]

= [0.66 0.34]

This means, after a week, the market share is expected to go upto 66%. We can generalise this formula as,

S1 = Si*P

S2 = S1*P — for predicting the state 2 weeks after the initial state

Sn = S(n-1)*P

Try the values by using the same probability matrix and see how the value changes week by week.

In the above example, we had 2 states. As the number of states increase, the matrix size will grow.

How do you implement Markov chain in Python

Python has loads of libraries to help you create markov chain. Since our article is about building a market simulator using Markov chain, we will explore our code keeping in mind our market simulator.

We have understood what is Markov chain and know that it is a concept that can be applied to various real-word scenarios to predict future outcomes based on current state. The data can be of any range — a week, a month, a quarter, a year — it is up to the accuracy that you need for your results.

You can get the datasets from anywhere, like Yahoo finance, or even Quandl that supplies free financial and economic datasets, just by using the quandl package of Python. This worked until 2018 but we can still use the data for learning purposes. Here is how to extract it –

We have used data from Microsoft Corporation. You can choose FB (facebook), Amazon (AMZN) or any others.

As we see, there are opening, high, low, adjusted closing and other values in the data. However, for prediction, we might not be necessarily interested in exact values. All we need to know are the ‘transitions’ — for example, if the value for coming 3 days has been 25.50, 28.00, 29.00, what will be the value on the 4th day? Will it go up or down?

But before sorting our data into high or low, we need to know the percent change of the close, high, low and volume values.

To use percent change function and many other data manipulation functions, we need to import the pandas package.

import pandas as pd
# Get the closing gap
Close_Gap = dataset[‘Close’].pct_change()
# The high percent change from the immediate previous row
High_Gap = dataset[‘High’].pct_change()
Low_Gap = dataset[‘Low’].pct_change()
Volume_Gap = dataset[‘Volume’].pct_change()
Daily_Change = (dataset[‘Close’] — dataset[‘Open’]) / dataset[‘Open’]
Outcome_Next_Day_Direction = (dataset[‘Volume’].shift(-1) — dataset[‘Volume’])

Let us create an empty new data set and fill it with the above data along with some other useful details –

new_dataset = []
new_dataset.append(pd.DataFrame({‘Close_Gap’:Close_Gap,
‘High_Gap’:High_Gap,
‘Low_Gap’:Low_Gap,
‘Volume_Gap’:Volume_Gap,
‘Daily_Change’:Daily_Change,
‘Outcome_Next_Day_Direction’:Outcome_Next_Day_Direction}))

Let us now print the new data set as -

print(new_dataset)

For better clarity, let’s get only few values using the head() method. But we have to first convert the list into a DataFrame object.

new_dataset_df = pd.concat(new_dataset)
print(new_dataset_df.head())

We don’t want the exact values. For our transition matrix and probability to work, all we need is the stable states. Based on the high low pattern, we can predict future market conditions.

This means that we can simply split the values into three categories — low, medium or high — or in short — L, M, H. This is called binning the data.

We are lucky that Python’s pandas package provides a function qcut that divides the data into equal sized bins.

# Close_Gap
new_dataset_df[‘Close_Gap_LMH’] = pd.qcut(new_dataset_df[‘Close_Gap’], 3, labels=[“L”, “M”, “H”])
# High_Gap
new_dataset_df[‘High_Gap_LMH’] = pd.qcut(new_dataset_df[‘High_Gap’], 3, labels=[“L”, “M”, “H”])
# Low_Gap
new_dataset_df[‘Low_Gap_LMH’] = pd.qcut(new_dataset_df[‘Low_Gap’], 3, labels=[“L”, “M”, “H”])
# Volume_Gap
new_dataset_df[‘Volume_Gap_LMH’] = pd.qcut(new_dataset_df[‘Volume_Gap’], 3, labels=[“L”, “M”, “H”])
# Daily_Change
new_dataset_df[‘Daily_Change_LMH’] = pd.qcut(new_dataset_df[‘Daily_Change’], 3, labels=[“L”, “M”, “H”])
With this information of L, M, H, we can find the event pattern which we get by concatenating the H L M values of close gap, volume gap and daily change –
new_dataset_df[‘Event_Pattern’] = new_dataset_df[‘Close_Gap_LMH’].astype(str) + new_dataset_df[‘Volume_Gap_LMH’].astype(str) + new_dataset_df[‘Daily_Change_LMH’].astype(str)
print(new_dataset_df.tail(20)

Now, we see a lot of changes that lead to these kinds of pattern — LML, MMM, MHH, MLH and so on. Let us compress this data to get the series of events as a string.

Reduce the data set, compress it to get only the most interesting and prominent variations removing the weak variations.

Once we have got this, we have to divide our data into the training and testing set. Most of the data has to go into the training set to train the model as accurately as possible.

This can be done by using train_test_split from sklearn package. Let us say we give about 80% of data for training, thus having 0.2 or 20% data for testing.

We can split the data based on the outcome_next_day_direction (positive or negative).

compressed_set_pos = compressed_set[compressed_set[‘Outcome_Next_Day_Direction’]==1][‘Event_Pattern’]
print(compressed_set_pos.shape)
compressed_set_neg = compressed_set[compressed_set[‘Outcome_Next_Day_Direction’]==0][‘Event_Pattern’]
print(compressed_set_neg.shape)

After getting the outcome direction, let us now build the markov grid.

• Find the unique event patterns from the compressed set. For eg, LHM, MLH etc…
• Form a pattern with from_event (eg. LMH) and to_event (eg. MLM)
• If a match is found, join the pattern values
• Create the to/from grid and the pivots
• Fill the NaN values with zero
• Build the transition grid with positive and negative compressed sets

Predicting the outcome is simple based on this matrix. We have to find the odds and create a confusion matrix with the actual and predicted values.

To get the code on building the matrix grid and predicting outcome, visit the code link by viralml. The video is based on a blog that details how markov chain works for stock market predictions.

Conclusion

Predicting stock market values is one of the most common real-time applications of the markov principle. In this article, we have tried to cover the basics and gone ahead to explain how to build a simulator. You can try running the code on your machine by using any dataset and also check the accuracy of the results. The main tedious part of using the markov chain is to clean and reduce the dataset to find and segment the datasets. The rest of it — just follows.