Algorithmic Trading 101 — Lesson 5: More Machine Learning
In our last lesson, we introduced supervised, unsupervised, and semi-supervised machine learning algorithms. As discussed, ML algorithms have great benefits related to automated execution. However, there are still humanistic decisions to factor in. Just like other algorithmic trading approaches, there is no one correct method — different data types and/or objective functions perform better or worse according to specific types of ML algorithms. Methods must be consistently back-tested in to find the optimal solution to your use case.
ML Framework
In this lesson, we’ll explore some specific techniques in greater detail. As we move down the list, here’s a general framework to keep in mind. When creating a ML algorithm:
- First, data-processing is required to be compatible with your chosen ML technique.
- Use the ML algorithm to generate predicted values on your “training set” — a subset of the data you want to use for initial model calibration.
- Compare predicted values to true values to create residual error function.
- Iterate! Update parameters and rerun your model to minimize that error function.
- Optimize and validate the algorithm to create confidence intervals.
- Finally, use your “final” algorithm on test data and put into production.
Types of ML Algorithms
Naïve Bayes
This method is helpful in determining the probability of categorical classification variables. E.g., if the price of Ethereum declined for the past two days, what’s the likelihood that the price will increase today? “Naïve” in this context implies that all the variables are independent of each other, and “Bayes” refers to Bayes’ Theorem, which relates conditional probabilities.
Mathematically, the Naïve Bayes algorithm looks like the following:
Here is an example of the model, using time of day as the event and whether the price increased or decreased according to the event:
Thus, if the given time is morning, we have a higher probability that there will be a decrease in the price. While this example uses a discrete random variable, we can model more complex probability distributions too — Gaussian, Bernoulli, or Poisson per say. However, the Naive Bayes formula will stay the same, just the calculation of the individual probabilities will differ.
To use this model to trade, you’ll need to decide at what probability level that you have sufficient confidence to act on that prediction. In this example, is 60% probability that price will decrease in the morning high enough for you to sell your positions?
For more on the Naive Bayes algorithm, see this article (Naïve Bayes Classification With Sklearn). In addition, this GitHub has an implementation of Naïve Bayes in Python that can be tailored to your needs.
Natural Language Processing
Today, the prices of assets fluctuate in relation to news. Articles, tweets, and other content can travel the world in seconds. Traders often read headlines and make decisions on their positions based on the interpreted sentiment. However, the time for a human mind to read, process, and trade based on a headline can often be too slow. Natural language processing (NLP)— teaching computers to read words and phrases, and understand their meaning — can help enhance the traders’ reaction speed. Furthermore, an effective algorithm can automatically take appropriate positions based on the news.
Let’s suppose there’s an article that states “ZRX is booming after fund manager buys stake.” If a person read this sentence, they’d recognize the connotation of this sentence implies that the ZRX will increase in value.
A NLP algorithm could parse this sentence into:
- Subject words = “ZRX” and “fund manager”
- Key words = “booming” and “buys”
- Filler words = “is”, “after”, and “stake”.
The algorithm then works similar to the decision tree algorithm; it evaluates each word within the phrase to determine a positive or negative relation. Some words like “hack” can be defined to always mean a negative effect. Some words like “inflation” are significant but meaningless without knowing the predecessor word (“higher” or “lower”). The Stanford Sentiment Treebank and similar databases already classify many commonly used words.
And we don’t have to rely on just one headline either. We could process many headlines — from many websites, Twitter feeds, or other sources — to count the frequency of keywords. And if there’s significant online activity of the same sentiment, we may predict how much a certain asset will increase or decease in value.
To read more about how to incorporate some of these approaches into your trading strategies, we suggest the following resources:
- Applying NLP to forum posts — Predicting Fluctuations in Cryptocurrency Transactions Based on User Comments and Replies
- Applying NLP to the cryptocurrency market and Twitter feeds: Impact of Social Media on Cryptocurrency Trading with Deep Learning
- On the Importance of NLP for Cryptocurrency Forecasting
- A Python implementation of NLP, used to forecast cryptocurrency prices in real time
Reinforcement Learning
Reinforcement learning methods rely on running many iterations of a model to check what the rewards or consequences are within each state. This is the technique utilized by AlphaGo and other similar projects to teach computers to beat humans at specific games or tasks. Within a game, each move has implications for future moves. When playing a game of chess, grandmasters often think numerous moves ahead to see what the future positions will be.
Similarly, the reinforcement learning algorithm creates a system where we will reward good decisions and punish bad decisions. The reward will be a quantifiable variable that we will try to maximize. At each state, an agent is allowed to take any fixed number of possible decisions — in a trading context, those would be the decisions to buy, hold, or sell. After your decision, the algorithm pushes you to the next state or time period and evaluates whether that decision is rewarded or punished. When the algorithm is first run, it essentially takes random guesses on what the action should be. However over multiple iterations, the algorithm learns which actions may be more beneficial in different contexts. Of course, if the model is run too many times, we would run into an overfitting issue. This is when the algorithm operates according to the given data points too closely and is unable to adapt to new data.
Let’s suppose we are looking at a two-step model where we are trying to go from point A to point C, and we are able to choose between B1 or B2 for the intermediate steps. Let’s assign the following rewards and punishments for the respective moves:
A → B1 = 10
A → B2 = -15
B1 → C = -25
B2 → C = 25
On the path A → B1 → C, we obtain a net result of 10-25 = -15
On the path A → B2 →C, we obtain a net result of -15+25 =10
So in this example, the path through B2 is the optimal path (since it ends in higher value).
Standing at state A, it is naturally more beneficial to go to state B1 as opposed to state B2, as the immediate result is 10 vs. -15 for the first action. However, we can also see that in the long run, it is more profitable to choose B2, as we end in a net result of 10 vs. -15.
In trading terms, you could think of the intermediate time B as our “unrealized PnL” and once we reach state C, we “realize” our final PnL. A reinforcement model, when run multiple times, can optimize which path to take — even if it appears that certain states are the less profitable option. That’s why it’s important to examine the longer time horizon in trading rather than worry about singular daily moves. An advantage of these models is end-to-end optimizations while also factoring potential drawdowns in the model. A disadvantages of these model is that as your nodes and actions grow, the time to estimate this model as increases.
To read more about reinforcement learning in a trading context, try:
- Applying the reinforcement model to Twitter data — Trading the Twitter Sentiment with Reinforcement Learning
- Trading Bitcoin with Reinforcement Learning
- An application to the stock market (Advanced) — Reinforcement Learning For Automated Trading
- A TensorFlow implementation for BTC trading
Deep Learning/Neural Networks/LSTM
These methods use previous inputs and outputs to determine how we can predict future outputs. Specifically, a neural network is a machine learning algorithm that is designed in the same way that your brain works, where more recent “memories” carry more relevance/weight than older memories.
In a recurrent neural network, we take inputs and run it through the algorithm to provide outputs. For each data point, we obtain weights of the algorithm to see how much impact it should have on the next data point. As we feed more and more time series data points into the model, the weights of the older points decrease until they become zero and get dropped from our model.
For example, let’s presume that you include the volume of trading activity as an explanatory variable for your model to predict a price. The volume of trading activity matters for the position that you are taking on day 1. In 10 days, day 1 volume’s impact will be less than it initially was for predicting the day 1 price, and in 100 days the impact of the day 1 volume will have a negligible impact on price. Instead, the day 99 volume will have the largest weight. The model recursively updates the weights each time to provide the best analysis at the given moment. For an example of neural networks applied to Bitcoin trading, please see this (Bitcoin technical trading with artificial neural network) study.
Another application of “deep learning” is the Long Short Term Memory (LSTM) algorithm. A trader may have an overall price view in the long term but also wants to keep track (and possibly take advantage) of price gyrations in the short term. In LSTM models, there is a “hidden” state that keeps tracks of the short-term states and weighs them relative to their long-term impact. These models can also be tuned to focus on the short-term state prediction if a good trading opportunity presents itself. Determining the right short- vs. long-term timeframes can be difficult, but if optimized correctly, these models can lead to accurate predictions in a live trading environment. See this (Machine Learning the Cryptocurrency Market) paper for a LSTM model applied to cryptocurrency trading.
Challenge #5 — Machine Learning Models
- Choose a machine learning model that fits your objective
- Implement that model on cryptocurrency data, and submit to The Ocean for feedback
*Remember, anyone that participates on Telegram or sends us a solution anytime during the course of our Algorithmic Trading 101 series is eligible to receive part of $5000 in cryptocurrency prizes.
__________________________
🤖 Links to Lessons 🤖
The Syllabus & How to Win
Lesson 1: Time Series Analysis
Lesson 2: Data, Strategy Design, and Mean Reversion
Lesson 3: Intro to Arbitrage Strategies
Lesson 4: Portfolio Management and Machine Learning in Python
Lesson 5: More Machine Learning
Remember to join our Telegram if you have any questions!
