Daytrader.ai — Machine Learning Applied to Intraday Trading
Daytrader.ai is applying machine learning to intraday trading strategies. This blog will serve to outline my notes and learning as I progress deeper into the abyss. There will be short blog entries, technical descriptions source code, and even free data to experiment with. Once you start attacking the problem you will find yourself hooked on coming up with new ways to reframe problems to computers. Consider yourself warned!
Sign up for the Beta at http://daytrader.ai
Let’s first start with a clear description of the problem we are trying to solve. We’ll then take a look into whether the problem is even truly solvable. After that we can begin to add constraints and dive deeper into the tasks.
Problem Description
The problems of financial forecasting can in most cases be simplified to this question: “Given the input value is X, can we predict the future price of something at time t?”
From there, the question to hone in on is this: given the history of securities on the NASDAQ-100, can we predict the price of a security 5, 10 or even 20 minutes into the future? There is a long list of organizations globally trying to apply machine learning techniques to this very problem. However most of them provide little value in answering the question. And almost all data testing and research is conducted secretively, behind closed doors .This does not yield any value. You could be thinking that the lack of value is inherent of the problem itself , believing, perhaps, that “there is no reasonable way to predict the value of the markets”. So before we continue I think we should further refine our premise.
Day trading is speculation in securities, specifically buying and selling financial instruments within the same trading day. Strictly, day trading is trading only within a day, such that all positions are closed before the market closes for the trading day.
A day trader looks to identify entry and exit position on a stock that has favourable conditions. These favourable conditions will yield many small term profits that add up to large gains. So we first need to convince ourselves of a few things:
- Are there people that day trade for a living?
- Do some day traders consistently beat out a buy and hold strategy?
- Are these traders identifying trends in the market that allow them to swing odds in their favour?
If we believe that there are people that are able to identify favourable patterns in the market then we should be able to train a machine to perform this task at superhuman levels. This is the premise and the goal of daytrader.ai
Process
We need to identify the strategies that successful day traders use to signal entry conditions into the market. As we will learn in future posts this technique can be split into 2 processes:
- A high level pattern description. Here we are simply looking to identify the entry semantics that occur for potentially hundreds of predefined strategies. This is done with a highly scalable and robust time series pattern matcher. (Apache Flink).
- Machine learning. Once we have triggered a pattern we can look through our historical data and find all the instances in past that pattern was triggered as well as the outcome (price after 5, 10, 20 min). We can then generate training example for phase 2 of the problem which we use machine learning to create a probability distribution over these past entries.
In order to accommodate the search, daytrader.ai has created a distributed platform to perform forward and back testing on the stock market. I will review the architecture and post some youtube videos of the progress along the way. The software allows us to define patterns, visualize and perform searches over the past 20 years of NASDAQ-100.
(Screenshot of technology)
Trading Pattern Overview
We can loosely analyze a stock in one of two ways:
- Fundamental Analysis
- Technical Analysis
“Technical analysis and fundamental analysis are the two main schools of thought when it comes to analyzing the financial markets. As we’ve mentioned, technical analysis looks at the price movement of a security and uses this data to predict future price movements. Fundamental analysis instead looks at economic and financial factors that influence a business. Let’s dive deeper into the details of how these two approaches differ, the criticism against technical analysis, and how technical and fundamental analysis can be used together.”
Source: https://www.investopedia.com/university/technical/techanalysis2.asp
Day Traders are mostly concerned with technical analysis and as such have developed a large grab bag of tools for analyzing price movement. These are referred to as “technical indicators”, and we will refer to a number of these during this blog series. For a full list of technical indicators take a look at this page here:
Useful indicators:
- Price
- Volume
- Simple Moving Average (SMA)
- Exponential Moving Average (EMA)
- Relative Strength Index (RSI)
- Stochastic Oscillator (STOCH)
- Moving Average Convergence Divergence (MACD)
Additional list of useful terminology:
https://www.investopedia.com/exam-guide/series-65/trading-securities/trading-terminology.asp
Also we will be referring to “Support” and “resistance” a lot in articles. If these terms are not familiar to you then you can take a look at this: https://www.investopedia.com/trading/support-and-resistance-basics/
These indicators are built right into most advanced trading packages. Day traders will set up their software in such a way so that it alerts them of a specific condition. Another way of saying this is the indicators are the building blocks that day traders use to define patterns. Here are some examples of places that have defined patterns:
Let’s dive into one of these patterns referred to as EMA crossover. In this pattern we are looking for when a faster moving average (say a 15 EMA) crosses over a slower moving average (say a 65 EMA). We are using this as an example mostly because of its simplicity. It easy to both visually see this on a chart as well as define this type of condition to a computer.
Summary
Now that we have framed our problem we can go ahead and start to define how we will collect data for these patterns. In the next post we will talk about some of the specifics for which to create a platform to forward and backtest. data This is an absolute must if you are serious about working with stock data. I will also be posting some public data sets from the platform that people try their own ideas on.
Sign up for the Beta at http://daytrader.ai
Some additional reading:
