Introduction to TradeMaster: A New Standard of Reinforcement Learning Framework for Quantitative Finance
We are releasing an open-source quantitative trading platform based on reinforcement learning: TradeMaster
TradeMaster, developed by Nanyang Technological University, is a unified, end-to-end, user-friendly quantitative trading platform covering four major financial markets, six trading scenarios, 15 reinforcement learning algorithms, and a series of visualization evaluation tools!
Platform address: https://github.com/TradeMaster-NTU/TradeMaster
Background
In recent years, artificial intelligence technology is playing an increasingly important role in quantitative trading strategies. Due to the outstanding decision-making ability in complex environments, applying reinforcement learning technology to quantitative trading has enormous potential. However, the low signal-to-noise ratio in financial markets and the instability of reinforcement learning algorithm training make it difficult for reinforcement learning algorithms to be deployed on a large scale in real financial markets. The specific challenges are as follows: 1. Complex development process with a large amount of engineering work, difficult to implement 2. Algorithm performance highly dependent on market conditions during testing, which is high risk and difficult to systematically evaluate 3. Design, optimization, and maintenance of algorithms have a high technical threshold, making it difficult to deploy on a large scale. The release of TradeMaster provides a software tool, an industry benchmark, and a class of industrial-grade product interfaces to address these three challenges.
TradeMaster Framework
TradeMaster consists of six core modules, covering the complete process of designing, implementing, testing, and deploying reinforcement learning algorithms for quantitative trading. We will introduce each of them in detail below:
Data Module: TradeMaster provides long-cycle, multi-modal (K-line and order flow) financial data at different granularities (from minute-level to daily) covering four major markets: China, US stocks, cryptocurrencies, and foreign exchange.
Preprocessing Module: TradeMaster offers a standardized pipeline for preprocessing financial time series data, consisting of 6 steps: 1. Data cleaning 2. Data filling 3. Normalization 4. Automatic feature mining 5. Feature embedding 6. Feature selection
Simulator Module: TradeMaster provides a series of data-driven, high-quality financial market simulators, supporting 6 mainstream quantitative trading tasks: 1. Cryptocurrency trading 2. Portfolio management 3. Intraday trading 4. Order execution 5. High-frequency trading 6. Market making
Algorithm Module: TradeMaster implements 7 state-of-the-art reinforcement learning-based trading algorithms (DeepScalper, OPD, DeepTrader, SARL, ETTO, Investor-Imitator, EIIE) and 8 classic reinforcement algorithms (PPO, A2C, Rainbow, SAC, DDPG, DQN, PG, TD3). Meanwhile, TradeMaster introduces automated machine learning technology to help users efficiently adjust the hyperparameters of reinforcement learning algorithms during training.
Evaluation Module: TradeMaster implements 17 evaluation metrics and visualization tools, providing a systematic evaluation from 6 dimensions: profitability, risk control, diversity, interpretability, robustness, and generality. Here are two examples:
Pseudo-code for the execution process:
TradeMaster is based on object-oriented programming principles, encapsulating different functional modules and achieving functional decoupling and encapsulation between different modules. It has good extensibility and reusability. The specific process includes the following 6 steps:
Test Results:
Taking the classic task of investing in the Dow Jones 30 Index as an example, the EIIE algorithm achieves stable positive returns and a high Sharpe ratio on the test set:
TradeMaster Tutorials
TradeMaster provides a series of reinforcement learning algorithm tutorials for different trading tasks across multiple financial markets, presented in the form of Jupyter Notebooks for easy user adoption:
Details can be found at: https://github.com/TradeMaster-NTU/TradeMaster/tree/1.0.0/tutorial
A reinforcement learning quantitative trading algorithm design competition based on TradeMaster will be held within this year to help everyone better understand and use TradeMaster. Stay tuned!