C++ Backtest Environment — Part 4: Structure and Data Manager
With the Bloomberg API successfully interfaced into C++ for easier data retrieval, it is now time to lay out the groundwork for the structure of the backtester as a whole. This will be heavily influenced by the event-driven system outlined by Michael Halls-Moore on QuantStart as well as Quantopian’s own algorithm development environment, but will diverge to allow for realtime simulation and intraday data implementation sometime in the future.
We are going to want each algorithm to be self-contained to potentially allow for multi-threading and simply keep our code concise. Thus, every algorithm class will inherit from Strategy which contains all of the functions needed for a user to iterate through events, pull history, place simulated orders, and keep track of holdings, positions, and performance statistics. We want to keep each element of the backtest as independent as possible from each other as well. A diagram of the basic historical strategy structure follows:
Thus, we can see the relationships between each of our upcoming classes. All the functionality for handling events will be contained in member objects Portfolio and ExecutionHandler depending on what type of event it is, and the data will be accessible from Bloomberg through our DataManager base class, which will have different implementations based on what type of strategy is being run (live, intraday, historical). Ideally, we want to only have to change a single variable to switch between these different modes.
With this foundation in place, we can get started on making an easily-accessible wrapper for the HistoricalDataRetriever we created in Part 3. This is necessary because we need to have a method that initializes the event loop in historical backtests with daily market events for performance calculation. We also need to implement an easy-to-use function for getting historical data in the algorithm itself, which must be done without using the parameter for current time we made in HistoricalDataRetriever so the user has no potential for look-ahead bias or grabbing data past the simulated current date of the event loop.
First, we need to build the abstract base class from which all different types of data managers will be derived, DataManager. The only thing needed is a reference to the simulated current time and a function called history() which returns a map of SymbolHistoricalDatas containing the fields specified for a set length of time before the simulated time. As DataManager is an abstract base class, we will not implement the history() function here.
// data.hppclass DataManager {
public:
explicit DataManager(Datetime* p_currentTime) :
currentTime(p_currentTime) {}
virtual std::unique_ptr<std::unordered_map<std::string, SymbolHistoricalData>> history(
const std::vector<std::string>& symbols,
const std::vector<std::string>& fields,
unsigned int time_units_back,
const std::string& frequency) = 0;
protected:
Datetime* currentTime{};
};
Our next class, HistoricalDataManager, will be a bit more complex. It is derived from DataManager so it must implement the history() function, as well as a function for populating the event loop with market events as is necessary for our historical EOD backtest performance reporting.
First, we must construct the class itself. The only member will be dr, a HistoricalDataRetriever which we can use to pull data. Thus, our constructor simply builds dr with a specification of what data is going to be requested.
// data.cppHistoricalDataManager::HistoricalDataManager(Datetime* p_currentTime) :
DataManager(p_currentTime),
dr("HISTORICAL_DATA") {}
The next easiest function to build will be the history() function, which is essentially just a wrapper for the pullHistoricalData() function we wrote in the data retriever in the part 3. It simply calls the latter without requiring the user to specify the current Datetime. The ternary is irrelevant right now, but will be useful when we implement the events which require up-to-date prices which may not necessarily be as low-frequency as daily.
// data.cppstd::unique_ptr<std::unordered_map<std::string, SymbolHistoricalData>> HistoricalDataManager::history(
const std::vector<std::string> &symbols,
const std::vector<std::string> &fields,
unsigned int time_units_back,
const std::string &frequency) { std::string freq = (frequency == "RECENT") ? daily : frequency;
Datetime beginDate = date_funcs::add_seconds(*currentTime, 24 * 60 * 60 * timeunitsback * -1); return std::move(dr.pullHistoricalData(symbols, beginDate, *currentTime, fields, freq));
}
Note: date_funcs::add_seconds() is a custom function which adds a set number of seconds to a Bloomberg::blpapi::Datetime. It does so through a series of time_t and struct tm permutations and can be given different parameters to specify special options, i.e. weekdays only or same week/month. The implementation is here on my GitHub for now, but it will be covered in a future part when we get to scheduling events.
Finally, we must create fillHistory() , a function that will generate MarketEvents for a set number of market days for performance statistics. This will allow the portfolio to be updated many times, rather than just when the algorithm’s scheduled functions are run and when it places orders. This simply requires getting daily EOD last price over the backtest period and building a MarketEvent for each. The events themselves will be introduced in the next part of the tutorial, but a MarketEvent simply keeps track of the last price (“PX_LAST” from Bloomberg) for several stocks on a given date. This function is easy enough to implement as well, though it looks complicated:
// data.cppvoid HistoricalDataManager::fillHistory(const std::vector<std::string>& symbols, const Datetime& start, const Datetime& end, std::list<std::unique_ptr<Event>>* location) { std::unique_ptr<std::unordered_map<std::string, SymbolHistoricalData>> data = dr.pullHistoricalData(symbols, start, end); for (auto i = data->operator[](symbols[0]).data.begin(); i != data->operator[](symbols[0]).data.end(); ++i) {
std::unordered_map<std::string, double> temp = {{symbols[0], i->second["PX_LAST"]}};
for (int j = 1; j < symbols.size(); ++j) {
temp[symbols[j]] = data->operator[](symbols[j]).data[i->first]["PX_LAST"];
}
location->emplace_back(std::make_unique<MarketEvent>(symbols, temp, i->first));
}
}
Although it looks like a mess, all this function does is pull historical data from the HistoricalDataRetriever, iterate through each date, and finally pull the prices at each date into common MarketEvents that are placed onto the main event loop stored as a pointer, location.
With that, we have finished the functionality of our HistoricalDataManager and are now ready to begin the more complicated aspects of the Event classes and the rules for scheduling functions only on market days and taking into account the holidays of the year.
The source code for this post can be found at my GitHub. If you have any other questions, feel free to email me at kirkilese@gmail.com!
