The OG Data Scientists: LTCM and Renaissance
All models are wrong, but some are useful. — George Box
Silicon Valley can learn a lot from the data scientists that came decades before the tech industry coined the term “data science.” There are some particularly rich lessons from two quantitative investing firms, both famous but for very different reasons: Renaissance Capital and Long Term Capital Management (“LTCM”). Since launching its flagship fund in 1988, Renaissance has achieved 39% average annual gains over 30+ years and is going strong today. Take your pick of famous investors — Berkshire Hathaway, Bridgewater, Blackstone — Renaissance outperforms all of them by a wide margin. LTCM’s gains were even more dramatic — 100% average annual returns between 1994 and 1998. Then it lost almost all of its value in two months, went bankrupt, and nearly took down the entire US financial system.
My perspective on these companies comes from two books, Gregory Zuckerman’s profile of Renaissance and its founder Jim Simons, “The Man Who Solved the Market”, and Roger Lowenstein’s profile of LTCM, “When Genius Failed”. Both are intricately researched pieces of non-fiction that read like novels. Renaissance is famously secretive, and Zuckerman’s book sheds only a little light on the firm’s actual investment strategies. Nonetheless, I’m mainly interested in these companies from a product perspective, and there are numerous striking product contrasts between Renaissance and LTCM. These contrasts are particularly relevant for FinTech companies (like the company where I work, Opendoor), but they’re also relevant to any company that makes decisions or recommendations based on algorithms… so pretty much every tech company.
Why are these companies fundamentally data science companies? First, for both their core competency was models. These models had to be highly accurate and robust in production — billions of dollars rode on them every day. Second, their cultures attracted and emphasized rigorous mathematical and statistical decision-making, something many tech companies are trying to emulate. LTCM’s founders included Nobel prize winners Robert Merton and Myron Scholes (inventor of the Black-Scholes equation) while Renaissance founder Simons was a pioneer in geometric topology before leaving academia.
Short-term predictions beat long-term predictions
From a product perspective, the first obvious difference between Renaissance and LTCM is that they did different types of trades, which had very different durations. LTCM mostly traded in pairs of government bonds (e.g. long on German bonds and short on Italian bonds, aka a convergence trade). These trades take a few months to play out, so they required forecasts accurate over a period of a few months. Renaissance, on the other hand, held its trades for only a few days (or sometimes hours), requiring only day-long forecasts.
Shorter is better because it means more historical instances to learn from. Consider trying to identify patterns in equity prices based on 100 years of data. To make 1 year predictions, there are only 100 non-overlapping instances to learn from. To make 1 day predictions, there are 36,500 non-overlapping instances. In other words, a shorter prediction period means more effective training data, all else equal, for time-series modeling.
A shorter forecast period also allowed Renaissance to quickly identify when its models went astray in the real world. One of the most dangerous ways a model can fail is to perform well in backtesting and poorly in production. Renaissance was able to notice that a backtest-verified model was failing in the real world within 1 or 2 days. It had many such failures in its early years, but since they were caught quickly, the losses from each were relatively small.
For LTCM, it required multiple months to discover when its models didn’t behave in the real world like they behaved in backtesting. The first time they noticed a major model failure it was already too late; they had accumulated many month’s worth of bad risk exposure and the firm was already spiraling into bankruptcy.
Deliberate Growth
Renaissance is unusual in how little it has grown its product lineup relative to its financial success. Renaissance started in commodities and currency trading, where it learned to generate sustainable, but relatively small profits by the mid-1980s. To grow, it needed to expand to a larger market: equities. Renaissance tested equities trading algorithms for nearly a decade, finding numerous algorithms that succeeded in backtesting, but failed in the real world.
Again and again, it dipped its toe in the water and pulled back. As the frustration mounted, some of the firm’s best people left — some believed Renaissance was too cautious on equities, while others believed equities couldn’t be solved. But a core group of scientists persevered. They finally found an algorithm that worked in the real world. It was the result of hundreds of incremental tweaks and bug fixes, not a singular “eureka” moment. Renaissance scaled this equities algorithm slowly, over the course of many years.
But even as it scaled its equities business, it took counterbalancing moves to moderate growth. In 2003, Simons concluded Renaissance was at risk of managing more money than its model could confidently allocate, so the firm stopped taking outside investors’ capital, a decision it hasn’t changed in the 17 years since.
LTCM, on the other hand, grew its product offering like wildfire. It originally focused on sovereign debt convergence trades, but other companies noticed its success and began to copy its trades. Within 24 months of LTCM’s founding, the influx of competition made the space less profitable and LTCM began to look elsewhere. The firm started to invest heavily in dual class stock companies and merger arbitrage, both radically different asset classes than bonds. To outsiders and the media, this was heralded as a sign of success. But, in reality, LTCM had failed to demonstrate sustainable profitability in their core product and was fleeing to new, even-less-validated products.
For a time-series prediction product, the best way to verify that success is real, and not just luck, is to demonstrate real world accuracy over a long period. Renaissance understood that; LTCM did not.
Never trust intuition over the model. And never fully trust the model.
When LTCM was founded, it focused on one product: algorithm-generated convergence trades in sovereign debt. The firm believed deeply in the power of its original algorithm; it was based on the work of two Nobel prize-winning founders and it had an incredible track record. Never mind that this track record was short, based on sufficiently few outcomes (due to the long-term nature of the trades) that it wasn’t differentiable from a lucky streak. The firm fully trusted its model.
However, the expansion from convergence trades into merger arbitrage and dual class stock companies forced LTCM to rely more on its partners’ intuition. Sure, LTCM built models for these new products. But there was no model for allocating risk across the different products (asset classes); that was done by committee. And the risk allocation ended up reflecting the committee members’ beliefs and internal politics. LTCM’s London office head gained influence within the firm and enthusiasm for a particular trade in Royal Dutch Shell — so LTCM put over two billion dollars behind it, despite the high risk concentration and lack of success with this new product.
At Renaissance, Jim Simons had been repeatedly burned early in his investing days when he overrode algorithmic recommendations. He learned he could and should identify when the model was seriously broken, stop trading on its recommendations, and fix it. But he also learned that if he was generally uncomfortable with the model’s decisions — but there was no obvious error in the model decisions — his intuition failed to generate better outcomes. At Renaissance, it became part of the culture that it was ok to turn the model off, debug it, and turn it back on, but it was not ok to override the model’s recommendation; any rationale for overriding a decision either had to be embedded in the model itself, or discarded.
Renaissance also benefited from having a monolithic model, which handled all the decision-making for the firm’s flagship fund. A monolithic codebase has become a dirty word in software development, but it had an interesting advantage for Renaissance: it forced allocation tradeoffs between products to be managed algorithmically. This differs from the man-machine hybrid at LTCM, where models picked the assets, but a committee decided how to allocate risk across asset classes.
Differentiated data
The last product factor that separated Renaissance and LTCM was Renaissance’s focus on differentiated data. Renaissance launched with a focus on a rather pedestrian data type: the daily close prices of commodities and currencies. This was data every other trader also used. But Renaissance realized that differentiated data did not only mean different types of data: it also meant higher quality and higher coverage of the same data type that its competitors had.
In fact, focusing on the same data type as its competitors made a lot of sense. The fact that all other trading firms valued this data type was evidence that the data type provided strong signal. And often it’s easier to amplify a known valuable signal than find an entirely new signal. Renaissance amplified the signal by collecting daily price data going years farther back in time than its competitors, combining data from more sources, and scrubbing and cross-checking the data to remove erroneous values.
As the firm grew, it acquired new data types: it went from daily close prices to intra-day prices, to company financials, to free text news stories… I’d bet Renaissance now uses social media and images and video, but I’m guessing.
With the benefit of hindsight, it’s easy draw contrasts where Renaissance looks good and LTCM looks bad. The truth is, Renaissance is far from unblemished — it has its fair share of weird and dismaying episodes. And LTCM actually had some deeply innovative, valuable ideas that were quietly adopted by other firms after its demise.
The real world is complicated, but there’s a final, simple lesson that applies even to startups that have nothing to do with finance or algorithms: nail your core product before you scale. And don’t confuse accolades from the press or investors, or even from customers, with the ability to generate consistent profits. At the end of the day, that’s what ultimately allows you to control your own destiny.
