Can market crashes really be predicted?

Maybe — if you’re clever enough to find the right measures that capture growing imbalances between those who want to buy and sell.

You don’t have to be a true believer in the lunacy of the Efficient Markets Hypothesis to accept that market movements are hard to predict. The mere fact that there are lots of smart people with ample funds out there trying to predict markets and profit from those predictions makes it certain that prices will generally move in unpredictable ways, even if there are persistent patterns as well. Trend following is a profitable strategy, which means there are verifiable trends in price movements. Some relatively simple patterns in movements have persisted over decades.

Still, it’s hard to predict market movements, and especially big movements — the large rallies of crashes that move markets by large percentages over short periods of time. But hard doesn’t necessarily mean impossible. I’m working my way through a new paper by physicists Jonathan Donier and Jean-Philippe Bouchaud which proposes a method for detecting moments when crashes are more likely. What makes it interesting, to my mind, is that it doesn’t take the usual approach of reading tea leaves, i.e. trying to find some weird and predictive mathematical signature in price times series. Rather, it looks at the much more promising data in the order book — the actual list of all the buy and sell orders entered into the market by traders.

Sensibly, Donier and Bouchaud argue that this data reflects, at least partially, how the crowd of traders is divided between optimists and pessimists, those ready to buy and those ready to sell. Because crashes (or rallies) happen at moments of great imbalance between these two groups, you might expect such imbalances to show up in the data, and provide useful signatures of large impending movements. The two physicists suggest that this is indeed possible, although the story is a little involved.

One problem is that the order book in most markets doesn’t truly reveal traders’ intentions, as traders often act to hide their intentions as much as possible, only posting orders when they need to. Why give others information on your intentions, which they might use to trade against you? What traders really think and feel only becomes evident as prices begin to move and they post new orders. The true intentions of traders, in other words, are only latent in the market, only hinted at by the actual orders listed in the book. They remain secreted away in peoples’ heads and not open to scrutiny.

This may seem like fatal problem, but Donier and Bouchaud have found a way around it by looking to the market for BitCoin, where (for reasons unknown) participants do tend to place orders far in advance. Here, as they show, it’s clear that relatively simple measures of buy/sell imbalance, especially a drop in the number of buy orders, can be used to project the size of a market crash. For example, the first figure below shows the Bitcoin price in US dollars (blue) in the run up to the crash on 10 April, and in the two weeks after. The price rocketed upward from $13 in early January to $260 just before the crash. The red curve is a “support price” calculated by looking at the actual price and correcting it with data from the order book; specifically, projecting the price that would result if a fixed fraction of the existing orders in the book were suddenly executed. The growing departure of the red and blue lines signals a bubble in Bitcoin, as buy orders grew increasingly scarce.

Indeed, this support price, based on a correction using the full order book, gives a nice prediction of the rough size of the crash which ultimately took place.

Of course, one curve isn’t so convincing. But the authors carried out a study of the biggest 15 or so single day drops between January and April 2013 in the BitCoin market, and found that price adjustments using the orders present in the order book just before predicted them all very accurately. This is clear from the figure below, right side, which shows the predicted drop versus the actual drop for these crash-like events.

So there you go: more than a dozen prominent crashes in this market predicted (in size, not in precise timing) from information present in the market just before they happened. Predictions made well after the fact, of course. But this information is available right now and should be usable to detect further events as well.

However, thinking about markets more generally, the authors admit that BitCoin is a bit special. This kind of analysis simply can’t be done for most other markets, where people don’t post their orders well in advance, so that no one has access to the full demand curve of potential supply and demand. The trading volume in the order book at any moment, for most markets, is only about 1% of the daily trading volume, meaning that most orders are latent — in peoples’ hearts and heads and only put into the book when prices begin to move. However, the rest of the paper is devoted to showing how the same information can be estimated, accurately, just from the orders actually present.

I won’t get into these details, which get fairly technical and invoke a number of earlier results on studies of price impact (how much single orders tend to move prices), and the dynamics of liquidity in markets. The end result is the suggestion that it’s possible, using readily observable quantities only, to detect legitimate early warning signs of an impeding crash. What is the key quantity? Well, if σ is the daily volatility in some market, and V is the daily volume, then the key measure of how illiquid and prone to a crash the market is a quantity, call it ILL, defined as σ divided by the square root of V. Extremely easy to calculate from public data for just about any market.

The paper notes that Bouchaud and two other physicists actually proposed ILL as an early warning signal three years ago, though not with such a strong foundation as presented here. Some economists have run some preliminary tests against stock market data (see their reference 23), and found promising results. What’s really new in this most recent paper in a much more solid founding of the theory behind ILL on the basis of analysis of the unique BitCoin data. Previously, as the authors note, is was “quite a leap of faith” to assume that this simple expression for ILL was the same as some much less easily calculable quantities. Now this seems secure.

All of which is, to my mind, pretty cool, and gives just the kind of result one might expect. The ability to predict a crash doesn’t come from any miraculous discovery of weird patterns in price movements, but from teasing out from liquidity signals real information on emerging buy/sell imbalances, and especially the drying up of buy side liquidity.

The authors admit that they still can’t say when precisely such a crash may happen, but this is probably to be expected. Conditions for a huge forest fire may be very real, but it still takes a random spark to set it off. Similarly, it may take the right coincidental series of sales to trigger a crash, even if this is only possible under certain conditions.

And, of course, the usual caveats apply. If this idea does turn out to work, and becomes widely known, then this may well alter behavior, changing the market again. But for now, the idea of predicting market crashes isn’t as far fetched as it may have once seemed. There’s no impenetrable mystery. It just takes a lot of work to find the right measures of collective human behavior.