The Most Difficult Puzzle in the World: A Brief History

Jon Taylor
Numerai
Published in
6 min readAug 20, 2019

--

Numerai is taking a different path to partially solve the most difficult puzzle in the world.

The First Law of Economics: For every economist, there exists an equal and opposite economist.

The Second Law of Economics: They’re both wrong.

You’d think that Finance is a well-researched topic, having solved the “big questions” of our time. Market efficiency suggests that an asset’s last-traded price, such as a stock, reflects its true value. If markets are efficient, then how can anyone outperform the average return of the market over a long time? Finance has been around longer than the scientific method¹ , but we have only just scratched the surface on market efficiency. Let’s look back in time at the discovery process of finance.

600 B.C.

One of the first philosophers was a trader. Thales of Miletus predicted a bumper crop of olives and so he negotiated with the owners of olive presses to pre-pay for their use at a future time. Thales is credited with creating the first recorded futures contract. He is also credited with being the first person in Western civilization to leave mysticism behind and begin thinking in terms of scientific philosophy.

1685 A.D.

A French mathematician by the name of Mathieu de la Porte is credited with providing the first use of the word arbitrage.

“L’arbitrage est une combinaison que l’on fait de plusieurs changes, pour connaître quelle place est plus avantageuse pour tirer et remettre.” — “La science des négociants et teneurs de livres”²

Translated loosely, arbitrage considers different exchange rates to recognize the most profitable places of issuance and settlement for a bill of exchange. The concept of market inefficiencies was evident; but no formal theory has been proposed.

1900

Louis Bachelier provided the mathematical process that we still use today. Bachelier was the first person to model Brownian Motion³ and is considered to be the Father of Quantitative Finance. Albert Einstein provided the mathematical proof five years later⁴. Some of the arguments in his thesis align closely with the assumptions of the Efficient Market Hypothesis, since a random walk would approximate the future price of a security when all the information about a security was available to everyone.

1952 — Modern Portfolio Theory

Harry Markowitz published “Portfolio Selection” in the Journal of Finance in 1952. He showed that a diversified portfolio exists which maximizes expected return and minimizes risk. His research has been cited 39,348 times⁵.

1970 — Efficient Market Hypothesis

Why did researchers take so long to formally define an efficient market? Eugene Fama proposed three forms⁶ of market efficiency:

Weak-Form Efficiency: Future prices cannot be predicted by analyzing prices from the past. In other words, making investment decisions based solely on historical pricing data will not generate excess returns in the long run.

Semi-Strong-Form Efficiency: Share prices adjust to new publicly available information quickly and in an unbiased manner. This form of efficiency means that trading on historic price data or fundamental stock analysis would not generate excess returns in the long run.

Strong-Form Efficiency: Share prices reflect all information available in the public domain and the information held by insiders, and no one can earn excess returns.

These hypotheses form the bulk of the foundations of empirical finance. Earning risk-free returns should be impossible. Hedge Funds should not be able to generate excess returns above their fee structure.

1973 — Black-Scholes Formula

Fischer Black, Robert Merton and Myron Scholes “did the math” to demonstrate that an option’s theoretical price can be calculated using five inputs: the strike price of an option, the current stock price, the time to expiration, the risk-free rate and the volatility of the stock. Their research eventually earned them a Nobel Prize. Unfortunately, their model does not account for “fat tail risk,” since their model assumes that stock returns follow a log-normal process.

Empirically, stock returns do not follow a normal distribution.

above: the left-tail of the histogram of daily stock returns

Fat-tail risk would eventually destroy Long-Term Capital Management, a hedge fund which Myron Scholes and Robert Merton served as Principals. Long-Term Capital Management required a bail-out organized by the Federal Reserve, who feared that contagion would bring down other titans of industry.

1992 — Fama-French 3-Factor Model

Factor trading strategies can thank the research done by Eugene Fama and Kenneth French, which identified two additional factors which explain stock returns beyond the traditional Capital Asset Pricing Model (CAPM). Value and Size are anomalies such that value stocks and small market cap stocks tend to outperform the market. Their paper has been cited 19,350 times⁷. Remember the EMH?

The second largest hedge fund in the world, AQR Capital Management, was founded in 1998 by Cliff Asness and three others. Cliff was Eugene Fama’s research assistant while Cliff was a PhD student at the University of Chicago.

We haven’t made much progress since 1992. In 2015, Eugene Fama and Kenneth French published a paper identifying five factors⁸ which explain asset prices. Even Cliff admits that there is more work to do, highlighting the challenges that quantitative strategies face even when the stock market is booming.

The Market is Not Efficient

Some may suggest that trading is gambling, but those people would be very wrong. Gambling at the casino is a zero-sum game; only one party wins with each roll of the die. Trading stocks is a positive sum game, where exploiting inefficiencies can lead to market beating risk-adjusted performance.

Enter Numerai

Numerai is a quantitative hedge fund trading global equities. Numerai doesn’t tell us more than that, and they don’t need to. After all, this is the most difficult puzzle in the world, due in part to the tremendous risks, and partly due to incentives for making discoveries and keeping them private.

Numerai rewards users for making predictions using their data. You don’t need to know what the data represents because even when academics know the data, they still can’t get it right⁹. Besides, as Marcos Lopez de Prado rightfully points out, empirical research on stock returns has issues with repeated trials¹⁰.

That’s why Numerai gives away their data. They are using predictions from users across the world, some who are trained in Astrophysics, Brain Imaging, Computer Science, Finance and many other disciplines. Some have no formal training at all. None of them know what features the data contain. Few of their users are trained in Modern Portfolio Theory or Quantitative Finance. They don’t have to know any of that; they just need to be able to create a predictive machine learning model which generalizes well.

Numerai is taking a different path to partially solve the most difficult puzzle in the world. An interesting feature of market inefficiencies is that when those inefficiencies are exploited, the market becomes more efficient. As Numerai grows, the market becomes more efficient.

Let artificial intelligence figure it out. Numerai provides a platform for people to work on this challenging puzzle, and pays them Numeraire for submitting good predictions. There’s no reason that the next great discovery can’t come from machine learning applications structured the way Numerai has done so. Why don’t you join the other users and help solve the most challenging puzzle in the world?

¹ The Code of Hammurabi (~1754 B.C.) contains rules about assigning owned property as debt collateral (Code 39); a rule for brokers and agents to “make good” their clients when the agent or broker incurs losses (Code 102)
² de La Porte, M. (1741). La science des négocians et teneurs de livres… Cavelier.
³ Bachelier, L. 1900. Théorie de la spéculation. Paris: Gauthier-Villars. Translated by A. J. Boness. In The random character of stock market prices, edited by P. Cootner. Cambridge: MIT Press.
⁴ Einstein, A. (1905), Über die von der molekularkinetischen Theorie der Wärme geforderte Bewegung von in ruhenden Flüssigkeiten suspendierten Teilchen. Ann. Phys., 322: 549–560.
⁵ Google Scholar citation count; observed July 29, 2019
⁶ Malkiel, B. G., & Fama, E. F. (1970). Efficient capital markets: A review of theory and empirical work. The journal of Finance, 25(2), 383–417.
⁷ Google Scholar citation count; observed July 29, 2019.
⁸ Fama, E. F., & French, K. R. (2015). A five-factor asset pricing model. Journal of financial economics, 116(1), 1–22.
⁹ See the First and Second Law of Economics
¹⁰ Bailey, D. H., Borwein, J., Lopez de Prado, M., & Zhu, Q. J. (2014). Pseudo-mathematics and financial charlatanism: The effects of backtest overfitting on out-of-sample performance. Notices of the American Mathematical Society, 61(5), 458–471.

--

--

Jon Taylor
Numerai
Writer for

Third year Finance PhD Student. I study FinTech and Bank Failures and other topics of interest. How do you interact with the Financial Markets?