Interview with Ahmed, PhD Candidate at INSEAD

Ahmed Guecioueur is a PhD candidate at INSEAD, a leading graduate business school. He used the Quantiacs platform for his research and we met him for an interview.

Hello Ahmed, can you tell us something about yourself and your study?

Hello, I am an academic researcher interested in the behavior of market participants. I find that trading (and finance in general) are great settings for studying human behavior because traders have very strong incentives to maximize their payoffs while controlling risk within the constraints that they operate under.

Even so, studies suggest that the decisions made by individuals can be subject to behavioral biases that may result in sub-optimal decisions. After all, we are human beings! For example, Richard Thaler won the Nobel Prize in 2017 for his work on psychology and economic decision making.

How did you use Quantiacs for your analysis?

In a recent research paper I examined leaderboards from past Quantiacs futures trading contests to study one specific element of traders’ behavior: how do traders take advantage of the data which are available to them? In a world where data are more and more abundant, this question has huge implications beyond Quantiacs contests.

My main statistical analysis makes use of the fact that between futures contests 7 and 8 the Quantiacs platform added a set of macroeconomic predictive variables to the trading API. I therefore compared the trading performance of contestants before that event to the trading performance of contestants afterwards. I measured performance using the out-of-sample (i.e. live period) Sharpe Ratio of each contestant’s best strategy, and also measured how experienced each contestant was by counting the number of contests they had taken part in so far.

In general, the Quantiacs framework has been helpful for my research because all traders are on the same footing: they have the same objective, the same trading universe and horizon, the same access to a set of predictive variables, and so on. Quantiacs maintains a set of fair rules for everybody; but for a researcher like me, having potentially confounding effects controlled for in this way (a contest where macroeconomic data are not available followed by a contest where macroeconomic data are available) has been very useful.

What are your findings?

What I found was in some ways to be expected, but in other ways surprising. I found that, in general, contestants do better the more they participate in trading contests: on average, a contestant’s Sharpe Ratio increases according to the number of contests she/he takes part in. This was to be expected, as prior academic research using brokerage or exchange data has found that stock market traders do better with experience too; see, for example, the 2010 paper by Seru, Shumway and Stoffman. By the way, I conducted a statistical analysis to account for selection effects, and it seems that this result is not just some artifact of traders joining or leaving the platform, which is reassuring.

So results improve with experience. Did you find that using more data also improves results?

I found that experienced traders performed better when the new variables were made available to them on the platform (controlling for market conditions). For these traders, it seems they benefited from having access to the new predictive variables that Quantiacs had added to the platform. It makes sense that rational investors should make use of all the tools at their disposal, including Bigger Data, to achieve their goals.

I ran a set of regression analyses in the paper which test for statistical significance, but one can also observe this fact visually in the following chart where group-level means are shown.

The y-axis is the average live-period Sharpe Ratio (SR), controlling for market conditions by subtracting a benchmark portfolio SR, and the x-axis is the number of contests a participant has taken part to. The comparison is between traders who didn’t have access to the new predictive variables (in earlier contests) and those who did (trading after they were introduced) at identical levels of experience. You can observe a steepening in the performance dynamics:

What was a little more surprising to me was the (lack of) effect of Bigger Data on inexperienced investors, such as those taking part for the first time. It seems that they did not benefit from having these variables made available to them: their performance change was not statistically different to zero when running a regression analysis. And yet if the experienced investors could take advantage of Big Data, why didn’t the inexperienced investors do likewise?

That is an interesting question. Do you have some idea?

I propose the following theory: inexperienced traders are more uncertain about what variables will predict the future evolution of asset returns, leading them to ignore some variables that they could have used to make better predictions. As they gain in experience, they become less uncertain and more likely to use more of the data that they have at their disposal. As a consequence, Bigger Data isn’t always helpful to investors (though it does not necessarily hurt either). As a formal theory, investors solve a robust portfolio choice problem using historical data but governed by a subjective model uncertainty threshold that leads them to discard certain predictive variables, and this threshold decreases with experience.

How did you test your idea using only public information from Quantiacs?

Testing out my theory was not straightforward, because Quantiacs protects the Intellectual Property of the users. What I do is to numerically estimate my robust portfolio choice model using the portfolio return data that are available on the Quantiacs web page. The evidence is consistent with my theory: it seems that on average experienced investors use more predictive variables than inexperienced investors do.

What is the takeaway of this theory for traders? Bigger Data can help traders perform better — if they don’t fear using it.

Ahmed would love to hear from Quantiacs traders about any aspect of their experience, or thoughts on the Quantiacs forum. Please leave a comment or ask him questions on the forum thread!