The Psychological Underpinnings of Why Systematic Trading is Superior to Discretionary Trading
Or what can an Austrian economist teach us about successful trading?
Before entering the exciting world of trading, I was fixated on the idea of how best human society may operate to achieve the greatest outcomes for the greatest number of people. This led me to reading texts by authors ranging from Max Weber and Karl Marx to Friedrich von Hayek and Karl Popper.
It was ultimately the Austrian-born economist Hayek that stuck with me through the years and had the greatest formative impact on me and curiously, my approach to trading. And before you get carried away thinking of your classic inflation-fearing Austrian School gold-bug, this is not what you think it is, so please read on.
As a student, I was lucky enough to study at the Albert-Ludwigs-Universität Freiburg, where Hayek had lectured for many years. While there, a very gifted professor, Dr. Viktor Vanberg, opened my eyes to the work of Hayek. Two of the most profound concepts from Hayek’s teachings that Professor Vanberg taught me were:
1) our “constitutional ignorance” as humans and how we have learned to deal with it
2) distinctions between social and physical sciences
Our constitutional ignorance
In 1952, Hayek published “The Sensory Order — An Inquiry into the Foundations of Theoretical Psychology”, in which he outlined his theory of the human mind. This interdisciplinary work was foundational and ahead of its time. So much so, that Joaquin Fuster, a prominent authority on the neuronal basis of memory at UCLA, was quoted as saying:
The first proponent of cortical memory networks on a major scale was neither a neuroscientist nor a computer scientist, but, curiously, a Viennese economist, Friedrich von Hayek… Hayek’s model clearly contains most of the elements of those later network models of associative memory (1)
One of the key points of “The Sensory Order” is that the mind is essentially “an instrument of classification.” In the words of Hayek, “perception is thus always an interpretation, the placing of something into one or several classes of objects” (2). That is, we can never comprehend the full complexity of reality in its entirety apart from the categories formed by the mind’s classificatory apparatus. Therefore, we cannot respond to the “inexhaustible complexity of everything” but instead must use the classificatory apparatus of our mind to selectively focus on only certain situational characteristics deemed as relevant and ignore the rest.
In order for humans to cope with this so-called “constitutional ignorance”, we have become accustomed to relying on rules as a means of determining the appropriate behavioral response to a given set of circumstances. Hayek argues rules are “a device we have learned to use because our reason is insufficient to master the full detail of complex reality” (3). Rules essentially simplify the number of factors we need to consider in a given situation, “singling out certain classes of facts as alone determining the general kind of action which we should take”(4). Rules can be understood as socially accepted patterns of behavior that are instinctual and learned via socialization, for there is so much inter-generational knowledge embedded in societal rules that have endured a process of cultural evolution. As Hayek eloquently puts it:
No single human intelligence is capable of inventing the most appropriate abstract rules because those rules which have evolved in the process of the growth of society embody the experience of many more trials and errors than any individual mind could acquire (5)
In an article entitled “On Hayek’s ‘Kinds of Order in Society’, Part 1”, Donald Boudreaux provided several useful examples of the use of rules in society:
Think of yourself in a big city… You know that when you come with your filled grocery cart upon a queue at the supermarket checkout lane, you take your place at the back of that line — and you’re content to do so because you know that other people will follow the same rule. You know that you aren’t allowed, without invitation, to enter premises belonging to other people. You know what green lights and red lights mean to motorists (6)
Such rule-oriented behavior allows for cooperation among a much larger number of individuals than any individual can personally know. Many of us conduct our lives adhering to such rules without an explicit awareness of them, as generally these rules have evolved through the spontaneous growth of society.
Following rules as a means of coping with our “constitutional ignorance” has notable implications for economics, which is primarily focused on self-interested, omniscient, rational beings making decisions to maximize their utility. For an agent to maximize their own utility it must know all possible circumstances, courses of action and consequences. In reality, such things are of course unknowable, not least due to our “constitutional ignorance”. Given humans are only able to perceive certain situational characteristics, Hayek argues that one must therefore compare:
· the selectivity of our discretionary situational judgement vs
· the selectivity resulting from following rules
Hayek poses the question whether humans do better trying to maximize their outcomes discretionarily on a case-by-case basis or by following simple rules of behavior. Given humans cannot know all possible facts, choices and outcomes relating to a given situation, have limited mental capacity and decision-making can be costly and hold up other areas of life, Hayek argues that on average we can do better by adhering to rules that have withstood the test of time and benefitted from the trial-and-error of generations before us.
This idea has been explored and confirmed by other academics, such as Ronald Heiner (7). Using a series of repeated games in which agents sought to maximize their payoffs by only either making discretionary decisions or adhering to simple “imperfect” rules, Heiner showed that non-omniscient agents achieve better outcomes by following rules.
In light of the above arguments, in the world of trading it would therefore make sense to investigate whether simple rules perform better than discretionary trades. In 2011, Brad Barber and Terrance Odean from the University of California published an article entitled “The Behavior of Individual Investors” (8). Some relevant conclusions from the article demonstrate that retail investors:
1) underperform standard industry benchmarks such as low cost index funds
2) sell winning trades early while holding on to losing trades
3) exhibit limited attention span and recency bias
These results were derived using data from different geographies and time periods, suggesting that they are cross-cultural and reflective of general human behavior. Retail investors are typically classified as ‘unsophisticated’ and do not utilize extensive data, models or rule-based systems, so the results are perhaps not so surprising. More surprising is the ample evidence that most institutional mutual fund managers underperform their benchmarks. Literature consistently documenting such underperformance is available beginning with Michael Jensen in 1968 all the way through to Carhart in 1997 and in K.J. Martijn Cremers more recently.
So, if retail and institutional investors both underperform relative to simple benchmarks, and if humans are likely to do better by following simple rules than trying to maximize outcomes discretionarily on a case-by-case basis, the answer for any aspiring trader should be clear: utilize a simple rules-based approach to trading and minimize discretion. In fact, simple investing benchmarks such as the S&P500 index are themselves rule-based systems, which is another reason why most investors cannot outperform them. Among other things, the S&P500 index requires that stocks meet a market capitalization threshold, which means that losing stocks necessarily drop out of the index and winning stocks remain in the index. This simple rule mechanism is why many investors would do better to invest in indices as opposed to ‘flying by the seat of their pants’ and succumbing to the many biases humans exhibit (recency bias, confirmation bias, prospect theory, etc).
Apart from standard index rules, there are many other simple rules available that have withstood the test of time. One only has to think of David Ricardo (1772–1823), who amassed a fortune with the credos “cut short your losses, let your profits run on.” Such statements provide the foundation for a profitable, well-documented, rules-based approach to trading called “trend-following,” which has outperformed most standard benchmarks over extended periods of time, particularly during bear markets. Michael Covel has done a superb job at documenting the success of such traders through the ages (9). There are a multitude of other successful rules-based trading strategies, such as those documented in Antti Ilmanen’s magnum opus, “Expected Returns” (10). Therefore, in light our limited mental capacity to discretionarily maximize outcomes on a case-by-case basis, both the theory and the evidence point to the general superiority of a rules-based approach.
This begs the question of how best to determine the appropriate set of rules, which brings us to the second insight from Hayek: distinctions between the social and physical sciences.
The delineation of simple and complex phenomena
Trading is necessarily a social science, as it is based on the results of human psychology and action en masse. To this day, human nature is not as scientifically well understood as physical phenomena in the natural sciences. E.g., if a physicist performs experiments on cubes of copper in a given setting 1000 times, he will obtain a high degree of certainty as to the specific outcome of the procedure. In contrast, if a social experiment were performed analyzing human behavior in a given setting 1000 times, there is typically a high degree of variability in the results and not often a high degree of certainty as to the outcome. One of the main reasons behind this is the heterogeneity in human personalities (social science) vs the homogeneity of the properties of copper (physical science).
Earlier studies of human behavior (particularly in the realm of economics) relied on statements of logic to understand the chain of events that led to a human making a particular decision. Hayek departed from this approach by favoring an empirical approach that relied on Karl Popper’s principle of “falsifiability.” This concept essentially claims that once we form a hypothesis regarding some phenomena, empirical data cannot prove this hypothesis to be true, but instead can only prove it not to be false. That is, via repeated experiments, we cannot claim that a working theory of some causal mechanism driving some outcome is true only because we have not found any observations to the contrary. We can only claim that it has not been proven false. The causal mechanism may in fact be different from what we believe it to be. It takes only one observation counter to a hypothesis to invalidate that hypothesis and prove it false. Therefore, scientific discovery is merely refining the number of possible explanations by excluding those that have been disproven as opposed to proving the truth of a given hypothesis.
Contrasting the physical properties of a cube of copper with the outcomes of human behavior, one can begin to see why it can be so challenging to create working theories of human behavior if this approach only requires one observation to the contrary to disprove a hypothesis. To deal with this, Hayek acknowledged the need for a separate approach in terms of possible conclusions from experiments in the social sciences compared to the natural sciences. In “The Counterrevolution of Science”, Hayek cautioned against “slavish imitation of the method and language of the physical sciences” and applying the mechanical “physics model” of the natural sciences to the social sciences (11). Initially, Hayek argued for a methodological delineation of the social and physical sciences. Later, he refined this argument in favor of differentiating sciences that study simple phenomena and those that study complex phenomena:
· Simple phenomena: “closed systems” with a “sufficiently small” number of interconnected variables
· Complex phenomena: “open systems” (of life, mind, society) where the outcomes of processes “depend on a very large number of particular facts, far too numerous for us to know in their entirety” (12)
Hayek argued that the level of complexity determines the extent to which we can draw conclusions. Specifically, for complex phenomena, due to our inability to know the facts and circumstances of each individual agent and their interaction with other agents, at best we will only be able to determine an “explanation of the principle” and a “prediction merely of the abstract pattern the process will follow” (13). In contrast to being able to predict a specific outcome with simple phenomena (e.g., a chemical reaction in a controlled environment), with complex phenomena we cannot hope for more than being able to account for the general principle. Therefore, with complex phenomena we will be restricted to general pattern predictions as opposed to specific outcomes of individual instances.
Clearly, financial markets are complex phenomena, being the aggregate outcome of the instincts and nature of millions of dispersed agents, each with their own individual circumstances, personality, objectives and risk tolerances, all of which is unknowable to any single being. Hence, in general, when forming hypotheses to e.g., determine profitable trading rules in financial markets, one can only hope to identify an “explanation of the principle” and a prediction of the “abstract pattern the process will follow.” Contrast this with a discipline such as physics, which seeks to precisely quantify motion through space and time and predict specific instances of situations. The application of such models is misplaced in an “open system” such as financial markets, with a vast number of unknown variables. At best, a profitable trading rule is expected to only work on average, in general and not in all circumstances. Essentially, the outcome of a trading rule applied to any specific instance is largely random, but as the trading rule is applied over a sample size of many trades, one can determine if it is likely to be profitable.
There are certain general patterns of human behavior or structural features of markets that can be exploited by the cunning trader. In order to save great heartache (and fortunes), one can thankfully obtain large datasets across countries spanning decades in order to test if rules can be specified that exploit such general patterns of behavior. However, again, note that based on the principle of falsification this does not mean we know with certainty whether the rule works or if our explanation is true, it is merely only an indication that based on the evidence available, it is not false.
There are four points worth mentioning in this regard. First, just because one has established a general pattern of behavior in financial markets and identified a trading rule that successfully exploited this in the past, does not mean that it is assured to work in the future. Therefore, one would need to have a separate risk management framework in place. This echoes the sentiment of Nassim Taleb in “The Black Swan”, which is based on the premise that just because an extreme event has not transpired in the data, does not mean it cannot happen. Second, as trading rules seek to generally capture the “abstract pattern” of some human behavior and hence are only expected to work on average, it would be prudent to diversify across trading rules and markets. Third, one needs to exercise care in developing an explanation for an exploitable pattern. Ideally, there should be some fundamental reasoning as to why a particular pattern should occur (e.g., behavioral biases, structural impediments, regulatory restrictions, etc) and this should be specified before examining any data. Finally, preference should be given to parsimonious models with fewer degrees of freedom to avoid perfectly fitting a model to the historical data. This increases the likelihood that the model will remain robust enough to handle unknown future data.
The path to successful trading
Hopefully by now, you’ve become aware of the necessity of the human brain to systematically ignore much of reality due to its inherent complexity. This allows our brains to filter information that is deemed important for survival. As a means of coping with this “constitutional ignorance”, humans tend to do better by adhering to rules as opposed to discretionarily maximizing outcomes on a case-by-case basis. The most appropriate rules tend to be those developed through cultural evolution, for such rules contain the wisdom acquired by generations of trial-and-error, more than any individual could acquire. This is particularly the case in the world of trading, where simple rules have been widely documented to perform better than the discretion of either retail or institutional investors. This result holds across geographies (space) and decades (time).
When examining complex phenomena characterized by a large number of unknowable variables such as financial markets, as opposed to simple phenomena such as the physical world, we can at best aim to explain concepts in principle and predict the general patterns a process may exhibit as opposed to specific outcomes. Therefore, in general, trading rules can only be expected to work “on average” as opposed to predicting the specific outcome of any single trade (as a discretionary trader may do). Furthermore, as trading rules are often designed based on historical data, which, given the concept of falsification, does not necessarily prove that one’s working hypothesis is true, it would be wise to couple any trading rules with a separate risk management protocol. Finally, one must take care in forming hypotheses when testing ideas on historical data to avoid perfectly explaining the past but not the future, and should utilize parsimonious models where possible.
Original paper available here: What Can An Austrian Economist Teach Us About Successful Trading? by Toby Carrodus :: SSRN
Read more of my work here: Author Page for Toby Carrodus :: SSRN
References
1. Fuster, J. (1995). Memory in the Cerebral Cortex: An Empirical Approach to Neural Networks in the Human and Nonhuman Primate. Cambridge: MIT Press, p. 87
2. von Hayek, F. (1952). The Sensory Order. University of Chicago Press, p. xviii
3. von Hayek, F. (1960). The Constitution of Liberty. University of Chicago Press, p. 127
4. von Hayek, F. (1964). “Kinds of Order in Society”, New Individualist Review (Winter).
5. von Hayek, F. (1978). New Studies in Philosophy, Politics, Economics and the History of Ideas. The University of Chicago Press
6. Boudreaux, D. (2021). On Hayek’s “Kinds of Order in Society,” Part 1, American Institute for Economic Research. https://www.aier.org/article/on-hayeks-kinds-of-order-in-society-part-i/
7. Heiner, R. (1983). “The origins of predictable behaviour”, American Economic Review.
8. Barber, B. & Odean, T. (2011). The Behavior of Individual Investors. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1872211
9. Covel, M. (2017). Trend Following. Wiley.
10. Ilmanen, A. (2011). Expected Returns. Wiley
11. von Hayek, F. (1942). “Economics and Knowledge: Scientism and the Study of Society”, Economica, New Series, 9 (35), 267–291, p.268
12. von Hayek, F. (1973). Rules and order. Law, legislation and liberty (Vol. 1). London and Henley: Routledge & Kegan Paul. p.23
13. von Hayek, F. (1973). Rules and order. Law, legislation and liberty (Vol. 1). London and Henley: Routledge & Kegan Paul. p.23
