Thermodynamics of Financial Markets
Information Theory — A Brief Primer
Information is physical. Landauer’s principle states that “any logically irreversible manipulation of information, such as the erasure of a bit or the merging of two computation paths, must be accompanied by a corresponding entropy increase in other non-information-bearing degrees of freedom”. Erasure of information is a dissipative process, which means that a minimal quantity of heat (Landauer bound) is released when a classical bit is erased, thereby increasing the entropy of the environment.
The seeming “interchangeability” between information and energy gives a physical flavor to its otherwise intangible nature. This important insight also extends the notion of information beyond the “1’s” and “0’s” in the realm of computing — to nature, our society, and of course financial markets.
The inception of modern information theory was when Shannon published his seminal paper “A Mathematical Theory of Communication” (1948) in the Bell Systems Technical Journal [1]. One of his most important contributions in the paper is quantifying the amount of information contained inside a stream of k statistically independent letters. The Shannon entropy associated with the probability distribution { p₁ ,p₂ , … , pₖ} is defined as:
where ∑ᵢᵏ pᵢ=1ᵢ. It measures our a priori ignorance, or equivalently the average information gain when we learn the value of a letter in the message.
One might ask the following question: how can we apply these information theoretic concepts to gain a better understanding of the complex nature of financial markets?
Financial markets are known to be stochastic, which arise from multiple economic driving factors and nonlinear interactions among its constituents. Information theory provides us with the tool to quantify “mixing” among constituents in the financial market (i.e., how much we can infer about an asset given the knowledge of some other assets), and how risk spillovers forms and propagate in the financial network.
Another aspect of the financial system is its non-stationary nature. The entropy production rate tells us how far the system is from equilibrium, which is inherently related to the structure of the underlying phase space in which the system resides. However, the market is known to display different behaviors in different business cycles. Dynamical entropy may provide us with a possible way of detecting the regime changes, based on which we can react and implement different strategies accordingly.
Dynamical Entropy
The dynamical entropy characterizes the stability of a dynamical process and measures the rate at which memory of the initial conditions is lost. One of the variants of dynamical entropy is the approximate entropy (ApEn) [2]. ApEn reflects how confidently we can predict the future based on the past by comparing the snippets of trajectories (which we refer to as “patterns”) across different times. Given a sequence Sₙ, we specify the two parameters m and r, where m is the pattern length and r is the tolerance parameter. By defining each pattern as pₘ(i) containing m measurements starting from Sᵢ, we say that pₘ(i) and pₘ(j) are similar if:
i.e., if the difference between any pairwise measurements in the pattern is less than r. By defining Φᵐ as the average information content in the matching of pattern of length m, the approximate entropy is calculated as:
If we find similar patterns in a series, the ApEn tells us the logarithmic likelihood that the patterns will no longer be similar if the observation length m is increased by 1. This comes from the intuition that the similarity of patterns have less predictive value if it is only a mere coincidence. In the case of highly irregular series, Φᵐ⁺¹ will be much less than Φᵐ, which gives rise to a higher ApEn. For a finite N, ApEn is bounded by 0 ≤ ApEn ≤ ln(N-m).
However, there are several well studied issues with ApEn: It is highly sensitive to the sequence length and also displays poor self-consistency. A similar regularity metric known as the sample entropy (SampEn) [3] was introduced which has fewer of these problems. It is defined as:
where B and A are number of patterns satisfying d[pₘ(i), pₘ(j)]<r of length m, i≠j which excludes the self-matching, in contrast to the pattern-wise approach in ApEn. We can also apply some physically motivated innovation in order to have a more well-behaved metric with higher discriminating power by calculating the similarity threshold r adaptive to the variations in the local moving window. This modification ensures that the SampEn has enough resolution power to “unfold” the dynamics in periods when the assets time series are very volatile.
How can we leverage these ideas in trading? A time series has smaller dynamical entropy when its dynamics is confined within a small region of the phase space, leading to repeating trajectories. The information required to describe the time series is low, which means that the future can be more readily predicted from the past. The observed characteristics in the dynamics tend to persist — meaning it is largely mean reverting (quasi-periodic) unless the system is suddenly perturbed, driving it towards previously unexplored regions. The opposite scenario would be a trend following behavior, which is seen as a continual growth or decline in the asset price. It can be observed that entropy-based trading strategies work particularly well during volatile periods. This is perhaps not very surprising, due to the sensitivity of entropy to transition across different dynamical regimes.
Untangling the Complex Financial Network
Financial markets are complex systems with a large number of interacting components, resulting in collective nonlinear behavior observed across different time scales. In order to make informed trading decisions, it is important to understand how the price of one asset is influenced by others, or if there exist common factors like economic data, monetary policies and market sentiments that drive price formation of various financial assets. Such inter-dependencies contains structural information regarding the market, in particular the systematic risks and the relationships of return behavior among different asset clusters. Uncovering this information is important for many applications such as portfolio construction and risk management.
Financial asset clustering involves placing a group of assets into subgroups according to their properties and allows us to represent the interconnections of stock markets visually on a network diagram, with each node representing an asset and the edges representing the strength of the dependencies. Such diagrams allow us to infer how different assets are clustered together as well as the transition from one cluster to another. In order to obtain a meaningful representation, we must first identify a set of properties and apply suitable measures of similarity (or dissimilarity). Here we shall adopt mutual information (MI) as the measure of nonlinear relationship. For the discussion of using the directional variant of MI (the transfer entropy), you are welcome to check out this awesome blog.
Given a set of random variables, the MI captures how much knowing some variables reduces uncertainties about others. For the bi-variate case, it is:
where S(X) is the self entropy of random variable X, and S(X,Y) is the joint entropy of X and Y, computed from the joint distribution function P(X,Y). The expression above can be regarded as the K-L divergence between the joint probability distribution and the product of marginal distribution of the two random variables. If they are equal (i.e., P(X,Y)=P(X).P(Y)), X and Y are independent and therefore I(X:Y)=0. This formalism can be generalized to include more processes. For the case of 3 random variables, the MI becomes:
The relation above is evident from the tripartite entropy Venn diagram:
In the diagram above, each circle represents the “uncertainties” of a random variable, and the central region where all three circles overlap represents the mutual information I(X:Y:Z). By using certain risk measures as input (e.g. the volatility), we can apply the MI to infer the dependence of different assets in terms of their underlying risk profiles. Similar ideas can also be applied to the return, draw-downs and dynamical entropy to infer the relationship in terms of these indicators and allow us to cluster the assets accordingly. By examining the connectedness of the network and the transitions between different clusters, we obtain valuable information about the risks and return characteristics of our asset universe:
The figure above shows the network diagram of around 90 assets selected from S&P 500 based on market cap, where the nonlinear dependency is captured by MI using the standard deviation of log return as input. The size of the node indicates the “Betweenness Centrality”, which measures how often a node appears on the shortest path between nodes in the network, whereas the color of the node indicates the “Degree”, i.e. the number of edges coming out from the node. Higher degree nodes have darker shades, and can be identified as the hubs in the network. The shade and the thickness of the edges indicate the amount of pairwise relationships, and the distance between the assets is smaller if they display higher inter-dependency. From the diagram we can easily identify multiple sectors, which we shall briefly describe below:
- Technology (top): Apple (AAPL.OQ), Nvidia (NVDA.OQ), Amazon (AMZN.OQ), Facebook (FB.OQ), Netflix (NFLX.OQ), Qualcomm (QCOM.OQ), Microsoft (MSFT.OQ)…
- Consumer and Telco (top right): Wallmart (WMT.N), Target (TGT.N), Costco (COST.OQ), Verizon (VZ.N), AT&T (T.N), P&G (PG.N), Colgate-Palmolive Co (CL.N), Coca-cola company (KO.N), McDonalds (MCD.N)…
- Biotech and Pharmaceutical (bottom right): Pfizer (PFE.N), Amgen Inc (AMGN.Q), Merck & Co (MRK.N), Abbott Laboratories (ABT.N), Celgene (CELG.N), Allergan plc (AGN.N), Gilead Sciences Inc (GILD.OQ)…
- Healthcare (bottom): United Health (UNH.H), Medtronic plc (MDT.N), Centene Corp (CNC.N)…
- Logistics and Aerospace (bottom): Lockheed Martin Corp (LMT.N), General Dynamics Corp (GD.N), United Parcel Service (UPS.N), Raytheon Co (RTN.N)…
- Energy (bottom left): Occidental Petroleum Corp (OXY.N), Kinder Morgan Inc (KMI.N), Conoco Phillips (COP.N), Halliburton Co (HAL.N), Schlumberger NV (SLB.N), Chevron (CVX.N)…
- Finance: Goldman Sachs (GS.N), MetLife Inc (MET.N), JP Morgan (JPM.N), Bank of America Corp (BAC.N), Wells Fargo & Co (WFC.N), Bank of New York Mellon Corp (BK.N), US Bancorp (USB.N)…
With a small mean path length of ~1.958 and a high mean clustering coefficient of ~0.786, we show that financial networks indeed demonstrate a small-world property despite a modestly sized asset universe. Notably, one can identify a chain of hubs (nodes with darkest hues) at the middle of the network, namely Visa (V.N), Mastercard (MA.N), American Express (AXP.N), Accenture (ACN.N), 3M Co (MMM.N), Danaher Corp (DHR.N) and Honeywell (HON.N), which are the major global merchant networks and conglomerate companies with operations spanning across various sectors. We also observe that assets in the finance sector generally display stronger connections, underlining high spillover risks in this sector. It is also no surprise that they are found near the central part of the network.
Risk Management
Returns of assets have varying performance over time, causing the portfolio to drift away from its original composition. Portfolio management is crucial in any trading strategy to mitigate risk and avoid overexposure. A conventional rebalancing strategy periodically resets a portfolio to a target allocation to maintain the initial risk-return profile, without adequately addressing the dynamical nature of the underlying asset dependencies. In contrast, using info-theoretic measures such as MI, we identify dynamic clusters by looking at the relative risk distances in the financial network at every rebalancing period. Dynamical portfolio rebalancing executed in this manner is not restricted to the original asset makeup, ensuring that the target asset is always rebalanced against its least correlated peers in terms of volatility.
Let us give an example to further illustrate the overall idea. From the price volatility time series of Conoco Phillips (COP.N), our clustering algorithm returns a list of the most correlated assets:
As expected, the group of the most correlated assets (CVX.N, OXY.N, HAL.N, XOM.N, SLB.N, KMI.N) to Conoco Phillips are the ones identified in the energy sector in the network diagram above. Since the clustering is based on the MI of price volatility, we can see that indeed the volatility of these assets display similar behavior. For the sake of risk diversification, it is not a good idea to have a high concentration of these assets in the portfolio.
What about the assets least correlated to Conoco Philips in terms of risk? In this case our algorithm returns AT&T (T.N), which is a telecommunication company. Interestingly, both Conoco Philips and AT&T lie almost at the two opposite ends of the financial network diagram we have created (despite two algorithms to be independent of each other). During rebalancing, it may be a good idea to rebalance the positions on this pair of assets in order to minimize overall risks.
In this article we have discussed leveraging ideas in thermodynamics such as the entropy-based trading strategies, financial asset clustering and risk management. In future posts we shall explore the possibility of employing other physical ideas to detect regime changes in financial markets.
References:
[1] C. E. Shannon, “A Mathematical Theory of Communication,” Bell System Technical Journal, Vol. 27, 1948, pp. 379–423 (Part I) 623–656 (Part II).
[2] S. M. Pincus, “Approximate entropy as a measure of system complexity,” Proc. Natl. Acad. Sci. USA 88, 2297–2301 (1991).
[3] Richman JS, Moorman JR. Physiological time-series analysis using approximate entropy and sample entropy. Am J Physiol Heart Circ Physiol. 2000; 278 : H2039–H2049.
Acknowledgement:
Special thanks to Jeffrey Barnhart, Teng Dan, Ilya Kulyatin and Sakyasingha Dasgupta for their valuable input and discussions.
About the author:
Yon Shin is a Research Scientist at Neuri Pte. Ltd.
Originally published at https://medium.com on April 5, 2019.
