It is beautiful if you could only see it…
I was once chatting with the geeky community on Discord, trying to find a solution to a Python programming exercise I couldn’t solve, when one person sent me a private message. I thought I received the solution on a ready plate and I was excited about it, only to read that this user found my pseudonym, t.logx, “very interesting and intriguing!” and asked for its meaning. “Can’t explain in a simple a message” I replied. “But I promise to write a blog piece about it whenever I find time and I’ll notify you.” That blog piece is what you are about to read.
We will go together in a -hopefully amusing- trip into the enchanting world of mathematics! Our destination is the enigmatic kingdom of logarithms, a place brimming with hidden wonders and arcane secrets. As we embark on this journey, we’ll take the magic-logic boat and sail through the mesmerizing rivers and lakes of entropy, where every wave’s ebb and flow reveals captivating tales of order and chaos, precision and possibility. So, prepare yourself for an adventure in which numbers come alive and the mysteries of the mathematical universe unfold before our eyes! It sounds like I am promising the world to you. I am.
Logarithms are gems of statistical mechanics.
Put data in a structure and you simply imbue it with a life of countless stories. Data finds its significance not in isolation, but through the intricate relationships forged within its structured framework. Data, much like the words of a sentence, hold little meaning when scattered and unorganized. Similar to how individuals find identity and purpose through their connections with other individuals, data gains depth and context through its relationships with other data points. Consider yourself: would you truly be alive without the bonds of parenthood and the relationships that define your presence in the wider society?
The arrangement of data itself forms a semantic structure, akin to how the order of words shapes the narrative of a sentence. This arrangement serves as the crucible where raw information transforms into actionable lively knowledge, unveiling patterns, connections, and insights that would otherwise remain dormant. By organizing data into a coherent structure, we unlock its potential. Thus, the structure not only gives context to data but also empowers us to navigate and understand the vast landscapes of information data could offer.
If you have a table of 5 rows and 2 columns then there are (5x2)! possible arrangements of data within that table. Let us call W the number of possible arrangements of data in this table. Then W=10!= 3 628,800 possibility of arrangement. This means that the data you’re looking at to analyze and understand has in fact 3 628,800 different configurations. Each configuration has its own life and its own story to tell.
Enter Entropy:
The entropy of the dataset indicates its level of (un)certainty. Differently put, the entropy indicates the amount of information required to describe a dataset with a number of possible configurations with absolute certainty. Given that the entropy and the number of possible arrangements are connected through the binary logarithm function, we can say that the entropy of this dataset, let’s call it E is equal to log2(W). Thus, E = log2(W) = log2(10!)= 21.8 bits of information
Before understanding what 21.8 actually means, let’s first have a better grasp of the meaning of entropy. I was first introduced to the concept when I came across an interesting book about the meaning of life, “Until the end of time” by Brian Greene. A witty and insightful book that I definitely recommend to the scientifically inclined reader.
Brian Greene explains that entropy, a measure of system (dis)order, naturally tends to increase because the number of disordered configurations vastly exceeds that of ordered ones. Using the analogy of tossing coins, he illustrates how each additional element exponentially increases possible outcomes, making high-entropy states (disorder) overwhelmingly more probable. This statistical tendency drives systems toward higher entropy, making a spontaneous return to a state of low-entropy exceedingly unlikely. Consequently, this principle dictates the arrow of time, guiding the evolution of systems from order to disorder.
I’m pasting a passage from his book below. It should explain things better. Brian, the floor is yours:
“Start with an analogy: Imagine you vigorously shake a bag containing a hundred pennies and then dump them out on your dining room table. If you found that all hundred pennies were heads, you’d surely be surprised. But why? Seems obvious, but it’s worth thinking through. The absence of even a single tail means each of the hundred coins, randomly flipping, bumping, and jostling, must hit the table and land heads up. All of them. That’s tough. Getting that unique outcome is a tall order. By comparison, if we consider even a slightly different outcome, say in which we have a single tail (and the other 99 pennies are still all heads), there are a hundred different ways this can happen: the lone tail could be the first coin, or it could be the second coin, or the third, and so on up to the hundredth coin. Getting 99 heads is thus a hundred times easier — a hundred times more likely — than getting all heads.
Let’s keep going. A little figuring reveals that there are 4,950 different ways we can get two tails (first and second coins tails; first and third tails; second and third tails; first and fourth tails; and so forth). A little more figuring and we find that there are 161,700 different ways to have three of the coins come up tails, almost 4 million ways to have four tails; and about 75 million ways to have five tails. The details of the numbers hardly matter; it’s the overall trend I’m driving at. Each additional tail allows for a larger collection of outcomes that fit the bill. Phenomenally larger. The numbers peak at 50 tails (and 50 heads), for which there are about a hundred billion billion billion possible combinations. Getting 50 heads and 50 tails in therefore about a hundred billion billion billion times more likely than getting all heads.That’s why getting all heads would be shocking.”
[…] If this material is new to you, you may not realize that we have now illustrated the essential concept of entropy. The entropy of a given configuration of the pennies is the size of its group — the number of fellow [micro-]configurations that pretty much look like [macro-]configuration. If there are many such look-alikesm the given configuration has high entropy. If there are few such look-alikes, the given configuration has low entropy. All else being equal, a random shake is more likely to belong to a group with higher entropy since such groups have more members.
[…]
because the number of configurations with more entropy is enormously greater than those with less entropy (by the very definition of entropy), the odds are enormously larger than random jostling — the relentless bumbing and vibrating […] will drive system toward higher entropy, not lower. The progression will continue until we reach a configuration with the highest entropy available. (pp.23–33)
Back to our original example, it is enough to understand that higher entropy corresponds to greater complexity, indicating more possible configurations. If E (representing the value of entropy) is high, the dataset has a large number of possible arrangements, implying high variability or randomness, which translate into low certainty and high uncertainty. If E is low, the dataset has a tiny number of possible arrangements, implying low variability or predictability, which translate into high certainty and low uncertainty.
What entropy in this regard means is that to uniquely specify one arrangement out of the 3,628,800 possible arrangements of the numbers in the table, you would need approximately 22 bits of information — whose role would be to accurately describe your table at that specific state or arrangement. These bits of information are used to describe a specific arrangement of the table uniquely. This means that each unique arrangement requires about 22 bits to be encoded distinctly from others. The bits of information do not describe the properties of arrangements but rather the distinctiveness of each arrangement in terms of information content.
Does it sound complicated? Bear with me. The previous example was a bit abstract. Let’s try a more tangible one
Go fetch a deck of cards from your shelf. Assuming the deck contains 52 cards. Imagine if you shuffle that deck to eternity, what is the final number of unique arrangements will you get in the end? Exactly! You will have a gargantuan W=52!= 8.0658 x 10⁶⁷ possibility. That means even if you spent your entire life shuffling, you wouldn’t scratch the surface of the potential permutations available.
Now, the entropy (or measure of certainty/uncertainty) of this colossal W is E — which is equal to? Yes, correct! E and W are governed by a logarithmic relationship. E=log(W)=log(52!)=226.2 This entropy value represents the amount of information required to uniquely identify the exact sequence of cards in the deck. So, if you have less than 226.2 information, none of the arrangements would be possible. Let’s remove one bit of information from the entropy and see what happens. Assuming we remove the bit of information that relates to the card’s position in the deck, would the sequence remain the same? Put your cards back to the shelf and come back again. There’s more to tell…
What “bits of information” here really means is the attributes or details (or dimensions if you want) we need to have in order to undestand the overall description of the deck’s order. Given that our deck contains 52 unique cards, the bits required to identify a specific card would be E=log2(52)=5.7 bits of information. What is then the total number of information needed for all 52 cards? Simply: 52 x log2(52) = 296.4 information.
A trip to complexity…
The more cards there are in our deck, the more possible arrangements we could have. Accordingly, the higher the entropy, the more information is needed to eliminate uncertainty about the exact configuration. And vice versa. This phenomenon underscores the principle that with greater disorder comes an increased requirement for information to accurately discern the precise configuration. Conversely, as order diminishes, uncertainty mounts, necessitating more data to reduce ambiguity.
Back to the grave: simplicity.
The existence of an infinite number of cards implies an infinite spectrum of possible arrangements. Consequently, the ordered arrangements do not surpass the disordered ones in quantity; rather, they share an equilibrium, coexisting on equal terms. Just as the characters in a timeless drama play out their roles, the concept of ever-increasing entropy meets its ultimate fate in the infinite. In this realm, 𝐸 = log(∞) eludes definition within the confines of the real number system, for infinity transcends the notion of number — it is the gateway where numerical values dissolve into abstract concepts. Infinity signifies unbounded growth rather than a finite value. Therefore, in the infinite, entropy itself becomes a moot point, as the conventional measures of order and disorder cancel each other out to ultimately lose their significance.
Entropy and certainty:
Here is a much better example to grasp how entropy serves as a measure of certainty in a specific system. We will play a game together with our friend Sabrina. She will choose a number between 1 and 1000. We will have to guess it. The probability for a correct answer would be one over a thousand possibility p(A) = 1/1000 = 0,001 = 0.1%
How would we proceed to get find an accurate guess?
Well, if there are 1000 possibility, then the entropy value would be E = log(1000)= 6.9 which approximates 7 information. This means that there are 7 properties that if we find them, you will be able to find the correct answer. It is pretty much like a chasse aux trésors. Each bit of information is a treasure that you need to hunt. The game isn’t as easy as you might think — or as it might appear indeed. But let us play together and guess the number our friend Sabrina is hiding. To do that, we need to ask 7 different questions:
In the first question, let us try to locate the exact position of the number in the sequence of available options.
- Does the number exist between 1 to 10?
- Yes.
We’re lucky! Else, we would have asked the same question repeatedly until we locate our number in a tinier set. Humm, the first piece of the puzzle is found. Let’s move to figure out the nature of the number.
- Is it real, integer, rational, irrational, complex?
- It is a rational number.
Second piece is found. Let’s get the third piece.
- Is it pair or odd?
- Odd.
Aha! This leaves us with yet another limited set of options. The available set of possible answers are: [1, 3, 5, 7, 9] I can see the entropy decrease and the order is now starting to reign slowly but surely. Let’s carry on the hunt of the 7 dragon balls.
- Is it a prime number?
- Yes.
Good. The puzzle is getting clearer. There are less options, less entropy, more certainty. Now the answer is either [3, 5, 7 or 9] Technically, there are 3 questions remaining to be able to accurately determine, but the solution is getting clear that we could predict now.
- Is it less or equal to 5?
- No.
Ah, mist! Okay, then it must be either 7 or 9. Two remaining bits of information. Let’s use relativity. The relationship between numbers in a dataset is a powerful indicator.
- Is the number divisible by 3?
- No.
Hurrrraaayyyy! With absolute certainty, we can say the answer is 7. But wait a minute…7 bits of information and 7 is the hidden answer!! Is that a matter of luck? Of course not! Any of the number in the 1000 possible answers has 7 unique features — which we called “bits of information”. Had Sabrina replied “Yes” to our last question instead, the answer would be 9.
A notable observation is that the logarithm base 2 of 1 is 0, indicating no information gain, as log2(1) = 0. From this, I understand that a single element, on its own, does not convey any growth or complexity. If considered individually, each singular entity bears a perfect sense of order. However, when a singular entity is considered in relation to a set of other entities, the concept of entropy or disorder comes into play. The arrangement and relationships among multiple entities introduce complexity, leading to variability and uncertainty. The singular is perfectly ordered, the plural is rather disordered.
The game we just engaged in closely mirrors an enlightening introductory philosophy course on logic I took at Stanford University. This course utilizes logical sentences to verify truths. In the exercise, four individuals (P1, P2, P3, and P4) are involved, with varying relationships — some like each other while others do not. Representing these relationships on a 4x4 table yields 2¹⁶ possible true-false combinations. (Bear in mind that 2¹⁶ represents all possible combinations of true/false states for the relationships between four people in 16 cells. 16! on the other hand, represents all possible permutations (arrangements) of 16 distinct items, not applicable to binary state problems.) Logical sentences limit the possible configurations of the world of options by splitting the set of potential scenarios into those where the sentence is true and those where it is false. Ideally, with sufficient sentences, we can precisely determine the state of affairs. The language of logic allows us to articulate incomplete information when that’s all we have, and complete information when full details are available.
I hope you come to appreciate the meaning of logarithms and their role in measuring the entropy of a system. And now to come full circle with how I came to label myself t.logx, I need to go back in time a decade ago or so when I was still studying at high school. We were studying logarithms and its inverse operation, the famous exponentiation. Although my understanding back then was quite superficial and limited to the basic rules and applications, I found them quite profound — but was not able to grasp their epistemological meaning. I was intrigued and wanted to learn more about them. Pondering on these mathematical tools over the years, I came to realize that, in a philosophical sense, that logarithms represent our human attempt to make sense of the immense and the infinitesimal, to find order in chaos. Moreover, I came to cherish them as a testament to our desire to understand the world around us, to our attempts to quantify and measure the world, and to unveil its underlying patterns.
And being a life observer, I found nothing that could represent me better than logarithms. I chose them to make part of my pseudonym mainly because they speak for the idea that through logic and reason, we can find clarity and insight into the fundamental principles that govern everything from the smallest particle to the largest galaxy.
In summary, “t.” symbolizes my name and identity, with the dot denoting multiplication (this is yet another story on the “multiplicative perspective”. I might write about it in the future. Let me know in comments or privately if you’re interested). “Log” represents logarithms, while “x” stands for the unknown. Together, my pseudonym signifies my quest to observe and understand the mysteries of the unknown — reflecting the existential answer I discovered when pondering my place and purpose on this planet. Essentially, it captures my pursuit of understanding life’s enigmas. Additionally, the pseudonym reads “t.logics” which further underscores my dedication to logic and rationality, affirming my adherence to reason and thoughtful/ unbiased inquiry.
Mathematics does not merely mirror the physical world, nor is it something we stumble upon like an ancient artifact. Instead, mathematics is interwoven into the very essence of reality itself. It isn’t just a language we use to describe our observations or a series of discoveries about the natural world; rather, it forms a fundamental part of the underlying structure of existence. I’m not talking about the invented mathematics — the subjective construct through which some have decided to interpret the world. I’m talking about THE Mathematics, the real one that is intrinsic to the physical world and that reflects the deep, inherent patterns governing all aspects of our reality.
I hope that by the end of this blog piece, you could look at any random sequence of data within a table and appreciate its diversity, for it is a tapestry woven from the threads of countless possibilities. I hope you you could cherish the relationships these numbers have between each other for these are lifes on their own. Can you listen to the symphony data are playing? Can you watch their fascinating dance? If you could, then this blog piece has delivered its promise… because…It is beautiful… magnificently beautiful…if you could only see it…
t.logx
