Benford’s Law : Finding a regularity in irregularities?
Now I know reading the word ‘law’ in the title makes it sound like this is a boring article about some stupid law which is extremely complicated and difficult to even comprehend. Trust me, that was my thought process too until I was bored one day and saw an episode of ‘Connected’ on Netflix about this very law and got so fascinated that I proceeded to read about this for another two days. Now, google Benford’s law and the official definition will come up but here I’m going to define it the way I understood it.
If you gather any random number of numbers say, the populations of the different countries or the gas bills of random people or the number of aeroplane tickets bought by people of a particular airway and so on, and in this group of random numbers, you strip away all the digits of these numbers except the first digit (which could be any random integer from one to nine), and count the number of 1s, the number of 2s and so on till the number of 9s you will notice that there are more number of 1s than 2s, more 2s than 3s, more 3s than 4s, all the way till 9 and the number of 9s will be minimum.
Now I’m sure the first thought would be that this would be random and the law itself doesn’t seem real at all. But there are multiple experiments which were conducted and what’s mind-blowing is that the law came true almost EVERY SINGLE TIME. The digits seemed to follow this pretty curve which the Benford’s law gave.
This phenomenon was first noticed by Simon Newcomb who, while flipping through pages of a book of logarithmic tables, noted that the pages at the beginning of the book were dirtier than the pages at the end and this showed that his colleagues, who shared the library, preferred quantities beginning with the number one in their various disciplines. The law was later rediscovered by Frank Benford and the name stuck (which I’m sure Newcomb probably doesn’t appreciate).
Now, coming to its application in the daily grind, one professor from The College of New Jersey was tinkering around with Benford’s in the subject he was teaching that is forensic accounting and auditing. He saw how the real world numbers follow this unusual phenomenon and he thought whether he could use this to detect manipulated numbers in cooked books. He then produced a groundbreaking article in 1999 Journal of Accounting where he introduced his research on this extraordinary law and how to use it to detect frauds, anomalies or errors in a variety of data sets. But upon further investigation, it was seen that Benford’s is not applicable to just tax numbers or accounting data sets. It seemed to leak into all types of data. Say you take musical notes of some of the world’s most popular music ranging from Beethoven to Adele and measure how long a particular note is played and apply Benford’s to those numbers you will get that logarithmically generated graph again. This ‘pattern’ is pleasing to our ears. But one can argue that humans purposefully create ‘patterns’ in subjects like accounting and music.
Well, surprisingly the law applies to things where we don’t make patterns(either consciously or unconsciously) as well such as sports statistics. It even applies to things which are seemingly god’s will such as death rates, sizes of celestial bodies and so on.
How can it help humanity?
- Not only does Benford help detect accounting frauds it can also help detect election fraud and can even be submitted in court as evidence in a few countries such as the US. It was used as evidence against fraud in the 2009 Iranian elections.
- It can also help detect social media fraud. In an article published by Jennifer Goldbeck, a researcher from the University of Maryland, she explains in detail how she and her team discovered a large Russian botnet just by applying Benford’s to their follower count. This can be so helpful since it can help detect which companies are using bots to spread false promotion and in general help consumers be more aware.
- In today’s day and age, filters have become so advanced that it has become extremely difficult to separate the real from the fake. Things like deep fakes can make false rumours look true and ruin people’s life. A computer scientist by the name of Hany Farid tried to use Benford’s to battle this very problem and he succeeded to a certain extent. In his paper, he mentions that non-following of Benford’s in image forensics can be used as a fingerprint of sorts and forgery can be found from the amount of deviation found from the law in compressed images.
As you can see this isn’t just an ordinary boring law. This is a pattern found smack dab in the middle of nature’s randomness considering anything is even random anymore. It’s a pattern which our brain unconsciously creates and applies to almost all the decisions we make to such a big extent that even if you decide to do the opposite of what you stand for, it will unmask your lies by giving you a graph where the boundaries appear to be pushed. It’s fascinating how just a simple curve can expose some of the world’s biggest scams or frauds and yet it remains underrated.
So, now what? Is anything real? Do we really have free will? Is everything predictable now? Does the Illuminati exist and is it connected to this law? These are some things I will be pondering on this weekend. What about you?
Read more at :
