The law of large numbers in trading and investing for non-statisticians

The law of large numbers is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed. [Wikipedia]

This law in trading and investing can be applied two ways:

1. To derive the true (expected) value from some tested event, we should gather as much event and its outcome(s) samples as possible.
2. We should test the event (or idea) only once, especially when seeing out of sample data, because further improvements of the strategy will increase probability of finding spurious relationships (correlations).

This is how it should be done, but as we know from Daniel Kahneman, human nature and our usual, fast thinking is poor statistician, consequentially, we are poorly applying this law in practice.

How does the law and those two basic rules apply in trading and investing?

1. Without enough event samples we don’t know the expected value, we can only rely on our own personal (and, as more samples from most of personal trading can show, — usually wrong) judgement.
2. With too much testing we would create systems, that are too complex, and as such will fail with unseen, new data, when those systems will encounter the future.

World of retail trader’s is full of erroneous applications without understanding this law, like “combine indicators and parameters to find the best fit”, “I saw this chart figure, it produced good results (few times), let’s make the system around it”, “I had used this system for a week, markets changed and it performed poorly, I need to improve it”.

Even if there are rare exceptions, simply put, because human nature doesn’t apply this law (and many more rules necessary in today’s trading or investing) well, we betray ourselves when we “believe in ourselves”, in our own personal (unfounded by any data) judgement. This mechanism is reinforcing, keeping traders to use same unfounded “analysis” techniques until their accounts usually are gone.