Simulating Wisdom of the Crowds

Hi, I’m an Ox. Can you guess my weight?

Introduction

In 1907, in a village south of Boston, Sir Francis Galton gathered a group of villagers for a competition. The goal was to guess the weight of an Ox. A total of 787 votes were cast, can you guess who got it right?

All of them. Well, all of them together. The average of the guesses was only 9 lbs or 0.8% off the actual weight! This is an illustration of the wisdom of the crowds. More formally, social learning, the ability to learn from others in a group. Social learning is a hallmark of our species and a very big reason for our survival.

However, there is a problem with social learning. In a recent paper, Courson¹ et al., point out that if social learning is the most beneficial strategy, then everyone ought to engage in it. Though, if everyone adopts this strategy, there is nothing being learned! Everyone is copying everyone who is copying everyone.

For the wisdom of the crowd to work, independent learners are needed. Dalton’s experiment only worked because each villager had relatively independent experiences and knowledge of an Ox’s weight. If the majority decided to copy, the estimates would become biased.

On the one hand, social learning works extremely well for survival, on the other hand, individual learning is necessary for social learning to succeed. The two strategies form a paradox. To resolve this, we can look at what the optimal proportions of followers and non-followers are in a population.

Illustrative Example

To explore the question of how success changes based on the number of followers, we can play a little game².

Let’s assume a world in which each person has a 1/2 probability of being right. However, the population is split. A proportion of people are informed, that is, they do not follow the crowd and form their own opinions. Because of this, they have a higher than chance probability of being right. The rest of the population are followers, they copy what the rest of the population thinks.

Here is the breakdown of the game:

1. We start with a list of initial guesses, random for followers, and better than random for informed agents.
2. Then, we pick a follower and ask them to endorse the answer of a subset of the population.
3. We repeat step 2 over and over again until all followers have adopted a new opinion

Here are the starting conditions:

N = 100 #the number of people in the population
p = 0.4  #proportion of followers
q = 0.52 #accuracy of the informed group
m = 11 #the size of the subset that the follower looks at

Let’s start with some initial guesses:

Each square represents an individual’s guess. Blue=1, White=0

Now, we pick a follower and have it observe the opinion of a group:

The follower (red rectangle) adopts the consensus of the group (green rectangle)

We repeat the process over and over until all followers have adopted new opinions, and get the following:

Distribution of opinions after all followers have played the game

Voila! A 13-point increase in accuracy just from copying others’ opinions.

Simulation

So far, we have seen some proof that the following strategy can increase overall population accuracy. But, we still don’t have the answer to our question, what is the optimal proportion of followers?

To answer this question, we repeat the game with different proportions of followers and aggregate the results, which look like this:

Each point represents a result from one game, 10 games per point on the X-axis. The red dotted line represents 50% accuracy

The results show that as the proportion of followers increases, the overall accuracy numbers go up. Though, not after a certain point. Do you notice what happens after around 0.6? As the proportion of followers increases past this point, the result is either really good or really bad!

This happens because as the proportion of followers approaches 1, the aggregated opinion becomes more and more a reflection of itself. As Courson et al., put it, “the imitation behavior spreads until the snake bites its own tail”. Eventually, the population is guessing at the chance, getting lucky half the time, and completely wrong the other.

To be sure, this happens even if the informed agents are 100% likely to get the answer right:

Informed accuracy, q, set to 1

So, we are left with an unsatisfying answer. Followers are good until they are not. Truth is, there is no magical number. What does seem important is the presence of individual learners.

Final Thoughts

What I’ve shown above is a very simple, yet powerful³ illustration of the benefits and perils of the wisdom of the crowds. Herding behavior can be beneficial up to a point, but overuse of this strategy, as sometimes seen in financial markets, can lead to increasingly variable outcomes.

These increasingly variable outcomes occur due to a reduction of diversity in opinion in the population. In their paper, Courson et al., offer strategies to maintain this diversity. One of them is a society that values original opinions, especially in contexts where the answer is unknown.

Code

Code for the simulations above can be found here

Citations

[1] de Courson, B., Fitouchi, L., Bouchaud, JP. et al. Cultural diversity and wisdom of crowds are mutually beneficial and evolutionarily stable. Sci Rep 11, 16566 (2021). https://doi.org/10.1038/s41598-021-95914-7

[2] This is known as the Curty-Marsili Framework,

Philippe Curty and Matteo Marsili J. Stat. Mech. (2006) P03013. DOI 10.1088/1742–5468/2006/03/P03013

[3] Bikhchandani, Sushil, David Hirshleifer, and Ivo Welch. “A theory of fads, fashion, custom, and cultural change as informational cascades.” Journal of political Economy 100.5 (1992): 992–1026.