Randomness is weird

Welcome to part 1 of a series of posts about workflow management for software developers. I’ve taken almost everything I know about this from Reinertsten, The Principles of Product Development Flow, but I think there’s value in approaching that book’s ideas in a different order and with more worked examples.

I want to show you something surprising about randomness. On its own, it might not look relevant to anything, but I promise: this is going somewhere, and there will be follow-up posts.

Let’s play a game. What we’re going to do is:

  1. Keep a cumulative total. It starts at 0.
  2. Flip a fair coin. If it lands on ‘heads’, we’ll add 1 to the cumulative total. If it lands on ‘tails’, we’ll subtract 1.
  3. Repeat step 2 a bunch.

What do you expect to happen with the cumulative total? Most of us would expect it to hover close to 0 and bounce around it a little bit, something like this:

Looks fine.

And we’re sort of right with this intuition. The most likely cumulative total is 0. Unfortunately, the more flips we do, the more likely it is that something like this will happen:

Erm… what?

Or even something like this:

Oh wow.

What on Earth is going on here?

Well, let’s imagine we flip the coin and it lands on heads twice in a row. That can happen: the probability of that is 0.25. Our cumulative total is now 2. What is the probability of it getting back to 0? We’d have to flip tails twice in a row. The probability of that is 0.25. But that’s the same probability as flipping heads twice in a row again, and getting to 4. And the most likely outcome from 2 flips of the coin is a change of 0. So we have a 75% chance of staying above 0. And the further away we get from 0, the less likely it is that we’ll ever get back to it. The coin flips are memoryless, but our cumulative total is not.

This behaviour of randomness, where we cannot rely on it to fix itself, has implications all over the place. I’m going to get into its implications for the way we organize work in a subsequent post. The next one, though, is going to be about something different: the economics of our work. If you stick around for long enough, I promise we’ll relate these things.

P.S.: If you’d like to play with randomness, I’ve written some truly awful Ruby to toss coins: https://github.com/rentalcustard/coin_flips. Run that stuff a few times to convince yourself I didn’t just pick cute examples. Do 1,000 flips 10,000 times and look at the distribution of cumulative totals. Then do it for 100,000 flips, then for 10,000,000 flips. How does the shape of this distribution differ?