Why big data and algorithms won’t improve business strategy
swardley
20613

Thanks for the great article, it made me reflect on the why and the where in gaining situational awareness.

Military strategy focuses on competition where there are always losers. Why not the more powerful cooperative, diplomatic strategies? Where do they go in the map? Hopefully you cover that in the book, because its a limiting perspective to take a solely adversarial stance.

I enjoyed the chess analogy, but (disclaimer I work as a big data engineer) there is a way to derive meaningful patterns from sequences of moves, but it involves gathering the right data.

To extend your metaphor, the right data would involve knowing the exact piece moved not just that a random pawn was moved. You also need the set of moves possible for each piece, i.e. that a king can move one unit in any direction. You would need to make the deduction that the playing space is quantized into units, however, knowing that the board is an 8x8 two dimensional space is not necessary. It could be derived experimentally.

With the right data, and the right technique, and the large volume of historical “sequences”, you could iterate over potential ways the sequences could be mapped to any space, starting with 2 dimensional space containing at least the number of units of space as there are pieces.

There exists other useful information, and the more you have the better the model simulations, but arguably you don't need all of it for sequences to become meaningful. For example the starting position of all the pieces, which would also hint at the board dimensions. That data is useful to reduce the simulation iterations required, but not strictly required.

The more data you have, the closer you are to knowing the rules and seeing the board.

That conclusion plays right into your argument about having situational awareness, but I would argue that big data is precisely an approach to gaining situational awareness, not in opposition to your theory. It lets you “see” the board. Experience ( gut feel, heuristics whatever you choose to call it) is derived from information gained from data.

If you remove data, then all you have left is randomness, as in your control panel where you only know that a random pawn was moved, because you are missing the data to inform you of exactly which one.