How to do Decision Theory

A gentle introduction

Hein de Haan
How to Build an ASI

--

Photo by Jan Genge on Unsplash

It’s Friday night, and you’re at a party with a hundred other people. At some point, a strange looking being walks in. The being calls himself Omega, and claims he is very good at predicting future events. Omega has a lot of money (which is unsurprising if his claim is actually true). He also likes to play a game, of which he explains the rules as follows:

Here are two boxes, A and B. A is transparent, and it contains €1000. B is opaque, and contains either €1.000.000 or nothing. You can choose to take both boxes, or only box B. The content of box B is fixed before the game starts. If I have predicted you choose only box B, then and only then does it contain €1.000.000. Otherwise, it is empty.

Omega plays the game with everyone at the party before he comes to you. About half the people won €1.001.000, and the others won €1000. Interestingly, however, it turns out Omega correctly predicted the choice of everyone at the party. Therefore, you know Omega is an (unreasonably) great predictor. But now it’s your time to play. Do you choose to take both boxes (“two-boxing”), or only box B…

--

--

Hein de Haan
How to Build an ASI

As a science communicator, I approach scientific topics using paradoxes. My journey was made possible by a generous grant from MIRI (intelligence.org).