Superrationality: How Decision Theory Resolves Any Dilemma

Chapter VI: Functional Decision Theory

Hein de Haan
Street Science

--

In the previous chapter, we saw how PLAYER — a computer program — managed to succeed at both Newcomb’s Problem and Smoking Lesion by considering the question: “Which output of PLAYER causes the best outcome?”

In doing so, it essentially follows Functional Decision Theory (FDT). Where CDT asks which action causes the best outcome and EDT asks which action is the best evidence of a good outcome, FDT asks:

Which output of this decision procedure causes the best outcome?

In the computer world of Chapter V, it’s obvious PLAYER should one-box. But if we follow FDT in real life, what does it recommend?

When Carl, Eve and their friends where participating in Newcomb’s Problem in Chapter I, it appears Omega was modelling their decision procedure in order to make her prediction.

What do I mean by modelling a decision procedure?

Let’s back up a bit. PLAYER was implementing a function: it had an input (e.g. Newcomb’s Problem) and a fixed output for that input (one-box or two-box). OMEGA could easily implement that same function (by running PLAYER), in order to make her prediction.

Carl, Eve and their friends aren’t computer programs like PLAYER. But they have some way of making a decision in Newcomb’s Problem. So they, too, are implementing a function, and we call this implementation…

--

--

Hein de Haan
Street Science

As a science communicator, I approach scientific topics using paradoxes. My journey was made possible by a generous grant from MIRI (intelligence.org).