How to do Functional Decision Theory

A gentle introduction

Hein de Haan
How to Build an ASI

--

Photo by Jan Genge on Unsplash

In How to do Decision Theory, I introduced the reader to the field of Decision theory. We looked at both Causal Decision Theory (CDT) and Evidential Decision Theory (EDT), and saw how both theories have their pitfalls: CDT doesn’t maximize expected utility in Newcomb’s problem, whereas EDT fails to do so in the Smoking lesion problem.

Functional Decision Theory, proposed by Eliezer Yudkowsky and Nate Soares, is an attempt to do better than CDT and EDT. The reader can look at FDT as follows: where CDT asks which action causes the best outcome (in terms of utility), FDT asks which output of her decision procedure causes the best outcome. (EDT, on the other hand, would ask which action would be great in retrospect, from an evidential standpoint). It may sound like the difference between FDT and CDT is just superficial, but it really isn’t. Let’s look at Newcomb’s problem to make things a bit more clear. Remember, in Newcomb’s problem, Omega (an unusually great predictor) says to you:

“Here are two boxes, A and B. A is transparent, and it contains €1000. B is opaque, and contains either €1.000.000 or nothing. You can choose to take both boxes, or only box B. The content of box B is fixed before the game starts. If I have predicted you choose only box B, then and only then does it contain…

--

--

Hein de Haan
How to Build an ASI

As a science communicator, I approach scientific topics using paradoxes. My journey was made possible by a generous grant from MIRI (intelligence.org).