Probabilistic reasoning in ReasonML

Imandra by Aesthetic Integration is a reasoning engine and a proof assistant that is able to formally verify statements about pure ReasonML/OCaml programs. For invalid statements, Imandra can provide explicit counterexamples refuting the claim. An amazing feature of Imandra is the ability to reason about functional inputs and generate functional instances and counterexamples. For example:

Here, we have introduced a variant type and a second-order function that evaluates its argument on each variant and computes the sum of the results. Then we’ve asked Imandra to come up with an instance of a function f that returns positive values and that produces 42 when calling sum(f).

A natural application of this ability is to apply it to formal reasoning about random variables and probability distributions over them. In the following, we’ll consider an application of this kind of reasoning to analyse the famous Monty Hall problem.

This post has a live Imandra Jupyter notebook version where you can evaluate and experiment with the examples live in the cloud. (There is also an OCaml version of the notebook.)

Monty Hall problem

In the Monty Hall problem, you are participating in a game where you are given three doors to choose from. One of the doors randomly contains a $1,000,000 prize. Let’s encode that in ReasonML — we’ll introduce the value of the prize and the variant type for the doors.

The game then proceeds as follows: first, you guess one of the doors, then the host opens another door — a door that, the host knows, doesn’t contain the prize. The host then offers you to change your initial guess or stay with it.

To encode that, we’ll introduce another variant type for the player choice and gather all the values that describe a single Monty Hall trial into a scenario record.

Probability mass functions

We want to treat scenarios described by the “scenario” record as random. This means, that we would like to deal with probability mass functions (PMFs) over the variables of the declared types. As a first example let’s make a PMF for the doors that distribute over the three doors with equal probabilities:

Notice that we are using the real type for numeric values in our examples. Internally, Imandra represents reals in an arbitrary-precision manner. The real-valued arithmetic operators are gathered in the Real module. Further, we’ll be explicitly opening the Real module to stress that we are dealing with the rational arithmetic.

Not any function can be a probability mass function — PMFs must satisfy the so-called “measure axioms”: all the returned values must be positive and the total probability over the whole function domain must sum to one. Let’s make a predicate that checks whether a function is a valid PMF over doors and use it on out pDoorsEqual to check that it is, indeed, a valid probability distribution:

The player controls the first_guess and thechoice fields of the scenario record. In general, the player can also choose his actions randomly. Following the game theory nomenclature, we’ll call the probability distribution over the player choices a “strategy”. For example, here is a strategy that chooses the first guess door randomly and then stays with that choice:

A strategy’s type is door => choice => real: it takes in which door the player chooses and which choice he makes and returns the probability with which that happens. To check that such a function is a valid probability mass function we introduce and apply a predicate for it:

Finally, we’ll need one last function that constructs a PMF over the whole scenario record. It shall take a door => realprize PMF, a door => choice => realplayer strategy and return a scenario => real PMF over the scenarios. Since we consider the host and the player choices to be independent, the total probability of the full scenario must be the product of the probabilities of the host and player choices:

Expected rewards

Given a scenario variable, we calculate a reward that the player gets. When the first guess is equal to the prize location, then we return the prize value if the player decided to stay. Alternatively, when the first guess is not equal to the prize location, then we return the prize value if the player decided to swap.

Given a probability distribution over scenarios, we want to calculate the expected value of the reward — the probability-weighted average of possible values of the reward for each outcome. We create a function that performs this averaging: it takes in a scenario PMF of scenario => real type and returns a real value for the expectation value:

As a final example, let’s use what we’ve built so far to calculate the expected reward for the scenario when the host hides the prize equally randomly and the player uses the random_then_stay_strategy:

We get an obvious result that if we choose a random door and stay with it, then the expected reward is 1/3 of $1,000,000. That seems reasonable and it is not very obvious that one can increase his chances of winning.

Reasoning about Monty Hall probabilities

Let’s ask Imandra if 1/3 of a $1,000,000 is the best outcome for any strategy the player can employ. To do this we’ll use Imandra verify directive to verify that, for any valid strategy, the scenario with equally probable prize placement gives an expected reward less or equal than 1000000/3.

Imandra refuted our statement and provided us with a counterexample strategy. Imandra suggests that the expected reward will be greater than 1,000,000/3 if the player chooses theDoorA and then Swaps his choice in 0.0003% of the cases. The counterexample strategy is defined in the CX module, so let’s see what is the expected reward with this strategy:

The counterexample strategy that Imandra produced increased our expected winnings by $1. This counterexample strategy employs swapping instead of staying. What would be our expected reward if we always swap?

By always swapping we’ve increased our chances of winning up to 2/3. Is this the best possible expected reward one might get in this game?

We’ve proven that the best possible expected reward in the Monty Hall game is 2/3 of a $1,000,000 in an (arguably, rather counterintuitive) result that one has to always swap his first choice to double his chances of winning.

Suppose now that we don’t know the winning strategy, but we do know that it is possible to win in 2/3 of the cases. Can we ask Imandra to synthesise the optimal strategy? For this purpose we can use the instance directive:

We’ve got an example of an optimal strategy — this one doesn’t randomize over doors, but suggests to always choose the DoorA and then swap.

Biased Monty Hall problem

Now let’s apply what we’ve learned about the “standard” Monty Hall paradox to a more complicated game setup — suppose that we somehow know that the host is biased towards the DoorA— the probability that the host will hide the prize behind the first door is 3 times higher than the individual probabilities for two other doors. Should one still always swap? Or, maybe, it makes more sense to always choose the DoorA and stay?

We encode our biased host in the biased PMF over doors (also doing a sanity check that it is a valid probability distribution):

In the previous section, we’ve proven that the maximal possible expected reward in the “standard” Monty Hall problem is 2/3 of a million. Again, we ask Imandra whether we can improve on this in the biased case:

Imandra suggests that we can do better than 2/3 if we use a mixed strategy of either choosing theDoorA and staying or choosing theDoorC and swapping. Let’s test both of these pure strategies and see what would be the expected rewards for each:

Choosing theDoorC and then swapping increases the chances of winning up to 80%! Again, is this the best outcome one can get?

And we’ve proven that the DoorC-Swap is the best strategy the player can employ in this setup. Intuitively the reason for that is that it is better to choose the least likely door, forcing the host to reveal more information about the prize location in the more likely doors.

Using this approach one can go further and analyse even more complicated Monty-Hall-like games (for example, the prize value might depend on the door and/or player actions). More generally, this illustrates how we can use Imandra’s ability to reason about functional values and real arithmetic to formally prove and refute statements about probability distributions. These abilities allow us to us to formally analyse many kinds of scenarios requiring the ability to reason about random variables.

This post has a live Imandra Jupyter notebook version where you can evaluate and experiment with the examples live in the cloud. (There is also an OCaml version of the notebook.) To learn more about Imandra, check out our other notebooks on try.imandra.ai and to stay in touch with our updates, please sign up on our website.