Statistical analysis of Etheroll’s random number generation

Tucker Cahill Chambers

Etheroll is a probabilistic gambling dApp built on the Ethereum blockchain. How does it work? Imagine a dice with 100 faces. You choose a number and bet that the dice result will be lower than your chosen number. For example, you bet that the dice result will be lower than 51, the dice is rolled, it shows a 43, and so you win. The game gives a 1% edge to the Etheroll contract, such that a winning bet receives 99% of what it would receive in an unweighted game (this is where the gambling part comes in). If you would like to look at some well-curated statistics from the game, check out myetheroll.com. Etheroll is designed to emulate random dice rolls via an Ethereum smart contract, and so the integrity of the game is dependent on the contract’s ability to produce sufficiently random numbers. However, random number generators (RNG) are an intriguing topic in blockchain, and at least on Ethereum a truly random RNG has yet to be devised — it is a hard problem. See this article for an in-depth explanation of different kinds of RNG that have been deployed on Ethereum, and their vulnerabilities.

Etheroll of course collides with this blockchain RNG issue. Etheroll’s solution is to outsource its RNG to an off-chain third party, Provable (previously known as Oraclize). When a player makes a bet by submitting a transaction to the Etheroll contract, the contract then sends a transaction to Provable’s contract, which replies in turn with its own transaction containing a privately-generated random number, which is then compared to the player’s chosen number and the bet is finally resolved either in favor of the player or the house. Using a third party to generate a random number is of course vulnerable to manipulation — how can a user know that the privately-generated number was in fact truly randomly chosen? Provable has their own rational interest in guarding the integrity of their RNG service, but the fact remains that users of Etheroll or any other contract using an offchain source of randomness are ultimately in the dark as to the mechanics of the RNG and the honesty of its result.

We can, however, look at the random numbers used by the Etheroll contract and perform some simple statistical analysis to see if any biases or patterns exist, which would suggest a non-random stream coming from the Provable oracle. So, does Etheroll roll a weighted die? Let’s take a look.

I analyzed 269,754 Etheroll results provided to its contract by Provable, starting in September 2018 at block 6290000. I pulled the results from the contract with some simple code via the Etherscan API. If you would like to download those results as a CSV, it can be found here: https://github.com/tuckertranslator/Etheroll, along with the code used to call the Etherscan API and parse the block data.

The first thing to look at is how many times each random number appears. In a truly random sequence of sufficient length, each number should appear the same number of times. Below are two histograms: the first, from the 269,754 Provable results; the second, from 269,754 random numbers generated by the Julia language rand() function.

Standard deviation: 28.888
Standard deviation: 28.863

The two histograms are quite similar, and their standard deviations are nearly identical. Next, let’s look at the probability that a random number n+1 is greater or less than the number n preceding it. To calculate this, I iterated over all of the numbers in both the Etheroll and rand() sequences, counting how many times each number between 1 and 100 was followed by a greater or lesser number. For example, we should expect that 50 is followed equally by greater and lesser numbers. The precise function used was to take the two counts — how many greater and lesser numbers — and divide the smaller of the two by the larger. For example, both 25 and 75 should return the same expected value of ~0.33.

Next we will look at the distances between consecutive appearances of the same number; that is, if we observe e.g. a 24, how many numbers appear before 24 appears again? With 100 possible numbers, we should expect the mean (or average) to be 100. The median should follow a geometric distribution, in which for 100 possible values, we should expect a median gap of 70 (for an explanation of why, see this Reddit post). As we can see below, both the mean and median gap sizes for each result are grouped around their expected values.

And for comparison, here is the same graphic produced with the rand()-derived set of 269,754 random numbers:

We can ask more specific questions as well, such as: how many consecutive numbers less than 11 do we observe? A number less than 11 should have a 10/100, or 10%, chance of appearing, and so two consecutive such numbers should have a 10/100 * 10/100, or 1%, chance of appearing. The code below iterates over all of the Provable results and counts appearances of n-consecutive numbers less than 11.

cnt_array = [] #initialize array for storing counts of consecutive appearancesfor i in 1:length(result_array)-1 #iterate over all Provable results
cnt = 0 #reset count
while result_array[i] < 11
cnt += 1
i += 1
end
push!(cnt_array, cnt) #store consecutive count
end
c = countmap([run for run in cnt_array]) #count chains of diff. lengthssort(collect(c), by=x->x[1])[3:end] #sort results and print----------[OUTPUT]
2 => 2485
3 => 256
4 => 28
5 => 2

From the OUTPUT we can see that the Provable results follow the expected pattern. And the same holds true if we look at consecutive appearances of numbers greater than 90.

2 => 2430
3 => 243
4 => 27
5 => 3
6 => 2

In conclusion, from this simple analysis, it appears that Provable’s RNG has at least superficially (i.e. beyond the depth of this analysis) provided sufficiently random numbers. However, the fact remains that Provable’s RNG is private and offchain, and the integrity of its previous outputs does not guarantee that its future results will not be compromised in some way.

Finally, if anyone is interested in running their own betting strategy simulations, I have published some simulator code here: https://github.com/tuckertranslator/Etheroll/tree/master. Here is an example output from one strategy simulated over the historical Provable results (remember, don’t overfit your models!):

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade