VRIQ Results

Testing Creative Problem Solving in VR

VRIQ is a 5-minute test of problem solving ability, published by 3DIQ on Steam as a study of VR education design. In its first week, we were able to collect the scores of 168 unique users. Scores for Vive and Oculus headsets were separated to account for differences between the two platforms. To prevent cheating and have as accurate data as possible, unique users only had their first score included in this dataset (we collected subsequent attempts in a separate variable, which we share the data for at the bottom of this page).

Vive Scores

Number of participants: 138

Mean Score: 18.09

Median Score: 18

Top 5% Score: 24

Top 13% Score: 23 or higher

Top 26% Score: 22 or higher

Top 35% Score: 21 or higher

Top 50% Score: 18 or higher

Bottom 25% Score: 15 or lower

Oculus Scores

Number of participants: 30

Mean: 18.43

Median: 20

Top 10% Score: 24

Top 37% Score: 22 or higher

Top 50% Score: 20 or higher

Bottom 33% Score: 16 or lower

Analysis

The Vive score distribution is fairly symmetrical, but rather than the hypothesis of a normalized distribution (1 peak with symmetry), there are 2 local peaks on opposite sides of the mean.

We have some hypothesis of why this might be the case, which spoil specifics of the test itself, so if you haven’t tried VRIQ yet, do so before reading our guesses here: VRIQ Score Breakdown

The Oculus score distribution resembles that of the Vive distribution, albeit with far less data points. The relatively low number of Oculus users comes as no surprise considering VRIQ was only published on Steam, which has fewer Oculus users. Additionally, the experience was designed and labeled as Roomscale, a setup which only a fraction of Oculus users have.

Multi-Try Scores

For fun, we also collected data from people’s attempts after their first. This data doesn’t differentiate between peoples 2nd, 3rd, and 4th tries.

As expected, the scores from repeated attempts are higher. Interestingly, the Vive scores still follow roughly the same pattern, with the exception of a lot more perfect scores. A possible reason for this might be that many of the multi-try scores on the Vive were friends of the initial user, thus trying it for the first time despite being recorded as a multi-try.

Full Paper

3DIQ Blog