The Problems of using Data from Virtual Reality Experiments

Is Research Data from Virtual Reality Experiments Automatically Invalidated?

Feb 12 · 14 min read
Virtual Reality promised us new experiences and immersion simply not possible via other means. Has it succeeded? The jury is still out on that one. Image source: Oculus / Facebook

Limitations of Virtual Reality

There are two key areas in which Virtual Reality hasn’t quite hit the mark with the current incarnation of virtual reality — immersion and interaction. These aren’t the sole reason why all research utilizing virtual reality should be invalided in my eyes but they are two key areas that should not be overlooked.


One of the marketing buzzwords that has followed Virtual Reality around since the dawn of the idea of virtual reality devices has been “Increased immersion”. The big selling point of VR has been the idea that you can only get so immersed in 2D images on a television screen whereas with these goggles, you can be transported into another world.

The film Ready Player One features an immersive virtual world known as the OASIS, accessible with technology that is designed extremely similarly to the virtual reality technology available today. Image source: Entertainment Weekly / Warner Bros.
  • Sensors such as accelerators and gyroscopes built inside of the headset track the position and rotation of the user’s head, sending this data to the rendering device to offer the illusion that the user is looking around the virtual world
  • Most virtual reality devices offer built-in headphones or audio emitters that help overwhelm another sense.
A mock-up image of the “Screen Door” effect using the video-game Team Fortress 2 as an example. Image source: Reddit


There are a number of virtual reality options available on the market at the time of writing. The biggest ones available at the moment are the Oculus Rift, HTC Vive, Google Daydream and Sony’s Playstation VR. Each one of these options offers different control methods:

  • HTC Vive uses motion controllers that also allow you to move your hands in 3D space.
  • Google Daydream is more limited than the previous two options and only features two interaction methods; a controller that allows 2D movement (translation but not rotation) and head-tracked context-sensitive actions (only applicable to specific applications).
  • Playstation VR allows for two different methods of control; either a gamepad or one of their Playstation Move wands that use a camera to track 3D movement (limited to a 180° angle from the tracking camera).
A brief overview of the many control methods available for the current popular virtual reality options.
  • Gamepad — Using a gamepad in virtual reality means that you can control your avatar or character in virtual reality using a similar control scheme to conventional video-games. The upside of this is that if you play video-games (either casually or more intensely), than this method will feel more familiar to you. In terms of immersion, this appears to be the weakest of all control methods for virtual reality currently.
  • Touch Controllers / 3D Wands — The quality and depth of movement from the Touch Controllers / 3D wands differ greatly from service to service. By design, working with 2D controllers, such as the controllers featured with the Google Daydream are extremely limited and offer functions that are not too dissimilar to using a computer mouse to interact with the world around you. Wands that offer 6 degrees of movement (6 DOF) offer greater immersion, able to replicate exact wrist positions as well as rotational data in 3D space.
Degrees of freedom has a massive impact on virtual reality experiences. In emulating hand movements, rotation is a key component. Imagine only being able to move your wrist without turning or any sort of rotation whatsoever. Image source: RoadToVR / Wikipedia
An example of a person dancing in the video-game “VR Chat”. Notice the parts where desyncronization of his real life actions to his in-game avatar occur. You can spot these when the arms and legs of the avatar unnaturally deform. Source: Yske (YouTube)

Limitations of Video-Games

Stepping away from Virtual Reality exclusive issues, video-games as a whole bring a number of important to note limitations — many of which can cause issues with data collected from virtual reality experiments.

On the top is a screenshot from a moral study performed in 2014 and on the bottom is a screenshot from the video-game Half-Life. Both the experiment and the video-game shared many of the same tools in their creation. Image source: HCILabUdine / Valve

Abstraction from Reality

One of the hottest debated topics in terms of video-games that has persisted over the years has been whether video-games influence the behaviour of the player, such as if violent video-games make players more violent or aggressive in nature.

Newspapers and studies into video-game violence directly conflict on idea of player detachment to their in-game characters and actions. Image source: Eurogamer / The Sun
The Grand Theft Auto series is a sandbox for players to live out their ideal life of crime. From stealing cars to selling drugs, almost anything is possible when it comes to ethically questionable choices in the world of Grant Theft Auto. Image source: DarkStation / TakeTwo
Jack Thompson famously debated the psychological effects of playing Grand Theft Auto. The BBC dramatized the debate in the docudrama “The Gamechangers” starring Daniel Radcliffe as Sam Houser, co-founder and president of Rockstar Games (the creators of the Grand Theft Auto series). Image source: BBC
Between the player’s desires and the video-game is the interface method. Is this layer of abstraction enough to alter what a person would do in specific situations? Image source: Business Insider

Psychological Behaviors of Video-Game Players (Diminished consequences)

Branching out from the core idea of the divide between player’s in-game actions and reality is the idea of diminished consequences. People will happily shoot enemies (and sometimes even “innocent” people) in video-games without feeling a drop of guilt. This is due to diminished consequences.

The video-game industry would be a lot different than it is today if players felt good for every gun they held or every bullet they used out of that gun. Image source: Electronic Arts / Respawn Entertainment
Even people’s perception of games has the basic understanding of you will either win or lose. Image source: Bandai Namco
These days it is impossible to gauge who has or hasn’t played video-games or even understanding their subconscious understanding of video-games as a whole. Image source: The Daily Mail / Newsteam


You could argue that the points raised are invalidated with more recent studies that tend to use 3D videos instead of video-games or gaming engines within the studies. This, I feel, only further cements the points raised.


A collection of game development related write-ups, break downs and a whole truckload of games industry analysis. There’s even tutorials and guides to help take your project to the next level!


Written by


Hello! Kitatus Here.



A collection of game development related write-ups, break downs and a whole truckload of games industry analysis. There’s even tutorials and guides to help take your project to the next level!