Virtual Reality Design: Future Trends
In my previous post, I talked a little about what I would consider when choosing between two different platforms: High End Mobile VR or High Immersion VR. However, given the current rate of R&D in VR, it’s worth considering the emerging technologies and their unique advantages.
Using the same example project about teaching the I2C protocol, let’s talk about how these technologies can impact project experience.
Instead of VR, how could augmented reality (AR) create a better learning experience for students?
In my opinion, given the availability of appropriate hardware and extensively tested implementation of the feature, AR would create a significantly better learning experience for students. For example, instead of a completely virtual environment, wouldn’t it be more useful if the app would instead superimpose relevant content onto a real, physical circuit a student is working on? In essence, the app would guide, explain, and troubleshoot the material with the student almost like Jarvis in Ironman.
However, there are significant hurdles in AR that don’t exist in VR, such as the need for high precision computer vision and higher quality HMDs. We can already see great progress on this front, as shown in Google’s camera translate feature, but probably not quite ready for comfortable, no-simulator-sickness user experiences.
Instead of tracking head rotation, how could eye tracking contribute to a better experience?
Unequivocally, eye tracking would significantly contribute to a better experience because of the addition of critical and insightful user information. Tracking eye movement and responses has many useful advantages: significant optimizations in rendering, better handling of focus cues, and real time understanding of users.
Rendering optimizations: The human eye is only focused on a fraction of a screen. The idea is to minimize computation for areas of the screen not focused by the eye, which is incredibly appealing for mobile platforms.
Focus cues: In layman’s terms, these cues are signals for our eyes to interpret 3D perception and depth estimation. Without proper handling, users experience fatigue and discomfort, which are huge red flags on the Simulator Sickness Questionnaire (SSQ). It’s an ever present problem in VR, but the field is moving fast on a solution.
Real time user intent interpretation: The human eye provides meaningful, but subtle psychological and physiological signals as demonstrated in their widespread application in much scientific research. For instance, through pupillometry, which is the measurement of pupil size and reactivity, Princeton University psychologist Daniel Kahneman showed that pupil size increases proportionally with task difficulty.
Wouldn’t it be amazing if the app knew exactly when a user is struggling mentally and offered guidance at that exact point? If this were exclusive to High Immersion VR, I’d switch platforms in a heartbeat.
How could more realistic haptics improve experience?
Background Info: Microsoft has done some cool research on creating “realistic haptic feedback” mechanisms.
Typically, physically feeling a virtual object can increase realism and user motor control and accuracy. For an educational application, fine motor control and accuracy could benefit the student, but this need could be eliminated using simple point and click actions. In essence, a student doesn’t need to feel a working circuit, but rather understand how a circuit works.
How important is graphical fidelity for the experience?
Higher fidelity is always a welcome feature, but in context of the app it’s not very important. Models can look more detailed, and various particle systems could be more realistic, but I think aesthetics at that level aren’t critical to the learning experience.
The I2C protocol is a fairly common communication protocol with many applications in engineering. With that said, it is critical that my target users learn this within the next two years. They would undoubtedly encounter the protocol at least once during their college career and many more times professionally. Accurate understanding is necessary for proper application.
Overall, as these future technologies become more refined and readily available, they present more opportunities for significantly boosting the efficacy, comfort, and experience of VR apps. When beginning the design process, it’s definitely worth asking whether there is value in adding these future technologies, and if so maintaining flexibility for implementation upon availability.