It sends them into detective mode, investigating the scene from a distance.
The Storyteller’s Guide to the Virtual Reality Audience
VR/AR Media Experiments

Distrust of your surroundings in VR is the equivalent of breaking the fourth wall. It’s a narrative convention that has to be used with intention, otherwise it’s unsettling to the viewer.