Concept 1: Ecological Validity

“In research, the ecological validity of a study means that the methods, materials, and setting of the study must approximate the real-world that is being examined.[1]”

Simply said, ecological validity indicates that any test or measurement should be done in context resembling as closely as the possible real-life situation in which the product is going to be used. Already today the Usability Labs — that are the exact opposite of this principle — are becoming less and less popular. But some situation simply cannot be reproduced in real-world, e.g. when they pose danger to the user, like car accidents.

The other reason why ecological validity is tricky to maximize in tests is internal validity of the test. To definitively interpret the data, the tester needs to have control over variables — and in real life, we usually have a lot of things we cannot control.

One solution would be to replicate these scenarios in Virtual Reality (VR). For example, we could build a city and import the CAD model of the car, so that the user feels as if (s)he would be driving the car model in question on the street. Then, the facilitator can trigger the events, like a pedestrian suddenly crossing the road or another driver suddenly stopping. And both the car and the driver (user) would react in this realistic situation. The car would activate its proximity sensors and display notifications. Then, the driver can evaluate the experience as if (s)he’d lived through it on the real street.

—

[1] Brewer, M. (2000). Research Design and Issues of Validity. In Reis, H., and Judd, C. (eds) Handbook of Research Methods in Social and Personality Psychology. Cambridge: Cambridge University Press.