User testing with brain activity readings
In recent years, I have attended a lot of sessions involving users, being it as an observer, or as a facilitator. These sessions had different goals and formats; contextual inquiry, interviews, observations, focus groups and other workshops formats, and even guerrilla testing at an airport's departure gates. All of them had 1 thing in common: the interviewed users had to give verbal feedback.
I consider this a great manner of getting feedback on whatever I am designing, in various stages of a project. It also doesn't harm the brand you're working for, since it is a great way to show they actively involve users in the design process. However, I have always made one consistent observation: users have difficulties describing and pinpointing their emotions and perception. I often see a conflict between my observation and a user's description.
Mapping subconscious perception
Recently, I did a study with a neuromarketing research company based in Amsterdam, the Netherlands, called Braingineers. What makes them so special, is that they capture brain activity during a study. This reveals a lot about the perception of a brand, and identifies issues without users having to think about it. Here's a video of them explaining more (turn on auto-subtitles if your Dutch is a bit rusty)
https://www.youtube.com/watch?v=JSQnO0O80Io&feature=youtu.be
These kinds of studies are not new, and they are finding their way into design processes more and more. This was however, my first experience with a user study measuring the emotional ups and downs. In this article I share my experience with this approach, and how exactly it differs from 'traditional' research methods.
How it works
This neuro usability study's approach didn't differ much from the traditional way of testing, except for the lack of the think-aloud method, and the presence of an EEG scanner, fitted onto the participant's head. Participants are asked to imagine a scenario, and perform some tasks with a mobile prototype. While participants perform the tasks, the EEG scanner captures the neuro-signals. After the test, participants fill out a questionnaire.
For the test we conducted, 3 parameters were being measured:
- Frustration: Level of discomfort with the current challenge. High frustration result in difficulty to complete the task.
- Excitement: An awareness or feeling of physiological arousal with a positive value.
- Engagement: An alertness and the conscious direction of attention towards task- relevant stimuli.
After the test has been conducted, the results are being analysed. About two days later, the participants receive a video of their test session, with some specific questions. These questions are specified during the analysis, and aim to dig deeper into the results of the EEG scan. For example, when the EEG output shows that a participant was suffering a lot of frustration at a certain point during the test, they have the ability to reflect on that, and provide insights in what was so frustrating. In the end, we end up with a comprehensive set of qualitative results.
My thoughts
As said, this was my first study of this kind. When we were setting up this study, I was very excited to try out this new testing method. I had heard good reviews about our research partner, and in the end I was not disappointed.
The good
- You get tangible data rather than user's opinions, thoughts, and impressions.
- The hardware requirements are surprisingly low, all it takes extra is an EEG scanner.
- The setup and format of neuro-usability tests are similar to the traditional testing format.
- The results and observed behaviours can be linked with psychology theory, which does not nearly get enough attention in UX design as it deserves.
The bad
- Since whatever you test needs to be real because simulations might affect the neuro-data, or look as real as possible, this testing method requires a lot of effort, or can only be brought in very late in the design process.
- Neuro-testing is not yet legal everywhere, i.e. France forbids these testing practices for now.
- There is quite a high threshold to take in this testing approach in-house, since you need the equipment, and trained people to apply the EEG scanner, operate the software, and analyse the results.
- How does it differ from the traditional manner of testing?
My conclusions
As far as I am concerned, this test was a good pilot for the company I work for, and for myself to get acquainted with this approach. I am convinced neuro-usability testing will become more mainstream, however I am also wondering if the output is that much different from the traditional user testing approach. When I got the results, I would have liked to see a benchmark of this approach vs. the traditional approach. Also, I would like to see how this method can be applied with more abstract concepts, rather than realistic mockups, or working products.
Do you have any experience with neuro-usability or neuromarketing? Please share your thoughts and experiences in the comments!