First Impressions: How Sharing Brain Activity Could Change How We Communicate
Researchers at Carnegie Mellon University investigated how viewing different visualizations of another’s brain activity influenced people’s impressions.
Human beings are naturally social creatures, and communicating our thoughts and feelings with others is an essential part of how we form meaningful relationships. Communication can occur in many different contexts via a variety of different means — an in-person conversation with a sibling; a text message to a friend; a video call with coworkers. It can be verbal, either vocal or in writing, but much of the information conveyed in conversation is nonverbal: our gestures, facial expressions, and posture are important cues we use to understand what others are thinking and feeling in face-to-face or video communication, and emojis have become a useful way of conveying emotion in text-based conversations.
Advances in wearable technology such as smart watches and consumer-grade EEG headsets open the door for new forms of nonverbal communication using biosignals such as heart rate or brain activity. But what would communication using biosignals look like, and what would they add to the conversation? What impressions would people make about the owners of such biosignal data? Computer science researchers at Carnegie Mellon University conducted a study investigating these questions.
Using a consumer-grade Muse EEG headset, the researchers measured the brain activity of a person listening to an evocative instrumental song and created six different visualizations (shown below) designed to convey information about three different types of brain waves: delta waves, which are associated with deep sleep; alpha waves, which are associated with relaxation; and beta waves, which are associated with active thinking.
The visualizations varied in their complexity — some, such as the Graph, showed the brain activity close to its raw form; others, such as the Swirl, were more interpreted, translating the raw activity into an abstraction that was more simplified.
The researchers recruited 32 participants (18 female, 14 male) to view each visualization, along with the music, in a random order, and asked them questions about their impressions of the visualizations as well as of the person whose brain data was being shown. (Participants were not told that the same underlying data was used for each visualization.)
Their results showed that people are willing to rely on expressive biosignals to make impressions about others. Participants readily made inferences about the immediate emotional and cognitive states of another person based on their brain activity, e.g., “They were… moving between very relaxed (deep sleep, green) and focused thinking (yellow). They were not focused on the music or enjoying it very much.” However, most participants were less open to inferring stable dispositional traits about the visualization target, with most pointing out that the listener was likely “an average individual.”
The impressions people made varied widely due to the ambiguity of some of the visualizations. In particular, interpreted visualizations, which manipulated the display of the data, exaggerated the prominence of certain waves. For example, when participants viewed the Swirl visualization, the most apparent feature that they noticed was the number of lines (representing beta waves), which was more noticeable than both the speed of the lines (delta waves) and the smoothness of the lines (alpha waves). As a result, they were more likely to report that the listener was concentrating, even though beta waves were actually the least active in the recording: “It was much easier to tell when the person was concentrating vs. when they were not. Other than that, it was difficult to tell when the waves were changing.” By contrast, the Light, which only displayed the color of the most prominent wave at the time, made all three waves more equally noticeable.
Additionally, while participants felt the visualizations were informative, about a third expressed concerns about privacy, believing that more complex visualizations such as the Sliders “might feel intrusive because [they] clearly [provide] a lot of information rather than general trends in brain activity.” Visualizations that were more abstract, such as the Colors, elicited fewer concerns.
They also noted that more complicated visualizations, such as the Graph, Sliders, and Swirl could be distracting in social settings, because people would focus too much on trying to understand them rather than attending to the other person. Simpler visualizations, such as the Emoji, were viewed favorably.
Based on these results, the researchers made suggestions for designing systems that use expressive biosignals such as brain activity. First, systems need to account for the fact that users’ interpretations might not align with the actual meaning of the data that is presented, and should ideally provide some way of disambiguating the true emotional or cognitive state of the biosignals that are shown. Second, systems should be designed to preserve users’ preferred level of privacy, so they can decide how much data they would like to share and to whom (e.g., publicly vs. privately, with the complexity of the data depending on who it is shared with). Finally, the signals need to be presented in a clear yet unobtrusive manner, to reduce the cognitive processing required for interpreting them.
While communication using biosignals is by no means ubiquitous, the options for doing so are becoming more and more common. This study explored what systems that allow users to share their brain activity might look like; time will tell which of these ideas, if any, will translate to the products that become available in the future.
Citation
Liu, F., Dabbish, L., & Kaufman, G. (2017). Can biosignals be expressive? how visualizations affect impression formation from shared brain activity. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1–21.
