The relationship between tap pressure and human emotion
Triple operates in the field of human computer interaction: We make digital products that come to life. Humanization of design and experience is important to us and we aim to incorporate it in everything we do. Today, I will tell you the story of my master thesis: An in-depth research of the correlation between tap pressure on a touch screen and emotion.
Why research the relationship between tap pressure and emotion?
Good question! In 1997, Rosalind Picard introduces the term affective computing, which is a special subsection of human computer interaction that focuses on the detection of, response to and expression of emotion by computers. She shows that emotion aware computers have the ability to be of the users’ assistance, and the capability to let computers make decisions that are influenced by the feelings of the user. The detection of emotion by smart devices has such a broad area of application: a possibility could be that a telecom provider app or website can contact customer support already in the background when it notices the user is upset about their bill. Another interesting case would be music applications promoting new and exciting songs when the user is happy, but slowed down and calm songs when the user is relaxed. Alternatively, games that have characters that can be helpful and friendly when progression is stalling, but can also suitably respond to aggression and anger, mitigating frustration or escalation. As a final example, imagine educational systems that can detect if the user is confused can provide additional examples, or when it notices boredom can provide more challenging exercises.
The current ways of measuring emotion of people by computers or smartphones often happens with sensors that are attached to the body, or sensors that use a lot of privacy sensitive data. For example, brain activity (EEG) is very good indicator of how someone feels, but measuring it requires several electrodes that are attached to the top of your head, often with special gels. Facial expression can also tell a lot about how someone is feeling, and the cameras and other hardware in our smartphones and computers are powerful enough that we can analyze the images and extract the emotional state of someone. There are many other ways of measuring emotion, but most of them have in common that they are invasive; they either require annoying sensors, or use privacy sensitive data like images or voice data. So, room for improvement!
That is why I decided to find another way of measuring emotion of people, without the use of attached sensors of the use of privacy sensitive data. While there are numerous ways of doing that, I settled upon using a relatively new sensor that is becoming more and more available: A pressure sensitive touch screen. My theory was that tap pressure would vary with different types of emotional states, just like micro-expressions that people show on their face with different emotional states.
How to measure emotion with tap pressure
I wanted to validate my theory: Tap pressure is connected to the emotional state of the tapper. This means that I need to know how people feel, and then measure the pressure they exert on the screen when tapping. I ended up using photos from the Geneva Affective Picture Database. These photos have been thoroughly researched for the emotional reaction they elicit from people when they see them. So now we have a way of knowing how people feel. On the other hand, we need to measure tap pressure. The measuring of tap pressure was done with the 3D touch screen that Apple incorporates in the iPhone 6s and newer. To get people to tap the screen when they saw a picture, I would let them solve a tiny puzzle after five seconds of looking at the picture. The puzzle was very simple and always required four taps. So in short: People see a picture, after five seconds they tap the screen four times, and I measure the tap pressures of those taps for that particular photo. Easy right?
51 test participants later..
… and I got all the data I needed to start analyzing the results! To get to the essence of data, I used an application called SPSS, which allows me to run complicated statistical analysis on the data. I ran a multiple linear regression test for those who want to know, but more importantly, it turned out that there is no relationship between tap pressure and emotion. What is interesting about that is that literature definitely seems to expect a relationship between the two. So why does my research contradict the expectations set by literature and intuition? One explanation is that the tap pressure was measured on a scale from 0.0 to 6.67, but al measurements were between 0.2 and 0.5. The full scale of possible measurements was never used! This can mean two things: Either the differences in tap pressure are more subtle and require a more sensitive sensor, or there just is no relationship between tap pressure and emotion. Another explanation may be that every person is different in how they are holding a phone, how big their hands are, and where they tap on the screen. These differences, combined with a small bandwidth of measurements, might be the source of the contradicting result. Hopefully a fellow researcher finds a non-invasive way of measuring emotion, either by repeating my experiment with the issues fixed, or by using an entirely different approach.
In order to receive the paper you can contact me at k.blom@wearetriple.com
