Probe

I feel the need to build something.

This is based on past experiences in projects as a key lesson was that it’s really good to get something built and implemented just to start exploring. It’s more based on a number of independent discussions I’ve had with people who are way more experience than I.

Each of those discussions concluded that it would be cool to build something that just sat in a meeting, gathering some key information from people, and doing something that indicated the ‘mood of the meeting’. It would just be interesting to see how people reacted to that. A similar idea to the Phillips Rationalizer (see my previous post), but for a group of people rather than an individual. And perhaps a little more ambient, so that it doesn’t actively seek to draw attention to it.

This neatly tied in with the literature that I have been reviewing.

There’s quite a bit of research looking at how to accurately identify specific emotions through a combination of biometric based measures. There were two papers that I tried to pull apart in a bit more detail.

Kolodyazhniy, Reibig, Gross, Roth & Wilhelm, 2011

This research sought to identify the emotions of sadness, fearfulness and neutral emotion through a combination of physiological measures from people.

I tried to breakdown their results as was revelant to me.

They used a number of physiological measures but then determined that a combination of five were most significant for identifying fear, sadness, and neutral emotion.

PEP (preejection period)

This is the period between contraction of the ventricle in the heart and when the valves actually open and the blood starts flowing into the aorta. The longer this timeframe is the more ‘relaxed’ the person is. It is influenced by the sympathetic nervous system and therefore is an indicator of stress or rather the lack of stress.

SRR (skin conductance fluctuations)

This measure is based on Galvanic Skin Response. It examines the number of fluctuations in skin conductance per minute.

GSR is a measure of arousal. The higher the conductance the more aroused a person is. The only issue is that it is non-specific as it does not give you an indication of what is the emotion behind that arousaland that could be a range of things like anger, nervousness, fear, etc.

pCO2 (Partial Pressure of Carbon Dioxide)

In particular, they looked at ‘end-tidal’ pCO2 — at the end of an exhale. A decrease in the pCO2 indicated an increase in respiration rate. This could be related to a number of factors, but in the context of the activity performed by the participants in the study it was related to an experience of fear or startle.

CS — contraction of the Corrugator Supercilii muscle

This is the muscle above your eye. It’s used when you frown.

ZM — contraction of the Zygomatic Major muscle

This is the muscle that sits at the top of your cheek. It’s used when you smile.

My whiteboard drawings indicate how the combinations of these measures assist with identifying the emotions.

Tognetti, Garbarino, Bonarini, & Matteucci, 2010

This research sought to find a correlation between physiological measures and player enjoyment of a game.

The researchers looked at five general measures. But within each general measure there were specific elements of that measurement.

GSR (Galvanic Skin Response)

This was the single measure with the greatest correlation with player enjoyment. The measure was the difference between a baseline measure and a measure whilst the user was playing the game.

They also measured the magnitude of the variation from the baseline and the duration of the variation from the baseline. These were important for the ‘combination’ of measures used to predict enjoyment.

BVP (Blood Volume Pulse)

The amount of blood flow in each pulse of beat of the heart. The lower the volume, the more enjoyment the person experienced (hence the negative correlation). This was the second highest correlation with player enjoyment.

RESP (Respiration)

This had the third greatest correlation with player enjoyment. There were a number of measures that made up this:

1. Inspiration time — the duration of breathing in;
2. Espiration (their spelling) time — the duration of breathing out;
3. Apnea in — the pause at the end of breathing in;
4. Apnea out — the pause at the end of breathing out; and
5. Respiration interval — the time it took to complete a respiration cycle from starting to breathe in to the end of breathing out.

The greatest correlation with player enjoyment was the apnea periods. The shorter the apnea periods the greater the level of player enjoyment.

TEMP (Finger temperature)

External temperature tested at the finger still had some correlation with player enjoyment. Player enjoyment is correlated to a drop in external temperature at the extremities.

ECG/HR (Heart rate)

The researchers found that an increase in HR was correlated with an increase in player enjoyment

Combined Measure

The researchers also looked for a combination of measures as an indicator of player enjoyment. They used a number of methods to determine this (most of which I do not understand). However, it seems that a matching of the following allows for enjoyment to be identified with a reasonably high level of certainty:

1. Blood Pulse Volume decrease from baseline
2. GSR difference from baseline
3. GSR magnitude from baseline
4. GSR duration of signal variation
5. Inspiration time decreasing
6. Expiration time decreasing
7. Apnea time decreasing

From here

There are two things from here.

The first is to investigate timing. It seems that, in an interaction or conversation, responses need to occur within 100–200ms in order for the interaction to be maintained (I need to define this a little better as it seems to be more than this). Which of these measures occur within this timeframe? If they do, can we detect them and produce an appropriate response within that 100–200ms period?

The second is to do a probe. Just create something and put it into a meeting environment to see how people react. It doesn’t have to be perfect, just a prototype.

My initial thoughts are to use GSR because it is relatively easy and is broad, as it as indicative of arousal generally rather than a specific emotion. As a probe, using a measure that is going to be reactive to a broad range of underlying emotions will be important to ensure people get an understanding of what the device does.