‘It Tastes Like Spring’: Beyond two senses with MulSeMedia

By Kris Henley

We were very fortunate to have our latest CoDE research seminar presented by Dr George Ghinea, a Reader in Computer Science at the School of Information Systems, Computing and Mathematics at Brunel University. Dr Ghinea is a pioneer in the field of Multiple Sensorial Media, and shared his perspective on this confluence of psychology, computer science, and media — what he called ‘the human aspect of computer science.’

Traditional multimedia applications have primarily engaged only two of the human senses — the aural and the visual — out of the five possible. With recent advances in computational technology, it is now possible to talk of applications that engage the other three senses, as well: tactile, olfactory, and gustatory. This integration leads to a paradigm shift away from the old multimedia towards the new mulsemedia.

In particular, Dr Ghinea discussed ‘Quality of Perception’, a user-centric metric that captures the enjoyment and information assimilation that make an app successful. When, for example, olfactory stimuli are incorporated into an app, the perception of quality and enjoyment is noticeably enhanced — with individual nuances of judgement and experience that point towards future areas of research for this innovative field.

Further, rather than simply replacing one sense with another, what happens when you integrate senses? Information is conveyed by the relationship between combined media as well as by independent media; how could this change or enhance meaning? There is a clear connection to synaesthesia, in which the qualities of one of the senses are transferred to/exchanged with another, and lead to descriptions such as ‘blue-coloured music’.

In a 90 minute talk Dr Ghinea touched on word-search puzzles, dining in the dark, and Proust’s iconic madeleine, engaging all of us with his enthusiasm.

Stay tuned for our next fascinating CoDE Research Seminar; we’ll let you know who’s coming!