COVID19 conundrum: To Touch or Not To Touch?
A contactless and immersive haptics touch interaction
The end of Touch Era?
The new norm of touch free social interaction since the COVID-19 outbreak raises questions around the survival of public touch screen kiosks[6]. At present, coming into contact with a touch screen has become inevitable in people’s daily lives, as hundreds of touch screens are used for self-service purposes in public areas such as in supermarkets, airports, hospitals and restaurants. Studies show that bacteria originating from people’s intestines, gut, nose, mouth, throat, and feces[1] can be found in many public displays, such as on airport check-in screens[2] and on touch screens in restaurants[3], hospitals and grocery stores[4]. There is widespread concern among consumers about the health risk of public touchscreens since the COVID-19 outbreak. A survey published by Ultraleap in the UK showed that 80% of the people now believe touch screens to be unhygienic and around 50% would not likely use them for interaction going forward. The global interactive kiosk market was valued at $23.11 billion in 2018 with a significant growth rate projection. Increased health concerns since the COVID-19 outbreak create significant commercial risk for retailers and public transport agencies who rely on these kiosks for customer interaction. Frequently wiping the surface of touch screens throughout the day is a common way to stop the spread of bacteria and viruses, but it is not effective enough as 95% of people aren’t washing their hands correctly[5]. This leads to a huge hygiene concern in the current COVID-19 scenario, making them a liability for retailers and public transport agencies that rely heavily on these interfaces for customer interaction [7].
New ways of interacting with public kiosks are needed.
Is contactless touch possible?
As it is difficult to guarantee the surfaces you touch in the public spaces are properly cleaned, the best option is touching nothing. However, it is impossible to avoid touch input if you want to communicate with software in computers in current digital environments. Our team started to think how we can complete the “Touch” operation without physically touching anything or wearing any devices? We found the answer in hand tracking interaction.
What is hand tracking interaction?
Hand tracking is an emerging technique, which has been widely used in Mixed Reality to enable intuitive interaction. It allows users to interact with virtual items in the interfaces. For examples, users can touch a virtual button, scroll a bar or input messages without touching or wearing any devices, but only via doing some hand gestures in the air.
Beside enabling contactless touch interaction, we want to stimulate the sense of touch to improve the touching experience as well. We utilise a haptic sensor to realise this purpose. The haptic feedback will be activated at the same time when operating, which uses ultrasound to create tactile sensations in mid-air, replicating the certainty of choice that users feel when using physical touch. Thus there’s no need for controllers or wearables, making gesture recognition an ideal input platform with such Mid-air interaction.
A contactless and immersive haptics touch interface
We created a contactless touch application based on the aforementioned the mid-air interactive technique to reduce the spread of COVID-19 and other communicable diseases on public kiosks.
How does it work?
The application is deployed as a WEB XR interface on a system integrated with Haptic Touch development kit hardware platform, which consists of a haptic sensor and a hand tracking sensor. The tactile sensor in the kit is made up of an array of 256 transducers. Each element can emit ultrasonic vibration with adjustable magnitude and frequency. These ultrasonic speakers are used to generate focal-points of ultrasound in mid-air strong enough to create haptic sensations. The kit connects to a PC to work in our demo and the Leap Motion Controller included in the kit is used to handle the hand tracking aspect of the game.
Key elements for good user experience
Lifelike visualisation
The virtual world of the Customer Feedback Kiosk is set up in Unity based on an New Zealand airport. 3D modelling of four coloured buttons representing the emoticons (happy, ok, bad, terrible) were designed and an interactive sequence was created for the click of an emoticon. On recognising the tap gesture over a specific button, the corresponding emoticon is chosen as an answer.
Intuitive interaction and realistic haptics feedback
The interaction between the user and the interface is controlled by gesture recognition and haptic feedback. To create a seamless connection between the virtual and real world, the interaction space has been set up in careful consideration of the Leap Motion controller’s specific tracking area. We chose four choice answers as the interactable space was limited. The interface design uses a transparent round button where the options are displayed to mark out the hand tracking area for optimising this hand interaction. A transparent digital hand is designed to mimic the user’s hand movements on the screen, closing the gap between the two worlds. The transducer array was also split into four parts to align with the interactable space. The array will be activated only when the leap motion controller recognises the hand gesture being made, such as press and release of the button. On experimenting with the haptic feedback points, a point vibrating vertically was found to be ideal to simulate a hovering button, and we used a group of points, rotating as they separate, to simulate a click as the user presses down. A feedback loop was created to make the selection happen virtually when the button is pressed and released.
To Touch or Not To Touch?
To investigate the usability, satisfaction and practicality of Mid-air haptics interaction, we conducted an experiment to compare our prototype with two other hand interaction techniques: Touch and Mid-air gesture touch. Both mid-air interactions caused less hygiene concern and gave participants more confidence to use a public kiosk, compared to touch screens. Compared with gesture interaction only, the haptic feedback gave participants a sense of touch and made the experience more enjoyable. Overall, 60% of the participants suggested that the touch experience is the best, while 40% of them commented that the Mid-air haptic touch experience is the best.
The results of questionnaires and interviews pointed out that before we practically deploy the mid-air haptic touch kiosk in a public area, we still need to improve the design of details of the interface, such as the distance between buttons or the size of the virtual hands to reduce error rate, and design proper training instructions to increase learnability of the interfaces. Nevertheless, the future of this application is promising as participants show positive attitude towards this prototype and look forward to the large-scale deployment of it in the future to keep everyone safe in the post COVID-19 world.
Thanks for reading.
References
[1] London Metropolitan University. (2018, December 4). Tests find traces of faeces on popular restaurant touchscreens. Retrieved June 9, 2020, from https://www.londonmet.ac.uk/news/articles/tests-find-traces-of-faeces-on-popular-restaurant-touchscreens/
[2] Insurance Quotes. (2018, January 22). Germs at the Airport. Retrieved June 9, 2020, from https://www.insurancequotes.com/health/germs-at-the-airport
[3] Smith, A. (2018, November 28). Poo found on every McDonald’s touchscreen tested Retrieved June 9, 2020, from https://metro.co.uk/2018/11/28/poo-found-on-every-mcdonalds-touchscreen-tested-8178486/
[4] Kaye, K. S., & Haas, J. (2018). Training infection prevention leaders for the present and future: United in patient safety. Infection Control & Hospital Epidemiology, 39(12), 1398–1399.
[5] Moser, C., & Tscheligi, M. (2015, June). Physics-based gaming: exploring touch vs. mid-air gesture input. In Proceedings of the 14th International Conference on Interaction Design and Children (pp. 291–294).
[6]Schönauer, C., Mossel, A., Zaiți, I. A., & Vatavu, R. D. (2015, September). Touch, movement and vibration: user perception of vibrotactile feedback for touch and mid-air gestures. In IFIP Conference on Human-Computer Interaction (pp. 165–172). Springer, Cham.
[7]Leap Motion. http://www.leapmotion.com/, 2013.
