Natural User Interface for Differently Abled
For years, human beings have been fascinated with the idea of removing the barriers between our bodies and technology and now there is a growing trend towards embodied interaction where people interact mentally and physically with technology. It can involve using your whole body, and not just your hands and eyes contributing to the design outcome.
Embodied interaction is a position in cogitative science and the philosophy of the mind emphasizing the role that the human body plays in shaping the mind.
- As humans we reject change because it takes up a lot of energy in the brain which can be utilized in more urgent and immediate tasks, and
- Even though we find technology rewarding, we are not prepared to abandon behaviours that we have mastered over the years in the hope of optimization and productivity.
Natural User Interface (NUI) has the ability to interact with the virtual world in a way that’s intuitive without having to learn how to use a complicated interface or abstracted set of controls. When we think about user interfaces that are natural and easy to use, we think of user interfaces where the interaction is direct and consistent with our ‘natural’ behaviour. The advantage of NUIs is that the user interaction feels fun, easy and natural because the user can use a broader range of basic skills compared to more traditional graphical user interface interaction.
Having to design something for physically disabled people, focusing on how they engage with their day through their body in the social context, I picked up a character, Abdul enacted by Mazhar Khan from a Bollywood movie Shaan which was released in the year of 1980.
Abdul’s role in the movie is, he is a city dweller who has undergone double amputee
(a person who has lost all or part of an arm, hand, leg, etc.,). working as a secret informer for Vijaykumar (Amitabh Bachchan).
Watching the movie, I was able to understand Abdul’s character as to how he engages with his day through his body in the social context. Based on that I created a persona of him.
Since the exact problem may not be obvious, I had to state all possible problems faced by him in his day-to-day life because of his disability in broad terms.
- Dependent on the wheel plates, he would have to put extra effort for his daily commutation.
- Using the wheel plates, he cannot climb stairs or commute in bad roads.
- He used his hands as substitute for legs which requires a lot of muscular energy to commute.
- Dependent on others and unable to care for himself.
- Difficulty to experience normal things like sports, travelling and so on.
- Might find it difficult to multi-task.
This has been an advantage for him, being unnoticed by the world which is helpful to be a secret informer.
Keeping this in mind, I had to come up with an idea which could remove the barrier between the body, technology and address the problems which was mentioned above.
Idea
Integrating Mixed Reality (MR) experience with the motorised power wheelchair, where the user will be able to control the chair with the gestures and make use of MR glasses to record, document and share the information using gestures and holographic display.
The motorized powered wheelchair works automatically and the user doesn’t have to push the wheels. With the help of joystick and other options, user is able to control the chair and the wheels.
Abdul’s height is a great asset for him to stay unnoticed and work as a spy. Considering that, the wheelchair could also have the leg elevations, seat elevations and tilting capabilities which can help Abdul to easily climb on the chair or get down. It doesn’t have to be a wheel chair, it could also be a wheel plate where the user is able to control it using MR glasses.
The MR glasses which have two cameras on the either side, give a binocular vision and the ability to track the three dimensions of an object. This enables the glasses to capture the gestures of the user. Digital objects and elements can be added to the real environment where the digital and physical objects could co-exist.
The MR glasses can capture the user’s hand movements, gesture and understands the user’s command. They send the information to the wheelchair and the wheelchair acts accordingly.
For example,
- When the user raises his hand and points straight ahead the chair moves forward.
- When he points in the right direction, the wheels turn towards right.
- When the user makes a fist, the wheelchair applies brake.
Video camera is a primary requirement for informers and spy. In case of Abdul it is difficult for him to carry it wherever he goes but with MR he will be able to interact with the real environment and capture information.
For example, when Abdul is viewing the real world through MR glasses and wants to record or capture any situation he could make a gesture as shown in the picture. This would capture the moment and save it.
As an informer, he should be able to easily share information and data.
The MR glass adds digital objects and elements to the real environment, where the digital and physical objects co-exist. For example, when Abdul wide opens his hands the MR glasses understand the gesture and opens up the holographic interface, where he can see or share his information to the agency he is working for.
Considering the unique characteristic of Abdul, I came up with the solution which could address the following problems currently faced by him,
- As the wheel chair works based on the gestures he doesn’t have to spend a lot of energy by pushing or pulling the wheels, commutation would become an easy task for him.
- Climbing uphill and downhill, travelling in bad roads is not a problem at all with the motorised power wheelchair.
- Recording important situations, collecting evidences become much easier with MR glasses.
- Reading and sharing information would not be a burden anymore.
- With help of MR glasses, he will be able to do his job efficiently.
People like Abdul are all around us. May not be a spy but differently abled? Yes. I just hope ideas like the one I have presented above helps them lead a life as normal as ours.
