SHIO~ your staircase companion
Context:
For this assignment, we want to look at the experience of climbing up and down the staircases of the Srishti N5 campus.
The way such a mundane and otherwise exhausting experience can be made a positive one would be by allowing the users of the staircase to interact with one another, or to have some kind of conversation facilitated with the help of an interface during the course of their usage of the stairs.
Experience focussed on:
Students walking up the staircases to their classes often have an unpleasant ascend to their destination, as they are worn out, exhausted or lazy to make the climb. They may even come across ‘awkward’ moments on the stairs when around other students they don’t know.
We hope to solve this by introducing a conversational agent that facilitates conversation either between the students or with the agent itself. It also distracts and occupies the users, giving a certain kind of incentive to use the stairs.
Motivation
As students, we have always felt the need to use the elevators on campus because we are tired and moreover bored of climbing around four floors on a daily basis. It is the same monotonous routine of ascending up and descending down. Since all of us have gone through this, we decided to choose this context to try and make the experience more wholesome.
Details
At the beginning of the staircases, on the ground floor of the N5 campus, there would be a small gadget that the students would make use of by swiping their identity cards. The staircase supports and connects itself with voice interface; a chatbot, named “Shio” and when the students swipe their card, the bot collects this data and addresses the people on the staircase.
For example, X and Y have swiped their cards and are using the staircase then, the bot addresses them by saying “Hey X and Y! How are you doing today?” This would be followed by the students answering the question.
To understand the context better, we have used three use cases:
1. When there is only one person using the staircase — As there will be lot of space available for the user, the chatbot can allow the user to play a game, for example piano tiles. This will let the users to accumulate points and make the experience of climbing the stairs fun.
2. When there are few people on the stairs (2–5 users) — Since there is no space available for the users to play any games conveniently, the chatbot can initiate a conversation with the users and also introduce people to each other on the staircase. This will reduce the number of awkward meetings on the stairs.
3. When there are many people on the stairs (peak hours) — The chatbot addressing each and everyone on the stairs will be a waste of time and hence instead of initiating conversations with specific people, it will address everyone and speak about common interests. For example, it will crack a joke or play music or even read the posters that are stuck on the wall so as to reduce the amount of time taken for people to stop and read the posters.
The users can also choose to ignore the chatbot if they wish as few of them may not be comfortable conversing in public.
Feedback report
The exhibition was attended by faculty, friends and master’s students. We had a wide range of opinions and compiling all that made us realise how people from almost the same stream have different opinions and perspectives. We spoke to around 20 people and most of them asked us, “What is the problem you are addressing?”. Once a few people asked us this, we realised that we had to state that our brief was not to solve a problem but to enhance an experience. Once we started stating this, the audience seemed more understanding of our topic. They could all relate to the tiresome experience of climbing up and a few also told us about how it is an extremely annoying experience but they would have never thought of it as an opportunity for a project. Few of them told us that our project is different from the rest which made us feel confident about our concept.
There were a few questions posed, such as
1. What if there are a lot of people entering the staircase at the same time? Won’t that cause a long line so as to swipe the card?
2. What if people do not want to have a conversation with the chatbot?
3. Can more than one student speak to the chatbot at once?
4. What sort of conversations will the users and chatbot have?
5. Can there be a game even when there are many people?
We also had a feedback form which mainly involved positive aspects of our project. There were very few who gave us suggestions. Few of the suggestions given were –
1. Try including more games
2. You could have more swiping machines in place to avoid a crowd
3. Try including facilitators and cleaning staff
4. The information on posters stuck on the wall can be extracted by the chatbot and can be read out to the users so that they do not have to stand and read
5. Conversations can involve aspects of classroom and class schedules
With questions such as above, we started pondering over the details which made us look at areas that we had overlooked before.
The details matter.
With all this feedback we looked at our concept note again and made appropriate changes as well as additions in order to convey our ideas and thought process better. This also allowed us to think about improving our demo video. The demo video consisted only of a brief conversation but in order to depict a better concept, we decided to reshoot the video.
With this we wrote down a few points to remember –
1. The chatbot should be able to continue a conversation started in the morning at any point of the day (reviving topics, just like any human would do).
2. The conversations should be multidirectional and open ended so that is feels human-like.
3. The chatbot must feel like a friend rather than a robot.
4. Code with respect to the context (number of people)
5. Test it with users to see the different kinds of responses so as to improve code
Another aspect that we missed out before the exhibition is the crowding near the swiping device (to enter the stairs). So in order to save time, many such devices will be installed so that many students will be able to use it at the same time.
What if many students talk at the same time?
The chatbot will answer sequentially (reply to the user who spoke first) in order to reduce confusion. It will also address the student (based on voice recognition) so that it is clear about who the response is directed towards.
Why a voice based agent?
Since voice based interaction is the most natural mode of conversation, we hope to use a voice based agents to allow the students to be engaged in a conversation while using the staircases. Any monotonous task can be made fun with the company of a friend, and our chatbot I being introduced as a friend so as to make the task fun.
Another level added to this chatbot, SHIO, would be when some users are casually walking up the stairs, they have the opportunity to play a game to pass the time that would ordinarily be wasted panting from the never-ending ascend. Stairs tiled with alternate black and white square blocks offer the users a chance to create their own tunes, by either following the black blocks, or white blocks, respectively.
Prototyping
While discussing our conversational agent, we used different methods to gain clarity over our chatbot. Few of them being — roleplay, crazy 8’s and mind map charts. When we started coding, we looked at voice as input. For this we installed win32client. We tried to branch out our conversation so as to provide a human like feel to the conversation. We tried to put down as many responses as possible so that any sort of inut could be recognized and responded to. Once we looked at voice as input, we tried to expand to voice as output. This seemed to be much more difficult than the previous challenge. We looked at various packages for speech recognition most of which were not compatible with our systems or created some sort of error. We looked at WIT, pygsr, Pyaudio, Snack lib and Tkinter etc. Snack lib and Tkinter. These libraries and packages go hand in hand. Snack is a Python library that can be used to create a User Interface. We installed the required packages, and when testing this on the python portal, we were able to import and use the Tkinter library, we tested it as well, but we were unable to get into the Snack library. There was an issue installing it on the system, as there was an error downloading on our Windows system.
We had issues with Pyaudio as well. The pre-requisites for Pyaudio are Microsoft Visuals C++. We had a few issues with this but we finally installed it. Once installed, we understood that it uses gtts(google’s text to speech). To check our code, we frequently used Jupyter which helped us rectify our mistakes.
Applications of SHIO:
- Students would prefer using stairs for the entertainment factor
- Encourages using stairs which in turn results in exercise motivation
- Acts as a database that tracks foot traffic at different times, generating data for the peak hours and mellow hours in which the staircases are used.
- The data can also be accessed by the cleaning staff, as it will help them know the timings most students will come and go. This will help them not waste their energy cleaning multiple times a day, restricting it to a minimum amount for their benefit.
- This data can also be provided to the canteen staff so they are equipped with enough eatery items for the number of students coming, to ensure that crowds don’t pile up and cause havoc for them.
- This can also allow students to sign in for the day, thereby giving them attendance when they swipe their card near the staircases when entering.
Demo video : https://youtu.be/-1QEn_SQwIc
