Designing a prototype of a Chatbot “Kiyo”

Pooja Konde

Published in

Design with code

8 min readAug 19, 2018

Group Members: Jayanti Dave, Nupur Patny and Pooja Konde

Brief:

To identify a context within Srishti Institute of Design, N5 Campus and create an experience or interaction through a conversational agent.

Abstract

Students in college frequently have to publicly present their work in front of their class and teachers. Especially, students of design, have to do presentations many times a year, in not just classrooms but also in front of a jury panel. In the field of design, most of what the students make has to be used by human beings, so if they are unable to present it well in front of people then it defeats the point of the product itself. This is quite a lot to think about and can often make students nervous or conscious about themselves, which can have further negative consequences on their presentation. For this reason and in an attempt to solve it in the context of the Srishti School of Design, Bangalore, we decided to make a service that can allow people to practice their speech and get better at it.

Research shows that practicing in front of a mirror is very beneficial and a useful tool in building confidence and self-correcting one’s mistakes, so practicing presentations in front of a mirror is something that should be encouraged in the college as well. However, even if the practice is done in front of a mirror, not everyone knows the correct behavior and manner of speaking required for a formal presentation, so looking at the mirror the person may not be able to observe his/her body language or voice tone.

This is why we thought it would be very interesting if we designed a conversational bot through the mirror itself that observed student’s actions and listened to them, and gave them personalised feedback on what behaviour was right and what could be improved for delivering a better presentation.

Concept:

Kiyo is an intelligent mirror that can give students feedback on how they present their work, in the context of either a classroom or in front of a jury. The name Kiyo came from Kioo which means mirror in Swahili.

The mirror is activated by the simple phrase “Hello Kiyo”, and a conversation would ensue between the student and the mirror, where the mirror would listen and advise the student on various aspects of their presentation. This would be an enhanced feature of the already useful mirror, and would also allow students to get advice from an extremely impartial and objective source.

Kiyo greets them warmly, gives them their time to begin their presentation and even cracks a few silly jokes in order to lighten the mood if the person is too nervous to present immediately.

Kiyo offers feedback on the student’s body language, their tone and volume after presenting and keywords from their project/presentation. After presenting in front of the mirror, Kiyo gives the user three tips on how to improve their presentation. We decided to limit it to this number because if we bombard the user with too many tips then they may get overwhelmed and not be able to process any of them fully. This limited number allows them to improve themselves one step at a time instead of trying to change too much in one go. If the user wishes to get more feedback than the first three, they have the option to ask for more and will be provided with two more tips.

The user can also practice again using the feedback to check if there was improvisation in their presentation. There is also an additional option where they can ask direct questions in case they do not want to present or are in a hurry. For example- How do I begin my presentation? Kiyo would then answer their question with interesting tips and exit after the user leaves. We have tried our best to make the conversation as lighthearted and humorous as possible so that the user doesn’t feel intimidated when doing their part.

Conversation Flow:

Process (Initial experiment — Feedback — Developing prototype) :

In our initial conceptualization of the service we looked at the spaces in the college (N5 campus), with available mirrors that people can access easily and found that these basically just existed in the bathrooms and the sickbay. However after a few rounds of feedback we realized that these spaces are too public for someone to quietly practice their speeches and would actually also disturb the other people around who are just minding their own business. For this reason we have decided to set our machine up in a private space of N5 Campus.

We demonstrated some of the conversation that would take place between the student and the mirror in the demo video, but after receiving a lot of feedback on it we decided to make some changes. The primary change is that the chatbot is a lot more casual than we originally thought it would be. It doesn’t use big, fancy words or long-winding sentences or bombards the tips on the users which could make them conscious and more nervous. Our prototype can demonstrate how the final machine will work, and cannot actually determine which is the right piece of advice to give. Thus, it randomizes it’s choices to make the conversation interesting so that the person doesn’t get the same advice each time. Throughout the process we have incorporated all the feedbacks we received and focused to stimulate a real human like experience for the users.

To achieve this, our first step was to develop a conversational flow to determine the feature set of our bot and how we can best reach its goal through the flow. The script was then implemented into code using Python. By the end of the third week we developed our very first version of the chatbot that demonstrated the interaction between the user and the mirror through a chat interface. We attempted to keep the conversation as human as possible, because we were trying to create a lighthearted digital personality for ‘Kiyo’ that had a few surprises (by having variations) to delight the users (instead of making them frustrated when it fails to understand them) and make them feel happy and comfortable so that they can present their work to the best of their abilities.

Once our chatbot was up and running on python, we were able to test it with a few users to track feedbacks and plan future developments. We worked on creating and building more paths the user could take to reach the end goal (like asking a direct question on how to begin a presentation, keywords from a project that are important to remember while presenting). We also kept in mind that some users would try their best to get the bot to fail. We designed more flexibility into our bot to make it more engaging and non-linear.

The major change in the second version of ‘Kiyo’ was the shift in interaction from being just text to being both, text and voice. We were able to successfully code a talking Kiyo by importing the system library on python from the Mac os. However, the mode of interaction for the prototype remained the same, a chat interface. We have elaborated our conversational flow for a more non linear path and have also demonstrated the same through Role Playing. We extended the storyline wherein the user can practice his/her presentation by incorporating the tips from the mirror. We aim to develop on this in the future by adding more storylines to improve the overall experience.

Overall, our conversational agent ‘Kiyo’ attempts to guide students as they practice the presentation of their work, and to make them feel confident before they go in for their actual exhibition.

Challenges in Future and Possible Tangible outcome.

The tangible technology involved in this smart mirror would be real-time, automated engagement detection using non-invasive sensors which look at the body language and gestures of the student such as if they are standing straight, moving too much or making enough eye contact. The microphones and speakers would detect the tone of voice and point out the most commonly used words so that the student could make adjustments to the words they choose. The mirror could also be replaced by a screen in order to simulate an audience, incorporate group presentation and respect the unique presentation style of each individual.

Summary

Our conversational agent ‘Kiyo’ attempts to guide students as they practice the presentation of their work, and to make them feel confident and happy before they go in for their actual exhibition. Our chat bot demonstrates how this will look in the language it uses and its response to the things that the user says. We ensured that the conversation is as natural and human as possible, so that the student feels as comfortable in the setting as possible. Ultimately the objective is — as it always was — about creating meaningful, personal interactions for the users. A lot of the changes we made to this from the first draft are based on the feedback that we received from our seniors and faculty, and we hope that this is now a step closer to how we imagined it would be.