Design for Tension
Lacey Gavala, Sean McDonough, Lukas Munoz, Caroline Whitman
Introducing Mr. Boto: A Chatbot Trying to Understand the Death Penalty
Mr. Chato Boto is a chatbot designed using Talkbot.io to help college students reflect on their beliefs surrounding the death penalty. An unbiased, overwhelmed bot, he seeks to better understand the death penalty and its consequences.
He does so by inquiring for the user’s insights through a series of questions surrounding four categories-cost, morality, deterrence of other crimes, and uncertainty of sentence. He asks for the user’s opinion on one of the four categories, presents them with a counter argument he heard elsewhere, and then asks for the user’s opinion on that counter argument. Through this process, the user is challenged to reflect and articulate the reasoning behind their beliefs, while being presented counter arguments from an unbiased, unargumentative source.
The Brainstorming Process
A variety of controversial topics were explored for Mr. Boto to talk to users about. Some of the topics explored included:
- Gun Rights
- Legalization of Marijuana
- Death Penalty
- Climate Change
- Internet Privacy
Ultimately, we decided on the death penalty conversation due to the abundance of arguments and literature surrounding both sides of the debate. We were concerned about tackling an issue that had too much information on one side of the argument and lacked information on the other, as this would bias Mr. Boto.
Mr. Boto first asks the user’s current stance on the death penalty — “yes”, “no”, or “I’m not sure”. After this question, he begins asking questions and presenting information for counter arguments. A different team member took each of these three routes and constructed flow diagrams for how the conversation would progress.
On the “yes” side, the topics for counter arguments explored included:
- Uncertainty of Conviction
- Conviction of Mentally Ill
- Location/Gender/Race Bias
On the “no” side, the topics for counter arguments explored included:
- Closure for Victim’s Family
- Crime Deterrent
- Prison Overpopulation
We soon realized that because Mr. Boto needed to present counter arguments for all of the topics and adjust conversation accordingly, we needed to standardize the topics Mr. Boto understood and discussed. This was a matter of defining scope, as discussed in Yogesh Moorjani’s “How To Design a Robust Chatbot Interaction”. We needed to decide what Mr. Boto would and would not cover with his discussions. The final four topics included:
- Deterrence of other crimes
- Uncertainty of Conviction
It was decided that the conversation would flow as follows:
- Introductions, otherwise known as onboarding so that the user clearly understands the purpose of the bot
- Ask the user whether they do, do not, or are unsure if they believe in the death penalty
- Ask them why they feel that way
- Present a counter argument, or clarify if the user did not mention one of the four categories Mr. Boto understands
- Ask the user how they feel about the counter argument
- Loop to ask about another one of the four counter arguments
User testing was done by having a group member act as a chatbot. There were two types of user testing, verbal and textual. Verbal user testing consisted of the fake chatbot, trying to match the flow charts as best as possible, speaking with the test user. This was performed because of the “Play Assistant” portion of “How To Design a Robust Chatbot Interaction”. As the article mentions, what better way to test a conversational interface than through a conversation?
For textual user testing, we used texting apps to message people the same way that our chatbot would message someone via Facebook messenger.
Notes were taken during both of these approaches so that insights could be used for further implementation. These included:
- I don’t know/I’m not sure is a commonly used answer
- There needed to be more depth on “yes” branch
- A user felt that there is no other punishment to fit the crime — this is one branch explored previously, but was not implemented in the final iteration of the bot
We noticed that user testing via messaging was easier to get feedback from, because we could look at the conversation multiple times to see where things weren’t working as we expected.
We also noticed that it was hard to guide the user down a path. Because of this, we decided to use the button feature from Talkbot.io. This was a matter of defining key user inputs, an important step in creating a chatbot interaction.
Building the Bot
Flow XO → Talkbot.io
In order to implement Mr. Boto, we made the choice to use Facebook Messenger, and so we looked at chatbot platforms with that capability. We initially began work in Flow XO, but soon dropped it in favor of Talkbot.io. Because the previously created design flows were somewhat non-linear, Talkbot’s 2d planar interface was much easier to use than Flow XO’s more linear columns.
With the more intuitive interface, we were able to copy most of our design flows as they were into Talkbot, with only branching and language recognition posing the occasional challenge.
We implemented language recognition by giving Talkbot lists of keywords to search for, and used the results to determine the direction of the conversation. Talkbot’s very visual interface also worked well for edits and fixes of conversation flow, language recognition, and lines of conversation.
Segment of Conversation
The Final Product
The following demo video provides a look into a conversation with Chato Boto. Mr. Boto introduces himself to the user, and asks if they agree with the death penalty. The user responds with their opinion, and he responds accordingly. The demo video shows three conversation threads with three different users: one in support, one against, and one who wasn’t sure.
Mr. Boto obeys a number of principles set forth by James Giangola’s “Conversation Design: Speaking the Same Language” on creating effective conversational interfaces. Some of the principles that Mr. Boto obeys include:
- Use of a persona
- Moving the conversation forward
- Brief and relevant messages
Mr. Boto exemplifies the persona of an unbiased, confused individual seeking to make sense of the various sources of information he uncovered in regards to the death penalty. No matter the user’s stance, he always asks if they could explain and help him. A persona is a necessary attribute of a bot, as it heavily influences the user’s feelings toward it.
Advancing the conversation
Mr. Boto makes sure to move the conversation forward as he presents counter arguments to the user’s initial argument, and then asks for the user’s opinion again. Because new information is displayed via the counter arguments, the conversation has a progression that corresponds to Mr. Boto’s discoveries. Moving the conversation forward is an important aspect of chatbots as if this is not done, the rule of conversation known as the Maxim of Quantity is broken. In this case, the bot will seem uncooperative and leave the user expecting more.
Short, sweet, & relevant
Mr. Boto’s messages are also short and relevant, as they ask for a user’s thoughts and then present only one sentence representative of a counter argument. Because he does not send a paragraph about the counter argument, the user is more likely to read and process what he says. Keeping the user engaged and active is necessary for the success of a chatbot, and if there is too much cognitive load placed on the user via a long, irrelevant paragraph, this will not happen.
After receiving feedback from classmates on demo day, Mr. Boto’s strengths and weaknesses were examined.
Strengths of Mr. Boto included:
- Topic choice
- Varied responses
- Streamlined conversation
- Use of facts and statistics
- Clear wording
- Questioning, not provoking the user
Weaknesses of Mr. Boto included:
- Buttons made for an impersonal feel
- Paths were too limited and guided
- Ability to become stuck in one conversation loop
- Use of “sorry, I didn’t understand that” after some users’ open-ended responses
We would implement a number of changes if we were to create a second iteration of Mr. Boto. Most importantly, we would refine our looping mechanism a bit more, as some users complained of becoming stuck in one loop repeatedly.
We would also leverage context not only for the looping mechanism, but for Mr. Boto overall, as sometimes the same topics would be repeated with the same user. It would be ideal if Mr. Boto could remember what he has and has not yet discussed with one user so that conversations could be more productive.
We would also minimize the use of buttons, and try to maximize the use of keywords for conversational direction.
Overall, we created a chatbot that challenges college students to think reflectively on their stance, or lack thereof, on the death penalty. Through the use of an unbiased persona, users feel welcome to engage in a stimulating conversation to aid in Mr. Boto’s understanding of the argument. The focus on four primary debate areas — cost, morality, deterrence of other crimes, and uncertainty of conviction — defines Mr. Boto’s scope and guides the conversation through relevant, brief, and productive facts and questions.