Argumentative Chatbot

Introduction

We were assigned to create a chatbot that would discuss a current tense topic with a human. The goal of assignment was to think of different ways to create a debate between the human and the chatbot. With the bot we created, we were able to asses the users’ knowledge about the subject, have a branching debate on the topic, and leave a certain impression on the user about the topic.

Brainstorming

First thing we had to decide on was the topic we were going to discuss. We wanted to discuss something that has recently surfaced as heated debate topics. Some of the topics we came up with were presidential elections, world war three, gerrymandering, religion, and instant gratification sanctioned by technology. After talking for some time we decided on discussing gerrymandering, because it is a pretty recent topic, not that many people know about it, but it is an issue with great impact in the United States electoral system.

What is gerrymandering?

Gerrymandering is a vote distribution system so that a party with less total votes can win more districts.

People recently started discussing gerrymandering because many states are in court because they changed the borders of their districts to utilize gerrymandering, and there are many who think this should not be allowed.

While brainstorming ideas for the bot, we wanted to give him a personality. We didn’t want the user to just talk to the bot and only absorb the content the bot provides. We wanted the user to have a reaction to what our bot says. The only way to do this is to evoke some sort of emotion in the user; either through the topic, or through the bot’s personality. Our goal was informative and argumentative rather than persuasive, so we decided to have an assertive, decisive tone. We also wanted to make him aware that he is a bot. This way the user will be encouraged to defend their ideas on the issue.

Initial design

Here is a sketch of our initial flow chart. This is just a flow of messages/arguments. The bubbles are messages by the bot and the arrows are choices made by the user. In this flowchart we focused on important points we wanted to make in our discussion. We wanted to have a branching flow. Which means a decision the user makes will end up in a different flow. We created four different flows, one per person, and combined them in one master branch.

Here is an example (my team mate’s) of a single argument flow branch. We wanted the user to input enough so that it didn’t feel like only the bot was talking. However if we had to create a new conversation flow for each choice we allowed them to choose. A way to deal with this problem is to give users the “illusion of choice”. This way you can make the user feel included in the discussion, and make them think their input mattered. Of course abusing this trick can end up with a bland one way conversation, and dis-encourage the user from talking to the bot because making the decision of which button to press would be pointless if they see their input doesn’t matter. We decided to have a flow skeleton as follows: Introduction of the argument, showing which side the bot is on, asking if they agree or disagree, start debating if they disagree, trigger a different argument flow if they agree.

While designing my flow I decided on writing a script of the conversation first. The topic of my argument was the ethical issues behind gerrymandering, and using computers to redesign districts. My script starts with an introduction to the topic of ethical issues, and then the bot gives an example of an ethical issue about gerrymandering. Which leads to a question to the user about their stance on the ethical issue. However, this is just to get the user to express their stance on the issue. The bot then talks about another ethical issue about politicians and map designers using gerrymandering. Then asks a question. This is a branching question. If the user disagrees, the bot will start a debate. I had to give the bot an assertive and dominant personality. I thought that throwing facts and numbers without giving sources, attacking counter-arguments instead of backing up his argument would be a good way to get the user more defensive and want to argue with the bot. If they agree with the first question, the bot will ask them “ Don’t you also agree that computer programs can do a better job of redesigning districts since we don’t see color, race, ethnicity, religion?” This is a more extreme point to get the user to disagree with the bot so the debate starts. If they still agree with that question, the bot will compliment the user and tell them they are smarter than their peers. This end condition is to show that the choices they make matter. The debate is basically the bot attacking counter arguments and asking the user for their ideas. At this point the user can type out an answer. This is again just to get the users input to make the user feel included in the debate.

Prototype

We used FlowXO for creating the chatbot. We all created our flows and met to create a master flow which connects all the different flows. It was pretty easy for us to share our flow with each other, and trigger different flows inside of one flow. Here is a video of someone using our prototype:

Demo Day

During our demos we had six different people test our prototype. Most of the feedback came from being overwhelmed by information about gerrymandering thrown at them. This was because they did not know anything about gerrymandering. We have tackled the problem of what to do about people who didn’t know anything about gerrymandering. That is why at the start of our discussion we ask the user if they know or have heard of gerrymandering. If the user chooses “No” we had the bot show bunch of information about gerrymandering so that they were able to debate the issue with the bot. Since we already made the bot arrogant and assertive, it would be likely of a bot spitting bunch of text and expecting the user to learn it instantly.

All users have commented on the personality of the bot. We have heard adjectives like sassy, informative, angry, salty, aggressive, etc. This meant that we were able to create an emotional response to the bot. We also observed that all of our users have typed out a reasonably long answer to our text response; however, it was not very obvious they were supposed to type out an answer.

Conclusion

I believe we did a good job in evoking an emotion in the user about the bot, and getting them into an argument with the bot. At one point one of our testers started arguing with us about what the bot was saying because he was not able to actually talk to the bot. Other than reducing the amount of text we show to the user, and not being as restrictive with the options, we have gotten positive feedback from everyone.