Design for Tension

Team: Cole Granof, Meijie Wang, Cem Alemdar and Brandon Navarro.

Building a chatbot to carry out an argument was a unique challenge for all of us. With something as dynamic as an argument, considering all of the possible branches was the first step in designing a convincing dialogue.

I suggested the topic of gerrymandering, which describes how district boundaries are manipulated to unfairly represent a party. Some of the most egregious examples of gerrymandering are happening in North Carolina.

After we settled on this topic, all of us did some research to see what unique arguments we could strike with our chatbot. One interesting proposal for designing fair districts was letting a computer algorithm draw the borders. These algorithms maximize certain attributes such as compactness and equality based on heuristics based on demographic information the model is initially provided with.

Focusing on the computer-aided solution to solve gerrymandering lead us to develop the personality for our bot. Since the user is essentially arguing with a computer, and the bot believes that computer will do a better job with designing districts than humans, we were able to leverage this to make the dialogue funny and engaging. The fact that the bot is making an argument for its own kind, claiming that humans are less objective and that “robots” can do a better job at designing districts, a “pompous” attitude for our bot was a natural fit. It was important to land on this personality before each team member split of to write a piece of the overall flow, and ultimately lead to a more cohesive experience when chatting with the bot.

Flow charts we made to guide our design of the logic

Initial User Testing

Using the structure of the argument based on the flow charts, I did some initial user testing with my brother, using iMessage to communicate, emulating responses that the bot might have. My goal was to see how the user answered the questions of a robot, such as the kind of wording, and how long the responses were. Since our final robot would mostly have to present “choices” instead of parsing freeform answers, this information helped guide how best to write those choices.

Transcription of me pretending to be a bot over iMessage

We decided to use the tool Flow XO for developing our final prototype since this would allow us to write flows separately and integrate them at the end. This required analyzing our main flowchart and deciding how to separate the argument into four main pieces. We eventually decided on this:

  1. Background information on gerrymandering
  2. Arguing the importance of gerrymandering
  3. Advocating for computer algorithms for districting
  4. Discussing the ethics of the previous solutions

The flow I worked on personally was the second item in the list above. The job of my flow was to make the case that gerrymandering was an important issue in today’s democracy. The argument would be supported with diagrams and maps as well as text with the bot explaining the significance of the images. Disagreeing with the bot would lead the bot to bring up more compelling evidence.

Final User Testing

The final prototype used for our day of in-class user testing can be seen here. Alternatively, if you are part of the class Slack channel, direct message @gerry and he will start a conversation with you. (You might have to follow the link instead of watching the embedded version if the embedded version cuts off some of the text for you.)

We got quite a few of our peers to talk with Gerry. We asked testers to think aloud and talk about how they felt. During testing, we also took textual notes as the user tested the bot.

Pictures of different students testing the bot

The people who got to argue with Gerry all felt that he had a lot of personality. It was great to see that the attitude we chose to give our bot came through during testing. One of the comments was that Gerry was “real sassy” which we consider a positive comment. Whether Gerry’s “sassiness” bordered on making the user feel “bullied” likely varies from person to person, but we realized that most testers did not take the bot’s comments too personally, and usually opted to play devil’s advocate to see what kind of retorts the bot would come up with.

Some of the other difficulties we encountered during user testing were simply a product of exporting our bot to Slack. Unbeknownst to us, Slack truncated the shortcut buttons, which the test console in Flow XO did not do. This was not too significant for our user testing since we were able to tell the testers what the full shortcuts should be saying, but this is a consideration we would take into account for a future implementation.

One of the most salient criticisms of our bot was that it required too much reading. This was a difficult thing to balance because we wanted our bot to be convincing and gerrymandering (combined with algorithmic solutions) is a complex topic that warrants a lot of explanation and justification. At the same time, the user testing revealed the importance of boiling down what the robot was saying and perhaps cutting out extraneous details.

Conclusion

Of course, you can’t have the same kind of spirited debate with a chatbot that you would with a human, since your options are much more limited in the former compared to the latter. At the same time, this doesn’t mean that chatbots are not a useful and entertaining way to learn about a challenging topic under the pretense of a “pseudo-argument” with a digital opponent. Interactivity is fun and engaging, and users are often willing to playfully anthropomorphize the bot when it is seemingly responding to your input in intelligent and occasionally witty ways. More than just adding an extra layer of fun. Furthermore, a chatbot has the unique ability to make your experience more personal, filtering out redundant information for users informed about a topic, and offering natural avenues to voluntarily explore a topic further in a dialogue-like flow. A thoughtfully designed chatbot with a strong personality has the potential to be a novel tool to engage with new ideas and explore the perspectives of others.

References

The following links are to articles used to research gerrymandering and gather pictures for the bot to display.

https://www.policymap.com/2017/08/a-deeper-look-at-gerrymandering/

https://www.npr.org/2016/03/10/469548881/north-carolinas-congressional-primaries-are-a-mess-because-of-these-maps

http://autoredistrict.org/

https://www.washingtonpost.com/news/wonk/wp/2014/06/03/this-computer-programmer-solved-gerrymandering-in-his-spare-time/?utm_term=.13271f9cd41a