Can Facebook Messenger help you build a deeper relationship with your audience?
Maybe, but it’s a lot of work. Here’s what Annenberg Media tried.
Chatbots seem to be popping up everywhere. You can find them on chat apps, on your SMS platform or in apps of their own. If you’ve used Siri or Amazon’s Alexa, you’ve used a chatbot.
Many news organization use chatbots to deliver the news, but few use it to have a conversation. The options for what a user can do beyond asking for the organizations’ top stories tends to be fairly limited.
At Annenberg Media, we wanted to develop a Facebook Messenger bot that would help us better connect with our audience. In 2016, the public’s trust in the media hit an all-time low, with 32 percent of people saying they trusted mass media. That number has risen since then, but a major challenge for news outlets right now is figuring out how to regain that trust. We thought we could use Facebook Messenger to help.
As a student-run newsroom covering the USC community, Annenberg Media obviously wants to be on top of the stories affecting students. We decided that allowing our audience to pitch us stories through Facebook Messenger could strengthen the connection between our journalists and the USC community by allowing students to give us a better sense of what’s happening on campus and allowing reporters and editors to correspond with them in a private, one-on-one space.
That’s where our chatbot journey began. Here’s what we learned after months of developing the voice and testing conversations with our audience.
Planning and prep: With our mission and goals for the bot set, we started mapping out the various directions conversations between the user and the bot could take. Since we wanted the bot to both send and receive stories, we started with a welcome message with two buttons that allowed users to choose between getting top stories or pitching a story. We also made several default answers that respond to all the messages we can’t plan for. (I’ll talk more about how we used those default responses later.)
We created several drafts of different greetings and replies, and workshopped those options with a group of about three people before testing them with real Messenger users. Here’s an example of our first welcome message:
That message was edited multiple times before we ever programmed it into the bot, so that screenshot is straight from Google Docs, which is where we drafted all the messages. This is what our welcome message says now:
In editing this message, we decided that we needed to be more specific about the bot’s capabilities so the user would understand exactly what they could do. We tried to keep a conversational tone but also make it clear that the bot has certain limitations. (The last part of the message is referring to two buttons that say “top stories” and “send us a news tip” immediately under the message. Stay tuned for more about those.)
Back to our decision to emphasize our bot’s limitations: While drafting these messages, we knew we wanted the bot to be fun and engaging, but we didn’t want it to sound too smart. If it sounded smarter than it is in reality, we were worried that users would be disappointed and become disengaged if it didn’t meet their expectations.
Once we had conversations mapped out, we began programming them into Chatfuel, a free bot CMS you can link to Facebook. We rely on both Chatfuel’s AI feature and buttons to guide users through a conversation. Once our language was in Chatfuel, it was time for users to start chatting with us!
Testing: As I outlined above, our welcome message leaves users with two options: sending us a news tip or seeing a curated list of our top stories. Since the language of our message refers directly to the options, we hoped our audience would know what to do.
So far, the “top stories” button is the more popular one, and we’re seeing less engagement with the button that invites users to share a news tip. We’ve tried letting our Facebook audience of about 54,000 fans know that they can use Messenger to send us tips by posting graphics on our page. However, the people who do click/tap that option so far have made no further actions.
The lack of engagement with this feature has left us asking a few questions. Are users simply curious about the feature, and they want to see what happens when they click/tap it? Or do they have a story they want to pitch, but, when they select the option, the directions for what they’re supposed to do aren’t clear enough? The language above may be clear to journalists, but does it make enough sense to resonate with non-journalists?
Since these conversations weren’t going anywhere, we decided to adjust the message the bot sends when the user selects “send us a news tip.”
By adding the second message, we hoped that we could make it slightly clearer to users what we’re asking of them. This change is relatively new, and no one has chosen the pitching option since we made the update. So it’s too soon to tell if this change is going to help.
Building the AI: Like any good bot CMS, Chatfuel has a section where you can program responses to keywords, such as “what’s up,” “usc,” “news,” etc. We can’t assume everyone will simply stick to the buttons, so the AI allows the bot to respond to miscellaneous messages.
We quickly learned based on the way people were behaving with the bot that we’d need not just a lot of keyword triggers but also a few default message responses to alleviate the affect of monotony. Here’s an example of where that default response came in handy:
This is obviously a silly example, since the user was clearly trolling the bot. However, it is a great demonstration of how it’s impossible to plan for every prompt a user may send our way. We have three different versions of our default message. The variation in these responses helps to make our bot seem a little smarter, or at least not extremely repetitive.
Takeaways: For me, the biggest takeaway from this experiment is that the initial process of mapping out various conversations between the bot and a user takes a lot of time since there are essentially endless possibilities. Planning these conversations involves trial and error, and it also requires that you think about the flow of simple conversations more than you probably ever have in your life. If you’re building a bot, you should be prepared to make changes and realize where your initial conversation predictions weren’t spot on as you gain more information about how real users engage with your bot.
I also learned that two-way conversation is harder than is sounds. We’re really going to have to encourage the audience to use the pitching feature if we want it to work as we originally planned, and there needs to be a clear reason why they’re coming to us to share the stories they want to see covered.
Testing the bot also made me realize that we need to watch out for more than just story ideas. We need to show our users that, when they send us messages, we care about what they’re saying and they aren’t wasting their time by reaching out to us, even if the message is outside of the sphere of what we do. For example, someone reached out to us for information on USC’s Ph.D. programs, which is obviously outside of the scope of our work. But we were still able to help.
This response was a very small gesture and took minimal effort to send. But small actions like this help to establish our bot as a place for users to go to receive information, and, hopefully, one day send us information in return.
Next steps: If we want to convince users that they should want to come to us to share the stories they care about, we need to work harder to establish a relationship with them. We decided we needed to foster a habit with our users before we expect them to feel comfortable chatting with the bot and sending it stories.
One aspect of this is pushing a weekly digest of stories to users to help them stay in the know on USC news and allow them to get more familiar with our coverage. Our hope is that, as they receive regular updates from our bot, they’ll become comfortable enough with it to send it their story ideas.
Ultimately, our goal is to build trust between Annenberg Media and our audience. Using a bot to accomplish this is an ongoing process. But, after months of testing, I think it has the potential to be a useful tool that provides users a private platform where they can engage with our work and share their own experiences.
