(Re)writing our bot for advertisers
When Messenger announced it was opening up its platform to allow developers to create bots in 2016, my advertiser growth team at Facebook jumped at the opportunity to build a bot. We aim to empower every business to be successful using their Facebook Page and by running ads. And an advertiser assistant in the form of a conversational bot felt like a natural way to provide activity updates, guide people to the right advertising solutions, and help them understand the return on their investment with Facebook Ads.
So we got to work. As the team’s content strategist, I knew this would be an exciting project because when your product is a conversation, the content is the product experience. So far, our team has experimented with a bot that helps advertisers understand their ad results, quickly extend campaigns, and find relevant help content.
Through careful data analysis and user research, we’ve learned that, in general, people find the updates we send in Messenger about their ads to be useful, and that the guidance we give helps them feel more confident about advertising on Facebook. But the real work is in fine-tuning the conversation. Iterating on content has helped us engage more people and deliver on the promise of the experience, which we want to provide value, show care, and build trust.
As more people interact with our assistant, the more we learn about how to evolve the experience. Here are some key lessons:
Start simple
If you asked me to loan you $100 right now, I’d probably say no. But if you asked me to loan you $20 and paid me back in a timely manner, I’d be way more likely to keep loaning you $20 here and there, even if it added up to $100 in short order. That’s because we started this give-and-take relationship with low stakes.
The same is true of kicking off a conversational module with Facebook advertisers in Messenger. Even though it’s important to deliver value up front, sending an overwhelming amount of info or asking a question that requires a lot of consideration right away can keep people from engaging in the conversation from the beginning.
This is the lesson my team learned when we first built our bot for people who are boosting posts and creating other ads from their Facebook Page. Our first approach was to take existing notification content, which tells the advertiser a couple results from their boosted post, and send it in Messenger:
We want to let you know that your boosted post “We’re so excited…” ended. It reached 13,423 people and got 1,232 post engagements.
Then we followed up with some “quick reply” options, which show up as selections at the bottom of the conversation:
What do you want to do next?
- View more results
- Keep boosting it
- Boost another post
But not enough people were answering this question, and we had a hunch it was because we were asking for too much too soon. Why didn’t give we people a chance to see how simple — and delightful — the interaction could be by asking a question that’s easier to answer?
Your boosted post ended and we can help you review the results. Want to see them?
By asking an easy yes/no question, we could help the advertiser see just how quickly we could respond. However, simplifying this step meant removing the ad results from the first message. We discussed this as a team and ultimately decided it was worth withholding high-level results at first in order to entice them to interact with the bot. But you may have noticed that wasn’t the only change we made…
Make your intent clear
In addition to simplifying the first question in the conversation, we also took a fresh look at its tone of voice. Were we doing enough to express our intent to help?
I used to work at Starbucks as a barista. On some shifts I’d be stationed at the espresso bar instead of taking orders at the register. When there was a backlog of people waiting to order and pay, I’d move down the line of customers to take their drink orders so they’d be ready by the time they paid. I tried a couple different approaches to helping customers in this scenario. Which of these questions do you think was more clear to people?
A: “Next in line?”
B: “I can help get something started for you. Do you know what you’d like?”
Approach B helped customers understand exactly what I was there to do. By being upfront about my intent to start making their drink, they knew I wasn’t asking them to step out of line and pay me. And they perked up upon hearing the words “help” and “for you” — sure, my whole job at Starbucks was to help people, but articulating that intention in a one-on-one encounter created a better customer experience.
Back on our product team, we talked a lot about how we wanted the conversations we were building to feel helpful. After all, one of our guiding principles is to help people learn and grow. We knew that many newer advertisers weren’t reviewing the more detailed results of their boosted posts, like the age, gender, and location of people who responded. This may be because they’re busy, or because they don’t know where to look.
By sending results in Messenger, we can help people uncover this info, which can help them unlock much more value from advertising on Facebook. But as we tested the original experience with our own ads, the content felt cold and distant, and it certainly didn’t express that we wanted to help them dig in. So we took another pass:
Three times more people responded to the revised question, which meant more people were engaging in the steps of the conversation that followed.
Invest in complete experiences
In addition to helping people review the results of their ad, we wanted the assistant to give people whose campaign had ended an easy way to keep advertising. We started by sending a message that invited advertisers to extend their campaign or create another one. If they chose one of those options we’d take them to their Page to complete the action.
This experience didn’t feel great. It reminded me of those self-serve grocery checkout lines that should get you out of the store faster, without the help of an employee, but sometimes fall short of that promise. I’ll choose the self-checkout when I’m in a hurry, but if I scan a bottle of wine or accidentally move an item from the bagging area before I’ve completed my transaction, it’s game over: my screen locks up and I have to wait for a cashier, who might not be available right away, to assist me.
In our bot, we noticed some advertisers were extending their campaigns despite the friction of having to leave the Messenger conversation to do so. This was encouraging, but we wanted it to be easier. That’s when we took the time to build out an end-to-end experience where we let the advertiser choose a budget and accept the terms and conditions for Facebook Ads without having to leave the conversation.
After this change, the number of advertisers updating their budget to extend their ad campaign skyrocketed.
Check your assumptions
We recently tested a conversation that kicked off when a person had started boosting a post on their Page but abandoned the flow. We assumed that when people abandoned, they got confused by something in the interface, or they had lingering questions about who would see the post and where it would appear on Facebook. We thought if we answered those questions and put their minds at ease, we could help more people feel confident about going back to finish what they started. So we showed this content:
We thought you might want to finish boosting your post.
- Finish boosting
- Get help
If the advertiser selected “Get help,” we’d offer up answers that appear in frequently visited help articles about boosted posts. But when not many people were selecting “Get help,” we considered possible reasons why:
Maybe they felt like “Get help” was a big commitment.
Maybe they ended up in the flow by accident and never intended to boost a post in the first place.
Maybe they didn’t really need help.
One of my favorite parts of my job is hearing our customers talk about their experiences with our products. I learn so much from trips to visit customers and observing UX research studies where we ask questions like, “How do you know if your ad was effective?” or “What do you expect to happen on the next screen?” The answers to these questions often challenge our assumptions, and they help us find new ways to help people.
We rarely get the opportunity to ask these types of questions in the product, but Messenger is well suited for digging a little deeper to understand people’s mindsets.
The team took a step back and realized we didn’t even know if people who abandoned had a question that could be answered by a help article. If we didn’t know, why didn’t we just ask?
When we changed the quick reply from “Get help” to “I’m not ready,” we noticed certain segments of Page admins responding more to it. And their responses to our follow-up question “why” helped us better understand their mindset and even respond directly to the reason the person provides. This makes the experience feel more like a conversation — where listening is just as important as speaking.
“The only kind of writing is rewriting.” — Ernest Hemingway
Looking back, some of the stuff in these experiments feels obvious: Of course we want to keep things simple, express our intent to help, build end-to-end experiences, and ask questions to learn more about our customers. But taking the time to get all of those elements right — through disciplined analysis, careful testing, and rewriting with intent — will make a big difference for the people who put this tool to work.
