Facebook’s Messenger Platform: Checking in on Poncho

I launched the pre-orders for my book “Designing Conversational Interfaces” this week. A couple days later I saw this tweet by Ben Lang.

This isn’t a coincidence. Chat interfaces are the cause of a lot of disillusionment. Most people think they need one but they don’t know why or how to make one that is good. I’m writing my book to help people answer those questions.

Facebook Messenger Bots

Facebook announced their messenger platform at F8 a while back and I tried out their bots right then. They wouldn’t really work properly so I left them and thought I would come back to them later. Of course that didn’t happen.

Ben’s tweet got me digging back into my history to see what ever happened to those bots. I’ve documented my journey here for you to follow along. There are a lot of lessons to be drawn even from a quick review.

You can jump to the bottom if you just want the lessons.

Poncho

Back then I was in the market for a way to keep up to date with the weather (now I’m a happy Weathertron user). I thought a conversational interface could be just the ticket and I tried out Poncho.

What was the bot called again?

Rediscovery of the bot isn’t easy since Facebook Messenger does not have a space for bots. I chatted with Poncho in the past but that had disappeared in my chat history. I remembered half of a name —something sounding like Ponyo— so I typed that in and quickly found our meteorological friend.

Not just the weather but recipes and movies as well.

I opened the chat and saw the messages where I had left off back then. This isn’t particularly relevant for me anymore and definitely stale for a weatherbot. Contrast this with the Lark or Quartz interfaces where there is a clear caesura between each episode.

Lots of help to be had.

I typed “Help” and I was quickly overwhelmed by a wall of text and so many controls that would not fit onto my screen.

“I’m not sure I understand.”

Then I got more controls with accompanying text that I could not make a lot of sense off without context. There seem to be two preset notifications that can be set and turned on/off.

Then Poncho didn’t understand me because I think I interacted with a control from the previous message. I gave an input for a context that we had already moved past so Poncho didn’t understand me anymore.

“Is this what you meant?”

Now we finally got an interaction that works the way I would expect it to work. I type something in, magic happens and we agree on a fact: that I am situated in Berlin.

Though why wouldn’t it use my phone’s GPS to figure out where I am and give me relevant weather information based on that? Actually I would expect it to open with that: weather based on my current GPS or Facebook location and build up from that.

Finally the weather!

Then I try to ask Poncho some questions which it doesn’t understand. It responds with something which may be better than saying “I’m sorry I don’t understand.” but only marginally so. It isn’t a conversation if the messages from either party don’t matter for what comes next.

Who uses Fahrenheit anyway?

Poncho gave me the weather but it was in Fahrenheit which is something most people in the world can’t understand. I asked to have it changed (which I think may be a very common request), but Poncho again does not understand.

Look at me talking here like some stupid person.

I kept trying to change the temperature unit but we’re not getting anywhere.

A message blob with all the settings just like an app.

Finally for some reason it spits out the settings message again and I can change to centigrade by tapping a button. How is this different or better than an app? If your interface has a bunch of settings that are not relevant for the conversation, you could save them for a separate sheet like the Quartz app does. I don’t think Messenger allows for that which is why they are all crammed inside a message.

Being able to change each of these settings in a conversational way would massively increase the size of the chat flow. Developing and testing those interactions is a lot of work and something that is totally understandable to shortcut like this.

Poncho now doubles as my alarm clock.

Another interaction that was surprisingly successful was changing the time of a notification. It understood my non-American time unit and normalized that into 8:30 am.

It didn’t understand that I wanted the notification in our current context to be turned on but it turns out that it already was.

Early in the morning on the next day I did get my forecast by Poncho with a message about the weather. I’ll try it for a couple more days to see whether this fits into my life.

How to do it better

Based on this quick review of the Poncho chatbot we can come up with a lots of improvements. Some of these are specific to conversational interfaces and some of them are timeless.

• Onboard people stepwise into your experience, don’t drop a huge introduction and settings screen on them.
• Open by giving people something interesting or valuable.
• Respect our mutual history and reintroduce yourself if it has been a while since we last spoke.
• Think about pace and don’t be afraid to take it slowly.
• Mixing custom control cards with a chat interface like this is not natural and contains lots of edges which result in weirdness. What happens if you scroll up and press a button on a card that’s no longer relevant?
  I think there is a reason why other applications try to not break the chat metaphor.
• Don’t pretend you can do things that you can’t. Users are going to notice.
• Test your application and if you have to choose, choose testing often over testing rigorously.
• Review your test results and monitor usage to actually improve your application. Iteration beats whatever you delivered at launch or are going to deliver in the future.

I’m writing a book about Designing Conversational Interfaces addressing many of the issues that come up when creating these kind of systems.