People > Bots for Complex Questions

“How would you know the hyperdrive is deactivated?”

I wish all of the bots I communicate with online could talk to me like this example of C3P0 calmly diagnosing a hyperdrive failure in The Empire Strikes Back:

When I hear the word “bot”, I instantly think of the sci-fi movies I grew up with where people talk to their friendly robot companions in plain english, expecting to have lively conversations where the robot answers (and queries) complex questions and answers. The reality of using bots today is somewhat different.

A Real World Query To A Bot

This week, I needed to know a statistic for landing page conversion so that I could know what to aim for when calculating my performance goals. This seems simple, but it’s a sophisticated question that will drive a lot of different marketing efforts.

If you’re like me, you’ve probably had this sort of experience when you need to answer a question — particularly a specialized business question — and end up with a ridiculous answer. I wanted to know how we were doing with our landing page conversion and had recently installed Growthbot in one of our Slack channels. This is an experimental software, so I didn’t expect to get 100% accuracy on the response, but since it is a sales and marketing tool, I expected Growthbot to handle a common closed question based on a metric marketers want.

How did the Bot Do At Answering the Question?

Growthbot failed here. The root cause might have been a simple miss of expectations, where I was pretty excited to ask a bot technical questions about SEO, landing pages, and conversion that I would otherwise need to point to a trained marketing expert. But I expected a bit more.

What could Growthbot have done differently?

As a bot, Growthbot could have responded in a few ways to gain confidence:

  1. Be explicit about what to do next when it doesn’t have an answer — here, it would have been very helpful to say “Oops, I don’t know — I’m asking someone to find out” and triggered a question to a human to look up the answer, tag that missed question, and start the process of understanding how often this question is asked?
  2. Do a fuzzy search of available information — I’m not sure yet whether search engines would be useful in this case, but setting the expectation for “I’m going to look for you and return in 15–20 minutes with a specific answer” would deliver more confidence than “I don’t have one”
  3. Return a list of links to known domain expertise — it would also have been possible to return a standard list of marketing links when a question is unanswerable.

Growthbot is a clever tool, but only when asked in a specific way about a specific problem domain. I expect it will get smarter in the future and perhaps use a natural language search to answer questions like these, but it seems quite a long way from being able to string together these sample queries into more complete and complex understanding.

What’s the Matter with Bots?

When you rely on robots to answer your question, they are only as smart as the algorithms you use to tell them how to learn.

For bots to deliver a better and faster experience than people, they must deliver a differentiated service experience that itself is better and faster than dealing with people.

Customer service questions — especially the complex ones — require more than just a search.

What would a person do when faced with this question? Google It.

Let Me Google That For You

The first organic link I found told me an answer: 7%. Do I have any idea whether this is correct or representational of my situation? Nope, although I have built up a cognitive bias to suggest that the top organic link found on Google is often a good one, especially when a sample of other nearby links has similar information.

But that link by itself has no value, since I don’t really know whether I am doing research or simply doing web search. Here’s where a smarter bot or a knowledgeable human could really help my experience and truly answer my question instead of simply providing an input I need to interpret to uncover the knowledge I need.

People > Bots For Service

If there was a fast and easy way to route my very specific and technical question about marketing to an expert who knew the problem domain (and perhaps was a marketer themselves) and to then route that answer to me, that would be extremely valuable.

Put another way, Growthbot (and other Bot solutions) are not useful yet beyond very simple situations because they are not good at anticipating the way that humans communicate.

We do not structure our queries as name-value pairs. We do not ask questions sequentially. We often don’t even make sense to each other when we formulate our questions.

People answer customer service questions better than bots because we are both good at pattern-matching, accuracy, and empathy. A bot answers a multiple choice question well and is highly accurate when placed in a particular decision tree, and may be entirely wrong because you went down the wrong rabbit hole of a question. A bot doesn’t respond well to sarcasm, emoji, or all of the other messy sentiment we put together when we communicate on the internet.

Technology makes it much easier for humans to answer your question with the precision and accuracy of a bot. A person, however, knows how it feels when someone asks you “why don’t you just Google it?” when compared with the satisfaction of knowing you received a great answer with a smile from an expert.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.