The not-so-obvious hard things about bots

Erik Kalviainen
Mar 9, 2016 · 5 min read
Image for post
Image for post
Is the third line going to say F***!?

…that maybe you didn’t realize until you tried to make one

In making Meya we’ve learned a few things about building bots. Not the type of learning you get from a great TED talk; more like the learning that takes place when you drop a brick on your toe as a child. Deep learning 😜

This is the age of the Internet after all, so without further ado, here is my list of hard things about bots:

  1. Getting started: Yak shaving instead of “Hello world!” You’ve just read an inspiring Medium post and you’re pumped to build a bot. So let’s make a hello world bot for Slack! Great, let’s go: Install npm package, but first fix Homebrew because last night’s OS X Yosemite update broke it. Find fix. Clone GitHub fix repo, run script. Enter 2-factor auth code since I’m on my new laptop. Bot installed locally. Nice. How do I deploy? Ok, connect to Heroku, login in to Slack and get my slack bot token. How do I do that? Look it up. Got it. Crap, my Heroku account VISA is out of date. Update the VISA on file. Try and deploy again. It didn’t work. Wait, what am I doing again? Let’s go for lunch.
  2. Setting up the development environment and testing. So your hello world is running. You showed it to a co-worker who says nonchalantly mutters “Cool” and goes back to reading ProductHunt. Let’s start to make something impressive. Wait? How do I create a development bot? Oh, I have to go through all of that again? Crap. Ok, so I’ve added my development bot a test Slack team I’ve created, spun up a local development server. Changed “Hello world” to “Hello world!” and try again. Nothing. No debug message. Just bot silence. Is my bot secretly mocking me?
  3. Inferring intent and natural language processing. So you get the idea to make the quintessential weather bot. It’s the todo list app of bots. Consider the use-case: “@WeatherBot, what’s the weather like this weekend?”. Ok, so do I make a regex looking for “weather”, but how do I know where they live? What about the time frame? When is “this weekend” exactly? Do they want want to know anything more specific? What if they misspelled something? If they ask the bot again tomorrow, I shouldn’t ask them where they live again. That would be annoying. What if they ask “Will it rain today?” or “Should I wear galoshes?”. Yikes, how many ways can you say rain boots?
  4. Dealing with impartial data, ambiguity and exceptions to the rule. In fielding general requests with Meya users sometimes give all the information you need “What’s the cheapest non-stop flight from NYC to SF leaving March 9th and return March 12th?” However, it’s much more common that you get an incomplete request like: “What’s the cheapest flight to SF?” or sometimes just “flight”. Sometimes users are ambiguous “I’m hungry.” Is that a statement or a request? Then there are exceptions, oh so many exceptions. Users want the special with sauce on the side, they want to fly for cheap unless it leaves too early in the morning.
  5. State management, rules engines and flow-programming. Many bots on the market are stateless for the most part. HUBOT is a good example. The user makes a request, and the bot responds. Done. Other bots like A.L.I.C.E. or Cleverbot are the equivalent of bot small talk. However, if you’re going to get into anything like commerce or transactional in nature, you’ll have to consider implementing a multi-step process. Take the sandwich ordering use-case: pick your meat, cheese, toppings, provide name, specify time, pickup/delivery, pay, etc. There are conditional steps in there as well. For example, you don’t ask for an address if they are choosing pickup. What if the store is closes in the middle of an order or runs out of a topping?
  6. Ensuring effective product development iterations. Bots are new so there aren’t a lot of established analytics best practices and tools. How are users using your product? It’s not about page/screen views, time-on-site/app, # of button clicks/taps. The mode of interaction is fundamentally different, so you need to think about your success criteria differently as well. Is a long conversation with a bot a measure of success or failure? It depends. Was the user trying to order from White Castle and the bot kept recommending stoner comedies? Or was the bot so efficient and friendly that the user fell in love? Most importantly, how do you make sure you cut what’s not working and double down on what is?
  7. Integrating and connecting to messaging apps. Your bot may be transcendent, but to matter in the real world it has to be integrated somewhere. But where? It depends on your users. You need to think about the available platforms: Slack, Kik, Telegram, SMS, Amazon Echo, WhatsApp, Messenger (fingers crossed). Maybe you want your bot to live on more than one. Gasp! Each platform has their own nuance in production and development environments.
  8. Hosting, deploying, and scaling bots. So you’ve put the finishing touches on a killer v1 of your bot and it’s ready for more than just you and your co-workers. Where are you going to host it? In the case of Slack, you need to maintain a separate real-time socket for each team that installs your bot. How are you going to monitor it’s uptime? When bots go down, they fail silently for the most part. You also need to think about deployment pipelines, testing, quality control, and continuous integration. It’s starting to feel like an app or website deployment.
  9. Making it feel natural and avoiding the uncanny valley. Should you try and pass the Turing test? Maybe not, because failure will leave your bot resting at the bottom of the uncanny valley. On the other hand, you don’t want to sound or come across so unnatural or awkward that it’s off-putting or frustrating to your user.
Kramer failing the reverse Turing test.

10. Making something that is meaningful, useful or fun that users actually want! This point can be overlooked because you’re busy dealing with points #1–9. Just because you can make a bot, doesn’t mean people want what your bot is offering.

We’re working hard on something that will hopefully help you with most of the points above if you are building or considering building bots. Send me a DM on twitter or email hi at to request early access or just to learn more. 💯

Think Outside the Bot

Thoughts from the team who makes a conversational AI platform known as Meya.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store