A Year With Orbit Operator
Last year, we accidentally built our own chatbot framework. Our colleagues at the Bakken & Bæck AI research lab in Bonn had already developed an AI service for understanding natural language in 2013/2014, before we got into writing robots in 2015/2016. It seemed like a good idea to put the two together by making a template for automated conversations: Orbit Operator.
Arguably, we started this project at the wrong end − with the technology instead of the problem − but the real push to actually build Orbit Operator mostly came from our chatbot-curious clients in service intensive consumer industries. And so it came to pass that, by the end of 2017, three virtual assistants went live on Orbit Operator:
- An infobot for the City of Oslo. The bot provides authorized parenting advice, curated by an experienced medical staff. Parents of young children often find themselves searching for answers online, a quest that usually ends up in getting ‘hearsay’ advice on parent forums or Facebook groups. The City of Oslo bot is based on professional expertise and provides quick answers to new parents’ FAQs. The infobot aims to free up time for the medical staff, so that they can spend more of their precious minutes on personal, face-to-face conversations. Furthermore, it aims to extend the municipal health station’s opening hours without increasing the staff’s workload.
- A customer service assistant for media and classified ads giant Schibsted. The chatbot answers FAQs from subscribers to five Norwegian newspapers and it provides assistance by performing simple customer service tasks, like solving login issues, sharing digital subscriptions and changing the delivery address for the newspapers. The bot is integrated with Schibsted’s call center technology stack.
- A bot that collects customer reviews for restaurants. While dining out, customers can upload a picture of their dish (hey, if it’s good enough for Instagram) and give an instant thumbs up or down to their food experience. This bot was developed with HORECA.digital, German retailer METRO’s digital arm, a subsidiary that equips independent restaurants, hotels and caterers with innovative digital tools.
Hold that line. Another chatbot platform?
With so many chatbot platforms out there, the legitimate question ‘why in the world should we make our own?’ naturally popped up.
The overwhelming number of chatbot frameworks that have emerged since bots became the new apps (thanks again, Microsoft!) provides a partial answer to this question: it is extremely difficult to come up with one generic recipe for building full-service chatbots that meet real user needs.
A ‘useful’ chatbot does at least three things well:
- It understands what the user wants by identifying questions, greetings, idle chit-chat, requests and answers.
- It thinks and makes decisions by processing the input from its users, asking them for clarification, searching through databases and checking business rules.
- It responds and acts by composing a reply, presenting a suitable predefined answer or performing tasks on behalf of the user (e.g. ordering a meal, changing the address for a delivery, etc.)
“Understanding” has become easier for bots thanks to commoditized, open source technology for natural language processing (e.g. Google Cloud Natural Language API). Once the bot understands, its “responding” mostly boils down to presenting a pre-defined text (similar to the standard answers one gets from a junior customer service representative).
The “thinking” part is more tricky. Finding a generic solution for chatbot “decision-making” is significantly more difficult than writing a recipe for understanding human language. The act of ordering an Uber is pretty self-explanatory, but the right way to handle a complaint very much depends on the organization that provides the particular service. An appropriate response for one company, could be an inappropriate response in a different company context.
Standardizing the “decision-making” part in a chatbot framework is nearly impossible because the inner workings of organizations (even in the same industry) are all slightly different. Training a chatbot to perform organization specific tasks, means diving into a whirlpool of business rules and legacy systems. This is one of the problems we chose to tackle by building Orbit Operator as a framework. This way, we could make our developers’ lives a bit easier, while at the same time providing different kinds of organizations the tools to customize the bots to their needs.
Half-Baked, not Half-Assed. What is Orbit Operator and what does it do?
Orbit Operator basically provides a chatbot development recipe on how to mix open source algorithms for natural language processing with custom integrations to enterprise systems. This combination results in a particular dish of averagely intelligent digital assistants, which have a sound understanding of user queries and are above-average efficient in solving them.
Orbit Operator consists of:
- A chat widget. Oftentimes a standard messaging app user interface like Facebook Messenger or WhatsApp will do. Sometimes, however, organizations prefer to embed a chat interface into their own communication channels (e.g. native app or customer account pages). Operator works for both scenarios, but a custom chat interface makes it easier to accommodate particular requirements e.g. data privacy concerns or branding/styling.
- Pre-trained machine-learning models for understanding user intents, which include text classification with language support for English, Norwegian and German and image recognition.
- An admin panel that lets enterprise users train the chatbot and set up dialogue templates, in order to effectively pass on their expertise (and boring tasks) to a digital assistant.
- Conversation flow logic that can be customized with customer specific elements like predefined responses, steps for process automation and business rules.
- Human handover. We all know how frustrating customer service conversations can be when the bot doesn’t get it right the first, second or eleventh time. So we made it easy for users to speak to a human being in case they need to.
- Hooks for integration with customer care technology stacks, e.g. Schibsted’s customer service bot that is integrated with Siebel CRM and the customer’s existing call center solution Puzzel.
- Customer insight gathered from Chatbase analytics.
The five most important lessons Operator taught us
Building digital assistants for media, retail and the government sector taught us a thing or two (or five) about chatbots:
1. ‘I want to speak to a chatbot’, said no one ever. Chatbots are first and foremost a productivity tool. While it’s true that customers − especially the younger ones − would rather text than talk, the real (and most valid) reason to invest in enterprise digital assistants for process automation is to increase cost efficiency. Being honest about this motive (and future perspective) makes it easier for customer service teams to adopt a chatbot as an important software tool in their work routines.
2. Training data is the gift that keeps on giving — but only if you give it enough time. Launching a chatbot is just the start of the journey. To reap the rewards of automation, organizations need to have a team/role in charge of monitoring bot responses and correcting bot behaviour by adding training data where needed.
3. Content is king, also for chatbots. We spent about one third of our time on defining the right content, in the form of both training data and predefined chatbot responses. This time investment was well worth it, first of all in terms of better-trained and more precise algorithms, but also in giving the organizations we work with an insight in how chatbots actually work. Involving the ‘chatbot trainers’ early on in testing the NLP models definitely paid off, as did visualising the test results. By asking experienced medical staff to curate the answers provided by our Oslo health infobot, we made sure that they were involved to such an extent that they actually cared enough to make it better.
4. Remember to check the right language box. At some point after launch, we noticed that the City of Oslo’s infobot was unusually hung up on ‘breastfeeding’. Granted, this is a topic of high interest among our users, it was overrepresented in our training data set as well, but still something seemed off. As it turned out, we forgot to check the language setting — the bot used English rules to understand Norwegian.
5. You need a panic button. There is no doubt that today’s technology for conversational agents has limitations. To acknowledge and prevent user frustration with chatbots ‘under training’, we designed a chat experience with a visible ‘take me to a human’- panic-button. The by-product of this feature is a way to measure the relative degree of automation, by comparing the number of customer requests fully solved by the chatbot with the number of cases that are handed over to a human customer service representative. If you can measure it, you can improve it!
There is a tendency among impatient techno-optimists to declare chatbots dead, even before they are fully-grown. Chatbot technology shortcomings are evident in the demise of all-purpose personal virtual assistants, like for instance Facebook M. We, however, like to think of ourselves as techno-realists, as we believe in the combination of AI and human capabilities –as already attempted by virtual assistants like Fin.
While the current status of enterprise chatbot technology is far from flawless, we do believe in its potential and therefore we’re not planning to give up on Orbit Operator any time soon. Viva La AIssistance!
