Conversational AI: 1,000 days and counting, building next-generation NLP

Śìvä Šâī N
May 21, 2020 · 6 min read

This month I will complete my first 1,000 days of living, sleeping, and dreaming conversational platforms. It is a good time to reflect on the successes and challenges so far, and those that are yet to come as we work to realize our original dream.

The story begins with my poor experience with telecom customer support. It’s still jarring and painful to get customer support for simple queries, despite the availability of tens of thousands of Customer service agents, trained with playbooks and scripts over many years. Not to mention the hours you might need to spend on hold for navigating complex issues in, say, your bill last month.

I kept thinking about deploying Virtual Agents to take away 80% of routine queries so that fewer and better-qualified agents could rapidly handle complex issues and escalations.

So when I found this opportunity at Active.ai to work on conversational AI platforms, I jumped on immediately for the thrilling ride. This is the longest in my career that I have worked on a single problem statement, and yet, I’m not satisfied that we have cracked the issue completely. It feels like we are on day 1 every single day.

This reminds me of a famous cycling lesson — “It never gets easier. You only go faster”.

Here are some reflections from those last 1,000 days of waking up to the adrenaline rush that is as exciting as the stock market these days.

Conversation Design First Approach

A key position that we took on Day 0, even before we hired our first team member, was to dream up our ideal experiences for conversational banking as end-users first. Designers were still naive in terms of what technology can or cannot do, and that turned out to be a great thing for us. It allowed us to set our ambition high enough that it will keep us on the toes for years to come.

I remember that even before we had our first customer or hired our first Python developer, we had scenarios identified like

Compound queries like “Transfer 500 to John and Smith” OR “Show me my balance and pay the electricity bill”

Asking “what is my balance” in the middle of a fund transfer flow

Executing long instructions like “Invest $5000 in Apple at $300 stop-loss 290 from my IRA account”

I will write separate articles covering these conversational scenarios later.

Market leaders at the time like IBM Watson and API.ai (now Google Dialogflow) didn’t have (and don’t have yet) such capabilities. Neither were customers mature enough to think of such scenarios. But we set the bar that we pursued relentlessly in years to come and continue to pursue today.

Replacing the mobile app

What made this journey exciting was this guiding North Star — “We aren’t here just to take over customer support or replace FAQ pages or reduce call center volumes. We are here to replace the Mobile App”.

This makes our journey very hard too! We have to constantly remind ourselves that a customer can do any transaction and get any information that a mobile application or website provides. Today we handle 200+ use cases in banking alone. Our research says we need to be able to handle about 3,000 scenarios to be able to replace the mobile banking app. We aim to achieve this by the end of 2021.

We constantly question fundamental approaches to building Virtual Agents today because of the scale of this ambition. For example, we might sunset something as basic as intent classification by the end of 2020 to allow our system to scale to 3,000+ intents.

Ambitious Full-Service Virtual Agents

With our guiding North Star in mind, we aimed to deliver complete experiences from the get-go — including FAQs, Transactions, and Queries. In 2017, our first customers went live with “Full-Service Virtual Agents” while our peers in this market were launching FAQ-only or menu-based ‘chatbots’.

In an ironic twist, we delivered our DIY “FAQ only” platform www.triniti.ai only in 2019, full two years after solving the harder problems first!

An early example of how we map out a domain in depth. circa 2016.

Ease of building a Virtual Agent

Let’s say we pivot to become a boutique ‘bot agency’. Using our product team’s knowledge and experience, we can ramp our state-of-the-art capabilities to build kick-ass & unique conversational experiences for one-off deployments. You can imagine how expensive this would be. This is not a scalable approach.

The party trick, if I may put it that way, is to simplify and make advanced features available transparently to an end-user, with a low/minimal learning curve. You shouldn’t need to hire AI engineers and data scientists to deliver a conversational agent with advanced capabilities.

Take Context Understanding, which enables the conversational AI to correctly answer a sequence of queries:

“Do you offer a car loan?”
“what’s the interest?”
“and max tenure?”
“for a personal loan?”.

Today, an Active.ai virtual agent developer does not have to worry about this module, beyond doing his usual tasks of defining FAQs, intents, entities, and building workflows. The advanced features work transparently, and therein lies the scalable value proposition.

Balancing Cost factors

This is one final factor that we are in constant battle with. Cost. It’s the cost for our customer to build and run the conversational platform that makes economic sense.

Many of the tools in our arsenal require high-end & costly GPU servers, which conflicts with our goal to democratize conversational AI. This means we have to keep tuning / researching ways to bring these capabilities to market within current constraints.

Over time, GPU costs will go down further and businesses will also start investing in the value delivered by virtual agents. Until then, it is a constant battle between what we can include in the product tech stack and the additional cost it will entail when processing millions of conversations a month.

We owe it to our demanding high-end customers

Financial Institutions have been a blessing for us as customers. They are willing to allocate a larger budget, dedicate team members, drive and monitor this with a fine-toothed comb.

They treat the Virtual Agent like their customer-facing employee and any flaw that can impact the brand will make everyone from Chief Digital Officer to Chief Marketing Officer breathe fire.

This means we can never settle for good enough, and continue to constantly innovate to make experiences better and management easier while staying within the cost constraints.

Wrapping up

In the mid-1990s, businesses were busy setting up feature-rich websites. Database driven websites needed large teams of experts spinning out custom code, generating millions of dollars for consultants and agencies. Then came Dreamweaver from Macromedia (now Adobe) and changed the game completely.

Dreamweaver enabled thousands of enthusiasts to design and build a data-driven website with clean generated code and democratized web development, unlike any other tool.

We want to be the Dreamweaver of conversational AI and enable the democratized development of AI-based virtual agents.

Here’s to the next 1,000 days.

Until next time. Stay Safe.

Active AI

Chatbots and AI delighting customers in text, IOT and voice

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store