Topolov’s chatbot score

Why not after all ?

So, my theorm about chatbots is that no fucking chatbot will ever be more userfriendly than a good UX until the general AI occurs.

To prove all you folks that I’m right, here is a simple rating system of any chatbot. Use it with your own or with any chatbot available out there, it will never score more than 10 points, until general AI is available.

A. Is your chatbot is able to immediately detect an off-topic phrase in a conversation, at any moment and to focus back to the main purpose ? (ex.: If I ask to a airplane tickets reservation chatbot about having a dinner, it should, politely and in a funny manner if possible, focus me back on departure dates…)

YES = 2 points

(Why it’s complex : scope definition is a complex thing. You cannot do it statistically, contextes have to be built in real time).

B. If date ranges (from/to) are needed, would your chatbot understand correctly the following sentence : “I would like to get a ticket for at least a week, with departure and arrival occuring during a week-end, anytime before chirstmas.”

YES = 5 points

(Why it’s complex : there is almost an unfinite amount of possible combinations. Only a system that can understand the concept of dates can manage this kind of tasks.)

C. Do your chatbot have a contextual memory for at least 20 phrases : if I start a conversation by saying ”i’m hesitating between Barcelona and Paris” and then 20 phrases later I say “well, finally i’m gonna for the second choice” is the chatbot able to understand you’ve mentioned Paris ?.

YES = 5 points

(Why it’s complex : the system must be able to understand, store and link items for at least 20 exchanges. It’s easier to have predefined context triggers (he gave me his budget YES/NO) rather than no store and analyse almost everyhing.)

D. Do your “book a table” chatbot is able to understand following : “at least two michelin star restaurant near Paris for two couples the day after christmas”.

YES = 5 points

(Why it’s complex : there are millions of possibilities for how to define something as complex as a restaurant (dark, gastronomic, chic, ….) you cannot label everything and making synonyms is not enough for precise abstract notions like “at least 2 michelin stars”. AI have to link the michelin stars with counters 1–2–3, you can surely hard-coded it, but you cannot hardcode everything.)

E. Your chat bot can understand a location described by “just behind the Eiffel tower”.

YES= 3 points

(Why it’s complex : “behind Eiffel tower” can be understood ONLY if you know what Eiffel tower is and how it’s architecurally oriented in space. Again it might be hardcoded in specific fields, but will fail generally)

F. For a data mining chat bot, can you complete the entire SQL syntax with natural language only.

(Why it’s complex : “SQL is a rich and complex language to query data from structured tables. Being able to replicate it with natural language is an extremely complex task due the potential nested structures”)

YES = 5 points

If your chatbot scores more than 10 points you’re the next billionaire. If not, invest in a good UX expert.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.