Dialog State Tracking in Conversational Interfaces

AILabs, Inc team is primarily focused on applying Conversational Commerce Research to closed domain fields. This includes Message Classification, Semantic Extraction, keeping dialog history in memory and more. And this means much attention on NLP tools like Syntax parser, Morphology parser, dependency parser.

Once you achieve “acceptable” results in Message(Question/Query) Classification(in closed domain), you start to think about tracking dialog state in order keep actual information in hands. For example, user may say “I would like to open bank account”, and as an example of next user message “Ok, show me the prices”. Here we must be able to handle fact that this query is related to operator response and history of user requests.

In our last experiment, we reproduced results from “A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems” paper, written by R&D company recently acquired by Microsoft, Maluuba.

Brief description: Synthetic dialog between operator and customer about restaurant recommendation is given. Operator starts dialog with “Hello , welcome to the Cambridge restaurant system? You can ask for restaurants by area , price range or food type . How may I help you?”

Each user message is marked by semantics: ‘ask’, ‘affirm’, ‘bye’, ‘hello’, ‘help’, ‘negate’, ‘null’, ‘repeat’, ‘reqalts’,‘reqmore’, ‘restart’, ‘silence’, ‘thankyou’, ‘confirm’, ‘deny’, ‘inform’, ‘request’.

Each operator message is market by semantics: ‘affirm’, ‘bye’, ‘canthear’, ‘confirm-domain’, ‘negate’, ‘repeat’, ‘reqmore’, ‘welcomemsg’, ‘canthelp’, ‘canthelp.missing_slot_value’, ‘canthelp.exception’, ‘expl-conf’, ‘impl-conf’, ‘inform’, ‘offer’, ‘request’, ‘select’, ‘welcomemsg’.

Message node may contain several semantics.

The goal is to predict semantics of user response, if last operator message semantics and dialog history are given.

The 2 layer RNN LSTM model is used for this task. Training process took less than 10 minutes on single CPU and test set showed 40% accuracy.

Indeed, this field deserves attention that it’s having. Going deeper into this problem, is definitely one of our near term priorities. Hope to see much improvement in this area and make our customers benefit from it soon.