Published in


How We Select Skills to Generate Hypotheses

In the previous article we observed the dialogue management in DeepPavlov Dream and the final response selection algorithm. In this article we will describe in detail how the Skill Selector works.

Dream Socialbot is built upon an open-source DeepPavlov Agent framework and based on DeepPavlov Agent Architecture. The pipeline contains a block of asynchronous Annotators that extract useful information about the current user utterance, the Skill Selector component, a block of asynchronous Skills in which a selected subset generates response candidates, and the Response Selector component choosing the final response. So, the Skill Selector forms a list of the most relevant Skills that would be called in the current dialogue context. The list of Skills could be formed with a margin. This is especially important if some selected skills don’t return a proper response candidates or even return none of them. At the same time, we also can not select too many skills in order to avoid an overload of the computational resources.

The overall strategy is to always pick up fallback and open-domain skills, while closed-domain skills are selected only in case of the triggers’ detection. Among these could be specific topics, entities, intents, and/or regular expressions. Sometimes we need to add restrictions. They could be added either in the Skill Selector or Response Selector, but the first scenario will enable us to do it quicker and save the resources.

The Skill Selector of the Dream Socialbot is a part of the Agent’s container which orchestrates the pipeline of the dialogue processing. Thus, the Skill Selector is not a separate docker container but a part of the Agent’s docker container. There are two main reasons for that. First of all, the Skill Selector consumes little resources. Secondly, by keeping it as a part of the Agent docker container, we minimize the risks of losing any information while transferring it from one container to another. The Dream Socialbot’s Skill Selector is a rule-based algorithm that chooses a list of Skills to generate response candidates. High-level approach is presented in Image below. Let us describe the algorithm in detail.

The main cases of the Dream Socialbot’s Skill Selector

First of all, we check if a high priority intent is detected in the user utterance. Those high priority intents are detected using Intent Catcher annotator which is a few-shot classifier. For the main English-based Dream distribution we consider the following high priority intents: “cant_do” (user requests Assistant’s commands that are not available for the Socialbot), “exit” (user requests exit from the conversation), “repeat” (user wants the Socialbot to repeat the Socialbot’s utterance), “what_can_you_do” (user requests what the Socialbot can do), “what_is_your_job” (user requests what the Socialbot does for living), “what_is_your_name” (user requests what is the Socialbot’s name), “where_are_you_from” (user requests where the Socialbot lives), “who_made_you” (user requests who created the Socialbot), “what_are_you_talking_about” (user requests what is the conversation about). The response to the last one is provided by the Grounding Skill that was designed to understand the user and connect with them. The rest of the high priority intents are processed by the Intent Responder. Most of the high priority intents were developed due to the rules of the Alexa Prize Challenge but we find them useful beyond the competition.

The next condition is a check for sensitive topics. Our Dream distribution tries to avoid reactions to user requests for the bot’s opinion on sensitive topics such as religion, politics, or sex. This rule has been originally created to comply with the rules of the Alexa Prize Challenge. However, we are now considering removing this restriction, because of the high quality of the toxicity classification that can help us avoid inappropriate responses. For this sensitive case we select only the open-domain template-based skills that do not provide any opinion responses. We add some closed-domain template-based skills (News, Coronavirus, Weather, Fun-facts, Short-story, Gossip, Small-talk Skills) based on the user utterance annotations.We also select skills that were linked to in the previous Socialbot’s utterance and scripted skill from the previous step if its continuation flag was not “CAN_NOT_CONTINUE”.

Finally, in the general case we select all open-domain skills including neural generative skills. We also have some rules to select closed-domain skills. For the majority of the closed-domain skills, the python function “turn_on_skills” from dream/common/ can be used to detect whether it’s appropriate or not to select them. This function utilizes triggers from “SKILL_TRIGGERS” dictionary where for each closed-domain skill the developer provides the following independent triggers: compiled_patterns (regular expressions for the current user utterance), previous_bot_patterns (regular expressions for the previous bot utterance,without restrictions on the current user utterance), topics (list of topics for the current user utterance), intents (list of Intent Catcher’s intents for the current user utterance). Similar to the previous case, we select skills that were linked to in the previous Socialbot’s utterance and scripted skill from the previous step if its continuation flag was not “CAN_NOT_CONTINUE”.

Besides that, we have a number of custom rule s that fix automatic annotations in specific cases. For example, several rules are related to the intents detected by the Intent Catcher: we remove detected intent “exit” in the beginning of the conversation if the user utterance is long enough, or we remove detected intent “cant_do” if the previous socialbot’s utterance contains questions about different activities and the user responds with a word “play”.

While creating a new distribution containing existing skills, you don’t have to change the Skill Selector to remove unused skills, the agent will call only the skills mentioned in the pipeline file. It is possible to create your own Skill Selector that will receive dialogues and return a batch of lists of skills to call. We are also working on the simplification of the Skill Selector given that we can maintain a smaller number of restrictions outside of the Alexa Prize Challenge.

To sum it up, the Dream Socialbot’s Skill Selector is the rule-based algorithm that always selects open-domain skills and also selects closed-domain skills based on the user utterance and its annotations. Adding a custom closed-domain skill can be done via triggers in “SKILL_TRIGGERS” from dream/common/

We’d like to hear what you think about the Deepavlov Dream. Please don’t hesitate to give us feedback on our DeepPavlov Community Forum.

You can test the Deepavlov Dream Socialbot and understand better how it works by following the link.




An open-source library for deep learning end-to-end dialog systems and chatbots

Recommended from Medium — New Insights to Help you Understand your Visual Content

Use machine learning to make categorization — introduction to Classification modeling

A Basic NLP Tutorial for News Multiclass Categorization

The Scientific Method in a Machine Learning Workflow

Freesound Kaggle: My 627th Place Solution

The Simplest Interesting (and Useful) Recurrent Neural Network

Detecting fraudulent credit card operations with machine learning algorithms

Tweet Sentiment Extraction

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dilyara Baymurzina

Dilyara Baymurzina

PhD in Technical Sciences. Researcher in DeepPavlov, MIPT (2017-current). DREAM Team Leader in Alexa Prize Challenge 4 (2020–2021).

More from Medium

On Interaction via Natural Understanding [Interactive Series]

Quantifying the Uncertainty for Speech Recognition

Yelp Review Classifier With NLP

Exploring K-pop and BTS Song Popularity with Spotify Audio Features & Song Lyrics