Automatic Slot-Filling in MindMeld

Make your conversational apps seamless and smarter without missing important information.

One order, but multiple steps to get to it. Image credit: ThoughtCo.

Whether we’re ordering food or talking with customer service, we often communicate information in pieces rather than all at once. Consider ordering a meal. You might choose the type of cuisine first, followed by a specific dish, then add extras or sides. Since this type of back and forth dialogue is such a natural part of our daily lives, it’s an essential capability for artificial conversational agents.

With the release of MindMeld 4.3, our conversational AI platform widely used by developers to build robust voice assistants and chatbots, we now support the ability to iteratively fetch information from users with minimal code and easy implementation for developers.

Before this release, developers could use MindMeld to create back-and-forth interactions with users through dialogue flows. A dialogue flow allows the user to be directed toward a specific goal over one or more turns. This enables well-guided conversations with the system to fetch all required information. However, using dialogue flows requires adding tedious code for permutations of missing entity slots and setting conditions for exiting the flow when the user isn’t responding as expected.

The following example shows how a supermarket conversational assistant used a dialogue flow to fetch the missing store name in user queries. The goal of this function was to return the working hours for the store in question.

This is the case when the only required entity is the store name. If fulfilling the intent requires multiple missing slots, the code would be longer and have more checks for the different permutations. With the new additions to our platform, this becomes much easier.

What’s new

In MindMeld 4.3, we’ve introduced automatic slot-filling — a simplistic form-based functionality to easily fetch missing data (read: entities). It allows developers to prompt users for any missing entities for an intent and validate their replies, all in one go.

This is achieved by defining a slot-filling form containing all the relevant information about the entities (slots) to fill and invoking the slot-filling functionality.

For our supermarket example above, the form would look like this:

We then invoke the slot-filling functionality by using the auto_fill decorator and passing this form as an argument. The decorated handler gets called after the slot-filling logic has captured and validated all the required entities, ensuring seamless execution of the handler functionality without any additional checks.

As you can see, the code inside the handler has reduced significantly. The dialogue state handlers now only need to define how they want the app to operate, given they have all the information. Let’s take a more detailed look.

Defining a slot-filling form

A form is a python dictionary consisting of four parts: entities to be captured, number of retries allowed per entity, an exit message when the user tries to leave the flow, and the keywords that they can use to do the same. More formal definitions and syntax can be found in our feature documentation.

Forms make use of the FormEntity object, which is a new data class introduced in 4.3. It’s used to provide information such as the type of entity (custom entity names or system entity names), the user prompts for filling each entity, and the logic for validating the user input.

Validation

User inputs may not always match the expectations for a slot, and that’s what makes input validation important. The validation for each entity can be defined inside the FormEntity object itself. There are three ways to validate user input:

1. System validation: Using internal system validation, which is built into MindMeld. This validates the system entities per the pre-defined definitions and custom entities based on the gazetteer mapping defined by the developer. Using this validation also results in completely resolved values being passed back to the handler. System validation is triggered by default unless configured otherwise.
2. Hints-based validation: There are possible cases when one would want only particular responses from the user to be accepted. Hints-based validation allows the developer to define an acceptable set of keywords or phrases for the slot. The user response needs to match at least one of them exactly.
3. Developer-defined validation: MindMeld slot-filling allows developers the flexibility of defining their own custom validation functions. If there are certain slots where the expected user input may differ from existing definitions of the entity, defining one’s own custom validation function would be the way to go. This function receives the processed query and all its metadata in the form of the request object and should return a boolean as an indicator of acceptability for the input.

The great thing about these validation options is that they can each be used alone or in any combination to provide stronger validation for each slot. The syntax for each can be found in our documentation.

Let’s look at some examples

Now let’s take a look at a few different ways we can use this functionality.

Banking Assistant

For the following examples, we show the use case of a personal banking application, similar to our newly released blueprint of a banking assistant.

We’ll talk about two specific use cases here:

• Transferring money from one account to another
• Checking the account balance

Using a single form with the @app.auto_fill decorator

As shown in the supermarket assistant example above, we first show how to use the auto_fill decorator for a money transferring use case. The form requires two ‘account_type’ entities, with one having the role of ‘account_from’ and the other having the ‘account_to’ role. It also needs a third entity: a system entity to capture the amount of money being transferred. Once this form is passed into the decorator, the slot-filling feature does all the work to obtain these entities from the user and subsequently provide them to the transfermoney_handler for further processing.

Using a single form with the invoke method

There are often cases where there is no need to know about slots or entities until a point somewhere in the middle of the dialogue handler logic. How do we ask for entities or slots in the middle of the handler now? Let’s take a look at the invoke method for slot-filling.

For this example, we’ll consider the use case of checking an account balance. The user needs to provide only a single entity for this: the account type. Before asking the user for the account they are interested in, we need to fetch their information from a backend system.

In the check_balance handler, we first fetch user information and then ask for the requested slot using the invoke method of the AutoEntityFilling class. In the call to invoke, we pass in a follow-up handler to which the flow gets redirected once the slot-filling is complete. This follow-up handler can continue to execute further business logic and respond to the user.

What is the difference between getting these entities at the beginning or in the middle? Reducing unnecessary questions for the user! This invoke call can be placed in the middle of the function, and the developer can choose to only call this when certain conditions are met, otherwise skip and continue.

For example, in the previous case, if we only want to show the account balance to users whose accounts are active, we can place a check in place for this and only look for the account type if the account is active. Otherwise, respond with a message.

Using multiple forms for dynamic needs

What if we want certain slots in the form to depend on the previous answers until that point? Dynamically obtaining information can be an important factor in developing a conversational app. We recommend using multiple forms or subforms for the same.

For example, in the money transferring use case, two of the three slots that we want to fill are the account to be transferred to and the amount to be transferred. Although now, let’s consider the case where we only want to fetch those two slots when we know that the first entity (account to be transferred from) is a savings account. For this, we split the original form into two and handle this checking logic inside the handler. We call the first form through the@app.auto_fill decorator and the second one through an invoke call. The slots received after the initial form should be stored in the responder frame for accessing them post the second form. See the ‘account_from’ slot below:

This also shows how both methods of calling this functionality can be used together to greater effect. More logical checks can be placed at various points in the dialogue management while fetching entities as the developer deems fit.

And that’s a wrap! We hope this post fills all the question slots in your mind with answers and gives you insight into how to use this new feature in your conversational apps. Here’s a brief recap:

1. With the release of MindMeld 4.3, we now support automatic slot-filling for entities as an enhancement to our dialogue management capabilities.
2. This functionality allows developers to fetch information from users in a seamless conversational manner while writing minimal code.
3. There is extensive in-built validation, along with the flexibility to define custom validations.
4. It can be called either using the @app.auto_fill decorator or a direct invoke call, or both to add dynamism.

We welcome every active contribution to our platform. Check us out on GitHub, and send us any questions or suggestions at mindmeld@cisco.com.