Add active learning to your chatbot’s QnA Maker KB

If you can’t already tell, the AtBot Team loves QnA Maker. It’s an amazingly simple-yet-powerful way to provide knowledgeable responses to common questions that would otherwise have to be created in Power Automate or custom-coded one-by-one, which would not be fun.

To add to this feature-rich service, QnA Maker recently got even better by including a little magic Microsoft calls active learning.

A primer on active learning

First off, active learning isn’t machine learning. The updates made through active learning are not automatic. Nor should they be. You likely have strict and professionally prepared responses to the questions in your KB. You don’t want an AI system playing around with that setup outside your control.

What is automatic are smart suggestions based on actual phrasings from users of your KB. It’s important to note that active learning has everything to do with input phrasings (Qs) and nothing to do with responses (As).

But first, to use active learning, you have to enable it and then filter your KB to see only the suggestions. If your KB is running on an old version of QnA Maker, there might be some technical updates required.

Enable and view active learning suggestions in the two steps shown above.

The feedback cycle

Basically, QnA Maker has become smart enough to know if some As are very close in score, based on what your users are asking.

Adding more Qs makes for a more robust experience, plus smarter and more accurate As. It’s become a best practice for proper KB maintenance.

When a user asks a Q that results in multiple As having similar confidence scores (the exact threshold is proprietary), QnA Maker will respond to the user asking them to clarify which Q they’re asking.

You can see the results of this if you jump into the Test pane of a trained QnA KB, ask a question, and click Inspect under the submission. The pane shows the primary A and other, similar- or lower-scoring As below it (if there are others). Each has its own confidence score.

In the example below, the question “how do i connect to wifi” connects to three As. The first has a 95 confidence score. But, it turns out, the second A also scores at 95. (The third comes in at 27.33.) The fact that both have similar (in this case the same) scores means there is ambiguity that could be cleared up. And that’s how active learning can help.

The question hits three possible answers, with the top two having similar confidence scores (second score not shown, but you can trust us).

Once enough users have used similar phrasing (this threshold is also proprietary), an explicit feedback suggestion will show up under the existing phrasings for the A those users chose.

You will want to train and publish your KB again if you accept any suggestions. Active learning is a nice way to fill in gaps in your KB in a natural way. After all, no amount of testing will predict every way a user will ask questions to your bot.

That said, the biggest takeaway on QnA Maker active learning is that it works best as your audience grows and uses the KB more frequently. More input means a larger variety in phrasings, which in turns means more opportunities to train your KB. And the more it learns, the better your KB gets. So make sure to maximize your audience!

For more information on QnA Maker active learning, check out its Microsoft documentation page.

AtBot and Active Learning

And of course we built support for active learning right into AtBot Premium. To accept explicit feedback, simply enable Use Active Learning Explicit Feedback in the QnA Maker tab of your bot’s management page and provide a follow-up question. Your bot is set to use active learning going forward! It couldn’t be any easier.

Also, if you haven’t already checked it out, you want to learn all the new things that come with QnA Maker’s multi-turn conversations. Multi-turn is a game-changer for knowledge management and chatbots, so don’t get left behind!

Find active learning in the QnA Maker tab of your bot’s management page in the AtBot Admin Portal.

AtBot brings AI within reach

AtBot is the premiere bot-as-a-service solution for the Microsoft cloud. Built completely within Azure, AtBot is your out-of-the-box, easy-to-configure bot for Teams, SharePoint, or the web. Teach AtBot tasks using Power Automate, make him your corporate source of knowledge with QnA Maker, help him understand almost anything your colleagues could ask thanks to LUIS, and manage his features with the AtBot Admin Portal. Get going with AtBot Free or start your free trial of AtBot Premium today.