DevOps for your Natural Language Processing model: Azure DevOps + LUIS

Let’s create our own Azure DevOps organization. If you don’t have one — you can start here for free. Note: in Azure DevOps you can create unlimited number of private Git repos.

Create new project for your NLU model. Let’s choose private repo with default settings (Git, Agile)

Choose `Repos` on the left-hand side of your screen. You will see how to clone this repo and how to push your changes.

Let’s clone this repo to your local machine. I will use Visual Studio Code for development.

Install this Azure DevOps extension to your organization. You will need to choose your organization from the list and proceed to the organization.

Let’s create NLU model using Microsoft LUIS cognitive service. Click on `Create new app` and add a name for your model, for example, `NLU`.

Click `Build` to add intents and entities to the model. We will create simple model that contains only two intents for now: `PlayMusic` and default one — `None`.

Click on `None` intent to add some sample utterances: book a flight to Ukraine, call the pizza place, penguins in the ocean, barking dogs are annoying, when does your store open?

Return back and click on `Create new intent` to create new intent: `PlayMusic`. Add sample utterance to `PlayMusic` intent: play music, start playing music, music please, play me some good music.

You can add more intents and entities later to this model.

Click `Train` first and then `Publish` buttons.

You need to commit your model to your Azure DevOps project repo as json file. You can download your model as json file from https://www.luis.ai/

Click on `Manage` →`Version`. Choose version of your model that you want to download. I will call my model `model01.json`. Let’s commit this model to our repo.

Let’s create test file for your model. It may look like that:

You can also add your own test cases. Format for test files is similar to the LUIS batch test format.

To find out more about format for test file go here: https://github.com/microsoft/NLU.DevOps/blob/master/docs/GenericUtterances.md

Let’s create pipeline.yml file to train and test our model.

Let’s commit and push this changes to our repo now.

After you pushed your changes — you will see them under Repos → Files.

Let’s create a build pipeline. Choose `Pipelines` tab on the left hand side and click on `New Pipeline`.

We have several options to choose from. Click on `classic editor`.

Choose branch where you have committed your pipeline to (for me it is a master ).

And then choose `yaml` option

Specify path to your yaml file and save.

There are some variables that we need to add to be able to train and test our Luis model.

Let’s create variable group and attach it to our pipeline.

Click on `Pipeline` → `Library` → `+ Variable Groups`

Specify `Variable group name` and add variables luisAuthoringKey, luisAuthoringRegion.

You can find those values on luis.ai portal under `Manage` →`Keys and Endpoint`

Default region — ` westus`

Now you need to choose your pipeline and attach these variables to your pipeline. To do that click on `Edit`

And then click on `Triggers`

Then choose `Variables` tab and under `Variables groups` click on `Link variable group`. Choose your group name from the list.

You can queue your pipeline now. You will see the following.

And now you can check your Luis.ai portal and you will see the new app with a random name and version 0.1.1

Using this extension you can create model via json file, train, publish and test your app against some test cases that you specify in your test.json file.

You can find your test results in the Test tab in your last successful pipeline.

You can add `clean` command after `test` command to your pipeline if you want to delete application that was generated. You still will be able to see how successful was the model from the json file your specify against the test cases in your test file and choose among different models or add some changes to existing model. This extension can help you to choose the best version or show failed tests, so you can analyze and improve your model accordingly.