Virtual Agent Lab with Watson
I recently put together a lab to help developers understand how to integrate the Natural Language Classifier, Retrieve and Rank, and Dialog services to build a virtual agent for support desk. We typically complete this lab (among others) during Architecture Workshops which are a formal offering of the Watson Ecosystem so the instructions are a little less explicit then you might expect, but no reason developers could not complete the lab independently if you work with the WDC documentation and API reference. The code can be found on github.
Virtual Agent is a self-service engagement application that helps a user access support information and troubleshoot issues in a self-service fashion. It follows an interaction between an end user and a virtual agent for a fictional company called Gemstone Medical. For a list of the interactions that are included check out SCRIPT.md
This lab uses the Dialog, Natural Language Classifier, and Retrieve and Rank services. It was designed to be completed during the Ecosystem Architecture Workshop. For additional sample apps check out the Watson Application Starter Kits
Let’s get started.
Creating an IBM Bluemix Account
- Go to https://ace.ng.bluemix.net/
- Create a Bluemix account if required.
- Log in with your IBM ID (the ID used to create your Bluemix account)
Note: The confirmation email from Bluemix mail take up to 1 hour.
Create Watson services
- Create the Dialog, Natural Language Classifier, and Retrieve and Rank services in Bluemix.
- Capture the name and credentials for each service.
Configure and test the dialog service
Note: Steps 1–5 are optional. They show you how to set up and use the separate dialog test tool.
- Test the sample dialog flow by deploying the Dialog Tool. While the dialog tool is not required to build this application, it will help you manage and test your dialog flows going forward.
- Follow the instructions in the Dialog Tool README to bind your dialog service to the dialog tool.
- Upload the gem_dialog.xml flow file through the dialog tool UI.
- Experiment with “Have a Conversation” to see how the dialog flow works in practice.
- Note: The Natural Language Classifier is not implemented in the dialog tool so the flow will not work as an end-user would expect. However you can observe the machine to machine interaction that will exist between the dialog service and application.
- Unbind your Dialog Service from the dialog tool application and bind it to your Virtual Agent application.
- Create a new dialog flow by calling the create dialog API endpoint with the gem_dialog.xml flow file. Create Dialog API Reference
- Capture the Dialog ID.
Train and test the Natural Language Classifier service
- Create a new classifier instance with the gem_training_questions.csv training file. Create Classifier API Reference
- Capture the Classifier ID.
Configure, train and test the Retrieve and Rank service
- Create a Solr cluster and capture Cluster ID. Create Cluster API Reference
- Review the contents of the gem_solr_config directory. Review schema.xml and notice the fields that are defined. When the cluster is READY, upload the solr config gemstone_solr_config.zip to your solr cluster. Upload Config API Reference
- Create collection. Create Collection API Reference
- Review the sample solr documents in fda_medwatch_docs.json. Notice how each document contains the fields that are defined in the schema.xml. Notice how the structure of the json file follows standard Solr API construction for adding and committing documents. Index the documents. Index Documents API Reference
- Create and train ranker. TBD. Create Ranker API Reference
- Download the application zip file from this git repository.
- Ensure that you have the Cloud Foundry CLI installed.
- Modify the manifest.yml file to include a unique application name and the name of the services you created
- Log in to Bluemix cf login
- From the app root directory, push app to Bluemix. This will copy all files to Bluemix and bind your services to the application. cf push
- Go to Environment Variables in Bluemix and create 3 user-defined environment variables: DIALOG_ID, CLASSIFIER_ID and SOLR_CLUSTER_ID. Input the IDs that you captured when configuring these services.
- Because you have defined environment variables, you do not need to hardcode credentials or IDs in the application code itself, however you should take note of where this can be done.
- Walk the application code, specifically welcome.py, dialog.py, nlc.py, and randr.py. Start with the initial GET request in the Flask server code and follow each function call.
Modify the dialog xml and classifier to create a new troubleshooting flow
- There are 3 simple dialog flows within the Live Content folder in the dialog xml: Product_Info, Troubleshooting and Issue_lookup. Create a new folder in the dialog xml called Troubleshooting_2.
- Write out a new dialog flow to represent a new issue that an end user might need to troubleshoot.
- Add that flow to your dialog xml. Update your dialog service with your new dialog xml and update your DIALOG_ID in Environment Variables.
- Add a new class to your NLC training data to represent the user intent that enters this new flow and train your classifier. Update the CLASSIFIER_ID in Environment Variables.
Once your application is deployed, follow the script in SCRIPT.md. Test your new dialog flow.
You have completed the Virtual Agent Lab! You are now ready to build your own virtual agent application powered by Watson