IBM Watson Assistant — Multiple Workspace Orchestration, Part One
Containing content in one monolithic workspace can be painful. Imagine a banking chatbot that can issue loans and provide general customer support. Chances are, you don’t want the content of the two tasks to intermix, especially if there are two teams responsible for the separate parts of the bot. A simple solution would be to create a different interface for each bot. However, two interfaces creates complexity for the customers. To avoid this complexity to the end user, developers can create orchestration logic to split workspaces and, when a user interacts with the bot, to combine workspaces. Creating this logical separation means a cleaner bot setup and a unified experience.
Now imagine this banking bot has three workspaces. One contains general customer support, another contains loan content, and a third contains credit checking content. There are three main ways to create the orchestration logic to connect these workspaces together: router, waterfall, and spray.
Router
A ‘router’ workspace selects which workspace to direct users to. There are two ways to configure the router: create one intent per workspace with all of the workspace’s examples, or copy all of the intents from the workspaces into the router. In the dialog tree, create dialog nodes where the response gives back the workspace ID to route the utterance to. If you create one intent per workspace, then this is just the general intent. If you create an intent for every intent in each workspace, then create ‘OR’ conditions for each of the intents.
After you receive the workspace ID of the child workspace you need to route to, pass the ID within your code to another /message call using the same utterance from the customer. If your dialog tree is rather flat, then you’re finished. If you have a multi-turn flow in the workspaces, you need a flag at the end of the flow so you can pass the call back to the router workspace. For example, in my credit check workspace I have 3 turns to ask the user for information to perform the credit check. I add a flag to the last node of the flow signaling the end. In my app, I listen for that flag to change the workspace ID back to the router ID. This could be when all slots are filled, I reach the final node in a flow, or when I return back to ‘root’ level in my dialog_stack.
Best used for: Bots requiring separate trainers/separation of ownership across multiple domains
Pros:
- The router can handle a large number of workspaces
- Maximum of two API calls per turn to get to correct workspace
- Sharing generic content in Router workspace across all teams
Cons:
- Requires additional training for router workspace
- On going maintenance to keep router and children workspaces in sync
Waterfall
The waterfall approach can be utilized when your various workspaces have a clear priority. For example, you might have two workspaces: one that handles your business processes, and another for general chit chat type questions. You want your users primarily interacting with your business processes, and only using chit chat as a fallback when your user is just chatting. There are two options for when to do a handoff. The first is when a specific intent is hit. Just like the router scenario, you would train an intent, or a number of intents on content related to the workspace to hand off too. The other option is when the anything_else node is hit, meaning no content was found, pass it over to this fallback workspace to find an answer. This orchestration could also be used if you have generic banking content that is only there to back up your loan bot. If users are on a loan website, and click a link to apply, you may want the generic content to only be backup in case they stray from the loan application.
You can implement similar logic here to return to the master workspace once the child workspace has finished a flow by listening for a flag to return to master.
Best used for: When you have content that is not a high priority for your use case, but still may be of some value to users.
Pros:
- Cheapest of the three options since most utterances result in only one API call
- Less work to keep the master workspace synced with children than the Router approach
- Users will typically end up in high value, high priority scenarios as they get first priority.
Cons:
- Does not work well when there are multiple domains of equal priority
Spray
The spray approach skips the parent / child relationship we have in the above two examples. Spray is sending the user utterance to all of your workspaces and giving the response back of the intent that scored the highest confidence. This requires the least effort to implement because you can maintain the workspaces separately and not build any handoff logic. However, it’s the most expensive since one user utterance is equal to however many workspaces you have.
Best used for: When content is rapidly changing, workspaces are being added/removed, and a router workspace could not be maintained.
Pros:
- Least complex to implement
- No additional training is required
Cons:
- Most expensive
- Need a way to share content between workspaces or a central workspace
Conclusion
These three orchestration approaches, spray, router, and waterfall make up the most common ways to implement multiple workspace orchestration. I personally find the router approach the most appealing because it causes less API calls than spray, and it’s an easier setup than waterfall. Of course, this is barring the fact that the script to keep the router in sync with child workspaces is a decent script to write. Feel free to reach out to me at chris.desmarais at ibm.com if you have any questions about Assistant or these approaches in general.
Part two gets into more detail on implementing the router approach.
Thanks to Mitchell Mason and Tammy for help in writing this post.
