Hey Watson, Can I Have Your Number?

Patterns for bridging Watson Assistant with your existing phone system

Photo by Anders Jildén on Unsplash

There is an insane amount of gobbledygook you come across when you’re working with telephony technology, old and new. This post explains various options for connecting your existing telephony infrastructure with Watson Assistant. We will demystify the complexities of connecting with a cloud-based virtual assistant platform, and provide some tips.

The patterns described here will focus on connecting to the public cloud version of Watson Assistant. “Bring Your Own SIP Trunk” is a broad category of integration patterns most vendors can work with, but just what the heck is a SIP trunk?

Session Initiation Protocol (SIP) is a Voice over Internet Protocol (VoIP) technology used to deliver telephone and communication services over the internet, or any IP-based network. The best way to explain it is to compare it to the Public Switched Telephone Network (PSTN) — the legacy telephony network made up of old fashioned phone lines.

Whereas the PSTN uses hardware — analog switches and phone lines — to route calls, SIP uses software — virtual “trunks” — to transmit and receive multiple phone calls at the same time. SIP is a standard to make sure different systems can communicate voice traffic over the internet.

In its simplest form, the flow for an end user calling Watson looks like this:

Watson Assistant Phone Integration Architecture

1. A user calls in over the PSTN, there aren’t many other options here because thats how all of our cell phones work.
2. A Communication Platform as a Service (CPaaS) bridges the PSTN with a SIP trunk to connect to digital platforms
3. The CPaaS provider opens a SIP trunk to Watson Assistant, and the user has a conversation
4. If needed, Watson Assistant integrates with any back end systems to have a personalized conversation
5. If needed, Watson Assistant can send a SIP referral back to the CPaaS provider, and they can open a new SIP trunk with a live agent service desk to connect the user to an agent.

There is a lot of valuable information to help you make the right decisions when deploying an assistant over the phone. We’ve broken it down into 3 core sections:

1. Public Internet SIP Trunking Myths — things you may have heard that have steered you away from going digital
2. Using the Public Switched Telephone Network to integrate with Watson Assistant — while it is possible, it comes at a cost, literally. We’ll explain why this is one of the simplest methods to get going, but in the long run likely isn’t worth the cost as you scale
3. Common integration patterns — you likely have an existing infrastructure, and we don’t want to make you overhaul your enterprise. We’ve helped many large clients and have some best practices to make sure you are successful with what you already have

If you just read the first two sections you’ll be ready to start thinking about how to deploy your assistant. Each pattern then includes advantages/disadvantages to help drive your decision making process.

Public Internet SIP Trunking Myths

Let’s lay to rest some myths about public internet SIP Trunking.

There was a time when the real-time nature of voice over the internet imposed certain quality restrictions on using the public internet for SIP trunks. This is simply no longer the case. In the age of real-time streaming services like Netflix and Webex, it is rare to run into scenarios where jitter and latency concerns justify the need for expensive, dedicated private circuits.

Security was another issue holding VOIP and SIP back. As long as the two endpoints of your SIP Trunk support secure Trunking (meaning SIPS and SRTP) along with SIP authentication, you can rest assured that your voice calls will be protected.

Using the Public Switched Telephone Network to integrate with Watson Assistant

I am often asked, “can Watson Assistant integrate with my vendor’s telephony product?” where the vendor might be Genesys, NICE InContact, etc. The answer I always give is “Yes, as long as the vendor’s product can connect to Watson Assistant via a SIP trunk.”

It’s important to understand that Watson Assistant is a terminating endpoint for SIP trunks. These SIP Trunks can be originated from any device or cloud-based service that has the ability to establish an outbound SIP Trunk over the public internet.

When direct peering over a public internet is not an option, clients often use the PSTN to reach Watson Assistant. This is when things can quickly turn south due to tradeoffs between speed of deployment and cost. A typical call can consist of several call legs or intermediate connections to complete the call. For instance, there is a call leg from the caller over the PSTN to the SIP trunking provider, then another leg to the interactive voice response (IVR) system, another to the live agent, etc. Each one of those legs costs money, which quickly adds up with large call volumes.

Here’s why: call minutes on the PSTN are expensive in general. What’s more, outbound call legs are more expensive than inbound call legs, and each direction is charged separately. Therefore, a single call with multiple PSTN legs can end up with multiple charges that get expensive in a hurry.

There’s also the issue of concurrency. The PSTN generally isn’t good at handling massive volumes of calls over one phone number. Individual phone numbers tend to only support a finite number of concurrent calls. To handle big volumes, it’s safer to provision multiple numbers and load-balance the calls to achieve the required concurrency.

Another issue: There is no way to exchange metadata (or context) about the call over the PSTN because it is built on protocols and standards that didn’t anticipate any need. This can cause issues when trying to share information collected by Watson Assistant with a live agent.

Some general rules of thumb about using the PSTN in the context of VoIP calls to Watson Assistant:

• ALWAYS try to limit the number of call legs that traverse the PSTN to at most one.
• If you find that you need more than one PSTN call leg, make sure you have a plan for reducing this down to one as quickly as possible.

Commonly Used Integration Patterns

In this section I will cover the most common Bring Your Own SIP Trunk network patterns that customers are using today to connect with Watson Assistant. I will break down each pattern and explain the various advantages and disadvantages. All the patterns outlined here assume that the caller starts with a number that is called on the PSTN and all patterns include the ability to transfer a call to a contact center that is hosting live agents.

Generally speaking, any pattern that relies on a single PSTN leg to complete the call flow should be considered a good pattern. The only patterns shown below that should be avoided are the last two due to the excessive dependency on the PSTN to support the integration with Watson Assistant.

1. Call transfer over Public Internet

Communication Platforms as a Service (CPaaS), like Twilio or Intelepeer, bridge the PSTN network with Watson Assistant. This allows Watson Assistant to be reached by calling a phone number provisioned on the PSTN. Customers can either provision a new number (either a Toll-Free Number or a local DID) or port an existing number to the CPaaS. These CPaaS providers also support the ability to transfer the call to either a cloud based or an on-premise contact center using standard SIP signaling sent from Watson Assistant or via an API called from a Watson Assistant webhook.

1. Call transfer over Public Internet

Here are the steps related to this call flow:

1. Call is made to Toll Free Number or local DID.
2. Call is routed to CPaaS over the PSTN.
3. CPaaS routes call to Watson over a public internet SIP trunk.
4. Watson transfers call by sending SIP REFER back to CPaaS.
5. CPaaS forwards the call to customer’s phone number (the transfer target) over a public internet SIP trunk (call leg from CPaaS to Watson Assistant is dropped at this point).
6. Customer’s Session Border Controller (SBC) forwards call to customer’s contact center.
7. After call is queued live agent picks up the call.

With this pattern the CPaaS is anchoring the call and is therefore in the call path until the call ends.

Advantages: This pattern is very cost effective because there is only one call leg on the PSTN, it’s also very flexible in that it allows metadata to be exchanged in the SIP signaling.

Disadvantages: Unfortunately, this setup is challenging primarily because it’s fairly rare to find an on-premise contact center’s SBC connected to public internet. It may also require numbers be ported to the CPaaS which can take time to complete.

2. Third-Party Contact Center as a Service (CCaaS)

One category of the Bring Your Own SIP Trunk pattern used to integrate with Watson Assistant is when a third-party CCaaS provider (e.g., NICE InContact, Genesys Pure Cloud, Twilio Flex, Five9, etc.) originates an outbound SIP Trunk to Watson Assistant. In this pattern, the CCaaS provider receives the inbound customer call off the PSTN and then forwards the call to Watson over a public internet SIP Trunk. Watson Assistant may transfer the call back to the CCaaS to handle it with a live agent. When the transfer completes, the call leg to Watson will be dropped. This diagram describes the pattern:

2. Third-Party Contact Center as a Service (CCaaS)

Here are the steps related to this call flow:

1. Call is made to Toll Free Number
2. Call is routed to CCaaS over the PSTN (typically the CCaaS has their own CSPs they work with for PSTN connectivity).
3. Call is then forwarded to Watson Assistant over public internet SIP trunk.
4. Watson Assistant transfers call either by sending SIP REFER back to CCaaS or using a CCaaS API to initiate the transfer.
5. CCaaS redirects call to customer’s call queue which is eventually handled by the live agent.

Advantages: There are many Watson Assistant customers using this flow and its very cost effective while also providing the ability to exchange metadata in the SIP signaling such as call context (e.g., pointers to session history).

Disadvantages: CCaaS controls all telephony routing which may reduce flexibility.

3. On-Premise Contact Centers peered via a Public Internet SIP Trunk

A variation of the CCaaS integration are cases where an on-premise contact center can integrate directly with Watson Assistant over a public internet SIP Trunk. This is another very cost-effective pattern that also supports the ability to exchange context but is, unfortunately, fairly rare, primarily because most on-premise contact centers are not setup to connect with cloud based solutions like Watson Assistant over public internet.

3. On-Premise Contact Centers peered via a Public Internet SIP Trunk

Here are the steps related to this call flow:

1. Call is made to Toll Free Number or the DID.
2. Call is routed to CSP over the PSTN and CSP forwards call to customers SBC, typically over some kind of dedicated private circuit (MPLS, TDM, ISDN, etc.).
3. Call is then forwarded to Watson Assistant over public internet SIP trunk. In this case the customer’s SBC has an interface on the public internet and can forward traffic to cloud based SIP endpoints via a SIP Trunk.
4. Watson Assistant transfers call by sending a SIP REFER back to the customer’s SBC to initiate the transfer. Customers SBC transfers the call to the on-premise contact center.
5. Call is eventually handled by the live agent.

Advantages: Very cost effective and allows for exchange of metadata via SIP signaling.

Disadvantages: On-premise customer SBCs are often managed by the customer’s Call Service Provider (e.g., AT&T, Verizon, etc.). This can contribute to the challenge of enabling public internet SIP trunks and often leads to the next pattern.

4. On-Premise Contact Centers peered via a dedicated private circuit

Dedicated private circuits broadly describe any pattern that relies on MPLS, dedicated fiber, meet-me places and cross-connects in order to integrate on-premise contact centers using SIP trunks with Watson Assistant. This is often the best long-term strategy but can take a long time to setup and is also expensive. It’s typically only used for customers that must support a fairly large number of concurrent calls. Project time constraints often lead customers to choose one of the less-than-optimal patterns described below but it’s important to consider this pattern as a long-term solution even if a pattern with multiple PSTN call legs is being used to quickly get into production.

This pattern can have several different flavors but typically involves the Call Service Provider setting up a dedicated circuit to reach Watson Assistant in one of its public cloud data centers. This diagram describes a typical setup.

4. On-Premise Contact Centers peered via a dedicated private circuit

Here are the steps related to this call flow:

1. Call is made to Toll Free Number or local DID.
2. Call is routed to CSP over the PSTN.
3. CSP forwards call to Watson Assistant over a dedicated private circuit. Often times the CSP will pre-qualify a call with a simple IVR before forwarding to Watson Assistant to determine if it’s an intent that should be handled by Watson.
4. Watson Assistant transfers call by sending a SIP REFER back to the CSP to initiate the transfer to the customers contact center.
5. CSP transfers the call to the on-premise contact center.
6. Call is eventually handled by the live agent.

Advantages: Guaranteed Qualities of Service and extremely high security are the two biggest advantages with this pattern. This pattern also provides the ability to exchange metadata in the call signaling.

Disadvantage: This pattern can be more expensive and is definitely more time consuming to setup than the above patterns that rely on over-the-top, public internet SIP Trunking. It also requires the purchase of CSP service features to handle things like SIP REFERs coming back from Watson Assistant or the ability to exchange metadata in the call signaling like pointers to the session history. It can also involve features like call forwarding in the CSP. The cost of these CSP features is well worth the functions they provide and will cost much less than traversing multiple call legs on the PSTN.

A variation on this pattern has the customer’s SBC interfacing directly to the private circuit as opposed to relying on services in the CSP to anchor the call and to handle SIP REFERs coming back from Watson Assistant. For customers that manage their own SBCs or have more control over the configuration of their SBCs, this can be a more cost-effective approach than relying on the CSP to anchor the call.

5. Hybrid patterns: Terminating Watson Assistant calls on premise using Voice Gateway

Often used when there are just no other reasonable options for integrating with Watson Assistant in public cloud via a SIP Trunk. With this pattern, the Watson Assistant Voice Gateway can be installed on-premise, directly within the data center where the customer’s contact center resides. The Watson Voice Gateway is a VoIP device that terminates SIP Trunks on one side and orchestrates Watson Speech To Text, Watson Text To Speech and Watson Assistant in public cloud on the other side, using only outbound HTTPs and web socket connections. These kinds of connection are typically much easier to make to public cloud services from an enterprise network and typically only require routing through an Application (forwarding) proxy or firewall configuration changes.

5. Hybrid patterns: Terminating Watson Assistant calls on premise using Voice Gateway

Advantages: Terminating the SIP trunk to Watson Assistant within the customer’s telephony network is often the only viable option to integrate. Also allows easy/secure access to backend systems from an on-premise Service Orchestration Engine.

Disadvantages: The obvious disadvantage with this approach is that it requires that customers install and manage another device in their network but Voice Gateway is very lightweight and provides a hybrid migration path to cloud without a lot of time and cost. Note that this is a very common pattern that many Watson Assistant customer use in production today.

To read more about the Watson Voice Gateway go here.

6. Call transfer over the PSTN from Watson (Bad)

Similarly to the first pattern, the CPaaS provides a new number or a number is ported to the CPaaS. However, now the only way to connect back to the on-premise contact center is over the PSTN:

6. Call transfer over the PSTN from Watson

Here are the steps related to this call flow:

1. Call is made to Toll Free Number or local DID
2. Call is routed to CPaaS over the PSTN.
3. CPaaS routes call to Watson over public internet SIP trunk.
4. Watson transfers call by sending SIP REFER back to CPaaS.
5. CPaaS forwards the call to customer’s Toll Free or local phone number over the PSTN (call leg from CPaaS to Watson Assistant is dropped at this point).
6. Customer’s carrier forwards call to customer’s contact center.
7. After call is queued live agent picks up the call.

Advantages: This pattern can often be stood up in a relatively short period of time.

Disadvantages: This pattern is where problems start to occur. First, there are two call legs on the PSTN which causes the customer to see three different PSTN charges which include an Inbound and Outbound CPaaS leg and an inbound leg at the Customer’s Call Service Provider. The cost of this call is substantially higher than the same call described in diagram 1. Secondly, it is often desirable to pass metadata collected by Watson Assistant to the live agent via the SIP signaling. This is just not possible when the call leg from the CPaaS to the customer’s contact center traverses the PSTN.

7. Call transfer over the PSTN to Watson (even worse)

There are cases when a customer does not want to switch Call Service Providers and port their existing Toll Free or Direct Inward Dial (DID) phone numbers to a CPaaS. Time constraints can also play a role here. This often leads to this pattern which is the least desirable of all the patterns listed in this blog:

7. Call transfer over the PSTN to Watson

Here are the steps related to this call flow:

1. Call is made to Toll Free Number or local DID
2. Call is routed to Customer’s Call Service Provider (CSP) over the PSTN.
3. Call is then forwarded by customer’s CSP over the PSTN to a CPaaS provisioned phone number.
4. CPaaS forwards call to Watson.
5. Watson transfers call by sending SIP REFER back to CPaaS.
6. CPaaS initiates a new call leg on a different DID back to customer’s Call Service Provider. This is often referred to as “hair-pining” the call.
7. CSP forwards call to customer contact center. After the call is queued live agent picks up the call.

Advantages: Often the quickest solution to stand up.

Disadvantages: It takes five PSTN call legs to complete this call. This is really the worst-case scenario. To begin with, the cost of this call is extremely high. There is also no chance of exchanging any metadata about this call in the signaling between Watson and the contact center. Calls like this can also be difficult to debug when something goes wrong. Needless to say, this pattern should be avoided whenever possible.

In some cases you can avoid hair-pinning the call when using this pattern. Some CSPs support a service that will allow call transfers to be initiated using DTMF tones sent from Watson Assistant all the way back to the CSP on the first PSTN call leg to Watson Assistant. These DTMF tones will trigger the CSP to drop the call leg to Watson and initiate a new call leg to the customers contact center. AT&T refers to this service as Transfer Connect. Unfortunately, this is a fairly expensive service and, while cheaper than the cost of an extra PSTN call leg, it’s typically avoided in favor of some of the other patterns described in this document.

Conclusion

If you made it this far you’ve probably come to the conclusion that VoIP network connectivity is a complex topic with a huge number of possible network configurations. In this blog I’ve tried to lay out some of the most common patterns, all of which are being used today to integrate with Watson Assistant but by no means is this list of patterns exhaustive. Also remember that there are subtleties to all these patterns that often require deep network planning with experts in the field to get it right.

It’s critical to understand the pros and cons of the patterns that are available to you before making an implementation choice. In many cases you will find that selecting one pattern is really just a temporary steppingstone to a different patten that is more cost effective and provides you more functionality. Planning for how to get to the ideal network pattern up front will save you time and money in the long run and should not be ignored. IBM can help you navigate these options to make sure you land on the best possible pattern. You can sign up for Watson Assistant to see the new phone integration in action. If you want to learn more, please contact your IBM representative or join the IBM Watson Apps Community to stay in touch.