WebRTC: Integrating Cermati’s CRM System with Telephony Infrastructure
Background
At Cermati, we’re serving our customers to offer our products and validating their financial service applications through phone calls. Thousands of daily phone calls are made from our call centers, going in and out of our Asterisk VoIP servers. Needless to say, the call center is one of the pillars of Cermati’s operations.
The asterisk servers are collecting call detail records (CDR) on a MySQL database. This is a precious source of data for our operational analytics, along with the records available on our customer relation management (CRM) system. We regularly pull data from both our Asterisk CDR database and CRM to analyze our operational performance and look for areas we can improve.
Here comes the challenge: We faced some difficulties when trying to analyze CDR data and relate them to records stored in CRM, since we have no reliable way to relate both of them. Our telephony system and our CRM system are disjointed. Both are two different systems from the very beginning. The only thing in common for both systems is the fact that they are used simultaneously by our call center operators, by switching from one to another according to their standard operating procedures.
Session Initiation Protocol
Before we dig deeper into the details, we should put a bit of introduction about the Session Initiation Protocol (SIP) for those who aren’t familiar with it. SIP is the de facto standard protocol for VoIP communication on the TCP/IP network’s side.
The details regarding SIP protocol is out of this article’s scope. The protocol definition is available in IETF’s RFC3261 for those who’d like to learn more.
This protocol is an important component to Cermati’s telephony system, as the telephony infrastructure is built around SIP as the communication standard.
SIP header is one of the most useful features provided by SIP in our context, considering the problem we have with data analytics. It can be utilized to pass data from SIP client to the VoIP server when a call is made.
Initial State
As described in the figure above, the operator interacts with two separate systems in order to perform their daily operational job. Those two systems are the CRM system and telephony system, with CRM web UI and SIP softphone as the respective interfaces from which the operator performs their tasks.
Now, let’s see how the call center staffs are using both the CRM and telephony system to perform a regular operational task. To validate an application record, a Cermati call center operator needs to perform steps as follows.
After the call is finished, Asterisk will write a new record to the CDR database to keep information regarding the call.
From the previous scenario, we can identify two problems.
- The operator keeps switching focus between CRM and telephony systems, which are two separate systems and requires separate clients to interact with (web browser for CRM, SIP soft phone for telephony).
- The operator never enters any reliable identifier regarding the processed application to the telephony system. Therefore, the telephony system doesn’t have any reliable key to relate the CDR records to their respective applications.
WebRTC
WebRTC is an open source project to allow real time communication using web browsers. It is supported by Google, Mozilla, and Opera. More information is available in WebRTC project’s official web site.
WebRTC allows SIP communication over WebSocket. Given the web browser supports WebRTC, it enables VoIP communication to be performed using a web page rendered on the browser acting as the SIP softphone clients.
We decided to implement WebRTC-based SIP client on our CRM system, with the following considerations.
- It’s relatively cheap and simple to implement a web-based SIP softphone in our CRM system, as there are already open source JavaScript WebRTC libraries available.
- Embedding the SIP softphone in our CRM allows us to have more control over the behavior of the softphone, which means we have more flexibility to design the UI and to define what data to collect regarding the calls.
- WebRTC is compliant to SIP standards, which allows us to utilize SIP headers to contain our analytics-related data to Asterisk. Asterisk can be configured to include custom SIP header key-value pairs when storing the CDR record.
Final State
As shown in the figure above, the role of the SIP softphone to interact with the VoIP server is now taken by CRM web UI. As the only interface the operator needs to focus on is the web UI, we no longer need the SIP softphone client we originally used.
The integration on the UI brings us two advantages.
- The operator no longer needs to keep switching windows between web browser and SIP softphone to read application data and perform calls. They can also perform call right from the CRM UI without re-entering the phone numbers provided by the CRM to the softphone, because the CRM already defined the numbers to be dialed to the dial phone number fields in the UI.
- The VoIP technicians no longer need to maintain and configure standalone SIP softphone clients installed locally on each call center operator’s workstation. Since the softphone clients are now embedded in the web-based CRM system, the configuration for the softphone clients are centralized on the CRM system’s dashboard and server settings.
While the simplicity of operational procedures is one of the gains we got from implementing WebRTC, it’s not the objective we had in mind when we decided to implement it. The real objective is improving our data analytics capabilities by adding an identifier to relate a CRM object to a CDR object.
Since we have telephony clients running inside the CRM, our telephony system can now retrieve unique identifiers required to bind the CDR data to CRM records associated with the call. At this point, we can send the unique identifier along with any other data to VoIP server by using SIP headers.
Our Asterisk server is bundled with Elastix, and most of the configuration generated by Elastix’ management scripts. Example configuration for our Asterisk to handle custom SIP headers is as follows.
exten => _00.,1,Macro(user-callerid,LIMIT,EXTERNAL,)
...
exten => _00.,n,Set(CDR(foo)=${SIP_HEADER(X-Foo)})
exten => _00.,n,Set(CDR(bar)=${SIP_HEADER(X-Bar)})
exten => _00.,n,Set(_NODEST=)
...The syntax Set(CDR(foo)=${SIP_HEADER(X-Foo)}) retrieves the content of X-FooSIP header, and set the value to CDR object on field foo. We need to alter the cdr table inside the asteriskcdrdb MySQL database to add the foo column, and Asterisk will automatically save the value contained in X-FooSIP header to the foo column. The same thing happens to the X-BarSIP header and barCDR field.
Since the configuration files are generated by Elastix, we also patched the Elastix module responsible with managing and generating Asterisk configuration files to include the configurations we need. Otherwise, the configuration we added to Asterisk for the purpose of handling custom SIP headers will keep being overridden by newly-generated configurations whenever the telephony system is reconfigured by Elastix’ server management scripts. The scripts are written in PHP.
We found the following line of code in core_do_get_config() function definition inside the /var/www/html/admin/modules/core/functions.inc.php on our Elastix machine.
...
$ext->add($context, $exten, '', new ext_set("_NODEST",""));
...The code will add lines that look like this to the Asterisk extensions configuration.
...
exten => _00.,n,Set(_NODEST=)
...So, to have Elastix generate the configuration to handle X-Foo and X-Bar SIP headers correctly, we need to modify the code in functions.inc.php into this.
...
$ext->add($context, $exten, '', new ext_set("CDR(foo)",'${SIP_HEADER(X-Foo)}'));
$ext->add($context, $exten, '', new ext_set("CDR(bar)",'${SIP_HEADER(X-Bar)}'));
$ext->add($context, $exten, '', new ext_set("_NODEST",""));
...With this, the modification we made to Elastix is done. Elastix now generates configuration with handlers for SIP header fields we defined. There are a few other configurations changes we need to make on Asterisk to migrate the call center client extensions from the standalone SIP softphone to our CRM-embedded WebRTC softphone, but we pretty much followed Asterisk’s documentations for that.
At the time we finished migrating the call center operators to the CRM-embedded softphone client, we already have CDR data records that can be directly mapped to application records in our CRM. Therefore, we achieved our objective on data analytics.
Conclusion
WebRTC enabled us to integrate our two originally separated systems: CRM and telephony. The integration leads us to solve our data analytics problem regarding the company operations, while also reduces the number of client components involved in a call center operator’s daily job.
While the reducing number of components in the telephony system is a positive outcome, embedding the SIP softphone into a web page also has its downsides. The implementation of WebRTC across different browsers doesn’t always follow the same behavior, and at one time we also experienced longer network delays during dialing after upgrading the browsers used by the operation teams. Addressing this problem requires strong standardization from our call center technicians regarding the workstations’ setups.
We still haven’t ventured into the video call capabilities of WebRTC, due to our lack of needs regarding video calls at this point. But WebRTC and Asterisk can be configured together to build a video telephony infrastructure on the web.
