PJSIP and RingCentral — Part 2: Handle Audio Medias

Welcome to the part 2 of the PJSIP and RingCentral article series! If you haven’t done so, please read part 1 first. In part 1, we covered some fundamentals, such as what PJSIP is and how to setup PJSIP to make phone calls via RingCentral. However, with just the knowledge from part 1, we won’t be able to make a meaningful phone call, because it doesn’t cover how to handle medias.

In this article, I am going to talk about medias. For this series the focus will be on audio medias, I don’t plan on covering video in this series. These are the topics I will be covering here:

1. The Audio Conference Bridge
2. AudioMediaPlayer and AudioMediaRecorder
3. Using audio files for RingCentral phone calls

Let’s get started with a very important concept: The Audio Conference Bridge

The Audio Conference Bridge

The conference bridge provides a simple but yet powerful concept to manage audio flow between the audio medias. The principle is very simple, that is you connect audio source to audio destination, and the bridge will make the audio flows from the source to destination, and that’s it. If more than one sources are transmitting to the same destination, then the audio from the sources will be mixed. If one source is transmitting to more than one destinations, the bridge will take care of duplicating the audio from the source to the multiple destinations. The bridge will even take care medias with different clock rates and ptime.

I really like this concept, because it makes handling media a lot simpler. It’s like a hub, you plugin all the audio sources and destinations onto this hub. And the hub will handle the data flows for you.

As a developer, especially when you are using the high level PJSUA2 API, you probably don’t need to manipulate the audio conference bridge directly. It will operate in the background and most of the time you can disregard it. But it’s still good know about its existence and how it works. That’s what I had in mind when I started this article.

AudioMediaPlayer and AudioMediaRecorder

AudioMediaPlayer player;
AudioMediaRecorder recorder;

AudioMediaPlayer is the audio data source while AudioMediaRecorder is the audio data destination.

In order to create an AudioMediaPlayer, you need to specify a wav audio file:

player.createPlayer("source.wav", PJMEDIA_FILE_NO_LOOP);

Pay attention to the PJMEDIA_FILE_NO_LOOP option. Without this option, by default the audio will be played again and again in a looping mode.

In order to create an AudioMediaRecorder, you also need to specify a wav audio file:

recorder.createRecorder("destination.wav");

This audio file is for saving data instead of providing data.

To make the audio data flow, you need an audioMedia object:

player.startTransmit(audioMedia); // audio flows from player to audioMedia
audioMedia.startTransmit(recorder); // audio flows from audioMedia to recorder

The audioMedia, in our case, could be an AudioMedia type object we obtained from an ongoing telephone call. In the next section we will show you a real demo.

Please note that, you can NOT make audio flow to AudioMediaPlayer or make audio flow out of AudioMediaRecorder. Because they do not support bi-directional traffic. However, there are certain kind of audioMedia which does support bi-directional traffic. You can both get audio from or send audio to it. We will see a real example in next section.

Using audio files for RingCentral phone calls

Step one — how can we obtain an AudioMedia object from an ongoing phone call? I posted all the code below and will explain in more detail right below the code.

We created a subclass of Call named MyCall. In this class we specified a callback method named onCallStat. This method will be invoked whenever a call start changes(for example, call connected, call ended…etc).

Then we checked whether the call had been answered:

if(callInfo.stateText == "CONFIRMED") { // call answered

If yes, we iterate all the medias:

for (unsigned i = 0; i < callInfo.media.size(); i++)

If the media is audio and we can actually get it:

if (callInfo.media[i].type == PJMEDIA_TYPE_AUDIO && getMedia(i))

Then we have the media:

AudioMedia *audioMedia = (AudioMedia *)getMedia(i);

Save remote audio to wav file:

audioMedia.startTransmit(recorder);

Please note that, it is possible to do more meaningful things than just saving the audio to a file. We can actually process the audio in real time and create some innovative applications. We will provide some examples in future articles of the article series.

Play wav file to remote peer:

player.startTransmit(audioMedia);

The audio data source doesn’t have to be a wav file. It could be the microphone. And the audio data destination also doesn’t have to be a wav file, it could be the speaker. But a microphone and speaker would require proper drivers to work. And drivers could be tricky if they don’t work out of the box. For servers there probably are no microphone or speaker at all. That’s why I didn’t use microphone and speaker as example.

Source code for your reference

Please check the demos from the official ringcentral-pjsip repository.

Summary

In this article, I introduced the concept of Audio Conference Bridge; I showed the usage of AudioMediaPlay and AudioMediaRecorder; I also showed you a real example about how to save phone call audio to a wav file and how to play that wav file to a phone call peer.

Now you should be able to do some meaningful things with the knowledge you gained from Part 1 and Part 2 of this article series. For example, you can create an app to send “telephony announcements” — call the customer automatically and play a recorded announcement audio.

There are quite a lot of innovative and exciting ideas in my mind. I’d like to show you more demo projects in the oncoming articles. Stay tuned for part 3 of the series!

To learn even more about other features we have make sure to visit our developer site and if you’re ever stuck make sure to go to our developer forum.

Want to stay up to date and in the know about new APIs and features? Join our Game Changer Program and earn great rewards for building your skills and learning more about RingCentral!