Group/Duo Video Calling in Android| Amazon Chime using JavaScript

Published in

Mobcoder LLC

8 min readDec 16, 2020

Mobile Applications with instant messaging and video conferencing based communications, builds an improved human interaction medium.

Moreover, in this pandemic situation by following social distancing rule, the most secure way to stay in touch with people right now is through a video conference, which resulted a uplift in video calling apps.
Through this post, I would like to show how to accomplish basic group-video calling tasks and handle real-time-scenario’s using Amazon Chime Android SDK. It will connect the user with other clients(Attendees) so that they can share a real-time audio-video chat session.

Amazon Chime Overview

Amazon Chime is a communications service that allows developers to use its communications infrastructure and services that power Amazon Chime, and add audio calling, video calling, and screen sharing capabilities directly to their applications.

Prerequisites

You need to have basic to intermediate understanding of Kotlin and Android development.
you need an AWS account and Identity and Access Management (IAM) role that has the AdministratorAccess policy. To build applications using the Amazon Chime SDK, development experience and familiarity with JavaScript is required. You have to read this Building a meeting application using the Amazon Chime SDK to understand the basic architecture of Amazon Chime SDK and deployed a serverless/browser demo meeting application.

AWS Security

Amazon operates AWS regions and networks to meet the requirements of some of the world’s most security-sensitive organizations.

Access Controls

To join a meeting on Amazon Chime you are required to have an authorization token. Get Authorization token from server along with meetingResponse and attendeeResponse objects. These objects contain all the information needed for a application using the Amazon Chime SDK for JavaScript to join the meeting.

Modern Industry Standard Encryption

When connected to an Amazon Chime meeting, your audio, messages, video, and content shared through screen sharing is encrypted while in transit using industry standard cryptographic protocols: Transport Layer Security (TLS), Datagram Transport Layer Security (DTLS), and Datagram Transport Layer Security-Secure Real-Time Transport Protocol (DTLS-SRTP).

Keeping Your Meetings Secure

Amazon Chime meetings are all based on 10-digit meeting IDs. In order to secure your meetings you need to ensure that only those that have been invited have the meeting ID.

Features for securing your meetings:

AWS Pricing

Amazon Chime offers pay-per-use pricing which lets you pay for features you use, on the days that you use them. You can switch between Basic features that don’t include a charge, and Pro features that do include a charge.

Key steps

Here is an outline of the key steps involved in integrating the Amazon Chime SDK into your Android application.

Configure your application
Create a meeting session
Access AudioVideoFacade
Handle real-time events
Render a video tile
Add RecyclerView to Render other attendees video tile
Cleanup

Configure your application :

The Amazon Chime SDK for Android is similarly comprised of a common binary and Kotlin wrapper. It supports Android projects written in Kotlin or Java targeting API Level 21 (Android 5.0) or greater.

Step 1. Open Android Studio and create New Project and set the minimum SDK for the app to be API Level 26.

Step 2. To add Amazon chime Android SDK, you must complete the following steps-
i. Download amazon-chime-sdk-media.tar.gz and amazon-chime-sdk.tar.gz

ii. Unzip the files and copy amazon-chime-sdk-media.aar and amazon-chime-sdk.aar into your application’s library directory.

iii. Open your project’s build.gradle and add the following under repositories in allprojects:

repositories {
    google()
    jcenter()
    flatDir {
        dirs 'libs'
    }
}

iv. Add the following under dependencies section:

v. Use Java 8 features by adding the following under the android section.

Step 3. Requesting permissions
Since our app uses audio and video from the user’s device, we’ll need to request audio and video permissions. We’ll use the Dexter Permission library to do this.

Add the Dexter Permissions library to your app’s build.gradle.

it will check if the permissions have been granted or not, if haven’t, we prompt the user for camera and mic permissions with the else will redirect to settings screen of device. Once permissions have been granted, this method will trigger method to authenticate meeting url and make api-call.

Next we walk you through creating Meeting(can consider a meeting as a chat room, people with same meetingID will join same meeting) in order to have a basic audio, video experience. You can refer to the API document or my GitHub repository for additional details.

nishusingh06/chimeDemo

You can't perform that action at this time. You signed in with another tab or window. You signed out in another tab or…

github.com

Enough talk, let’s get coding.

Create a meeting session

You need to make a API call(POST request) to meetingUrl to create a meeting (room) and an attendee by default. The meetingUrl is the URL of the serverless demo meeting application you deployed(see Prerequisites). Else you can get meetingResponseJson from server and directly use it for session configuration.

2. We will use the response of above request to construct a MeetingSessionConfiguration. Convert the JSON response to the pre-defined CreateMeetingResponse and CreateAttendeeResponse types by using Gson, this can be done in a number of ways.

don’t forget to add Gson dependency to your app’s build.gradle.

//Gson dependency
implementation 'com.google.code.gson:gson:2.8.6'

Request Audio Focus (Real time scenario) :

Our meeting session is created, now before accessing AudioVideoFacade we need to ensure that audio focus is granted to our application i.e, no other application running on our device should occupy it if we are in a meeting session, else that sound will also be passed in audio streams. So just request focus by adding below lines:

and register audio focus change listener, it will get called when audio focus changes after being requested with AudioManager#requestAudioFocus(), and until being abandoned with AudioManager#abandonAudioFocusReques().

Access AudioVideoFacade :

Now we can access the AudioVideoFacade instance and use it to control the audio and video experience.

//get audioVideoFacade instance
audioVideo = meetingSessionModel.audioVideoaudioVideo.start()

audioVideo.start() -will start the meeting session. This method will initialize all underlying components, set up connections, and immediately start sending and receiving audio.

The video streaming does not start automatically. Call the following methods to start sending local video and receiving remote video.

// Enable remote video to start receiving streams.
 audioVideo.startRemoteVideo()//share the local attendee’s video with remote attendees
 audioVideo.startLocalVideo()//Switch between front and back camera in the meeting.          audioVideo.switchCamera()

You can control local audio on and off by calling the mute and unmute methods on the facade

// Mute local audio input.
audioVideo.realtimeLocalMute()// Unmute local audio input.
 audioVideo.realtimeLocalUnmute()

Handle real-time events :

We have to handle various real-time events during the meeting to update UI accordingly. Events are triggered when attendees join or leave the meeting, audio is muted or unmuted, or video is enabled or disabled. The Amazon Chime SDK for Android provides several observer interfaces including AudioVideoObserver, RealtimeObserver and VideoTileObserver. You can implement these as per your requirements.

1. AudioVideoObserver — provides status of audio and video client

AudioVideoObserver is used to monitor the status of audio or video sessions. The following code demonstrates some examples of the callbacks from AudioVideoObserver.

override fun onAudioSessionStarted(reconnecting: Boolean) =
    logger.info(TAG, "Audio successfully started. reconnecting: $reconnecting")override fun onAudioSessionStopped(sessionStatus: MeetingSessionStatus) =
    logger.info(TAG, "Audio stopped for reason: ${sessionStatus.statusCode}")override fun onVideoSessionStarted() =
    logger.info(TAG, "Video successfully started.")override fun onVideoSessionStopped(sessionStatus: MeetingSessionStatus) =
    logger.info(TAG, "Video stopped for reason: ${sessionStatus.statusCode}")// Register the observer.
audioVideo.addAudioVideoObserver(this)// Un-Register the observer.
audioVideo.removeAudioVideoObserver(this)

the onAudioSessionStopped callback is triggered when streaming stops due to poor network, so we will stop meeting and clear resources in this callback.

2. RealtimeObserver — provide attendee status

// Notifies when attendees joined the meeting.
override fun onAttendeesJoined(attendeeInfo: Array<AttendeeInfo>) {
    attendeeInfo.forEach {
        logger.debug(TAG, "Attendee join. attendee Id: ${it.attendeeId} external user Id: ${it.externalUserId}")
    }
}// Notifies when attendee(s) being dropped due to network.
override fun onAttendeesDropped(attendeeInfo: Array<AttendeeInfo>) {
    attendeeInfo.forEach {
        logger.debug(TAG, "Attendee dropped. attendee Id: ${it.attendeeId} external user Id: ${it.externalUserId}")
    }
}// Notifies when attendee(s) being removed.
override fun onAttendeesLeft(attendeeInfo: Array<AttendeeInfo>) {    attendeeInfo.forEach {
        logger.debug(TAG, "Attendee left. attendee Id: ${it.attendeeId} external user Id: ${it.externalUserId}")
    }
}// Notifies when attendee(s) whose VolumeLevel has changed to muted.
override fun onAttendeesMuted(attendeeInfo: Array<AttendeeInfo>) {    attendeeInfo.forEach {
        logger.debug(TAG, "Attendee muted. attendee Id: ${it.attendeeId} external user Id: ${it.externalUserId}")
    }
}// Notifies when attendee(s) whose VolumeLevel has changed from muted.
override fun onAttendeesUnmuted(attendeeInfo: Array<AttendeeInfo>) {  attendeeInfo.forEach {
        logger.debug(TAG, "Attendee unmuted. attendee Id: ${it.attendeeId} external user Id: ${it.externalUserId}")
    }
}// Notifies when signal strength changes for attendees whose SignalStrength has changed.
override fun onSignalStrengthChanged(signalUpdates: Array<SignalUpdate>) {
signalUpdates.forEach {
        logger.debug(TAG, "Attendee signal strength changed")
    }
}// Notifies when volume levels changed.
override fun onVolumeChanged(volumeUpdates: Array<VolumeUpdate>) {
    volumeUpdates.forEach { (attendeeInfo, volumeLevel) ->
        logger.info(TAG, "AttendeeId: ${attendeeInfo.attendeeId} externalUserId: ${attendeeInfo.externalUserId} volumeLevel: $volumeLevel")
    }
}// Register the observer.
audioVideo.addRealtimeObserver(this)// Un-Register the observer.
audioVideo.removeRealtimeObserver(this)

We will receive above callbacks for local and remote user both, we will update recyclerView adapter for Remote user callbacks. Once you get a list of attendees, you can fetch the attendee name from the externalUserId by the following code.

val attendeeName = attendeeInfo.externalUserId.split('#')[1]

3. VideoTileObserver — video tile track

Some applications only need audio, so video events are handled by a separate observer. If you’re not going to render video, you can skip this step and the render video tile step below.

By implementing onVideoTileAdded and onVideoTileRemoved, you can track the currently active video tiles.

// Register the observer.
audioVideo.addVideoTileObserver()// Un-Register the observer.
audioVideo.removeVideoTileObserver()

Render video tile :

To render a video tile (both local and remote), you have to define a VideoRenderView in the layout resource file.

<com.amazon.chime.sdk.media.mediacontroller.video.DefaultVideoRenderView
    android:id="@+id/video_surface"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />

To display the new available video tile, bind the view to the tileId. In onVideoTileAdded(tileState: VideoTileState) observer callback

audioVideo.bindVideoView(view.video_surface, videoTileState.tileId)

Binds the video rendering view to Video Tile. The view will start displaying the video frame after the completion of this API.

Add recyclerView to Render other attendees video tiles :

We will add a horizontal recyclerView on top of SurfaceView that we used to render local users view on full screen. Add below code in activity_meeting.xml :

Add adapter for our recyclerView. We will pass audioVideo facade and mutable collection of VideoAudioCollectionTile in constructor. audioVideo facade will be used to bind video tile with Surface View.

Some other real time scenario’s :

**1. Receiving a phone call in between meeting (Register TelephonyManager)**

We need to monitor PHONE STATE in order to observe incoming calls in between a meeting session, and manage audio streaming accordingly else that conversation will also be passed in our meeting streams. So just register PhoneStateListener provided by TelephonyManager by adding below lines:

val telephonyManager: TelephonyManager =
    getSystemService(Context.TELEPHONY_SERVICE) as TelephonyManager
telephonyManager.listen(phoneStateListener, PhoneStateListener.LISTEN_CALL_STATE)

we will mute and unmute local audio stream, accordingly onCallStateChange callbacks:

2. Surface View of remote user hides behind Surface View Local user

When creating a layout with a “picture in picture” type layout for remote user’s video stream, when the local video comes through, the video feed covers over the remote video of the person you’re having the meeting with as well. The “Wrapper” for the remote video feed continues to show up, it’s just the feed that gets overridden.

You need to callsetZOrderMediaOverlay(true) on the DefaultVideoRenderView of the remote video tiles.

view.attendeeVideoSurface.setZOrderOnTop(true)

Cleanup :

We need to clear resources we occupied and Unregister all listeners in onDestroy() of our activity.

NOTE: Don’t forget to delete the two AWS CloudFormation stacks created in the prerequisites, if you don’t want to keep the demo active in your account and wish to avoid incurring AWS charges. These stacks can be found in the AWS CloudFormation console.