Packet Loss Runtime Comparisons for iOS Video SDKs with Apple’s Network Link Conditioner: Agora v. Twilio, TokBox, & Facetime 🏃♂️🏃🏻♀
Most product specialists take an ad-hoc approach to field-testing their SDKs under different conditions (if at all). In in this step-by-step tutorial, I will show an alternative. It is a runtime comparison with a bona fide network testing utility. Apple’s Network Link Conditioner is just such a utility that allows developers on macOS and iOS devices to accurately and consistently simulate adverse networking environments, which run the gambit from 100% Loss, 3G, DSL, EDGE, High Latency DNS, LTE, Very Bad Network, WiFi, all the way to WiFi 802.11ac. The result is a simulation tool for packet degradation.
In your runtime Comparisons for iOS Video SDKs, you will run different codebases tested side-by-side to see how either one fares, as network conditions begin to fail, if not terminate entirely. You will run one codebase alongside another, while simulating the effects of packet degradation with Apple’s Network Link Conditioner. You will demonstrate how to use Apple’s Network Link Conditioner, make side-by-side comparisons in terms of diagnostics, as well as see the effects of packet degradation in real time. Concurrently, you will become a specialist in how to test demo applications for the three major competing forces in real time communications, Agora, Twilio and TokBox.
Enabling the Network Link Conditioner
Enabling the Network Link Conditioner changes the network environment across the system according to the selected configuration, limiting uplink or download bandwidth, latency, and rate of packet loss.
You can choose from one of the following presets:
- 100% Loss
- High Latency DNS
- Very Bad Network
- WiFi 802.11ac
Alternatively, you may create your own preset. The Network Link Conditioner provides the following levers for creating “Custom Profiles”:
- Downlink Bandwidth
- Downlink Packets Dropped
- Uplink Bandwidth
- Uplink Packets Dropped
- DNS Delay
Enabling Network Link Conditioner on Xcode with iOS Devices
To utilize the Network Link Conditioner, you have to make sure that your iPhone or iPad is connected to your Apple MacBook.
- Connect your iOS device to your Mac
- In Xcode navigate from Window > Organizer
- Select your preferred device in the sidebar
- Click “Use for Development”
Alongside your Xcode configuration you have to configure your iOS device. iOS device under Settings > Developer > Networking.
Runtime Comparison with the Network Link Conditioner
Your next step is to do a runtime comparison. In your runtime Comparisons for iOS Video SDKs, you will run two different codebases tested with web integrations side-by-side to see how either one fares, as network conditions begin to fail, if not terminate entirely. For the background, you will use Hammacher Schlemmer’s electric jellyfish.
Custom Packet Loss
Why custom packet loss? Results based on 100% Loss, 3G, DSL, EDGE, High Latency DNS, LTE, Very Bad Network, WiFi, WiFi 802.11ac are largely inconclusive or, rather, tell us nothing at all about how to differentiate the abilities of competing SDKs. To really appreciate how competing SDKs perform under adverse network conditions the packet loss has to be successively negative. In a double uplink packet-in loss scheme where packets are dropped from the network at 10% increments, the packet loss is successively negative. To implement this type of scheme click “Add profile” in the Network Link Conditioner. Under name write in the name of the packet loss percentage as 10, 20, 30, etc… In packet loss add 10, 20, 30, etc… During testing toggle on and successively degrade network quality.
Agora vs. Twilio, TokBox, FaceTime
In testing Agora against Twilio, TokBox and FaceTime, the results were that these latter three SDKs fell off on or before 60% double uplink packet-in loss, whereas Agora continued to stream with little to no interruption well past 60% of the same. Agora’s stream is clear, stable, and continuous. The frames per second is rolling. There is no visible resolution degradation. Here is the video:
Our analysis, as in what we see at Agora, does not focus on specific network faults, uncovering detailed failures in resolution or frames per second but what the naked eye sees under visible network workability. In the four contrasted videos, both sets of which are competitor multi-party platform specific iOS demo applications, Agora’s SDK outperforms TokBox’s on, around or at 40 percent packet loss and it outperforms Twilio’s on, around or at 40 percent packet loss. FaceTime crashes on, around, or at the same percentage.
Do not let your lava lamp of interconnected application users, stop. If you haven’t adopted Agora yet, check out the following guides that help you switch from TokBox or Twilio. If you want to withstand adverse network conditions, chose Agora. Agora’s software is software that runs on a Software Defined Real-Time Network. Agora’s SD-RTN™ optimizes optimizes its uplink, downlink voice, video or live broadcasting traffic algorithmically..
Connect with us!
If you have any questions or would like to reach out to us at Agora directly, feel free to join our Slack channel agoraiodev.slack.com.