My Experience Building a Live Streaming App with Wowza GoCoder SDK for iOS

Building a live streaming capable application is not an easy feat. There are many moving pieces that make up an app like this. Master knowledge of AVFoundation, AudioToolbox, VideoToolbox, and other tools is required if you want to build your own audio and video encoders/decoders. Many of these frameworks are complete black boxes, especially VideoToolbox.

There are some third-party options that make all the hard work for you, but they’re not everything they advertise. We’re going to take a look at the GoCoder SDK by Wowza Media Systems. Wowza is a major provider for live streaming solutions. I’d like to mention that I’m not affiliated, endorsed or sponsored by Wowza. This SDK is proprietary, and no access to its source code is provided.

When I faced the challenge of building the live streaming portion of an app for a client, I evaluated some options on how to tackle the problem. The first one was to build my own encoders, but the deadline wouldn’t allow me to complete the task on time. They also had all their server-side infrastructure powered by Wowza, one of the reasons we decided that it was appropriate to use their mobile SDK for the app as well.

The Documentation

GoCoder comes with a complete API reference and some sample applications that demonstrate some of the SDK’s capabilities. They also have an app in the App Store that you can configure with your streaming server parameters and go live in minutes. Initially, this was very good, but I quickly found out that the examples were almost “Hello World” type of applications, and that the API reference doesn’t provide anything else apart from listing classes, properties, and methods. I was on my own.

Setting Up the SDK

Setting up the SDK is straightforward: you just pass properties to a configuration object with settings for video, audio, and the streaming server instance you need to connect to. Then assign this to a shared instance of GoCoder’s main class, call the startStreaming method with a callback delegate object, and you’re golden.

The Challenges

Through the course of development, I found out that implementing GoCoder brought a set of unique challenges, that unfortunately, not even their developers took into account when writing the SDK.

Providing a solid mobile live streaming experience is not easy. There are many variables in play that need to be considered, and the following are the ones you have to pay special attention to:

  1. Performance and battery life
  2. Connection stability
  3. Interruptions
  4. Wired and wireless headsets
  5. Background execution
  6. General error handling

If you run Wowza’s examples or use their own app in the App Store, the live streaming experience works only if following the happy path: no interruptions during your broadcast, no connection loss or bandwidth variations, no external accessories connected.

All of these scenarios led us to discover that the SDK was far from been tested by a QA team that considered everything our team did. I take a moment to thank my QA team for all their hard work in helping me discover and work around many of the issues that the SDK team didn’t.

About Performance

The SDK is performant and conserves battery well, considering that audio and video encoding are intensive tasks for the hardware. One of the problems we faced regarding this is that when starting and stopping a broadcast, threading issues made the app crash, even their samples and their own app suffer from this. We were “lucky” enough to get that particular issue resolved after 8 months of submitting a ticket with additional issues, that yes, as of the writing of this story, are not yet fixed by Wowza. DO NOT use any version of GoCoder below 1.5.1.x, they fixed the threading issue there.

Connection Stability

Mobile live streaming is very susceptible to networking conditions and connection quality. GoCoder provides a rudimentary adaptive bitrate and frame rate system that tries to accommodate for variations in bandwidth. It doesn’t work as expected all the time.

If you start a broadcast using WiFi and switch to cellular, get ready to implement your first workaround: the SDK doesn’t transition well and an error is thrown. It’s up to you to use a reachability listener to validate that the error thrown is indeed caused by a connection loss or change, and restart the broadcast.

Some streaming architectures suffer from this, since they provide simultaneous broadcast functionality, and a broadcast with the same stream name cannot be restarted once it ends. Yes, I was the lucky winner on this one too. Our server-side streaming team had to put a substantial amount of extra work to accommodate functionality to “resume” a broadcast if it was ended within a few seconds timeframe. Make a mental note about this resuming feature, it will be mentioned a lot in the next sections.

Interruptions

Get ready to be surprised about this one: ANY audio session interruptions will KILL the broadcast audio and it will never recover. An alarm, reminder, Siri, phone call, and pretty much anything that would trigger an AVAudioSessionInterruption will make your audio completely go away. This isn’t handled internally by the SDK in any way. When I submitted a ticket with the issue, the answer was to stop the broadcast and restart it after the interruption ended.

Any developer with basic knowledge of AVFoundation knows that a began interruption notification has no guarantee that will also receive an ended one. We’ve been lucky so far and all interruptions we’ve tested have sent both.

Basically, you MUST implement an AVAudioSessionInterruptionNotification observer and start/stop the broadcast accordingly. Hopefully, your streaming architecture won’t suffer from having to implement “resuming” logic on the server-side.

A special case has to be considered among all interruptions: phone calls. These CANNOT be treated as a regular AVAudioSessionInterruption, so you MUST implement logic to stop/start a broadcast using a CXCallObserver object from CallKit. This means that your app cannot support iOS versions below 10.0.

Wired and Wireless Headsets

You’d think these would work out of the box right? Well… NO! Connecting and disconnecting wired/wireless headsets in any combination will KILL your audio. Once again, Wowza’s solution was to “stop and restart the broadcast.”

For this one, you’ll need to implement your own AVAudioSessionRouteChangeNotification observer and stop/start the broadcast accordingly.

It’s worth mentioning that if you don’t configure an audio session using the .videoChat mode, forget about using Bluetooth headsets, they simply DO NOT WORK correctly with GoCoder. There are internal issues in the GoCoder audio engine, which is based in the AudioQueue services instead of AudioGraph. This is also the reason why all of the above audio issues are present.

There are tradeoffs of using the .videoChat mode: using the built-in mic will cause your broadcast volume to be lower than expected, This is due to the device’s tonal equalization and automatic gain control kicking in, and once this happens, you cannot set the input gain manually anymore.

To overcome that, you need to use the low-level AudioUnit API (in C) from AudioToolbox, and tap into the Voice Processing I/O unit. I’m still in the middle of doing that as of the writing of this story. I’ll update it later to shed some light on how to do it.

Background Execution

So far, you’ve been a good developer and implemented all the required handlers for networking, audio interruptions, and audio route changes. GoCoder provides a property that allows you to keep broadcasting in audio-only mode if, while live streaming, you send the app to the background. This sounds wonderful until you receive an interruption while in the background and your handlers attempt to start and stop the broadcast.

GoCoder’s startStreaming and endStreaming methods just go bananas when called in the background. By this time, your code looks horrible already with all of the above handlers. I decided that it’s just not worth the pain, and disabled background audio-only mode. Guess what I did? A start/stop cycle when notifications for UIApplicationDidEnterBackground and UIApplicationDidBecomeActive are fired.

You’ll also have to validate all of the above conditions for audio interruptions/route changes and phone call handlers with it. Flags everywhere!!!

General Error Handling

When GoCoder throws an error, you’d expect some error codes and messages describing what happened. Be ready to receive the exact same error messages for ALL errors. You missed or put an invalid value as your audio sampling rate? You get the same error message as if you got a connection loss. Let’s not forget the fact that there’s ZERO documentation on error codes in their API Reference, or anywhere.

You’ll have lots of fun with this, trust me, I barely slept and ate for 3–4 weeks trying to handle them, believing that I was the problem. In the bottom of my heart, I knew it wasn’t me, but found really hard to digest all of these findings and reporting them to QA and my CTO. I still find it hard to believe, and in some form, I feel scarred by it now. I spent all possible options inside my code to try to verify it wasn’t me, but them.

My Takeaways and Recommendations

By no means, GoCoder is a silver bullet for mobile live streaming on iOS. The framework needs to mature a lot more, to go through a lot more development, testing, and debugging. Don’t get me wrong, when it works, it produces beautiful live streams with amazing quality and device performance. This MUST be acknowledged, kudos for the team at Wowza for it. I only hope the product keeps improving, for us developers, and for our end users as well. GoCoder is a proprietary solution, and obtaining a license isn’t cheap. The least I’d expect is premium support because they advertise it as a premium product. We also paid good money for it.

Other Alternatives

There’s nothing better than open source software. It’s reliable, supported, and if something is broken, you can contribute and fix it yourself. Also, there’s transparency as to what’s happening and how things are done. I’ve recently been playing with an open source live streaming framework called HaishinKit. It looks promising and the developer is friendly and open to solve issues on GitHub. It might be a better alternative, but again, you’d need to evaluate it thoroughly to determine if it’s your best choice.

Let Me Help Share the Pain

If you or your company decide to build a live streaming app, I want you to know that I’m an expert implementing GoCoder/HaishinKit. I’ve already gone to hell and back with it, and know exactly what to do to make it sing. Don’t hesitate to contact me for additional information about my consulting services.

Image credit: Wowza Media Systems, LLC