Debugging audio glitches on Android
In this article I will explain how to identify the causes of audio glitches in your app using systrace and the Android Studio profiler.
Background: I wanted to build a one hundred oscillator synthesizer app (inspired by this amazing project from LOOK MUM NO COMPUTER). A perfectly reasonable thing to do, right?
I had already built an app which would use a single oscillator to produce a sine wave. I updated it to use one hundred oscillators and then summed the outputs of each one.
The result: horrible clicks and pops in the final audio aka audio glitches 😩
The problem is, each oscillator uses a few computations to generate its data, and with one hundred of them my CPU usage was enough to push me over my audio callback deadline.
In my case, each callback would request 96 frames of audio data for a 48kHz audio stream. This means I had 96/48000=0.002s or 2 milliseconds to generate 96 frames of data. So my app was probably taking longer than 2 milliseconds to generate those 96 frames, at least some of the time.
Well, that was my hypothesis, but how could I know for sure?
Measuring callback times
Enter systrace. Systrace is my #1 go to tool for debugging audio problems. It shows exactly what’s going on across processes and CPU cores. The key is knowing how to use it.
Here’s the command I use:
systrace.py --time=5 -o trace.html -a com.example.myapp audio sched freq
This will run a trace for 5 seconds, putting the output into trace.html. The trace will include audio buffer status, CPU frequency and CPU scheduling information.
Here’s the trace:
Again, knowing what to look for is the key. The thing I’m most interested in is the audio buffer status. This is shown in the aaRdy line for my app, which presumably stands for AAudio ready and shows the number of frames which are ready to be read from the AAudio internal buffer.
Incidentally, I’m using Oboe for my audio library which will use AAudio on devices which support it and OpenSL ES on everything else. If OpenSL ES was being used I would look for a row named fRdy.
Let’s zoom in on the aaRdy line:
By clicking on any of the individual slices we can see the number of audio frames which are in the buffer at that time. In the slice above there are 192 frames. I’m using a buffer size of 192 frames (which is twice the minimum buffer size and is usually a safe default), so the buffer is currently full.
But then the buffer starts empty, first dropping to 96 frames, then….oh dear…to zero! At zero we are guaranteed to have an audio glitch because there’s no data in the buffer.
Now, we could look at everything else that was going on in the system at this time and it’s likely we’ll see a CPU core migration, some other process come in and stomp all over our audio process or a CPU frequency drop causing a delay in execution of our oscillator’s code.
But we can’t really control any of that without jumping through some fairly dangerous hoops (setting thread affinity, CPU load stabilization etc).
Instead, let’s take a look at our own code and see what’s taking so long to execute. For this we can use the wonderful new native CPU profiler in Android Studio. Here’s the output from a Sampled (Native) trace, in the Flame Chart view:
Well, look at that, my app is spending an awful lot of time in sinf, a method calculating sine values. This is puzzling because sinf isn’t exactly computationally expensive and whilst 100 oscillators might sound like a lot, any modern CPU should be able to handle this load without breaking a sweat.
Unless the compiler didn’t optimise the code.
Compiler optimisation
So, how do we know whether the compiler optimised our build? Well there’s a simple way to check. Open the following file:
app/.externalNativeBuild/cmake/debug/<architecture>/build.ninja
Now look for a line starting with FLAGS, here’s mine:
FLAGS = -isystem /Users/donturner/Library/Android/sdk/ndk-bundle/sysroot/usr/include/aarch64-linux-android -D__ANDROID_API__=21 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security -std=c++11 -O0 -fno-limit-debug-info -fPICAnd there you have it -O0 no optimisations.
This actually makes perfect sense for a debug build because we might want to use step debugging and we don’t want the compiler to mess with the order of code execution.
But now we can try various different optimisation flags to see whether they affect performance. To do this I added the following line to my app/build.gradle file in the android.defaultConfig.externalNativeBuild.cmake block:
cppFlags "-Ofast"Then recompile and run. And the result was…..
No difference :(
I’ll be honest I was stumped at this point and ended up trying all sorts of different things: faster implementations of sin , switching to a square wave, copy-pasting code from stackoverflow, shouting at Android Studio. Nothing had any effect. Out of interest I re-checked my build.ninja file to see if there were any obvious flags I’d missed and to my amazement I found this in my build flags:
-Ofast -O0
That’s right. Android Studio, CMake and the NDK work together to override your build.gradle settings and the compiler will take the last optimisation flag, meaning -Ofast is completely ignored. This is a bug, and it comes from the file $ANDROID_NDK/build/cmake/android.toolchain.cmake.
To disable this behaviour I searched for ANDROID_COMPILER_FLAGS_DEBUG and commented out any code relating to optimisation flags.
Recompile, run and…..success!! The app runs like clockwork.
I can now listen to the sweet, sweet sound of the one hundred mega-drone synth with no horrible glitches 😃
UPDATE 21st May: One of my colleagues showed me a different workaround which doesn’t involve changing `android.toolchain.cmake`. All you need to do is add the following to `CMakeLists.txt`:
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -Ofast") Conclusion
If you’re experiencing audio performance problems, use systrace and the Android Studio profiler to figure out what your code is doing. And always check your build flags.
If you’re interested in building real-time audio apps on Android be sure to check out the Oboe library (full disclosure: I am one of the authors).
Got audio performance problems or questions? Would love to hear about them in the comments.
