Play any sound in an Android or iOS app with JUCE (1/3)

9 min readMar 10, 2023

Dealing with sound in a mobile app is a full time job by itself. There are plenty of solutions, going from basic but limited ones to more versatile ones that are often a burden to implement, especially when the documentation is poor or almost non-existent.

Story of my life

I worked for quite a long time on a streaming music app dedicated to party. That app unfortunately didn’t make it to production but gave me the opportunity to test and implement some solutions that allow me to play music the way I wanted (and I could test it during some parties and that was fun!).

But what do I mean by “the way I wanted”? Since that app was capable of making transitions between music tracks like a real DJ, I needed to:

get the lowest level access possible to the audio waveform
process the audio signal in near real-time (for example to speed up or slow down tracks, synchronize them, filter them, …)
redirect the resulting audio signal to the audio output, again in near real-time

So I first implemented everything using OpenSL ES and the Ogg Vorbis libraries. It’s a bit old now, but you can find how I generated those libraries for Android here. I won’t lie to you, that was tricky. But it worked!

But then I decided to port that app to iOS. Unfortunately, OpenSL ES didn’t seem to be available on iOS (or at least the code that I had managed to get to work), so I had to make some research again…

And I found JUCE. JUCE is a framework for audio application and plug-in development, written in C++, that you can use in many ways. But we will focus here on how to make a JUCE library, how to implement it and how to play a sound with it, on both Android and iOS.

Let’s get started

JUCE is definitely an awesome tool. Unfortunately, the official documentation doesn’t explain how to implement it on an existing Android or iOS app. And it took me ages and way too much pain before I managed to get it to work.

You’ll have a genuine smile in your face the first time it works on your app, I guarantee it! But be careful and don’t miss any step, otherwise you’ll have to deal with obscure error messages that can drive you mad for way too long!

So to make it as clear as possible, that (long) tutorial will be divided in 3 parts:

How to create and export a C++ JUCE library for Android and iOS
How to implement that library in an Android or iOS app
How to play a basic sound in C++ using JUCE in an Android or iOS app

If you don’t know how to call C++ code from Kotlin or Swift, or just need a small refresh, you should definitely check my article about it first, especially since I’ll start from there.

How to create a C++ JUCE library

We start by first installing Projucer. Projucer is a basic IDE that allows you to generate standalone apps, plugins and libraries. We use it since it does quite a lot of work for us when it comes to creating C++ libraries.

Projucer

In order to properly create a C++ library project for both Android and iOS, carefully follow the steps below:

First, create a new project. A new window opens.
On the left, choose Static Library.
On the right, for Project Name, enter a name for your library project. For example: juceaudio.
Below, for Modules, select:
- juce_audio_basics
- juce_audio_devices
- juce_audio_formats
- juce_core
- juce_data_structures
- juce_events
Below, for Exporters, select:
- Xcode (iOS)
- Android
Click on Create Project…
Choose the location you want for your project.

Now that the project is created, we still need to set it up. First, you should see an interface that looks like this:

First go to the Project Settings by clicking on the small purple wheel on the top right of File Explorer. You should see an interface like the one above.
You can enter the information you want, like the Company Name for example.
A little further down you should find Use Global AppConfig Header. Put it to Enabled.
And for Bundle Identifier, enter the package name (or bundle identifier) of your Android (or iOS) app. It will be easier if you have the same package name for both your Android and iOS versions of your app. Here I entered: com.juceaudio.juceaudiodemo.
Now on the left of your screen, click on Exporters > Android.
Scroll down and search for Minimum SDK Version, and enter 21. From now on, we assume that the minSdkVersion of your app is 21.
Right below, you should find Target SDK Version. Enter 33, or even higher if a newer version of Android is available.
Now on the left of your screen, click on Exporters > Xcode (iOS) > Debug.
Search for iOS Deployment Target and enter 12.0.
Now on the left of your screen, click on Exporters > Xcode (iOS) > Release.
Again, search for iOS Deployment Target and enter 12.0.
Save the project (File > Save).

Some basic notions

Before we start coding, I just want to clarify some basic notions and vocabulary that I’ll use from now on.

A sound can be represented by a waveform like the one below:

Each sound played by an app is not a curve, but rather a representation of it, most of the time an array of float values (in rarer cases integers), where each float can have a value between -1 and 1. We say that the sound is sampled.

Each float value is called a sample, and simply represents the amplitude of the sound at a specific moment in time.

The sound above is a mono sound, but most of the time, we deal with stereo sounds. We can then represent that sound in multiple ways, including:

two array of float values, each one representing a mono sound like the one above
one array of float values, where the first float value represents the first sound amplitude on the left, the second float value the first sound amplitude on the right, and so on (left, right, left, right, … you get it)

I’ll use the second option but both are fine.

Also, since we will deal with stereo sound for our project, I’ll also use the term frame. A frame is simply a couple of samples, one for the left and one for the right, at the same moment in time.

Finally, depending on the desired sound quality, we may want to play more or fewer frames every second. For example, when you play a CD, the sound is played at 44100 Hz, which means that 44100 frames are played every second (or 44100 samples for each channel). That what we call the sample rate. On current devices, the sample rate is usually 41000 Hz or 48000 Hz, but in rarer cases, it can go higher.

Let’s code the library

Now we can code the library. First things first: Always add or remove files using Projucer, in the File Explorer section. Never ever use Android Studio or Xcode. Note that this rule only applies to our library code.

The only purpose of the code below is to redirect audio samples to the audio output of the device.

But before we dive into it, some important things to know:

The audio frames are not redirected one by one to the audio output but rather by blocks, usually a few hundreds of them. So our app’s job is to provide, when asked by our library, a block of frames of the desired size (which will also be provided by the library). We will see an example of it later.
Never assume that the block size is always the same overtime. Surprisingly, that size vary more often than you could think. So we will always take that size into account each time our app needs to provide a new block of frames, even though it could make things a bit trickier.
To avoid any naming conflict, we can use our own namespace (in the example below, I use jad, like Juce Audio Demo).

OK so first, I created a listener, that we will use in our app to know when the library needs us to provide audio frames, but also set up our app or release resources:

#pragma once

namespace jad
{
    class AudioSamplePlayerListener
    {
        protected:
            AudioSamplePlayerListener() = default;
            ~AudioSamplePlayerListener() = default;
        public:
            virtual void prepareToPlay(int samplesPerBlock, double sampleRate) = 0;
            virtual void releaseResources() = 0;
            virtual float* getNextAudioSamples(int numberOfFrames, int numberOfOutputChannels) = 0;
    };        
}

Then I created an AudioSamplePlayer class:

the header:

#pragma once

#include "JuceHeader.h"
#include "AudioSamplePlayerListener.h"

namespace jad
{
    class AudioSamplePlayer : public juce::AudioSource
    {
    public:
        AudioSamplePlayer();
        void setListener(AudioSamplePlayerListener *listener);
        void setupPlayer();
        void closePlayer();
        void prepareToPlay(int samplesPerBlock, double sampleRate) override;
        void releaseResources() override;
        void getNextAudioBlock(const juce::AudioSourceChannelInfo& bufferToFill) override;
    private:
        AudioSamplePlayerListener *listener = nullptr;
        std::unique_ptr<juce::AudioSourcePlayer> player;
        juce::AudioDeviceManager deviceManager;
        JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR(AudioSamplePlayer)
    };
}

the class itself:

#include "AudioSamplePlayer.h"

namespace jad
{
    AudioSamplePlayer::AudioSamplePlayer()
    {
        player = std::make_unique<juce::AudioSourcePlayer>();
        player->setSource(this);
        deviceManager.addAudioCallback(player.get());
    }
    
    void AudioSamplePlayer::setListener(AudioSamplePlayerListener *listener)
    {
        this->listener = listener;
    }
    
    void AudioSamplePlayer::setupPlayer()
    {
        deviceManager.initialiseWithDefaultDevices(0, 2);
    }
    
    void AudioSamplePlayer::closePlayer()
    {
        deviceManager.closeAudioDevice();
    }
    
    // override
    void AudioSamplePlayer::prepareToPlay(int samplesPerBlock, double sampleRate)
    {
        if (listener != nullptr)
        {
            listener->prepareToPlay(samplesPerBlock, sampleRate);
        }
    }
    
    // override
    void AudioSamplePlayer::releaseResources()
    {
        if (listener != nullptr)
        {
            listener->releaseResources();
        }
    }
    
    // override
    void AudioSamplePlayer::getNextAudioBlock(const juce::AudioSourceChannelInfo& bufferToFill)
    {
        if (listener == nullptr) return;
        int outputChannelsNumber = bufferToFill.buffer->getNumChannels();
        
        // get the audio samples to redirect to the audio output (numSamples represents the total number of frames needed to be played):
        auto* samplesInline = listener->getNextAudioSamples(bufferToFill.numSamples, outputChannelsNumber);
        
        // map the block of audio frames stored in samplesInline to the audio output:
        for (auto channel = 0 ; channel < outputChannelsNumber ; channel++)
        {
            // get a pointer to the start sample in the buffer for this audio output channel :
            auto* buffer = bufferToFill.buffer->getWritePointer(channel, bufferToFill.startSample);
            
            // fill the required number of samples :
            for (auto a = 0 ; a < bufferToFill.numSamples ; a++)
            {
                buffer[a] = samplesInline[a * outputChannelsNumber + channel];
            }
        }
    }
}

The two main important things here are:

prepareToPlay(int samplesPerBlock, double sampleRate). This function will automatically be called once when we call setupPlayer() and will provide us one useful piece of information: the sample rate of the device.
getNextAudioBlock(const juce::AudioSourceChannelInfo& bufferToFill). This function will automatically be called many times as soon as we call setupPlayer(). As you might guess, it will actually be called each time our library needs more frames to play.

How to export a C++ JUCE library

Let’s see now how to export our library for both Android and iOS.

Export for Android

In Projucer:

On top of the interface, you’ll find Selected exporter. In the drop-down list, choose: Android.
Click on the icon on the right to open Android Studio.

In Android Studio:

Click on the SDK Manager icon, the Preferences window opens.
Click on the SDK Tools tab and update (or install) NDK and CMake.
Back to the main interface, click on the Build Variants tab on the bottom left, and choose the desired Build Variant: debug_Debug to export the library for development, release_Release for production.
Build the library by clicking on the Make Project icon (or Build > Make Project).
The library will be available for 4 CPU architectures: arm64-v8a, armeabi-v7a, x86 and x86_64. You’ll find the 4 versions of that library in the following directories (<Project> is the location of the library project, <auto_gen_dir> is an auto generated directory that looks like 3z234567):
- debug :
<Project>/Builds/Android/lib/build/intermediates/cxx/Debug/<auto_gen_dir>/obj/
- release :
<Project>/Builds/Android/lib/build/intermediates/cxx/RelWithDebInfo/<auto_gen_dir>/obj/

Export for iOS