Creating a Fluidsynth Hello World App for Android
Fluidsynth is the most popular software synthesizer in the world. It’s used in lots of music programs today, such as DAWs, MIDI sequencers, etc.. If you want to produce high-quality music from your applications, Fluidsynth is a must.
However, I struggled quite a bit to integrate Fluidsynth into Android. As surprising as it may sound, there wasn’t any Hello World example app around the Web to use as a guide. Creating one was difficult, with many missteps and stumbles along the way. So I decided to create such guide. Here are my findings, laid out in a clear step-by-step way.
If you get lost while following the steps, you can check the fully working GitHub repository at https://github.com/HectorRicardo/fluidsynth-android-hello-world.
- Download and extract the latest Fluidsynth Android and Windows64 binaries from https://github.com/FluidSynth/fluidsynth/releases. Yes, even though you’re developing for Android, you also need to download the Windows 64 zip file. You’ll see in a minute why.
If you open the android zip file, you will see that there are 4 folders inside it: arm64-v8a, armeabi-v7a, x86, and x86_64 . Each of these 4 folders represents an architecture of an Android device.
Let me explain. Fluidsynth is written in C language. Code that is written in C tends to be extremely efficient. And Fluidsynth has to be efficient because it is a software synthesizer: it’s synthesizing music sounds on the go, running lots of complex mathematical sine and cosine functions. Synthesizing music is not an easy task, so Fluidsynth had to be written in C so it ran quickly and didn’t consume too much system resources.
The disadvantage of C code is that it has to be compiled for every architecture that you want to run it on. So when you compile C code, you are only compiling it for a specific target architecture. In Android, there are 4 major architectures,arm64-v8a, armeabi-v7a, x86, and x86_64. That’s why you see folders for each of these architectures: Fluidsynth was compiled for each one of those.
If you open any folder, you will see lots of .so files (shared object files). Those files contained Fluidsynth’s compiled code for that particular architecture whose folder you just opened.
Let’s keep going…
2. Create a Native C++ project in Android Studio. Name the project “Fluidsynth Android Hello World”. Choose Java (instead of Kotlin) and use the default C++ standard toolchain.
3. Create a folder called fluidsynth under app/src/main/cpp .Then, create a subdirectory called lib under this new fluidsynth folder.
4. Now, move the extracted Android binaries (the 4 architecture folders) to the lib subdirectory.
Up to this point, you should have these files: app/src/main/cpp/fluidsynth/lib/${ANDROID_ABI}/all the .so files where ${ANDROID_ABI} stands for either arm64-v8a, armeabi-v7a, x86, orx86_64.
Remember, you can check the Github repository if you get lost in these steps (link at the top).
Ok. Time to confess something. I told you a lie in step 1. You don’t really need the Windows binaries. However, you do need the library header files (.h files).
Header files are necessary because they are the link between your application source code and the binaries of a library you want to use. In other words, if you don’t have the header files, you won’t be able to call the functions contained in the .so compiled code.
I don’t know why there were not included in the Android zip file (maybe the Fluidsynth team forgot to include them?). But good thing that you downloaded the Windows zip file.
5. Open the extracted Windows zip file, and copy the include folder to the app/src/main/cpp/fluidsynth folder in your app.
So now, the fluidsynth folder contains two subdirectories: lib and include. As we saw, the lib folder contains the binaries, or shared object files, and the include contains the header files which will be used to access the functions contained in the .so binaries. Note: it is a good practice to put all your header files into a folder called include.
Ok, cool. We have included Fluidsynth in our app. Time to play some music! Right?
Not so quick, pal. As I said before, Fluidsynth is written in C. However, Android code is in Java. How do you connect the dots ? Through something called JNI, Java Native Interface.
In a nutshell, JNI lets you mix C or C++ code with Java code. In order to call C code from Java code, JNI requires you to specify in Java which methods will have a native implementation (native implementation = implementation in C or C++).
If you created the Android project as a Native C++ project (step 2), then there’s already an example of JNI. If you open MainActivity.java, you will see two bits of important code there:
// Used to load the 'native-lib' library on application startup.
static {
System.loadLibrary("native-lib");
}..../**
* A native method that is implemented by the 'native-lib' native library,
* which is packaged with this application.
*/
public native String stringFromJNI();
The comments are pretty self-explanatory. Basically, the method stringFromJNI is implemented natively (in C++), and Java will load the native library (the native code) called native-lib to find the method’s implementation.
But where is this native-lib library located? It turns out that there’s a file called CMakeLists.txt that defines the native libraries that will be used in Java (along with their dependencies and instructions on how to compile this libraries). If you open it up, you will see the following snippet
add_library( # Sets the name of the library.
native-lib #line A # Sets the library as a shared library.
SHARED # Ignore this for now # Provides a relative path to your source file(s).
native-lib.cpp # line B
)
Line A specifies the name with which you’ll refer to your C++ code from Java. That name will be the argument of the System.loadLibrary method call. Line B specifies the name of the file in which your code is located. In this case, it’s in the file native-lib.cpp. Cpp extension means it’s a C++ source code file.
Now, if you opennative-lib.cpp, you will see this:
#include <jni.h>
#include <string>
extern "C" JNIEXPORT jstring JNICALL
Java_com_example_fluidsynthandroidhelloworld_MainActivity_stringFromJNI(JNIEnv* env, jobject) {
std::string hello = "Hello from C++";
return env->NewStringUTF(hello.c_str());
}There’s lots of JNI syntax here. I won’t delve into the details here, but it suffices to say that this is the the implementation of stringFromJNI method. There’s an optional challenge at the end of this guide where you will implement other methods in C++ using JNI syntax.
6. Back to Fluidsynth. As I said before, besides defining the native libraries, CMakeLists.txt also specifies the dependencies and instructions on how to compile such libraries. We therefore need to specify that our native C++ code will be calling Fluidsynth functions. I won’t delve too much into the details, but for that end, here’s the complete and finalCMakeLists.txt (we won’t modify the file anymore):
# For more information about using CMake with Android Studio, read the
# documentation: https://d.android.com/studio/projects/add-native-code.html
# Sets the minimum version of CMake required to build the native library.
cmake_minimum_required(VERSION 3.4.1)
# Create a variable fluidsynth_DIR to specify where the fluidsynth library is located.
set(fluidsynth_DIR ${CMAKE_CURRENT_SOURCE_DIR}/fluidsynth)
# Our code (native-lib.cpp) will be calling fluidsynth functions, so we add the fluidsynth binaries as dependencies of our code.
add_library(libc++_shared SHARED IMPORTED)
set_target_properties(libc++_shared PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libc++_shared.so)
add_library(libcharset SHARED IMPORTED)
set_target_properties(libcharset PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libcharset.so)
add_library(libffi SHARED IMPORTED)
set_target_properties(libffi PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libffi.so)
add_library(libFLAC SHARED IMPORTED)
set_target_properties(libFLAC PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libFLAC.so)
add_library(libfluidsynth SHARED IMPORTED)
set_target_properties(libfluidsynth PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libfluidsynth.so)
add_library(libfluidsynth-assetloader SHARED IMPORTED)
set_target_properties(libfluidsynth-assetloader PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libfluidsynth-assetloader.so)
add_library(libgio-2.0 SHARED IMPORTED)
set_target_properties(libgio-2.0 PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libgio-2.0.so)
add_library(libglib-2.0 SHARED IMPORTED)
set_target_properties(libglib-2.0 PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libglib-2.0.so)
add_library(libgmodule-2.0 SHARED IMPORTED)
set_target_properties(libgmodule-2.0 PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libgmodule-2.0.so)
add_library(libgobject-2.0 SHARED IMPORTED)
set_target_properties(libgobject-2.0 PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libgobject-2.0.so)
add_library(libgthread-2.0 SHARED IMPORTED)
set_target_properties(libgthread-2.0 PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libgthread-2.0.so)
add_library(libiconv SHARED IMPORTED)
set_target_properties(libiconv PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libiconv.so)
add_library(libintl SHARED IMPORTED)
set_target_properties(libintl PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libintl.so)
add_library(liboboe SHARED IMPORTED)
set_target_properties(liboboe PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/liboboe.so)
add_library(libogg SHARED IMPORTED)
set_target_properties(libogg PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libogg.so)
add_library(libsndfile SHARED IMPORTED)
set_target_properties(libsndfile PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libsndfile.so)
add_library(libvorbis SHARED IMPORTED)
set_target_properties(libvorbis PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libvorbis.so)
add_library(libvorbisenc SHARED IMPORTED)
set_target_properties(libvorbisenc PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libvorbisenc.so)
add_library(libvorbisfile SHARED IMPORTED)
set_target_properties(libvorbisfile PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libvorbisfile.so)
add_library(libz SHARED IMPORTED)
set_target_properties(libz PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/libz.so)
add_library(preloadable_libiconv SHARED IMPORTED)
set_target_properties(preloadable_libiconv PROPERTIES IMPORTED_LOCATION ${fluidsynth_DIR}/lib/${ANDROID_ABI}/preloadable_libiconv.so)
# Library that will be called directly from JAVA
add_library(native-lib SHARED native-lib.cpp)
# Specifies the directory where the C or C++ source coude will look the #include <yourlibrary.h> header files
target_include_directories(native-lib PRIVATE ${fluidsynth_DIR}/include)
# Link everything alltogether. Notice that native-lib should be the first element in the list.
target_link_libraries(
native-lib
libc++_shared
libcharset
libffi
libFLAC
libfluidsynth
libfluidsynth-assetloader
libgio-2.0
libglib-2.0
libgmodule-2.0
libgobject-2.0
libgthread-2.0
libiconv
libintl
liboboe
libogg
libsndfile
libvorbis
libvorbisenc
libvorbisfile
libz
preloadable_libiconv
)Build your Android project to check that there are no build errors up to now.
7. Go to native-lib.cpp and include fluidsynth.h header.
#include <jni.h>
#include <string>
#include <fluidsynth.h> // Add this lineNow fluidsynth is correctly integrated into our project. (You might need to rerun a Gradle build for Android Studio to recognize the fluidsynth.h header, or even a full restart). Let’s try adding some Fluidsynth code to test it out. I will adapt code obtained from http://www.fluidsynth.org/api/example_8c-example.html. (By the way, you can check that website to see all the commands -in C- that you can give to fluidsynth). I will also rename the method stringFromJNI to fluidsynthHelloWorld and changed its signature to not return anything (remember to do this change both in Java and C++).
#include <jni.h>
#include <fluidsynth.h>
#include <unistd.h>
extern "C" JNIEXPORT void JNICALL Java_com_example_fluidsynthandroidhelloworld_MainActivity_fluidsynthHelloWorld(JNIEnv* env, jobject) {
// Setup synthesizer
fluid_settings_t* settings = new_fluid_settings();
fluid_synth_t* synth = new_fluid_synth(settings);
fluid_audio_driver_t* adriver = new_fluid_audio_driver(settings, synth);
// Load sample soundfont
fluid_synth_sfload(synth, "wait a minute...what soundfont?", 1);
// succesfully loaded soundfont...play something
fluid_synth_noteon(synth, 0, 60, 127); // play middle C
sleep(1); // sleep for 1 second
fluid_synth_noteoff(synth, 0, 60); // stop playing middle C
fluid_synth_noteon(synth, 0, 62, 127);
sleep(1);
fluid_synth_noteoff(synth, 0, 62);
fluid_synth_noteon(synth, 0, 64, 127);
sleep(1);
fluid_synth_noteoff(synth, 0, 64);
// Clean up
delete_fluid_audio_driver(adriver);
delete_fluid_synth(synth);
delete_fluid_settings(settings);
}Don’t run this yet!
Notice two things: first, in order to play notes with Fluidsynth (with any software synthesizer in reality), you need to send a noteOn MIDI event, then sleep (wait) for some time (in this case, 1 second), and then send a noteOff MIDI event. If you don’t sleep for sometime, the note will be played and then instantly stopped, so no sound will be produced.
The second thing to notice is that we’re missing a soundfont. In order for Fluidsynth to produce sound, you need to specify a soundfont (aka soundbank). A soundfont is a file that contains thousands of samples (super small audio files) used to simulate the sound of musical instruments or other sound effects. There are lots of soundfonts around the Internet to synthesize really cool music.
8. Here’s a simple soundfont file you can use to test Fluidsynth: https://github.com/HectorRicardo/fluidsynth-android-hello-world/blob/main/app/src/main/assets/sndfnt.sf2. Add that file to your assets folder (app/src/main/assets/example_soundfont.sf2).
How do you make Fluidsynth use that soundfont? The problem with assets in Android is that everything you add as an asset, it gets compressed into the final APK, so you can’t access it directly as a file. In fact, it doesn’t even exist as a file, it exists as a series of bytes. In order to overcome this, we’re going to do the following 2 steps:
- First, from Java, we’re going to make a copy of the soundfont file to a temporary file we have access to.
- Second, we’re going to pass the name of this temporary file as a parameter to the C++ function, so Fluidsynth loads the soundfont into the synthesizer. This one’s tricky, because it implies to change the signature of the native method both in Java and C++, and some additional JNI syntax. I will add the code below
MainActivity.java
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
try {
String tempSoundfontPath = copyAssetToTmpFile("sndfnt.sf2");
fluidsynthHelloWorld(tempSoundfontPath);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private String copyAssetToTempFile(String fileName) throws IOException {
try (InputStream is = getAssets().open(fileName)) {
String tempFileName = "tmp_" + fileName;
try (FileOutputStream fos = openFileOutput(tempFileName, Context.MODE_PRIVATE)) {
int bytes_read;
byte[] buffer = new byte[4096];
while ((bytes_read = is.read(buffer)) != -1) {
fos.write(buffer, 0, bytes_read);
}
}
return getFilesDir() + "/" + tempFileName;
}
}
// add parameter to native method signature
public void String fluidsynthHelloWorld(String soundfontPath);native-lib.cpp
extern "C" JNIEXPORT void JNICALL Java_com_example_fluidsynthandroidhelloworld_MainActivity_stringFromJNI(JNIEnv* env, jobject, jstring jSoundfontPath) {
...
// Load sample soundfont
const char* soundfontPath =
env->GetStringUTFChars(jSoundfontPath, nullptr);
fluid_synth_sfload(synth, soundfontPath, 1);
...
}And we’re done ! If you run the app, you will hear a trumpet play 3 notes.
What’s next? Well, even though the app’s working and sound is coming through our device’s speakers, the code can be cleaned up. We can add additional JNI methods in our C++ code that correspond to each fluidsynth command you can do. I challenge you to do that. For that end, I recommend these two pages where you can read more about JNI
I will write soon another story where I show how to do this challenge. Until then, stay tuned…
