Building FFmpeg 4.0 for Android with Clang

Originally published at my blog here.

Preamble

This article is the continuation of my article about building FFmpeg for Android. If you haven’t read it, please, take a look here: Building FFmpeg for Android, since I will skip some basic stuff that already was described here.

In this article, I will concentrate on the specifics of building FFmpeg with Clang. GCC toolchain inside NDK has been deprecated for quite a while now and it’s highly advised to migrate to Clang (since GCC is not maintained).

Important notes

Performance and size

It’s reported that sometimes Clang tends to produce larger and less-performant binaries (libs) than GCC does. I haven’t noticed significant difference in case of FFmpeg, however if you would link FFmpeg with external library (like libx264 for instance), things might get ugly. Keep that in mind.

Assembler

It seems that both GCC and Clang distributed inside most recent (r17b, at the moment of writing) NDK have troubles with ASM code of FFmpeg, which unfortunately forced me to turn it off. Obviously, it will hurt the performance of library quite a bit, so if performance is critical for you — you might want to fall back to earlier versions of NDK and FFmpeg. However, I should aware you that even with enabled asm time of processing of larger videos with complex filters is still quite long. So if time is crucial for you — consider using MediaCodec instead. Hopefully, I will be able to ship article explaining basic processing in a few weeks.

Development Kit

As with previous article, this one ships along with an update to FFmpeg-Development-Kit, which should ease the process of building FFmpeg for mobile platforms. If you not actually interested in digging into details of building and would rather get ready .so files and template to work on — you can skip rest of the article and proceed directly to the GitHub.

Host OS

While it is technically possible to build FFmpeg on Windows — I highly recommend you to use Linux or OSX as the host system. If you only have Windows machine — consider installing Linux as the second OS or use VM. Building of open-source projects on Windows always tends to be painful and cross-building is usually twice as painful. So unless you have time to deal with all kinds of strange errors happening all around you — consider using *nix based machine for the building process.

Setup

Downloading and extraction

First things first: you will need NDK and FFmpeg. I used r17b and 4.0.2 versions respectively. You’re free to use any versions you would like to, however, it might fail to build. FFmpeg is actively developed and NDK is actively developed as well, so there is always a possibility of conflicting changes that would ultimately result in failed integration. In case you run into such case — fallback to the earlier versions. Download NDK from here and FFmpeg from here. Note that Android Studio allows you to download NDK in SDK Manager, however, I personally prefer to use separate distribution and suggest you do the same.

After you’ve downloaded NDK and FFmpeg, unpack NDK to the disc and make sure to put FFmpeg under NDK/sources directory (so it would be something like android-ndk-r17b/sources/ffmpeg).

Android project

Usually, you want to use libraries directly in some project with JNI. I tend to place such project in the same folder where NDK is located (that allows to automate the building process slightly better). A bit more about this in Building section.

Change in FFmpeg sources

Important note: I highly recommend you to don’t change code or files of FFmpeg unless you absolutely have to. Unfortunately, I actually had to do few changes manually, those changes are listed and explained below.

libavdevice/v4l2.c

Open this file and find the following line:
int (*ioctl_f)(int fd, unsigned long request, ...);

to

int (*ioctl_f)(int fd, unsigned int request, ...);

This change is required to avoid conflict with ioctl function inside of NDK, which have the different signature.

configure

Open configure script with any text editor and navigate to line #5021 or:
SHFLAGS='-shared -Wl,-soname,$(SLIBNAME)'
inside “android)” case in “case $target_os in” switch.
Change the line to:
SHFLAGS='-shared -soname $(SLIBNAME)'

It has to deal with linker and soname of the library being created. Essentially, it seems that FFmpeg should pass flags to CC (which is Clang in our case), however, it actually passes those flags directly to the linker which causes the linker error. To resolve that issue we fix the flags assuming that they’re passed directly to the linker.

fftools/ffmpeg.c

Navigate to line #4852 or:
exit_program(received_nb_signals ? 255 : main_return_code);
and change it to:
ffmpeg_cleanup(received_nb_signals ? 255 : main_return_code);

FFmpeg is command-line program essentially and it has the tendency to call exit(…) or abort() functions under the hood to close the program and it actually places one call to exit(…) just before the program would end normally with “return main_return_code;”. And if FFmpeg is used as binary everything is totally fine, however, in the case of Android native code calling any of those functions will cause a silent crash of the application with no message. In order to avoid that, we actually just calling the cleanup function, without the call to exit(…).

Building

Clang vs GCC

There are few differences between Clang and GCC. First of all, GCC cross compiler is always a binary specially prepared to target a specific platform. If you look into “toolchains” folder inside NDK you will notice a lot of folders like “$arch-4.9”. Each of those folders contains specific GCC toolchain for a particular architecture. Contrary to GCC, Clang has only one binary, which is used to build any of the supported targets.

Script

I highly recommend you to use Bash script for the building. You, obviously, can enter all commands in terminal directly, however, a script will allow you to iteratively test different configurations and persist your work across multiple days (and you want that, believe me).

Variables

Let’s start with setting up variables:

DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # get script directory
PARENT="$(dirname "${DIR}")" # parent directory
NDK="$(dirname "${PARENT}")" # NDK directory
PROJECT="$(dirname "${NDK}")/{project_name}/app" # place your project name instead of {project_name}
PROJECT_JNI="$PROJECT/jni"
PROJECT_LIBS="$PROJECT/libs"
SYSROOT="$NDK/sysroot" # sysroot for the cross-building
HOST= # place your host system here, like "darwin-x86_64" for OSX
LLVM_TOOLCHAIN="$NDK/toolchains/llvm/prebuilt/$HOST/bin" # location of clang binary

Note that this script expects Android project to be located in the same folder where NDK is located. It’s not a strict requirement, you can use any path you want. If you want to use a custom path make sure to set variable PROJECT to your project “app” folder.

CC and linker flags

Next, let’s define common flags for CC and linker:

CFLAGS="-O3 -fPIC"
LDFLAGS="-lc"

-O3 flag tells the compiler to try to optimise generated code to be faster, however resulting binaries/libraries might be larger in size. If size is crucial, set the flag to -Os, which will optimise for size by the cost of speed.

-fPIC tells the compiler to generate Position Independent Code, which is required by Android security standards (non-fPIC libs will not be loaded on Android 6.0 and higher).

-lc flag is passed to the linker and tells linker to search for libc.

Building function

When I was initially integrating FFmpeg I wrote 6 scripts (5 for every architecture and 1 global). However, after reviewing all the paths and configurations I decided that it will be better to create a function, which will mangle parameters by only a few input arguments.

Let’s start the definition:

function build {
    ARCH=$1
    LEVEL=$2
    CONFIGURATION="--disable-asm --enable-cross-compile --disable-static --disable-programs --disable-doc --enable-shared --enable-protocol=file --enable-pic --enable-small $3"
    LIB_FOLDER="lib"

This function expects architecture (as the string, with following values: armeabi-v7a, arm64-v8a, x86, x86_64) as first argument (saved in ARCH variable) and platform level as the second argument (as the string, the value must be in following ranges: 14–19, 21–24, 26–28) saved in LEVEL variable.

CONFIGURATION contains common flags passed to the FFmpeg configure script. All are pretty much self-explanatory.

LIB_FOLDER essentially is a workaround for the strange decision of NDK team to put lib files for all architectures inside “lib” folder except for x86_64 (libraries for this architecture for some reason reside in “lib64” folder).

Handling architectures

Next, let’s handle each particular architecture by creating switch for $ARCH:

case $ARCH in
esac

armeabi-v7a

Let’s start from “armeabi-v7a”:

"armeabi-v7a")
    TARGET="arm-linux-androideabi"
    CC_FLAGS="-target thumbv7-none-linux-androideabi -mfpu=vfpv3-d16 -mfloat-abi=soft"
    LDFLAGS="--fix-cortex-a8 $LDFLAGS"
    PLATFORM_ARCH="arm"
    TOOLCHAIN_FOLDER=$TARGET
;;

TARGET contains the string of target which is used by NDK in internal paths. It will be different for each architecture.

CC_FLAGS contains flags passed to the CC (Clang in our case). We’re interested in -target flag since it defines target we’re compiling for and specifying the wrong target is going to produce incompatible code and thus build will fail.

LDFLAGS — we’re adding –fix-cortex-a8 as a workaround for the bug in Cortex A8 CPU.

PLATFORM_ARCH — contains the string to create the correct path to the platform for particular architecture.

TOOLCHAIN_FOLDER — is a workaround for x86 architecture toolchain (folder actually called “x86”, while TARGET is “i686-linux-android”, for all other architectures this folder name equals target name).

arm64-v8a

"arm64-v8a")
    TARGET="aarch64-linux-android"
    CC_FLAGS="-target aarch64-none-linux-android -mfpu=neon -mfloat-abi=soft"
    PLATFORM_ARCH="arm64"
    CONFIGURATION="$CONFIGURATION --disable-pthreads"
    TOOLCHAIN_FOLDER=$TARGET
;;

There is the only thing that is really different from armeabi-v7a and it is appending –disable-pthreads to CONFIGURATION. It’s needed because linker for arm64 does not support pthreads command passed by FFmpeg and it produces broken libraries (with “read” in soname).

x86

"x86")
    TARGET="i686-linux-android"
    CC_FLAGS="-target i686-none-linux-androideabi -mtune=intel -mssse3 -mfpmath=sse -m32"
    PLATFORM_ARCH="x86"
    TOOLCHAIN_FOLDER=$PLATFORM_ARCH
;;

Note difference in TOOLCHAIN_FOLDER definition. As mentioned x86 toolchain folder name doesn’t match target name, thus we need to define it to have the correct value.

x86_64

"x86_64")
    TARGET="x86_64-linux-android"
    CC_FLAGS="-target x86_64-none-linux-androideabi -msse4.2 -mpopcnt -m64 -mtune=intel"
    PLATFORM_ARCH="x86_64"
    LIB_FOLDER="lib64"
    TOOLCHAIN_FOLDER=$PLATFORM_ARCH
;;

Note LIB_FOLDER redefinition. As was mentioned above for some reason only x86_64 have its libs in “lib64” folder, not in “lib” folder as every other architecture.

Building toolchain variables definition

TOOLCHAIN=$NDK/toolchains/$TOOLCHAIN_FOLDER-4.9/prebuilt/$HOST/bin
CC=$LLVM_TOOLCHAIN/clang
CXX=$LLVM_TOOLCHAIN/clang++
AS=$CC
AR=$TOOLCHAIN/$TARGET-ar
LD=$TOOLCHAIN/$TARGET-ld
STRIP=$TOOLCHAIN/$TARGET-strip

TOOLCHAIN is a base path to GCC toolchain (Clang falls back to GCC LD and other components).

CC — path to Clang (C compiler).

CXX — path to Clang++ (C++ compiler).

AS — assembler, we will use Clang as assembler as well.

AR — binary for working with archives (packing/unpacking).

LD — linker.

STRIP — binary for stripping unnecessary symbols.

FFmpeg configure

Let’s finally configure the FFmpeg itself:

PREFIX="android/$ARCH"
./configure --prefix=$PREFIX \
$CONFIGURATION \
--ar=$AR --strip=$STRIP --ld=$LD --cc=$CC --cxx=$CXX --as=$AS \
--target-os=android \
--extra-cflags="$CC_FLAGS -I$SYSROOT/usr/include/$TARGET $CFLAGS" \
--extra-ldflags="-L$NDK/toolchains/$TOOLCHAIN_FOLDER-4.9/prebuilt/$HOST/lib/gcc/$TARGET/4.9.x -L$NDK/platforms/android-$LEVEL/arch-$PLATFORM_ARCH/usr/lib $LDFLAGS" \
--sysroot=$SYSROOT --extra-libs=-lgcc

We start by defining PREFIX variable which will define the destination folder for built libraries (android folder in our case).

We pass include directory in –extra-cflags and we pass actually two lib directories in –extra-ldflags. First one is for libgcc and the second one is platform one, which contains implementations of functions specific to the platform and arch.

Note that -lgcc actually passed to the –extra-libs, which make FFmpeg append it to the end of the building command. It’s a bit strange how it works, because Clang generates code that uses GCC built-in functions, but doesn’t provide implementations for them, which requires us to add libgcc manually.

Building of .so files

In order to avoid possible problems, we’re going to use “make” that is distributed inside the NDK:

$NDK/prebuilt/$HOST/bin/make clean
$NDK/prebuilt/$HOST/bin/make -j2
$NDK/prebuilt/$HOST/bin/make install

Make that usually distributed with *nix systems would work fine in most cases, but as an extra precaution, it’s better to use NDK’s one.

Final touches

There is final step left: we need to set up proper module for the NDK and do actual ndk-build of our JNI to create the final library that will use FFmpeg:

export NDK=$NDK
export ARCH=$ARCH
export PLATFORM="android-$LEVEL"
export NDK_PROJECT_PATH=$PROJECT
yes | cp -rf Android.mk "$PREFIX/Android.mk"
$NDK/ndk-build
if [ ! -d "$PROJECT/out" ]; then
mkdir -p "$PROJECT/out"
fi
yes | cp -rf "$PROJECT_LIBS/$ARCH" "$PROJECT/out/$ARCH"}

First of all, we’re exporting variables to make them global (i.e. accessible in ndk-build and according *.mk files). Note NDK_PROJECT_PATH variable which is pointing to the project, which will be built.

Then we force copy Android.mk to the configure destination folder (for details check out the previous article and/or GitHub repo).

Then we perform actual ndk-build.

As a final touch, we check if “out” folder exists, create it, if it doesn’t, and then copy built set of libraries from project “libs” folder to the “out” folder.

Running the function

Now, after the function is ready, we just need to call it for every architecture. It’s quite trivial:

build "armeabi-v7a" "14"
build "arm64-v8a" "21"
build "x86_64" "21"
build "x86" "14"

Note that building can take quite some time, so you can make yourself a cup of tea or coffee, while you waiting for it to finish.

Bonus!

This section primarily for the people, who interested in additional details or tricks that might help in the process of FFmpeg integration.

Redirecting stdout and stderr

FFmpeg (as pretty much any library) prints its logs into either stdout or stderr. By default, both of them are redirected to /dev/null, which means that this information is not accessible.

In theory, in particular case of the FFmpeg you can declare custom logging function and set it as primary via “av_log_set_callback”, however, for some reason it was causing SIGSEGV (it seems that somehow strings are losing null-terminator and it was causing memory access error), so I implemented another solution based upon this post: How to use standard output streams for logging in android apps.

I haven’t really changed much, except for the fact that I already had logging header (logjam.h), so I created C file (logjam.c) and exposed function:

int start_logger(const char *app_name);

in the header.

Also, the original post was missing headers, so here are headers for the .c file:

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <pthread.h>

I’m calling this function in:

jint JNI_OnLoad(JavaVM* vm, void* reserved)

to spawn redirection thread at the very start of the FFmpeg usage.

Note that FFmpeg by default logs a lot of information, so you might want to set log level to a lower one.

Diagnosing FFmpeg build errors

When you’re trying to experiment with FFmpeg configuration (to make binary smaller, for example), you usually get a lot of errors and need a lot of trials to make everything work.

In order to diagnose what exactly went wrong in building process, navigate to ffbuild/config.log. This file contains logs of configure process and most of the time you’re going to find the cause (or at least lead to the cause) of the problem here.

Usually, if you were able to successfully build FFmpeg with NDK toolchain, ndk-build itself should not bring problems. However, sometimes you can stumble upon undefined symbols problem. If there is such a problem — make sure to check soname of the library and symbols presence with the readelf tool. Also, make sure that target triple is correct: if you built FFmpeg for the wrong triple — ndk-build will not recognise library files and load them.

Summary

It’s not an easy thing to integrate native code in Android. It always has a plenty of pitfalls and hidden problems, which quite frequently appear only in runtime on specific architecture or even device.

However, I hope that this article will make at least process of building and integrating itself easier, than it was for me.

On this positive note, I wish all readers good luck with Android development and waiting to see you in the next articles!