How to build a Flutter app with C/C++ libraries via FFI on Android and iOS including OpenCV, Libsodium, Cmocka and Eigen

Flutter can greatly help developers in maintaining the same code base for developing both Android and iOS applications. However, plenty of Flutter plugins and packages for popular libraries that are originally produced in C/C++ language are either incomplete or not actively maintained by the Flutter community. Moreover, there is a lack of clear guidance and instructions on compiling C/C++ libraries into Flutter.

Therefore, this is a guide to help you port such C/C++ libraries into your Flutter app on both Android and iOS via dart:ffi with examples of porting OpenCV, Libsodium, Cmocka and Eigen from C/C++.

Setup

To build static libraries or XCFramework bundles on iOS, Xcode will be required. For both platforms, CMake, Ninja, Android SDK, Android NDK, JDK, Python 2, Python 3, and C compilers (Clang, Clang++, LLVM) are required.

For Flutter and Dart, a plugin is required to house all these static libraries which can be created following the official guide here.

Any Library with Android and iOS Build Instructions

Libraries with their own guides to build static libraries on Android and framework/XCFramework on iOS are the easiest to port from C/C++.

For Android, a successful static library build will result in 4 samplelib.a files built for arm64-v8a, armeabi-v7a, x86, and x86_64 Android ABIs along with its headers (.h) files. Following the guide stated in Setup to create the CMakeLists.txt file, we can link the libraries into the Flutter plugin by simply adding the following code in the CMakeList:

add_library(samplelib STATIC IMPORTED)
set_target_properties(samplelib PROPERTIES    INTERFACE_INCLUDE_DIRECTORIES ${CMAKE_CURRENT_SOURCE_DIR}/samplelib/include/
IMPORTED_LOCATION
${CMAKE_CURRENT_SOURCE_DIR}/samplelib/${ANDROID_ABI}/samplelib.a)
target_link_libraries(samplelib)

Note: Ensure in externalNativeBuild for cmake and minSdkVersion under defaultConfig in build.gradle are set correctly to avoid compile errors.

For iOS, a successful build will result in a .framework/.xcframework folder or iOS/iOS simulator .a static library with their headers. For the former case, add the folder into root/ios directory and add the following into podspec:

s.vendored_frameworks = 'samplelib.xcframework', 'samplelib2.framework'

Note: If Xcode has errors finding the frameworks, you can try adding it manually in Xcode as in this guide.

For the latter case, we will need to preprocess the .a files and headers into a .xcframework using the following command from here:

xcodebuild -create-xcframework -library <path> [-headers <path>] [-library <path> [-headers <path>]...] -output <path>

OpenCV with Contrib Full Build

Firstly, clone both OpenCV and OpenCV contrib,

git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git

For Android, set $ANDROID_HOME (SDK path) and $ANDROID_NDK_HOME (NDK path) in terminal and run the following:

python3 path/opencv/platforms/android/build_sdk.py \
--sdk_path $ANDROID_HOME \
--ndk_path $ANDROID_NDK_HOME \
--extra_modules_path path/opencv_contrib/modules/

Once done, the static libraries will appear in OpenCV-android-sdk/sdk/native/staticlibs/ and OpenCV-android-sdk/sdk/native/3rdparty/libs/ while header files are in OpenCV-android-sdk/sdk/native/jni/include/. We can then link OpenCV with an example CMakeLists here.

For iOS, we can run the following to get the xcframework:

python3 path/opencv/platforms/apple/build_xcframework.py \
--contrib path/opencv_contrib \
--iphoneos_archs arm64,armv7 \
--iphonesimulator_archs arm64,x86_64 \
--build_only_specified_archs \
-o opencv2.xcframework

Note: The instructions above are only tested on NDK18 and Android CMake 3.10.2 with OpenJDK 8 for Android and Xcode 13.2.1 for iOS.

To test whether OpenCV works, for both Android and iOS, add the following cpp file to check the number of pixels of an image.

flutter_ffi_examples/NativeOpencv.cpp at master · khaifunglim97/flutter_ffi_examples

To call the C++ function, we can write a simple FFI wrapper:

static late final _opencvImgPixelsPtr = nativeExampleLib.lookup<NativeFunction<Int32 Function( Pointer<Uint8>, Int32)>>('opencv_img_pixels');
   
static late final opencvImgPixels = _opencvImgPixelsPtr      .asFunction<int Function(Pointer<Uint8>, int)>();

Then, you can add the following code into example/lib/main.dart:

ByteData bytesData = await rootBundle.load('images/sample_photo.webp');
_preOpencvPixels = bytesData.buffer.asUint8List(        bytesData.offsetInBytes, bytesData.lengthInBytes);     
final _preOpencvPtr = _preOpencvPixels.allocatePointer();    
int numPixels = FlutterFfiExamples.opencvImgPixels(     _preOpencvPtr, _preOpencvPixels.length);    
calloc.free(_preOpencvPtr);

For an image with 800x600 resolution, you would expect an output of:

OpenCV pixel count sample output in Flutter via FFI

Libsodium

Firstly, clone the libsodium repository and run autogen.

git clone https://github.com/jedisct1/libsodium
cd libsodium
./autogen.sh -s

For Android, set $ANDROID_NDK_HOME and export LIBSODIUM_FULL_BUILD=1 for full build. Inside libsodium directory, run the android build scripts in the build-dist directory.

./dist-build/android-${ABI}.sh
# Where ABI = x86, x86_64, armv7-a, armv8-a 

This will output libsodium-android-${arch} where arch = i686, westmere, armv7-a, armv8-a+crypto respective to ABI that is to be copied into their respective x86, x86_64, armeabi-v7a, arm64-v8a and include folders following the instructions above.

For iOS, running the following build script results in libsodium-apple/Clibsodium.xcframework that can be copied into root/ios.

./dist-build/apple-xcframework.sh

Note: The instructions above are only tested on NDK23 and Android CMake 3.18.1 with OpenJDK 8 for Android and Xcode 13.2.1 for iOS

To test whether libsodium is correctly linked into the Flutter app, the following file can be added to randomly generate a number from libsodium:

flutter_ffi_examples/NativeLibsodium.cpp at master · khaifunglim97/flutter_ffi_examples

The FFI wrapper for the random generator function can be written as follows:

static late final _libsodiumRandomPtr = nativeExampleLib.lookup<      NativeFunction<Uint32 Function()>>('libsodium_random');   
static late final libsodiumRandom = _libsodiumRandomPtr      .asFunction<int Function()>();

Calling the libsodiumRandom() function will give the following output:

Libsodium random number generator in Flutter via FFI

Cmocka

Firstly, get the 1.1.5 version of Cmocka here and set $ANDROID_NDK_HOME. For Android, inside cmocka run the following script in terminal:

mkdir build
cd cmocka/build    
cmake -DCMAKE_SYSTEM_NAME=Android \   
-DCMAKE_ANDROID_ARCH_ABI=${ABI} \    
-DCMAKE_ANDROID_NDK=${ANDROID_NDK_HOME} \    
-DCMAKE_ANDROID_STL_TYPE=c++_static \    
-DWITH_STATIC_LIB=TRUE \    
-DCMAKE_INSTALL_PREFIX=$(pwd) \    
..
# Where ${ABI} = x86, x86_64, armv7-a, armv8-a
make && make install

An output of lib/libcmocka-static.a inside the build directory will be required to be copied into the respective Android directory format along with their include files as shown in the second section.

For iOS, we can create a new build folder and run the following script:

cmake -DCMAKE_SYSTEM_NAME=iOS \
"-DCMAKE_OSX_ARCHITECTURES=${ARCH}" \    
-DCMAKE_OSX_SYSROOT=${SYSROOT_PATH} \    
-DCMAKE_OSX_DEPLOYMENT_TARGET=9.0 \    
-DCMAKE_INSTALL_PREFIX=$(pwd) \    
-DCMAKE_IOS_INSTALL_COMBINED=YES \    
-DWITH_STATIC_LIB=ON \    
-DWITH_EXAMPLES=OFF \    
-DBUILD_SHARED_LIBS=OFF \    
-DCMAKE_C_FLAGS=-fembed-bitcode \    
..
# Where ${ARCH} = "armv7;arm64", etc
# ${SYSROOT_PATH} = "$(xcode-select -p) /Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk"

The command will needed to be repeated for both iOS and iOS simulators to have both static libraries (.a) files and header (.h) files. All of these files will then be required to build the xcframework following the xcodebuild command in section 2.

Note: The instructions above are only tested on NDK23 and Android CMake 3.18.1 with OpenJDK 8 for Android and Xcode 13.2.1 for iOS

To test whether Cmocka is correctly linked into the Flutter app, the following file can be added to perform a null test that is similar to Cmocka’s sample:

flutter_ffi_examples/NativeCmocka.c at master · khaifunglim97/flutter_ffi_examples

The FFI wrapper for the null test function can be written as follows:

static late final _cmockaNullTest = nativeExampleLib.lookup<      NativeFunction<Int Function()>>('cmocka_null_test');   
static late final cmockaNullTest = _cmockaNullTest      .asFunction<int Function()>();

Calling the cmockaNullTest() function will give the following output:

Cmocka null test in Flutter via FFI

Note: Further info of the test case can be found running in Xcode for iOS and logging via android/log.h for Android.

Eigen

Since Eigen is a header only file, there is no need for any compilations. Therefore, after cloning the Eigen repository, we can copy the entire Eigen directory inside the clone repository into Flutter plugin’s root/ios directory and into a new Eigen directory.

For Android, you would need to add the following into CMakeLists.txt for the header files to be included during build:

include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../ios/Eigen/)

For iOS, you would need to add the following into podspec for the header files to be included during build:

s.resources = "Eigen"  
s.xcconfig = { 'USER_HEADER_SEARCH_PATHS' => '"${PODS_TARGET_SRCROOT}/Eigen"/**' }

To test whether Eigen is correctly linked into the Flutter app, the following file can be added to create an Eigen matrix that is similar to Eigen’s first sample:

flutter_ffi_examples/NativeEigen.cpp at master · khaifunglim97/flutter_ffi_examples

The FFI wrapper for the print matrix function can be written as follows:

static late final _eigenMatrix = nativeExampleLib.lookup<      NativeFunction<Pointer<Utf8> Function()>>('eigen_matrix');   
static late final eigenMatrix = _eigenMatrix      .asFunction<Pointer<Utf8> Function()>();

Calling the eigenMatrix() function will give the following output:

Eigen print Eigen Matrix in Flutter via FFI

Other Libraries

With CMake build capabilities, you can follow the sample for building Cmocka for both iOS and Android and modify with CMake options following the CMake cross compiling guide in the official documentation.

Without CMake build capabilities, you can follow the samples in libsodium/dist-build and modify android-build.sh and apple-xcframework.sh to build Android and iOS. In general, the build commands of ./configure are to be replaced with the intended library build command in android-build.sh and apple-xcframework.sh along with the input and output names.

Header only C libraries, like Eigen, you can follow the instructions for integrating Eigen in Flutter above.

Own/Simple library, alternatively, instead of building static libraries, you can include all of the source files into root/ios/Classes, modify CMakeLists and directly port your C/C++ functions into Flutter manually or using ffigen.

Conclusion

This is a complete guide to build integrate popular C/C++ libraries like OpenCV, Libsodium, Cmocka and Eigen in your Flutter application.

A full example and further in depth instructions can be found in this Git repo.