Integrating C library in a Flutter app using Dart FFI
What would you do if you wanted your Flutter app to be as much performant as possible, or there was no package in pub that you need? Probably, you would use native Java/Objective-C libraries and communicate with them using Platform Channels. But what if both platforms share the same business logic? Surely, nobody wants to write the same code twice in two different languages.
The purpose of this article is to help Flutter developers set up their project to use a native C or C++ library and write a single code base that uses it. As an example, the article describes the process of adding the OpenCV library to a Flutter app project.
What is it all about?
Dart Foreign Function Interface (FFI) allows us to call native C/C++ functions on both iOS and Android platforms without using Platform Channels or making Java/Objective-C bindings. That leads to having a single code base written in C with zero platform-specific code.
OpenCV is a computer vision library that contains a lot of services used for image processing. It is mainly written in C++.
Creating a plugin
Even though it’s not required, it’s better to create a Flutter plugin to separate all task-specific stuff from the main application. Run flutter create — template=plugin native_opencv command to create it.
Next, we need to update the app’s dependency list in pubspec.yaml with a newly created plugin and ffi package:
dependencies:
native_opencv:
path: native_opencv
ffi: ^0.1.3ffi package comes in handy for working with C UTF-8/UTF-16 strings.
Setting up the plugin on iOS
OpenCV is distributed as a framework and it has to be included in the plugin along with C++ library. OpenCV also requires AVFoundation framework. Place the framework in plugin’s ios folder (or use a symbolic link) and add lines to .podspec file:
# telling CocoaPods not to remove framework
s.preserve_paths = ‘opencv2.framework’ # telling linker to include opencv2 framework
s.xcconfig = { ‘OTHER_LDFLAGS’ => ‘-framework opencv2’ } # including OpenCV framework
s.vendored_frameworks = ‘opencv2.framework’ # including native framework
s.frameworks = ‘AVFoundation’ # including C++ library
s.library = ‘c++’
If your library doesn’t come as a framework, you need to only include C++ library:
# including C++ library
s.library = ‘c++’Setting up the plugin on Android
Things are a bit harder for Android platform because it doesn’t support C++ out-of-the-box, but rather with a help of the Android Native Development Kit (NDK).
Native library and its headers to be included in the plugin’s android module. Shared libraries should be (but not necessary) placed in jniLibs folder. Headers can be placed anywhere (e.g., in the root folder of the plugin).
sdk/native/libs/* -> native_opencv/android/src/main/jniLibs/*
sdk/native/jni/include -> native_opencvLibraries placed in jniLibs folder are automatically included in apk. If you want to use any other directory, you have to specify in build.gradle of the plugin:
sourceSets {
main {
jniLibs.srcDirs = ["libs"]
}
}At this point plugin’s structure looks like this:
native_opencv
- android
- src
- main
- jniLibs
- arm64-v8a
- libopencv_java4.so
- armeabi-v7a
- libopencv_java4.so
- x86
- libopencv_java4.so
- x86_64
- libopencv_java4.so
- include
- opencv2
- ...In order to build and link the library and our code from native_opencv.cpp must be added to cmake (or ndk-build). Create CMakeLists.txt configuration file in android folder of the plugin:
cmake_minimum_required(VERSION 3.4.1)include_directories(../include)
add_library(lib_opencv SHARED IMPORTED)
set_target_properties(lib_opencv PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/src/main/jniLibs/${ANDROID_ABI}/libopencv_java4.so)add_library(native_opencv SHARED ../ios/Classes/native_opencv.cpp)
target_link_libraries(native_opencv lib_opencv)
And specify this config, as well as some flags and arguments in build.gradle:
android {
defaultConfig {
externalNativeBuild {
cmake {
// Enabling exceptions, RTTI
// And setting C++ standard version
cppFlags '-frtti -fexceptions -std=c++11'
// Shared runtime for shared libraries
arguments "-DANDROID_STL=c++_shared"
}
}
}
externalNativeBuild {
cmake {
path "CMakeLists.txt"
}
}
}If your library doesn’t provide shared libraries, you have to link it statically and use C++ static runtime.
Your C code has to be placed inside of plugin’s ios/Classes folder because CocoaPods doesn’t allow including source code from anywhere above .podspec file, though it’s not a problem for Android build system. Files placed in Classes folder are automatically compiled by the Xcode build system; therefore we don’t need to specify them explicitly.
Let’s create a .cpp file with some sample code:
To make sure OpenCV’s core, imgproc, imgcodecs modules are all linked and working, process_image function will process an image: load it, binarize it, find contours, draw contours on a binarized image and save it. version function will return a pointer to a string that represents OpenCV’s version.
Afterward, we need to “connect” with Dart code by binding it to native code. Create .dart file in the plugin:
Now it’s time to call these functions and check the result:
Yay, we can see OpenCV’s version and processed images now. There were no linking/compilation errors.
Giving UI thread a rest
Heavy processing might take some time (sometimes even dozens of seconds) and block the application’s UI thread, furthermore, an app stops responding to any user actions and rendering new frames. This problem is pretty easy to solve with dart:isolate library. A new isolate can be created to do some massive work (e.g., complex image processing). The user will still be able to interact with an app.
Let’s define Isolate’s entry point and arguments:
And add a simple flag to show a loader while an isolate is working, spawn it and wait for the signal:
To simulate actual heavy work image processing will iterate ten more times:
Adding platform-specific code
Sometimes there’s still need in adding platform-specific code. For example, I wanted to log code execution time, but Android sends stdout data to /dev/null, i.e., printf call gets ignored. To avoid that, the Android logging library has to be used.
Editing CMakeLists.txt to link a logging library:
find_library(log-lib log)target_link_libraries(native_opencv lib_opencv ${log-lib})
Android NDK defines __ANDROID__ flag by default, so it can be used in preprocessor directives to detect the platform:
And now logging works on both platforms.
Dart FFI is a powerful tool that gives an opportunity to use one of many C libraries and make your apps really fast while sharing code between different platforms. But keep in mind that dart:ffi is still in beta and missing some wanted features like support of nested structures or inline arrays.
Thank you for reading and feel free to leave any feedback. The source code for the app is available here:
https://www.twitter.com/FlutterComm
