Integrating C library in a desktop Flutter app using Dart FFI
This article is a follow-up to my previous article where I discussed the integration of a C++ OpenCV library in a mobile Flutter application on Android and iOS.
It’s been a year and things have changed since then. The new second major version of a framework has been released. Flutter 2.0 introduced not only a stable release of Dart FFI and long-awaited null-safety support, but also desktop support, so now Flutter apps can be built for 3 more platforms — Windows, macOS, and Linux. Even Canonical itself endorses Flutter nowadays.
And again, the purpose of this article is to help Flutter developers set up their project to use a native C or C++ library and write a single code base that uses it. As an example, the article describes the process of adding the OpenCV library to a Flutter app project. But this time — it’s all desktop: Windows, macOS, and Linux.
What is it all about?
If you have no idea what Dart FFI is and why do you need it, please, refer to my previous article. In this one, I’ll assume you have a project set up and ready (at least, the Dart part of it), and the same C++ source file will be used. Or you can just use the git repository.
Creating a plugins
First of all, make sure you have both the framework and the project set up for the Flutter desktop. The instructions are available on the official website.
A desktop Flutter plugins have to be created separately for each platform. Let’s create them:
flutter create --template=plugin --platforms=windows native_opencv_windows
flutter create --template=plugin --platforms=linux native_opencv_linux
flutter create --template=plugin --platforms=macos native_opencv_macosThe project’s structure should look like this now:
PROJECT_ROOT
... - native_opencv
- native_opencv_linux
- native_opencv_macos
- native_opencv_windows
Our main plugin can now include each platform’s plugin, so including a single main plugin in a project will lead to including plugins for every platform. This kind of plugin is called an endorsed federated plugin. To make it work, update a pubspec of the main plugin (native_opencv):
flutter:
plugin:
platforms:
android:
package: com.example.native_opencv
pluginClass: NativeOpencvPlugin
ios:
pluginClass: NativeOpencvPlugin
macos:
default_package: native_opencv_macos
linux:
default_package: native_opencv_linux
windows:
default_package: native_opencv_windowsSetting up the plugin on Windows
OpenCV for Windows is distributed as a self-extracting .exe archive that contains pre-built .dll libraries. After extracting it, set up an environmental variable OpenCV_DIR that leads to the build folder of the library:
(If you use CMD) setx -m OpenCV_DIR D:\src\opencv\build
(If you use PS) [System.Environment]::SetEnvironmentVariable('OpenCV_DIR','D:\src\opencv\build')Originally, we had our C++ source file in native_opencv\ios\Classes\native_opencv.cpp . And now we are going to use the same file by making a hard link to it:
(CMD) mklink /H native_opencv_windows\windows\native_opencv.cpp native_opencv\ios\Classes\native_opencv.cpp
(PS) New-Item -ItemType HardLink -Name native_opencv_windows\windows\native_opencv.cpp -Target native_opencv\ios\Classes\native_opencv.cppAdd the link to the .gitignore file as well.
Earlier, I used the image_picker package for selecting an image on mobiles. As it doesn’t support desktops yet, I’ll use the file_picker package to show a native dialog for choosing image files:
Now we’ll set up project building. Just like on Android, the cmake is used for building a project, so open up CMakeLists.txt located in native_opencv_windows\windows\CMakeLists.txt.
First, define .dll libraries names, so we can include them in the build later. You can find DLLs in the opencv\build\x64\vc15\bin folder.
NOTE: Lines that have to be added are bold.
set(PLUGIN_NAME "native_opencv_windows_plugin")set(OpenCV_RELEASE_DLL_NAME "opencv_world454.dll")
set(OpenCV_DEBUG_DLL_NAME "opencv_world454d.dll")
Add C++ source code file (precisely, a hard link to it):
add_library(${PLUGIN_NAME} SHARED
"native_opencv_windows_plugin.cpp"
"native_opencv.cpp"
)To distinguish between different build configurations for choosing the right library, add these lines:
target_compile_definitions(${PLUGIN_NAME} PRIVATE FLUTTER_PLUGIN_IMPL)
target_include_directories(${PLUGIN_NAME} INTERFACE
"${CMAKE_CURRENT_SOURCE_DIR}/include")target_compile_definitions(${PLUGIN_NAME} PRIVATE
OpenCV_DLL_NAME=
$<$<CONFIG:Debug>:${OpenCV_DEBUG_DLL_NAME}>
$<$<CONFIG:Profile>:${OpenCV_RELEASE_DLL_NAME}>
$<$<CONFIG:Release>:${OpenCV_RELEASE_DLL_NAME}>
)
And, finally, add the OpenCV library to the target:
target_compile_definitions(${PLUGIN_NAME} PRIVATE
OpenCV_DLL_NAME=
$<$<CONFIG:Debug>:${OpenCV_DEBUG_DLL_NAME}>
$<$<CONFIG:Profile>:${OpenCV_RELEASE_DLL_NAME}>
$<$<CONFIG:Release>:${OpenCV_RELEASE_DLL_NAME}>
)set("OpenCV_DIR" $ENV{OpenCV_DIR})
find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})target_link_libraries(${PLUGIN_NAME} PRIVATE ${OpenCV_LIBS} flutter flutter_wrapper_plugin)# List of absolute paths to libraries that should be bundled with the plugin
set(native_opencv_windows_bundled_libraries
""
"${_OpenCV_LIB_PATH}/${OpenCV_DLL_NAME}"
PARENT_SCOPE
)
In the Dart part, open the corresponding dynamic .dll library:
Windows plugin setup is complete but if we run the compilation now it’s going to fail. The reason is that another compiler is used on Windows — MSVC. It doesn’t support some things that are available in the GNU compiler.
__attribute__ is not available in MSVC and has to be replaced with __declspec(dllexport), so the functions are exported into a .dll file. Go to native_opencv.cpp, create a FUNCTION_ATTRIBUTE macro, and use it for required functions:
Also, printing to debug output with a simple printf won’t work — just like in Android. Messages to the debugger are sent by the OutputDebugStringA function instead. Create a macro to detect WinAPI, include windows.h, and call OutputDebugStringA:
Aaaand it’s ready to go:
Setting up the plugin on macOS
OpenCV releases don’t contain pre-built libraries for macOS and Linux, which means that we need to build them ourselves. First, download OpenCV’s source code or clone its repository. Instructions on how to build an Xcode framework for macOS are available in a readme file located inopencv/platforms/apple/readme.md. In short, just run the python script like this:
python3 ~/dev/lib/opencv/platforms/apple/build_xcframework.py --macos_archs=x86_64,arm64 --build_only_specified_archs --out ./build_xcframeworkCreate a hard link to native_opencv.cpp file from the iOS plugin:
ln native_opencv/ios/Classes/native_opencv.cpp native_opencv_macos/macos/Classes/native_opencv.cppAfter that, the instructions are similar to iOS. Copy (or create a symlink from) opencv2.xcframework to native_opencv_macos/macos. Edit native_opencv_macos.podspec:
s.platform = :osx, '10.11'
s.pod_target_xcconfig = { 'DEFINES_MODULE' => 'YES' }
s.swift_version = '5.0' # telling CocoaPods not to remove framework
s.preserve_paths = 'opencv2.xcframework' # telling linker to include opencv2 framework
s.xcconfig = { 'OTHER_LDFLAGS' => '-framework opencv2' } # including OpenCV framework
s.vendored_frameworks = 'opencv2.xcframework' # including native framework
s.frameworks = 'AVFoundation', 'Accelerate', 'OpenCL' # including C++ library
s.library = 'c++'
end
The only difference from iOS’ podspec is in adding two more native frameworks: Accelerate and OpenCL.
In case you set up the app for iOS, no changes need to be made to native_opencv.dart and native_opencv.cpp files. In case you don’t — just follow the instructions for iOS in my previous article.
Setting up the plugin on Linux
Same as for macOS, OpenCV releases don’t contain pre-built libraries for Linux. Instructions on how to build them are available on the official OpenCV website.
Set up an environmental variable OpenCV_DIR that leads to the build folder of the library by adding it to the “.XXXrc” file (for example, .bashrc):
export OpenCV_DIR=/home/west/dev/lib/opencv/buildCreate a hard link to native_opencv.cpp file from the iOS plugin:
ln native_opencv/ios/Classes/native_opencv.cpp native_opencv_linux/linux/native_opencv.cppNow edit the native_opencv_linux/linux/CMakeLists.txt:
...
add_library(${PLUGIN_NAME} SHARED
"native_opencv_linux_plugin.cc"
"native_opencv.cpp"
)set("OpenCV_DIR" $ENV{OpenCV_DIR})
find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})apply_standard_settings(${PLUGIN_NAME})
set_target_properties(${PLUGIN_NAME} PROPERTIES
CXX_VISIBILITY_PRESET hidden)
target_compile_definitions(${PLUGIN_NAME} PRIVATE FLUTTER_PLUGIN_IMPL)
target_include_directories(${PLUGIN_NAME} INTERFACE
"${CMAKE_CURRENT_SOURCE_DIR}/include")
target_link_libraries(${PLUGIN_NAME} PRIVATE flutter)
target_link_libraries(${PLUGIN_NAME} PRIVATE PkgConfig::GTK)
target_link_libraries(${PLUGIN_NAME} PRIVATE ${OpenCV_LIBS})
...
Debugging native code on Linux/Windows in VSCode
Open the project as usual. Generate a build folder by simply starting an app. Install CMake Tools and C/C++ extensions:
Call a CMake: Configure action. It’ll tell that there’s no CMakeLists.txt file found on the root. Press Locate and choose CMakeLists.txt file from a platform’s folder (e.g., linux/CMakeLists.txt):
Now create a new C/C++configuration. For Linux it’s “gdb Launch”, for Windows it’s “Windows”. A configuration file .vscode/launch.json has a property called “program”. Point it to an executable file:
(Windows)
${workspaceFolder}/build/windows/runner/Debug/flutter_native_opencv.exe
(Linux)
${workspaceFolder}/build/linux/x64/debug/bundle/native_opencvOn Linux, make sure you have installed OpenCV after building it by calling cmake install. Create a symlink to the headers:
sudo ln -s /usr/local/include/opencv4/opencv2 /usr/local/include/opencv2On Windows, add OpenCV’s include folder to includePath parameter in .vscode/c_cpp_properties.json. Here’s the final c_cpp_properties.json for both platforms:
Run a configuration and everything should be working: OpenCV docs, breakpoints, debug output, etc.
Note that to apply changes to .cpp files you have to re-build/re-run the Flutter project.
With a power of Dart FFI, we were able to share the same C++ source code file and the same library across 5 different platforms.
Thank you for reading and feel free to leave any feedback. The source code for the app is available here:
