OpenCV on Android — Tiny with Optimization Enabled
This article is also published on LearnOpenCV
My OpenCV Android SDK = small size library
If you choose OpenCV for production, your primary goal is to bring down the size of the library and also make it performance packed. OpenCV is an awesome library with tons of algorithms but you must be using a very small subset of these algorithm in your application, hence it makes perfect sense to include what is required and leave out the rest.
Library can be compiled statically along with your application code or can be dynamically linked at runtime and this is completely application development decision. Here we will create shared object (.so) for this demo.
Android is a versatile OS and can run on multiple hardwares, from mobile phones to IOT devices, Raspberry Pi, to various single board computers and therefore it become important to cross compile the code for that particular computer architecture called ISA. As we are building it for Android, we would need its built tools. Android has a fancy name for its build tools called, NDK (Native Development Kit), anyways, but that’s what we need.
Following are the list of software and their respective version we are going to use:
- cmake — 3.7.2
- NDK — r14b
- OpenCV — 3.4.1
- Host — Ubuntu — 16.4
- Target — armeabi-v7a (ARM based)
- Android SDK 25.2.5 tools.
Step 1:
Download NDK and unzip it to your work area. There are various version of NDK, hence it is important to read the release notes and pick the one you need for your project. But for this exercise its Android NDK Release 14b on Ubuntu inside Docker container. But this can be on any Linux box.
root@dc:/opt/android-ndk-r14b# ls
CHANGELOG.md ndk-build ndk-gdb ndk-which prebuilt shader-tools source.properties sysroot
build ndk-depends ndk-stack platforms python-packages simpleperf sources toolchainsWe will build the standalone toolchain for OpenCV compilation,
root@dc:/opt/android-ndk-r14b# ./build/tools/make_standalone_toolchain.py \
--arch arm \
--api 23 \
--install-dir /tmp/my-android-toolchainThis command will create toolchain based on --arch and --api that i need for my project and install them in /tmp/my-android-toolchain directory.
Step 2:
Often ANDROID_NDK path is set to the prebuilt toolchain, that comes along with the NDK. But we will use the one that we built in the previous step, we need to let cmake know this important fact and hence set the environment variable.
$ export ANDROID_STANDALONE_TOOLCHAIN=/tmp/my-android-toolchain/Thats pretty much our build environment needs to know, there are no other variables required. If you already have ANDROID_NDK set, please unset it.
Step 3:
Building for Android also needs few important packages called Ninja and Ant, we would install them as well:
$ sudo apt-get install ninja-build antStep 4:
Download OpenCV and there is no better place to find download steps than Satya Mallick’s blog. Either you can compile the way the blog suggested or just download all the packages, OpenCV repository and leave it there. Because we are compiling for Android so we can skip the compilation part mentioned in the blog. If you need contrib modules, download them for this article I am skipping the,
Anyways, let us move to OpenCV directory.
$ cd opencv/
$ mkdir build
$ cd build
$ cmake \
-DCMAKE_TOOLCHAIN_FILE=../platforms/android/android.toolchain.cmake\
-DANDROID_STL=gnustl_shared \
-DANDROID_NATIVE_API_LEVEL=23 ..Note: make a fail if the SDK is higher than 25.2.5 for OpenCV 3.0. Please make sure you have appropriate Android SDK in place.
The output will show the configuration for this build and this is the step where you can change the configuration of your likes.
For ARM based devices, optimization I usually enable using:
- NEON
- VFPV3
Check out the preamble of platforms/android/android.toolchain.cmake for various configuration options.
$ make -j $nprocStep 5:
Hopefully, the build goes fine and you have ready ELF output in the form of .so (shared object), you can also generate output as .a, for this, you need to enable appropriate flag in the cmake. The output would look like following:
root@dc:/opencv/build/lib/armeabi-v7a# ls
libopencv_calib3d.a libopencv_flann.a libopencv_java3.so libopencv_shape.a libopencv_video.a
libopencv_core.a libopencv_highgui.a libopencv_ml.a libopencv_stitching.a libopencv_videoio.a
libopencv_dnn.a libopencv_imgcodecs.a libopencv_objdetect.a libopencv_superres.a libopencv_videostab.a
libopencv_features2d.a libopencv_imgproc.a libopencv_photo.a libopencv_ts.alibopencv_java3.so is what you need for your android project, but if you look at the size of it, its around 9.5MB:
root@dc:/opencv/build/lib/armeabi-v7a# du -h libopencv_java3.so
9.4M libopencv_java3.soThat is because, it has all the modules included. To verify this, run following command:
$ strings -a libopencv_java3.so | grep .cppStep 6:
Lets trim the library for our needs. Let us say we need just two modules core and imageproc:
root@dc:/opencv/build/lib/armeabi-v7a# /tmp/my-android-toolchain/bin/arm-linux-androideabi-g++ \
-shared -o libopencv_tiny.so \
--sysroot=/tmp/my-android-toolchain/sysroot/ \
-Wl,--whole-archive libopencv_core.a \
libopencv_imgproc.a -Wl,--no-whole-archivenow check the file type and size of libopencv_tiny.so
root@dc:/opencv/build/lib/armeabi-v7a# file libopencv_tiny.so
libopencv_tiny.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, not strippedroot@dc:/opencv/build/lib/armeabi-v7a# du -h libopencv_tiny.so
8.3M libopencv_tiny.so
This is non-striped, let us strip it:
root@dc:/opencv/build/lib/armeabi-v7a# /tmp/my-android-toolchain/bin/arm-linux-androideabi-strip --strip-unneeded libopencv_tiny.soroot@dc:/opencv/build/lib/armeabi-v7a# file libopencv_tiny.so
libopencv_tiny.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, strippedroot@dc:/opencv/build/lib/armeabi-v7a# du -h libopencv_tiny.so
3.4M libopencv_tiny.so
Optional Step: ‘Bonus round’
You can further drill down and pick each .o file rather than .a file, that can be achieved using ar tool:
$ /tmp/my-android-toolchain/bin/arm-linux-androideabi-ar x libopencv_core.aar tool extract all the .o from .a file and later you can make .so from them the same way we created before, the command would look like following:
root@dc:/opencv/build/lib/armeabi-v7a# /tmp/my-android-toolchain/bin/arm-linux-androideabi-g++ \
-shared -o libopencv_tiny.so \
--sysroot=/tmp/my-android-toolchain/sysroot/ \
-Wl, — whole-archive *.o -Wl, — no-whole-archiveStep 7: Create .so for your application
Linking is where the effort ends, and let us cross that milestone too. As this is cross-compiled for a specific platform, it is important to include a few extract libraries in the .so, these extra libraries are completely 3rdparty and platform-specific. If you do not include them you will get a linking error, hence let us do it:
root@dc:/opencv/build/lib/armeabi-v7a# /tmp/my-android-toolchain/bin/arm-linux-androideabi-g++ -L /opencv/build/3rdparty/lib/armeabi-v7a -lz -llog -ljnigraphics -ltegra_hal -lcpufeatures -lm -lgnustl_shared -fexceptions -frtti -fpic -shared -Wl,-soname,libopencv_java.so -o libopencv_java.so -Wl,--whole-archive libopencv_core.a libopencv_imgproc.a -Wl,--no-whole-archiveDependences should look like the following, few of these extra libraries are .a so they won’t show up in the dependences.
root@dc:/opencv/build/lib/armeabi-v7a# readelf -d libopencv_tiny.soDynamic section at offset 0x3615d8 contains 33 entries:
Tag Type Name/Value
0x00000003 (PLTGOT) 0x362b18
0x00000002 (PLTRELSZ) 2488 (bytes)
0x00000017 (JMPREL) 0x3f674
0x00000014 (PLTREL) REL
0x00000011 (REL) 0x28cb4
0x00000012 (RELSZ) 92608 (bytes)
0x00000013 (RELENT) 8 (bytes)
0x6ffffffa (RELCOUNT) 11570
0x00000006 (SYMTAB) 0x148
0x0000000b (SYMENT) 16 (bytes)
0x00000005 (STRTAB) 0xaf78
0x0000000a (STRSZ) 97135 (bytes)
0x00000004 (HASH) 0x22ae8
0x00000001 (NEEDED) Shared library: [libz.so]
0x00000001 (NEEDED) Shared library: [liblog.so]
0x00000001 (NEEDED) Shared library: [libjnigraphics.so]
0x00000001 (NEEDED) Shared library: [libm.so]
0x00000001 (NEEDED) Shared library: [libc.so]
0x00000001 (NEEDED) Shared library: [libdl.so]
0x0000000e (SONAME) Library soname: [libopencv_tiny.so]
0x0000001a (FINI_ARRAY) 0x35e7b4
0x0000001c (FINI_ARRAYSZ) 8 (bytes)
0x00000019 (INIT_ARRAY) 0x362554
0x0000001b (INIT_ARRAYSZ) 132 (bytes)
0x00000010 (SYMBOLIC) 0x0
0x0000001e (FLAGS) SYMBOLIC BIND_NOW
0x6ffffffb (FLAGS_1) Flags: NOW
0x6ffffff0 (VERSYM) 0x27690
0x6ffffffc (VERDEF) 0x28c58
0x6ffffffd (VERDEFNUM) 1
0x6ffffffe (VERNEED) 0x28c74
0x6fffffff (VERNEEDNUM) 2
0x00000000 (NULL) 0x0
Make sure you strip the newly created .so before checking the size.
Now, your shared object ( libopencv_tiny.so ) is ready for android deployment. Often it is given libopencv_java3.so name where 3 is the version but do not worry its just a file name, it will still work or make it android complaint and name it as libopencv_java3.so.
Step 8: Linking with sample code
Let us say we have following code in the file demo.cpp inside directory ./demo, that you want to link with created .so:
#include "opencv2/opencv.hpp"
#include<iostream>int main()
{
std::cout << "OpenCV Version " << CV_VERSION << std::endl;
}
We MUST use the same toolchain to compile and link demo.cpp, can you guess why?
Ok! now that you are thinking, let me quickly start the compilation:
root@dc: /tmp/my-android-toolchain/bin/arm-linux-androideabi-g++ -L/demo -I /demo/include/ -Wall -o demo demo.cpp -lopencv_tinyI have moved the generated libopencv_tiny.so to folder ./demo and all the opencv includes (i mean .hpp), this is just to demonstrated that we are in isolation and not using /usr/local/include
Now, coming back to my question, why same toolchain?
root@5c7fe9a72d74:/demo# readelf -h libopencv_tiny.so
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: DYN (Shared object file)
Machine: ARM
Version: 0x1
Entry point address: 0x0
Start of program headers: 52 (bytes into file)
Start of section headers: 3561860 (bytes into file)
Flags: 0x5000200, Version5 EABI, soft-float ABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 8
Size of section headers: 40 (bytes)
Number of section headers: 27
Section header string table index: 26Did you get it? Ok, let me just say it: The .so is cross compiled for a specific arch type and that is ARM where as out host machine is x86 based linux. This is the reason, you need to run the executable file only on the same target machine type.
One final step:
If you build it on x86, you must also tell where your .so resides, you can keep it anywhere you like, i mean in any folder of your project or workarea, as this run time linking, ld should be able to find it. The best way to do this is by updating an environment variable:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<your .so path>Now, say you have become very generous and want to share it with everyone on the system. For that you need to have root privileges. You will need this for two reasons:
- To put the library in a standard system location, probably
/usr/libor/usr/local/lib, which normal users don’t have write access to. - You will need to modify the
ld.soconfig file and cache. Asroot, do the following:
cp /demo/libopencv_tiny.so /usr/local/lib
chmod 0755 /usr/local/lib/libopencv_tiny.soWe need to inform ld that it’s available for use, hence let’s update the cache:
ldconfigNow, we do not need LD_LIBRARY_PATH, so let’s clear it too. CAUTION: you clear your own path from LD_LIBRARY_PATH only.
unset LD_LIBRARY_PATHAnother Option : Compiling using OpenCV Makefile
I am including reply suggested by Yk Order. Beauty of using OpenCVMakefile, is that all dependences are taken care and you need to pass the appropriate build tools for your platform.
Example for Android :
cmake -DCMAKE_TOOLCHAIN_FILE=../platforms/android/android.toolchain.cmake \
-D ANDROID_ABI=arm64-v8a \
-D CMAKE_BUILD_TYPE=Release \
-D ANDROID_NATIVE_API_LEVEL=23 \
-D WITH_CUDA=OFF \
-D WITH_MATLAB=OFF \
-D BUILD_ANDROID_EXAMPLES=OFF \
-D BUILD_DOCS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_TESTS=OFF \
-D ANDROID_STL=c++_shared \
-D BUILD_SHARED_LIBS=ON \
-D BUILD_opencv_objdetect=OFF \
-D BUILD_opencv_video=OFF \
-D BUILD_opencv_videoio=OFF \
-D BUILD_opencv_features2d=OFF \
-D BUILD_opencv_flann=OFF \
-D BUILD_opencv_highgui=OFF \
-D BUILD_opencv_ml=OFF \
-D BUILD_opencv_photo=OFF \
-D BUILD_opencv_python=OFF \
-D BUILD_opencv_shape=OFF \
-D BUILD_opencv_stitching=OFF \
-D BUILD_opencv_superres=OFF \
-D BUILD_opencv_ts=OFF \
-D BUILD_opencv_videostab=OFF \
..On your Linux or MacOS: (It will include imgproc and highgui modules in the build)
cmake
-D CMAKE_BUILD_TYPE=Release \
-D WITH_CUDA=OFF \
-D WITH_MATLAB=OFF \
-D BUILD_ANDROID_EXAMPLES=OFF \
-D BUILD_DOCS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_SHARED_LIBS=ON \
-D BUILD_opencv_objdetect=OFF \
-D BUILD_opencv_video=OFF \
-D BUILD_opencv_videoio=OFF \
-D BUILD_opencv_features2d=OFF \
-D BUILD_opencv_flann=OFF \
-D BUILD_opencv_ml=OFF \
-D BUILD_opencv_photo=OFF \
-D BUILD_opencv_python=OFF \
-D BUILD_opencv_shape=OFF \
-D BUILD_opencv_stitching=OFF \
-D BUILD_opencv_superres=OFF \
-D BUILD_opencv_ts=OFF \
-D BUILD_opencv_videostab=OFF \
-D BUILD_opencv_highgui=ON \
-D BUILD_opencv_imgproc=ON \
..You can pick your modules you need in final build, in fact i may have missed quite a few.
Enjoy the power of dynamic linking!
