Packaging Native Code with Prefab: A Solution for Imported Prebuilt Binaries

By: Nir Moshe

Introduction

Since we started writing Android applications at Lightricks, we used a monorepo to store our projects. With a monorepo it’s very convenient to share both knowledge and code between teams, which increases the pace of development. Recently, due to performance issues we began breaking down the monorepo into standalone components — AARs (Android Archive).

The standard way to ship Android modules is using AAR files. Packaging Java/Kotlin-based modules into AAR is trivial, however, packaging native libraries (C/C++ code) is not and we used Google’s Prefab to do so. During the process of packaging native modules as AARs, we encountered the problem that Android’s Prefab plugin does not allow packaging modules with imported prebuilt binaries. In this blog, I will present a simple method for dealing with this issue.

Please note that I assume that the reader is familiar with Android, and specifically with Gradle and building native libraries using CMake.

Introduction to Prefab

In Android, the standard way to package libraries or Gradle modules is using AAR (Android Archive) files. These files are similar to APK files and contain Java/Kotlin packages, as well as other files such as resources, assets, and localizations. By default, when building an Android Library, Gradle will package it as an AAR file.

If your library contains native code which is part of the interface, you will need to provide pre-built files for static/dynamic linking (such as *.so or *.a files) and all relevant “include” files (such as *.h files) for each supported ABI (Application Binary Interface). Prefab is a tool for generating build system integrations for prebuilt C/C++ libraries, it was developed by Google specifically for this purpose. It has Android Gradle plugin support and can easily package all necessary artifacts into an AAR file with minimal configuration.

Example: Let’s say that we want to publish a simple native library “myLib”. We simply add to our “build.gradle” file the following section:

android {
   ...


   externalNativeBuild {
       cmake {
           path "CMakeLists.txt"
       }
   }


   buildFeatures {
       prefabPublishing true
   }


   prefab {
       myLib {
           headers "src/main/myLib/include"
       }
   }

That’s it!

Now, an application that wants to use packages compiled with the Prefab plugin needs to add the following snippet to their “build.gradle” file:

android {
   ...


   buildFeatures {
       prefab true
   }

and to the “CMakeLists.txt”:

find_package(myPackage REQUIRED CONFIG)


...


target_link_libraries(
   someLib
   myPackage::myLib
)

Packaging imported libraries using Prefab plugin

We discovered that the Prefab plugin does not support packaging native code that includes imported prebuilt binaries. For example, at Lightricks we use a static version of the Botan library and do not compile the code ourselves, but rather use a file that has already been built for us. When building our applications, we link against libbotan-2.a.

However, activating the Prefab plugin with a “CMakeLists.txt” file containing an imported library (like the following example) does not work as intended. When attempting to package the library with the Prefab plugin, the resulting AAR file will be missing the libbotan-2.a file.

add_library(
        botan-2
        STATIC
        IMPORTED
)

set_target_properties(
        botan-2
        PROPERTIES IMPORTED_LOCATION
        /lib/${ANDROID_ABI}/libbotan-2.a
)

After checking online, we found that this is a known issue with Prefab plugin.

I used the Android code search engine to get a better understanding of what’s going on, and to see if there are hidden or undocumented options for packing imported prebuilt binaries. Android code search found a small amount of relevant files within the “Android Studio” project, and one particular sentence in the documentation caught my attention:

It seems like the Prefab plugin is building a directory structure before packing it into AAR. I wanted to learn more about how Prefab plugin handles the ‘payloadIndirection’ parameter, so I examined the code and discovered the following function:

When ‘payloadIndirection’ is set to false, the Prefab plugin simply copies the ‘abiLibrary’ file (the compilation product) into the new directory structure.

To sum up, after examining the Prefab plugin’s code, we determined that the Prefab plugin expects all build products to be located in the products directory.

How do we solve this issue?

As a solution to the problem described above, we attempted to build a CMake script that takes the binary files and places them in the products directory, tricking the Prefab plugin into thinking they are the actual build products.

For example, let’s say the file we want to package is libbotan-2.a. We can create a new “CMakeLists.txt” file that performs the following actions:

1. Create a new empty file: `dummy.cpp`.
2. Define a new library by compiling the `dummy.cpp` file. The compilation result will be a binary file named libbotan-2.a, but without the implementation we desire.
3. Immediately after the build, perform a copy operation that takes the binary file we want to use and overwrites the build of `dummy.cpp` (copying to CMAKE_CURRENT_BINARY_DIR).

Here is an example of such a “CMakeLists.txt”:

cmake_minimum_required(VERSION 3.10)


# Specify the location of the .so file to replace the shared library
set(FILE_TO_COPY <some_location>/libbotan-2.a)


add_library(botan-2 STATIC dummy.cpp)


if(EXISTS ${FILE_TO_COPY})
   message("Using .a file: ${FILE_TO_COPY}")
   add_custom_command(TARGET botan-2 POST_BUILD
           COMMAND ${CMAKE_COMMAND} -E copy ${FILE_TO_COPY} $<TARGET_FILE:botan-2>
   )
else()
   message(FATAL_ERROR "Can't find file ${FILE_TO_COPY}!")
endif()

In the “build.gradle” file:

android {
   ...
  
   defaultConfig {


       ...


       externalNativeBuild {
           cmake {
               arguments "-DANDROID_STL=c++_shared",
                         "-DANDROID_CPP_FEATURES=rtti exceptions"
           }
       }
   }


   externalNativeBuild {
       cmake {
           path "CMakeLists.txt"
       }
   }


   buildFeatures {
       prefabPublishing true
   }


   prefab {
       "botan-2" {
           headers "src/main/botan/include"
       }
   }
}

This allows the Prefab plugin to properly package the desired binary file, libbotan-2.a, in the AAR file.

Conclusion

We were able to address the issue of the Prefab plugin not supporting the packaging of native code with imported prebuilt binaries. By using a CMake script that tricks the Prefab plugin into thinking the desired binary is the actual build product, we are able to successfully include the file in the generated AAR package. This breakthrough was made possible by examining the Android source code, which serves as a powerful resource for developers. If you have any questions or need further assistance, don’t hesitate to reach out to me.
—
Create magic with us
We’re always on the lookout for promising new talent. If you’re excited about developing groundbreaking new tools for creators, we want to hear from you. From writing code to researching new features, you’ll be surrounded by a supportive team who lives and breathes technology.
Sounds like you? Apply here.