Create SPM module for MEGA SDK C++ code

By Javier Navarro Melchor, iOS Team Lead, Mobile, MEGA

Let’s review how we managed to extract our Objective-C++/C++ SDK from our main iOS project to a Swift Package Manager module.

Swift Package Manager (SPM) is a versatile tool that enables efficient modularisation of Swift projects and simplifies dependency management. Unlike traditional approaches, such as adding new targets, which often involve complex project configurations, SPM streamlines the process, allowing you to create separate modules for distinct parts of your codebase with ease.

General overview of MEGA SDK

At MEGA, we are using a library in C++ for our iOS, Android and desktop applications, the MEGA SDK. The MEGA SDK has some external C++ dependencies: libcurl, libuv, libsodium, etc.

In Android, we’re able to use C++ code through the Android Native Development Kit and the Java Native Interface framework. As for iOS, we have a binding implemented in Objective-C++.

Previously, we had an Xcode project to build the SDK (C++) code and the Objective-C++ binding. Now, we extracted Objective-C++/C++ from our main iOS project to Swift Package Manager module. With this new solution, we can set up a dependency in our data layer from the repository SPM module to our SDK data source.

MEGASDK (Objective-C++/C++) Before: Xcode project. Now: SPM module

It’s worth mentioning that before, if you wanted to use the C++ library from Swift iOS application you had to write an Objective-C++ binding, as you can’t mix Swift and C++ code. But since 2023–06–05 it is possible mix Swift and C++ code: https://www.swift.org/documentation/cxx-interop/

We will investigate how to mix Swift and C++ in our project soon. We will not cover how to write an Objective-C++ binding for C++ library.

Swift packages

Swift packages are reusable components of Swift, Objective-C, Objective-C++, C, or C++ code that developers can use in their projects. They bundle source files, binaries, and resources in a way that’s easy to use in your app’s project.

The structure of a Swift package typically consists of several key components and directories, organised in a specific way to facilitate package management and integration into Swift projects. Here is a common structure for a Swift package:

Package manifest (Package.swift)

This is the heart of the package and defines the package’s metadata, dependencies, and targets. It’s written in Swift and serves as a configuration file for the package. Here’s an example of what it might look like:

import PackageDescription
 
let package = Package(
    name: "MyPackage",
    products: [
        .library(name: "MyFirstLibrary", targets: ["MyFirstLibrary"]),
        .library(name: "MySecondLibrary", targets: ["MySecondLibrary"]),
    ],
    targets: [
        .target(name: "MyFirstLibrary", dependencies: ["Dependency1", "Dependency2"]),
        .target(name: "MySecondLibrary", dependencies: ["MyFirstLibrary"]),
        .testTarget(name: "MyLibraryTests", dependencies: ["MyFirstLibrary"]),
        .binaryTarget(name:"Dependency1", path:"path/Dependency1"),
        .binaryTarget(name:"Dependency2", path:"path/Dependency2"),
    ]
)

Source code (Sources/)

This directory contains the source code files for your package. You can organize your code into different subdirectories and modules within the Sources/ directory.

Sources/
├── CppSources
│   └── MySource.cpp
│   └── Include
│   	└── MySource.h
└── Objective-CppSources
│   └── OtherSource.mm
│   └── Include
│   	└── OtherSource.h

Tests (Tests/)

This directory is where you place your unit tests for the package. Each module in your package can have its corresponding test module.

Tests/
├── MyPackageTests
│   └── MyFileTests.cpp
└── AnotherModuleTests
    └── AnotherFileTests.cpp

SPM package for Objective-C++/C++ MEGA SDK code

This is how our Objective-C++/C++ package for MEGA SDK is just now. Let’s dive into the steps we followed:

// swift-tools-version: 5.8

import PackageDescription

let package = Package(
    name: "MEGASDK",
    platforms: [
        .iOS(.v14)
    ],
    products: [
        .library(
            name: "MEGASdkCpp",
            targets: ["MEGASdkCpp"]),
        .library(
            name: "MEGASdk",
            targets: ["MEGASdk"])
    ],
    dependencies: [
    ],
    targets: [
        .target(
            name: "MEGASdkCpp",
            dependencies: ["libcryptopp",
                           "libmediainfo",
                           "libuv",
                           "libcurl",
                           "libsodium",
                           "libwebrtc",
                           "libzen"],
            path: "Sources/MEGASDK",
            exclude: ["examples",
                      "tests",
                      "doc",
                      "contrib",
                      "bindings",
                      "src/win32",
                      "src/wincurl",
                      "src/mega_utf8proc_data.c",
                      "src/thread/libuvthread.cpp",
                      "src/osx/fs.cpp"],
            cxxSettings: [
                .headerSearchPath("bindings/ios"),
                .headerSearchPath("include/mega/posix"),
                .headerSearchPath("bindings/ios/3rdparty/webrtc/third_party/boringssl/src/include"),
                .define("ENABLE_CHAT"),
                .define("HAVE_LIBUV"),
                .define("NDEBUG", .when(configuration: .release))
            ],
            linkerSettings: [
                // Frameworks
                .linkedFramework("QuickLookThumbnailing"),
                // Libraries
                .linkedLibrary("resolv"),
                .linkedLibrary("z"),
                .linkedLibrary("sqlite3"),
                .linkedLibrary("icucore")
            ]
        ),
        .target(
            name: "MEGASdk",
            dependencies: ["MEGASdkCpp"],
            path: "Sources/MEGASDK/bindings/ios",
            cxxSettings: [
                .headerSearchPath("../../include"),
                .headerSearchPath("Private"),
                .define("ENABLE_CHAT"),
                .define("HAVE_LIBUV")
            ]
        ),
        .binaryTarget(
            name: "libcryptopp",
            path: "Sources/MEGASDK/bindings/ios/3rdparty/lib/libcryptopp.xcframework"
        ),
        .binaryTarget(
            name: "libmediainfo",
            path: "Sources/MEGASDK/bindings/ios/3rdparty/lib/libmediainfo.xcframework"
        ),
        .binaryTarget(
            name: "libuv",
            path: "Sources/MEGASDK/bindings/ios/3rdparty/lib/libuv.xcframework"
        ),
        .binaryTarget(
            name: "libcurl",
            path: "Sources/MEGASDK/bindings/ios/3rdparty/lib/libcurl.xcframework"
        ),
        .binaryTarget(
            name: "libsodium",
            path: "Sources/MEGASDK/bindings/ios/3rdparty/lib/libsodium.xcframework"
        ),
        .binaryTarget(
            name: "libwebrtc",
            path: "Sources/MEGASDK/bindings/ios/3rdparty/lib/libwebrtc.xcframework"
        ),
        .binaryTarget(
            name: "libzen",
            path: "Sources/MEGASDK/bindings/ios/3rdparty/lib/libzen.xcframework"
        )
    ],
    cxxLanguageStandard: .cxx14
)

Step 1: create XCFrameworks for external C++ third party libraries

In the beginning, we had fat static libraries (.a) for our C++ external dependencies. Basically, it consists of:

1. Compile the library for each architecture supported. When we started at MEGA with the iOS application, the architecture supported were i386 and x86_64 for simulators, armv7 and arm64 for devices. Now, we support x86_64 and arm64.
2. Merge the libraries into one fat library using lipo command

You can’t use .a static library as binaryTarget in the package.

let package = Package(
   name: "MyPackage",
   ...
   targets: [
      .target(
         name: "MyPackage",
         dependencies: ["libcurl"]
      ),
      .binaryTarget(name: "libcurl", path: "path/libcurl.a"), // This will cause an error
      ...
   ]

We had to create XCFramework using the command command: xcodebuild -create-xcframework

xcodebuild -create-xcframework -library ${CURRENTPATH}/bin/libcurl/libcurl.a -headers ${CURRENTPATH}/bin/libcurl/iPhoneSimulator${SDKVERSION}-arm64.sdk/include -library ${CURRENTPATH}/bin/libcurl/iPhoneOS${SDKVERSION}-arm64.sdk/lib/libcurl.a -headers ${CURRENTPATH}/bin/libcurl/iPhoneOS${SDKVERSION}-arm64.sdk/include -output ${CURRENTPATH}/xcframework/libcurl.xcframework

The process basically is:

1. Compile the external C++ code from library for each architecture
2. Create the XCFramework

And now, in our package we can use: .binaryTarget(name: “libcurl”, path: “path/libcurl.xcframework”).

We had to create XCFrameworks for every C++ external dependency we had in our MEGA SDK: libcurl, libuv, libsodium, cryptopp, libmediainfo and libzen

Currently, for dependencies, we are using local path packages, but you can use remote packages:

.binaryTarget(
 	name: "SomeRemoteBinaryPackage",
    url: "https://url/to/some/remote/xcframework.zip",
    checksum: "The checksum of the ZIP archive that contains the XCFramework."
),

Step 2: mixing Objective-C++ and C++ in a module

Mixed languages are not supported in the same SPM target. Our framework contains both Objective-C++ and C++ codebase. So, we need to separate our MEGA SDK into two targets with their respective folders. Objective-C++ target depends on C++ target.

import PackageDescription

let package = Package(
    name: "MEGASDK",
    platforms: [
        .iOS(.v14)
    ],
    products: [
        .library(
            name: "MEGASdkCpp",
            targets: ["MEGASdkCpp"]),
        .library(
            name: "MEGASdk",
            targets: ["MEGASdk"])
    ],
    targets: [
        .target(
            name: "MEGASdkCpp",
            path: "Sources/MEGASDK",
        ),
        .target(
            name: "MEGASdk",
            dependencies: ["MEGASdkCpp"],
            path: "Sources/MEGASDK/bindings/ios",
        ),
    cxxLanguageStandard: .cxx14
)

Step 3: create “include” folder for public headers

This is not strictly necessary, but advisable. We had the headers (.h) and the implementations (.mm) for the Objective-C++ binding in the same folder (binding/ios). When you create a target, you can specify the path for a target’s source files. By default the public headers path is the path you specify for the target, appending /include.

Step 4: exclude files from compilation

SPM works opposite to the Xcode project. In the Xcode project we had before, when the MEGA C++ team added a new source code file to the SDK git repository if we wanted to include that code in the build we had to add it to the Xcode project. SPM, on the other hand, adds to the compilation all the files that are in the target’s path, so it is necessary to indicate which files or folders we want to exclude from the compilation. In our case, we want to exclude the following files:

...
targets: [
	.target(
		name: "MEGASdkCpp",
		exclude: ["examples",
				"tests",
				"doc",
				"contrib",
				"bindings",
				"src/win32",
				"src/wincurl",
				"src/mega_utf8proc_data.c",
				"src/thread/libuvthread.cpp",
				"src/osx/fs.cpp"],
...

Step 5: add cxxSettings

We had problems with some headers: file.h file not found. For that reason, we had to add cxxSettings for each target Objective-C++ and C++. The headerSearchPath: provides a header search path relative to the target’s directory (the path must be a directory inside the package), while define: defines a value for a macro.

For our package, this is the cxxSettings for each target:

cxxSettings: [
	.headerSearchPath("bindings/ios"),
	.headerSearchPath("include/mega/posix"),
	.headerSearchPath("bindings/ios/3rdparty/webrtc/third_party/boringssl/src/include"),
	.define("ENABLE_CHAT"),
	.define("HAVE_LIBUV"),
	.define("NDEBUG", .when(configuration: .release))
],

cxxSettings: [
	.headerSearchPath("../../include"),
	.headerSearchPath("Private"),
	.define("ENABLE_CHAT"),
	.define("HAVE_LIBUV")
]

Step 6: add linkerSettings

Finally, we had to deal with link issues: undefined symbols:

_OBJC_CLASS_$_QLThumbnailGenerationRequest, referenced from:
       in MEGASdkCpp.o
_OBJC_CLASS_$_QLThumbnailGenerator, referenced from:
       in MEGASdkCpp.o
_res_9_getservers, referenced from:
      mega::HttpIO::getDNSserversFromIos(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) in MEGASdkCpp.o
_res_9_ndestroy, referenced from:
      mega::HttpIO::getDNSserversFromIos(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) in MEGASdkCpp.o
_res_9_ninit, referenced from:
      mega::HttpIO::getDNSserversFromIos(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) in MEGASdkCpp.o
_u_foldCase, referenced from:
      mega::SqliteAccountState::icuLikeCompare(unsigned char const*, unsigned char const*, int) in MEGASdkCpp.o
      mega::SqliteAccountState::icuLikeCompare(unsigned char const*, unsigned char const*, int) in MEGASdkCpp.o

For link settings, you can use system frameworks and system libraries. In our case, for the C++ target we need to add the following system libraries and frameworks:

linkerSettings: [
	// Frameworks
	.linkedFramework("QuickLookThumbnailing"),
	// Libraries
	.linkedLibrary("resolv"),
	.linkedLibrary("z"),
	.linkedLibrary("sqlite3"),
	.linkedLibrary("icucore")
]

Conclusion

We have now covered how we created Swift Package from MEGA C++ library, the problems that we faced during the process, and how we solved them. We hope this guide helps you if you need to create a Swift Package for C++ code. The problems you may encounter may be similar to the problem we had, depending on the state of the library’s C++ code, whether it has external dependencies on other C++ code, whether it has Objective-C++ bindings or not, and other factors.

For our part, our next step is to investigate interoperability between Swift and C++ in our MEGA SDK project.

Our git repositories are public, you can check the iOS code here and the SDK code here.