AKA “The best Mobile Lightning Wallet for Android is about to become the best Mobile Lightning Wallet”.
You asked (like a thousand times :)), we listened… Phoenix on iOS is now officially a thing: we are releasing a testnet version for TestFlight, for a limited number of beta testers at first, which we’ll open up later. To receive a TestFlight invite please contact us at phoenix@acinq.co
A mainnet version will follow in Q1 2021.
If all you want is to use Phoenix on iOS, then just be patient. It’s coming your way.
But if you want to know how we “ported” Phoenix to iOS (and why it took so long) please read along.
Background
Most people know ACINQ through Phoenix or eclair-mobile, but our flagship product is actually eclair, a fast, reliable implementation of Lightning that powers the biggest Lightning node, our own ACINQ node.
eclair runs on the JVM, which we believe is the best platform for running servers that are live 24/7, and is developed in Scala which we believe is one of the best languages for writing complex systems.
Our Lightning Wallet, Phoenix and Eclair Mobile, are also based on eclair and run on Android’s JVM.
So our problem was simple: how do we port a Scala application to iOS ?
Additional constraints
The app we’re building is a mobile Lightning Wallet. We’ve been building such apps for Android since 2017 and there are things we learnt:
- We want a “native” app i.e. an app that has the look and feel of an iOS app, which means that the main GUI should be a Swift app
- We’re building a non-custodial wallet: it is an actual Lightning Node, not a remote control app for a remote node or a client that just calls a remote API. Lightning is a fairly rich protocol and we’ll need very good network, cryptographic, and concurrency libraries
- We want to control as precisely as possible how Lightning is implemented, and use the protocol extensions we developed for Phoenix
- Phoenix tries to provide users with the best possible UX without being custodial. This includes providing good user support which is much harder for mobile apps than it is for server/desktop apps
What we tried: find a JVM for iOS ?
Since our Lightning Engine runs on the JVM, the simplest option would be to find a JVM that runs on iOS. There have been very few attempts, which all failed for a simple reason: Apple does not want apps to execute “data” which is basically how the JVM works. So, no JVM on iOS.
What we tried: compile our existing library into a native executable ?
The next easy solution would be to find a way to compile our existing Lightning library into a native iOS executable: this is what GraalVM promises.
GraalVM is a very ambitious project which, using its ahead-of-time compiler, promises to turn any JVM application into a native executable for all platforms that it supports.
However, using GraalVM is not straightforward yet. Configuration is quite complex, as well as integration with iOS. But after a lot of hard work we built an actual app that ran on an iOS device and implemented the “Lightning Handshake” defined in BOLT #8 (i.e it could connect to another Lightning Node and say hello). However it was very unstable, crashed randomly and was basically impossible to debug. Not a very good option for a mobile wallet…
So we run out of “easy options”, and it looks like we’ll have to actually do some work and write some code…
If reusing what we currently have is not possible, our plan becomes:
- build a native iOS app in Swift (the UI)
- connect the UI to a new embedded Lightning Engine
For that new Lightning Engine, we can:
- start from an existing low-level library (in C or Rust).
- write a new LN implementation from Scratch in Swift
- or ?
What we did not try: use a low-level LN library
Using such a library means that there’s a lot of additional work that needs to be done: connection management, persistence, logging, interaction with the Swift GUI…
It also means that we end up maintaining 2 very different code bases, so it would be something we would do only if there were no other options.
What we did not try: build a new Lightning implementation in Swift
Integration with the Swift GUI is no longer an issue, we can reuse a lot of native libraries (encryption, networking, logging, …) but still end up maintaining 2 different code bases. No one at ACINQ knows Swift, and since we’ve already built a LN implementation we know how much work it takes to get something that you can trust with real money, so again this is something we would do only if there were no other options.
What we did: use Kotlin Multi Platform
At the beginning of 2020 we learned that there were serious efforts to build a Kotlin compiler that would target, among other platforms, iOS.
It immediately got our attention:
- Kotlin, which we already use, is a really cool new language on the JVM, and very close to Scala (though not as powerful, and also not as complex)
- Jetbrains, the makers of Kotlin, have a very impressive track record. Among other products, they make Idea, the best JVM IDE you can find. It has become the de-facto Scala IDE (even though there was an “official” Scala IDE based on Eclipse) and their Scala plugin has 18 million downloads ! Which means that they know quite a bit about Scala and used that knowledge when they designed Kotlin
- Building a production ready LLVM-based compiler for Kotlin, a Garbage Collector and a set of standard libraries is very serious work, but not as complex as what GraalVM is trying to accomplish. And Kotlin MultiPlatform explicitly targets iOS
- Since Kotlin runs on the JVM we can mix it with Scala code and reuse our very extensive Lightning test suite.
Step #1: Port our Bitcoin Library
To learn more about Kotlin and evaluate how much work it would be to “port” our code, we started with our bitcoin library. It implements everything you need to build, sign and verify Bitcoin transactions and work with the Bitcoin blockchain, yet its code remains quite simple. It also uses Bitcoin Core’s secp256k1 library, through a custom JNI interface, so it will be a good test for evaluating how well KMP interacts with native code.
The result is there:
- secp256k1-kmp, a KMP library that encapsulates Bitcoin Core’s secp256k1 library
- bitcoin-kmp, a feature-complete bitcoin KMP library that uses secp256k1-kmp
Both libraries target the JVM, Android and iOS. They will also work on Linux and macOS but these platforms are not production targets for us (for now just use the JVM).
What we learned in the process is that:
- Kotlin is very close to Scala. In many cases it’s just a matter of renaming .scala files to .kt and fixing a few minor syntax differences, but sometimes actual refactoring will be necessary as Kotlin’s type system is not as powerful as Scala’s. But learning Kotlin will not be a problem for us
- KMP tooling is already quite good and mature, so is IDE integration (with IDEA). Using github’s CI is straightforward and we have automated integration tests on the JVM, linux and iOS
- We can reuse our test suite very easily, first because porting tests was even easier than porting the main code, and also because we can mix Kotlin and Scala code and build a Lightning Node that uses our new KMP bitcoin library
- The Kotlin compiler is not fully optimized yet, and compiled Kotlin code is sometimes a bit slow. In our case it is not an issue since for time-critical operations (i.e signing and verifying transactions) we use Bitcoin Core’s secp256k1 library through secp256k1-kmp
- Compiling Kotlin code means that your code has to be ..err.. Kotlin. Pretty obvious, but if you’re using third-party libraries you’ll have to port them to Kotlin too or find KMP equivalents (and there are not many KMP libraries yet).
So we were quite happy with our evaluation of Kotlin, and were ready to proceed to the next step.
Step #2: Build a KMP Lightning Prototype
At this point KMP looks very promising and we begin to think that it is probably what we’re looking for.
Overall design
We have a very simple design for our Lightning Node, based on state machines and actors.
Channels are state machines:
(State, Event) -> (State, Actions)An Event could be a message received over the network, a block tick, a notification that a transaction has been confirmed/spent/…
An Action could mean “send a message”, “watch this transaction”, “publish this transaction”, …
These state machines are encapsulated behind Actors (one per Channel) and exchange messages with other Actors (network, storage, blockchain monitoring, payment relay…).
This design has worked very well for us: our node has 99.99% uptime, manages more than ten thousand channels, and I don’t think we ever got hit by a concurrency bug.
A Lightning Wallet does not have the same constraints and will hopefully not manage 1000s of channels but we’d like to reuse the same design if possible.
It turns out that even though KMP does not have “Actors” per se, they have coroutines which is the next best thing. For our use case there’s actually very little difference.
So now our next step was to build a prototype that implements enough of the Lightning Protocol to connect to a Lightning Node, create an actual channel and receive a payment:
- We get to see how coroutines work, how to handle serialization of Lightning messages, channel data, and port Scala code that is more complex than our bitcoin library.
- We also get to build an actual iOS app, and study how best to interface the Kotlin and Swift layers.
It took about a week to implement a POC that could be used to setup real LN channels and send payments. It was very hackish, lacked proper logs, error handling but it validated that KMP was the way to go.
A new Lightning Engine in Kotlin !
It took a few months to get from a POC to something that we could release on testnet (and it will take at least a few more weeks before Phoenix is ready for mainnet) but here we are:
We have:
- lightning-kmp, a multi-platform Lightning engine that can be used on iOS, Android and the JVM (as well as Linux and macOS, but not for production app) that reuses the same design as our Scala Lightning engine, and supports all the UX features that Phoenix pioneered in 2019: on-the-fly instant channels, unified balance (which relies on zero-reserve channels and MPP), “backupless backups” (aka encrypted peer backups), trampoline routing…
- A real native iOS app, developed in Swift, which is just a very thin layer on top of our Kotlin engine (Phoenix iOS codebase is 95% Kotlin!)
Yes, we need to maintain a new Kotlin codebase, but it actually makes things simpler for us. Our “android” branch had been drifting away from our main branch, both for functional reasons (there are significant differences between a “routing” Lightning node and a Lightning wallet) and technical reasons (we have to use older versions of some of the libraries we use on Android).
We get a clean new codebase that was specifically designed for Lightning wallets, in a language that is so close to Scala that there’s basically no learning phase (almost everyone at ACINQ was able to contribute to our new Kotlin engine).
In fact, our plan is now to retire our current Android app and replace it with our new MultiPlatform design.
Conclusion
Kotlin Multiplatform and Kotlin Multiplatform Mobile are awesome and if you’re developing “rich” applications where most of the work is done client-side you should really have a look, especially if you already have code that runs on the JVM.
We’re close to a mainnet release:
- our low-level Bitcoin libraries pass all our reference tests
- our new Lightning engine is almost ready, its design is sound and there are no major issues left
- but there is still some UX work to be done in the iOS app, which is why we’ll first go through a few iterations on TestFlight
Phoenix iOS, and the development of a new multi-platform Lightning engine specifically designed for mobile wallets is a major milestone for us. As explained above, we will maintain 2 specialised code bases:
- Scala + Akka for high-performance routing nodes
- Kotlin Multiplatform for mobile Lightning wallets
And we will rewrite Phoenix Android to use our new mobile stack. We’re looking for people to help us, so if you want to work on one of the coolest projects in Bitcoin today using one of the most promising multi-platform development stacks please contact us at jobs@acinq.co
- When iOS ?
- 2 weeks
