Dinghy: Painless Rust tests and benches on iOS and Android

Snips uses Rust to develop libraries that run on mobile devices. We developed dinghy, a tool to streamline testing and benching of libraries on the phones themselves. We are now sharing this with the Rust community at large, hoping many library developers will want to try and make their work portable.

TL;DR

  • Run cargo test and cargo bench on your phone.
  • A dinghy is a small boat that a big ship, like a cargo (!) ship, carries or
    tows around.
  • https://github.com/snipsco/dinghy

Rust for Snips

At Snips, we believe in Artificial Intelligence and Privacy. And we believe
even stronger that a user should not have to choose one over the other.
We are convinced that your own mobile phone is the best place to store and
process your data, so we bring AI to your phone, instead of bringing your
personal stuff to huge data-centers out of your control.

In order to make the best use of the relatively limited resources available
on a mobile phone, we are developing part of our toolbox using Rust. In the same way a user should not have to chose between a better experience and their privacy, we refuse to choose between portability, performance and safety. We want them all, so we picked Rust.

Rust tools for mobile development

While Rust as an ecosystem is still young, most of the heavy lifting to code
libraries for mobile devices is here: by using the LLVM compiler framework,
Rust takes advantage of mature code generation and optimisation for the ARM processors that make most smartphones tick.

Besides the compiler, Cargo provides the dependency management and build system. Rustup takes care of managing Rust toolchains and precompiled standard libraries, making cross compilation relatively painless. Finally, Rust support conditional compilation through a simple set of attributes, making it possible for libraries to adapt to the platform their are compiling to.

Unit testing is still a different matter, though. Cargo built-in testing capabilities work well, as long as you want to run tests on the computer you are developing with. Most of the bugs we encounter daily are trivial enough to manifest themselves in the same way on a laptop than on a phone. But sometimes we need to get closer to the metal to squeeze every bit of available performance. When dealing with specific processor features and conditional compilation, surprises are always possible. And of course, benching on actual hardware is also a must.

Cargo and rustc compile tests (and benches) to simple command line executables. They embed basic test-filtering logic, a simple result formatter, and can naturally be called from the command line.

When you have a command line.

Running tests on phones

On Android, things are relatively straightforward. Cargo can be cajoled into compiling executables for the Android platform. Then adb can be used to push the binary to the phone and run it.

Manually running tests on Android

iOS is a different story: the only way to run code on a device is to make it an app. And an app has to be signed. Then it has to be properly installed, and can finally be run using LLDB’s remote capabilities. In order to sign an app, a developer needs an account and certificate from Apple. It no longer requires a paying developer account, a developer can now obtain a free certificate valid for their own phone.

But we were bored going through these error-prone steps “by hand”. And we
wanted to lower the cost to make it easier for other developers as well.

Dinghy

Dinghy is a cargo extension. For iOS, the setup takes only a few minutes of
clicking through XCode to obtain a certificate. Less than that if you already
are a iOS developer, or if you are an Android user. Anyway, it’s a one time cost. Once it’s done, it will be used for all your cargo projects.

Running a test suite on a connected phone becomes as easy as:

cargo dinghy test

Running TSS tests on an iPhone and an Android phone

And of course the same apply for benches:

cargo dinghy bench

We have done our best to cover the same command line format as cargo, so… just add “dinghy” between “cargo” and “test”, and you’re done. Feature flags work, as well as test arguments (for filtering).

Examples from the Rust ecosystem

We have tried to run the tests of a few popular Rust projects, and a few
of them that put hurdles in our way in the past weeks. Some projects work out of the box. Others have external dependencies that they try (and sometimes fail) to cross-compile. Some will not even bother trying and will fail miserably.

  • byteorder works out of the box
  • nom works (a few tests are relying on external files that are not
     shipped to the phone)
  • num-bigint works out of the box
  • we have made ramp work (PR pending)
  • flate2 works, except for two tests involving external files
  • sodiumoxyde does not work out of the box (the build script fails to cross compile)
  • *ring* fails to cross compile
  • hyper just works (if you disable SSL)
  • backtrace fails (does not even compile, but it’s a known limitation)
  • core-foundation works out of the box

Conclusion

Many portability issues are relatively easy to fix, but most of the time nobody is actually aware of their existence. We think Dinghy allows a developer to test on a phone, find an issue, reproduce it, report it, discuss it, and ultimately get it fixed. Unit tests and test cases are the true keystones of collaborative development process. CI would be ideal, of course, but, let’s face it, it’s hugely expensive to CI on phone hardware.

But if you are a Rust library developer, install Dinghy and obtain a certificate, plug your phone in, go to the project you are working on right now, and give it a shot.

Let’s get Rust everywhere.

If you enjoyed this article, it would really help if you hit recommend below :)

Follow us on twitter @kalizoy & @snips

If you want to work on AI + Privacy, check our jobs page!