Designing the Rust Unleash API client
Designing the Rust Unleash API client
One interesting question that our post about the Unleash API client for Rust provoked is “What were the Rust language specific design considerations?” We think that is a great question and (with the caveat that, as we haven’t released v1.0 yet, the story is perhaps an incomplete answer) one we’re delighted to address.
In this post we’ll cover a) side effects during otherwise pure function calls, b) async and sync code, c) performance and correctness with an API that is entirely driven by Strings everywhere, and finally d) the UX for users of the API.
Side effects
The Unleash client SDK specification covers the expected behaviour of API clients using JS as a reference point.
Each client is expected to register with the API server on startup, then download the entire set of feature toggles, run for some period of time giving locally calculated answers, and then download updated feature toggles from the API server. Metrics on which toggles were locally consulted and whether they resulted in an on or an off result also get uploaded to the API server periodically.
That is: querying a toggle — a conceptually const operation, needs to perform mut on some data structure in the client state.
There are two possibilities here: Either the mut is exposed to the caller, in which case only one alias can make is_enabled calls at a time — because it needs a `&mut Client` — or we provide for interior mutability. In this case we need thread-safe interior mutability, so something like an Arc<Mutex> is in order. The Java, Python, dot-Net, and Node.js clients don’t need to deal with this aspect of API design, making it a Rust-specific consideration.
A similar but much more limited choice consideration applies in other languages that have const vs non-const pointers: whether to make is_enabled a const method or not. It is more limited because few other languages enforce the restriction that only one alias to a struct may be mutable.
This then is the shape of the is_enabled function (full source).
It uses the arc_swap crate, which allows multiple readers of a data structure (which we need for the strategy evaluations), and we use primitive AtomicBools to update the metrics. Note the return type — more on that later.
We need to run more performance analyses to see whether it is the atomic operations that are the remaining scalability issue and perhaps move those into a thread-local accumulators which we reap once per polling cycle. If not, we need to solve whatever the issue is.
Async and sync
In Rust, async and sync code are asymmetrically compatible. Async code can be run from sync code via a task::block_on call (or similar in Tokio, etc.); sync code run from within async code may cause deadlocks at worst and certainly can be expected to impact performance as other Futures do not get polled.
One approach is to write two clients, or to write plain IO-free sync code for the most part and then async- and sync-specific modules that layer on top of that. I chose to write a single client and provide the IO management directly within it, as the amount of IO is very small and strictly contained. We might change this if there’s a lot of discomfort from threaded server users about depending on, e.g., async-std. Changing this would involve having a new struct that does the registration and polling IO, driving the IO-free code, and then a new sync client-based implementation as a Cargo feature.
In most languages this asymmetry exists, though a crucial factor as to whether one can always block on an async Future is the contract made around the future itself. In Twisted, for instance, because the reactor might be a non-reentrant GUI event loop, futures (‘Deferred’) can’t be safely blocked on always, only sometimes. Although the Rust definition does suffer the same ambiguity, all the evolving runtimes make stronger guarantees, so we think we’re clear of that difficulty.
Performance and correctness with Strings
In Rust, performance and correctness are both valued. Unleash has a difficult problem, though: It has an untyped API — the parameters for every strategy are the same : Hash<String, String>. And while in principle a schema could be provided to clients, this could make things more fragile — and it would certainly make the UI more complicated. But in the absence of a schema, how do we provide performance and correctness to the Rust code?
To an extent we cannot. The job of the Unleash client is to be programmed by externally defined data, changing over time. Part of this definition is that a bad API definition on the server should fail safely. One definition would be to fall back to default (that is, bad definitions act like missing definitions); the current implementation has bad definitions that act like empty definitions, which is also reasonable, if different.
Further, the strategy signature for evaluation wants to take `parameters: Hash<String,String>` which would be shared-state — another potential cross-thread issue — at a minimum. Since this is Rust, the lifetime would need to be managed, making writing new Strategies harder.
To solve both these safety issues and address much of the performance issue of dealing in Strings, we compile the API data into Fn closures that convert the parameters data into a HashSet (or whatever is appropriate) and own it themselves. These are then stored in a Box.
This is similar to what the Unleash Python client does, except that the Python client caches in a field on an object, which is mutable state, rather than returning a separate object, and is depending on GIL or GIL-emulation in non-CPython Pythons to make that safe.
A related source of potential bugs is the name of the feature toggles: each toggle is identified by a String:
All it takes is a simple typo, and a user’s feature will never fire appropriately. The wrong feature will show up in the Unleash API server metrics as queried. The developer will know to search for it, but this is less correct than ideal.
Additionally, because we accept a String, the client has to do a String lookup in a data structure to query the toggle every time through the code path.
A possible future refinement would be to move to a sum type — an enum — rather than a String. This would permit compile time checking that all the call sites for a given feature ‘x’ used the same value rather than depending on the developer typing it in the same way, if the Client struct was parametrised by the enum type. It would also permit having a known size at compile time for the feature set — additional features sent by the API server could be discarded. Whether that will make a performance difference isn’t clear yet, though more benchmarking when some idle time presents itself will be useful in answering that question. If for some reason we wanted to track features that the compiled code didn’t know in advance, then one of the enum values could hold the existing dynamic map of String names.
UX for users of the API
We wanted to keep the overhead of writing strategies as low as possible. Until more are written outside of the crate itself, this is somewhat of an open question, but the current closure approach makes for very little boilerplate, and strategies can reuse code effectively. Rust’s reuse of code amongst trait objects is quite different to other languages; default implementations and blanket implementations are very useful, but building deep hierarchies of refined meaning isn’t.
As before we’d love your feedback. We can be reached at our GitHub repository or in the #unleash-client-rust channel on the unleash Slack.
