Generic Methods in Rust: How Exonum Shifted from Iron to Actix-web

The Rust ecosystem is still growing. As a result, new libraries with improved functionality are frequently released into the developer community, while older libraries become obsolete. When we initially designed Exonum, we used the Iron web-framework. We’ve now decided to shift the Exonum platform to the actix-web framework.

In this article, we describe how we ported the Exonum framework to actix-web using generic programming.

Exonum on Iron

In the Exonum platform, the Iron framework was used without any abstractions. We installed handlers for certain resources and obtained request parameters by parsing URLs using auxiliary methods; the result was returned simply in the form of a string.

The process looked (approximately) like the following:

In addition, we used some middleware plugins in the form of CORS headers. We used mount to merge all the handlers into a single API.

Our Decision to Shift Away from Iron

Iron was a good library, with plenty of plugins. However, it was written in the days when such projects as futures and tokio did not exist.

The architecture of Iron involves synchronous requests processing, which can be easily affected by a large number of simultaneously open connections. To be scalable, Iron needed to become asynchronous, which would involve rethinking and rewriting the whole framework. As a result, we’ve seen a gradual departure from using Iron by software engineers.

Why We Chose Actix-Web

Actix-web is a popular framework that ranks high on TechEmpower benchmarks. It has an active developer community, unlike Iron, and it has a well-designed API and high-quality implementation based on the actix actor framework. Requests are processed asynchronously by the thread pool; if request processing panics, the actor is automatically restarted.

Previously, concerns were raised that actix-web contained a lot of unsafe code. However, the amount of unsafe code was significantly reduced when the framework was rewritten in a safe programming language — Rust. Bitfury’s engineers have reviewed this code themselves and feel confident in its long-term stability.

For the Exonum framework, shifting to actix solved the issue of operation stability. The Iron framework could fail if there were a large number of connections. We have also found that the actix-web API is simpler, more productive and more unified. We are confident that users and developers will have an easier time using the Exonum programming interface, which can now operate faster thanks to the actix-web design.

What We Require from a Web Framework

During this process we realized it was important for us not to simply shift frameworks, but to also devise a new API architecture independent of any specific web framework. Such architecture would allow for creating handlers, with little to no concern about web specifics, and transferring them to any backend. This conception can be implemented by writing a frontend that would apply basic types and traits.

To understand what this frontend needs to look like, let’s define what any HTTP API really is:

· Requests are made exclusively by clients; the server only responds to them (the server does not initiate requests).

· Requests either read data or change data.

· As a result of the request processing, the server returns a response, which contains the required data, in case of success; or information about the error, in case of failure.

If we are to analyze all the abstraction layers, it turns out that any HTTP request is just a function call:

Everything else can be considered an extension of this basic entity. Thus, in order to be independent from a specific implementation of a web framework, we need to write handlers in a style similar to the example above.

Trait `Endpoint` for Generic Processing of HTTP-requests

The most simple and straightforward approach would be declaring the `Endpoint` trait, which describes the implementations of specific requests:

Now we need to implement this handler in a specific framework. For example, in actix-web it looks like the following:

We can use structures for passing request parameters through the context. Actix-web can automatically deserialize parameters using serde. For example, a=15&b=hello is deserialized into a structure like this one:

This deserialization functionality agrees well with the associated type Request from the `Endpoint` trait.

Next, lets devise an adapter which wraps a specific implementation of `Endpoint` into a RequestHandler for actix-web. Pay attention to the fact that while doing so, the information on Request and Response types disappears. This technique is called type erasure — it transforms static dispatching into a dynamic one.

At this stage, it would be enough just to add handlers for POST requests, as we have created a trait that is independent from the implementation details. However, we found that this solution was not quite advanced enough.

The Drawbacks of the `Endpoint` Trait

A large amount of auxiliary code is generated when a handler is written:

Ideally, we need to be able to pass a simple closure as a handler, thus significantly reducing the amount of syntactic noise.

Below we will discuss how this can be done.

Light Immersion into Generic Programming

We need to add the ability to automatically generate an adapter that implements the `Endpoint` trait with the correct associated types. The input will consist only of a closure with an HTTP request handler.

Arguments and the result of the closure can have different types, so we have to work with methods overloading here. Rust does not support overloading directly but allows it to be emulated using the `Into` and `From` traits.

In addition, the returned type of the closure value does not have to match the returned value of the `Endpoint` implementation. To manipulate this type, it must be extracted from the type of the received closure.

Fetching Types from the `Fn` Trait

In Rust, each closure has its own unique type, which cannot be explicitly indicated in the program. For manipulations with closures, we use the `Fn` trait. The trait contains the signature of the function with the types of the arguments and of the returned value, however, retrieving these elements separately is not easily done.

The main idea is to use an auxiliary structure of the following form:

We have to use PhantomData, since Rust requires that all the generic parameters are indicated in the definition of the structure. However, the type of closure or function F itself is not a generic one (although it implements a generic `Fn` trait). The type parameters A and B are not used in it directly.

It is this restriction of the Rust type system that precludes us from applying a simpler strategy by implementing the `Endpoint` trait directly for closures:

In the case above, the compiler returns an error:

The auxiliary structure SimpleExtractor makes it possible to describe the conversion of `From`. This conversion allows us to save any function and extract the types of its arguments:

The following code compiles successfully:

Specialization and Marker Types

Now we have a function with explicitly parameterized argument types, which can be used instead of the `Endpoint` trait. For example, we can easily implement the conversion from SimpleExtractor into RequestHandler. Still, this is not a complete solution. We need to somehow distinguish between the handlers for GET and POST requests at the type level (and between synchronous and asynchronous handlers). In this task, marker types come to our aid.

Firstly, let’s rewrite SimpleExtractor so that it can distinguish between synchronous and asynchronous results. At the same time, we will implement the `From` trait for each of the cases. Note that traits can be implemented for specific variants of generic structures.

Now we need to declare the structure which will combine the request handler with its name and type:

Next, we declare several empty structures that will act as marker types. Markers will allow us to implement for each handler their own code to convert the handler into the previously described RequestHandler.

Now we can define four different implementations of the `From` trait for all combinations of template parameters R and K (the returned value of the handler and the type of the request).

Facade for the Backend

The final step is to devise a facade that would accept closures and add them into the corresponding backend. In the given case, we have a single backend — actix-web. However, there is the potential of additional implementation behind the façade. For example: a generator of Swagger specifications.

Note how the types of the request parameters, the type of the request result, and the synchrony/asynchrony of the handler are derived automatically from its signature. Additionally, we need to explicitly specify the name and type of the request.

Drawbacks of the Approach

The approach described above, despite being quite effective, has its drawbacks. In particular, endpoint and endpoint_mut methods should consider the implementation peculiarities of specific backends. This restriction prevents us from adding backends on the go, though this functionality is rarely required.

Another issue is that we cannot define the specialization of a handler without additional arguments. In other words, if we write the following code, it will not be compiled as it is in conflict with the existing generic implementation:

As a result, requests that do not have any parameters must still accept the JSON string null, which is deserialized into (). This problem could be solved by specialization in C ++ style, but for now it is available only in the nightly version of the compiler and it is not clear when it will become a stable feature.

Similarly, the type of the returned value cannot be specialized. Even if the request does not imply a certain type of the returned value, it will still pass JSON with null.

Decoding the URL query in GET requests also imposes some unobvious restrictions on the type of parameters, but this issue relates rather to the peculiarities of the serde-urlencoded implementation.

Conclusion

As described above, we have implemented an improved API, which allows for a simple and clear creation of handlers, without the need to worry about web specifics. These handlers can work with any backend or even with several backends simultaneously.