Airframe HTTP: Building Low-Friction Web Services Over Finagle
airframe-http is the latest addition to Airframe, lightweight building blocks for Scala. airframe-http is a library for mapping HTTP requests to functions in Scala. This is not a web framework, rather, it can be used as a thin-wrapper over HTTP server libraries, such as Finagle.
To define REST APIs on top of a low-level HTTP server library like Finagle, we usually needed an additional web framework, such as Finatra, Finch, etc., each of them has its own coding style. For example, Finatra uses Google Guice for injecting necessary modules. Finch adopts purely-functional style, so it might not be appropriate depending on your team’s coding preference.
Airframe already has a built-in dependency injection library, which redesigned Google Guice from scratch to be more Scala-friendly. Using Finatra, which depends on Google Guice, is sort of a drawback for me. So I built airframe-http to provide the best practice for building web services using Airframe.
Defining Web APIs as Scala Functions
To define web service APIs, we basically need a set of functions which receive HTTP requests, then return HTTP responses. The content body of HTTP requests and responses often has JSON data (or MessagePack for efficiency), representing some message objects. Manually writing mappings between HTTP requests/response and message objects (e.g., Scala case classes) are cumbersome, so we need to simplify this mapping process.
In airframe-http we use Endpoint annotation to define a mapping from an HTTP request path to a function in Scala. This example shows how to map an HTTP request for /user/:name to a function call getUser(name):
It is also possible to return Future[User] for building asynchronous APIs. For example, if you need to access external resources inside the API implementation (e.g., accessing another Web API, or making database requests), Future responses will be useful to concatenate these resource accesses in a sequence.
A good thing about airframe-http is that no DSL is required other than the Endpoint annotations. The above example is just a regular Scala trait that is independent from any web framework, so we can write test code for the implementation without running web servers. When we use this trait with airframe-http, it will translate HTTP requests to corresponding function calls in Scala, and returns its result as JSON HTTP responses.
Starting A Finagle HTTP Server
airframe-http-finagle is an extension of airframe-http to use Finagle as the web-server backend. airframe-http-finagle is available since Airframe version 0.66:
To start an HTTP server backed by Finagle, you need to define Router and Airframe design as follows:
When building a FinagleServer instance, a server thread for receiving HTTP requests will start at the specified port. If Airframe session terminates, the HTTP server will automatically terminate. This is using the life cycle management feature of Airframe. To add more REST endpoints, you can use Router.add[X] method.
That’s it! You can find more advanced examples in our documentation. After seeing this example, you might be wondering why existing web frameworks (e.g., Finatra, Finch, akka-http, etc.) needed to define complex DSLs for HTTP request mapping. In the following section, we will see why airframe-http can provide such a simple interface without creating DSL on top of Scala.
A Secret Source: Airframe Surface
To map HTTP request parameters to function calls, we need to know function argument names and their concrete types. However, during compilation from Scala code into JVM byte code, function argument names and types will be removed (type erasure). So existing web frameworks (e.g., Finatra, Finch, akka-http, etc.) needed to define DSLs to complement the information that will be lost after the compilation. For example, annotations to function arguments are frequently used to tell the frameworks about the method argument names and types.
What we can do to avoid such heavy annotation usage? Our answer is using Scala Signatures (ScalaSigs), which are the detailed type information embedded by Scala compiler as hidden data inside class files. By reading Scala signatures, we can know even generic type parameters, which will be lost after type erasure. For example, Seq[User] type will be compiled to just Seq[java.lang.Object] inside the class file. If we read Scala signatures, we can find its original type Seq[User]. We can also read method argument names and types as well.
airframe-surface is a library for reading Scala signatures by using Scala reflection (For Scala.js, it uses Scala Macros to extract type information at compile-time). airframe-surface enables reading detailed type information from classes and methods. This information makes easier to define mappings between HTTP requests to Scala functions.
Mapping HTTP Requests to Method Calls
airframe-http uses airframe-surface to check the method signatures of a given web API class. After finding methods that have Endpoint annotations, we read method arguments to define HTTP request routing table.
HTTP request/response data are string (or JSON) values, but method arguments can have arbitrary types, such as Int, Double, Boolean, or more complex case classes. To perform data conversion between these types, we usually need some serialization libraries, such as Jackson, play-json, circe, etc. In airframe-http, we have a built-in serialization library, so we even don’t need to think about selecting one of such serialization libraries.
airframe-codec is another module of Airframe, for serializing data through MessagePack format. airframe-codec contains a collection of MessagePack-based data conversion rules for regular Scala types, including primitive types, Scala collections, case objects, etc. So for almost all data classes, we can generate data conversion codecs (e.g., JSON -> case class mappings) by combining these pre-defined rules.
For interested readers, here are the detailed steps on how airframe-http calls methods from HTTP requests:
- Given an HTTP request, airframe-http finds a matching method from the routing table.
- For each function argument of the target method, airframe-http will find a MessagePack codec, which translates given MessagePack data into the corresponding argument type data.
- Convert the HTTP request path parameters and query parameters into MessagePack Map value (e.g., Map(“name” -> “xxx”)). If the HTTP request body contains JSON data, it will be converted to MessagePack as well.
- By using the prepared MessagePack codecs in airframe-codec, generate method call argument values.
- ControllerProvider will find an instance of the target Web API class registered in the Airframe Session. Then call the method of the instance using the method arguments generated in the above step.
- ResponseHandler reads the method return type, and finds an appropriate MessageCodec to convert the return value into JSON format. First the returned objects will be converted to MessagePack by using the MessageCodec, then converted into JSON data.
This HTTP request mapping process might be intimidating at first glance, but individual steps are implemented by using small libraries (e.g., airframe-surface, airframe-codec, airframe-json, etc.) that are already available in Airframe.
Use Cases & Future Direction
airframe-http is designed to be an web-framework agonistic library, so it’s possible to create more adapters not only for Finagle, but also for other web frameworks. A reason we supported Finagle first is that Finagle is already matured, and it has been used in our production without any major problems.
A practical use case of airframe-http would be creating a lightweight web server for micro-services. In this context airframe-http is simple enough and can save the learning cost, compared to using existing feature-rich web frameworks.
At the same time, we also understand that web frameworks need to address more complex problems (e.g., authentication, request filtering, HTML rendering, rate control, etc.). In this sense, airframe-http is still in an early phase library, so it will not be an immediate replacement of major web frameworks (e.g., akka-http, Play!, Finatra, etc.). Because of time constraint we may not be able to add rich features that exist in other web frameworks, but we highly appreciate contributions (e.g., PR, bug reports, fixes to documentations, etc.) to improve the quality of the code.
Airframe will continue to be lightweight building blocks for Scala. airframe-http is just one of the examples to show the power of such building blocks. We are planning to add more useful modules to Airframe so that it can make your daily programming in Scala much easier.
