Airframe RPC: A Framework for Using Scala Both for Frontend and Backend Programming
Airframe RPC is an open-source framework for building RPC services, using Scala as a unified RPC interface between servers and clients.
Why Airframe RPC?
Airframe RPC enables calling Scala methods at remote servers. You don’t need to worry about how to encode your data into JSON, nor how to define HTTP REST endpoints. Airframe RPC abstracts away these details; the framework generates code for serializing your data objects into JSON or MessagePack and calls appropriate HTTP endpoints on your behalf.
While gRPC has been a popular approach for building RPC services, it’s built around Protobuf technology for defining data structures and RPC methods. This means, you and your collaborators need to use Protobuf ecosystem almost for everything to enjoy the benefits of gRPC. And also, gRPC heavily uses HTTP/2 features, some of them are not supported in web browsers, so if you need to write web applications using gRPC, an additional proxy like gRPC-Web is required.
In 2020, Scala.js, which can compile Scala code into JavaScript, finally became 1.0.0 after 7 years of development. This has paved a way for using Scala both for server (Scala JVM) and client (Scala.js) programming. We explored an approach for using Scala’s functional interfaces as RPC endpoint definitions, and successfully created Airframe RPC on top of technology stack of 20+ Airframe modules.
Although Airframe RPC is a relatively new project started on January 2020 inside Arm Treasure Data, this project has proved various advantages. For example:
- Free from REST. We can just use Scala’s functional interface for defining servers. Google’s REST API Design Guide has been useful resources for defining clear REST API endpoints, but we’ve found using programming language’s native interface is much easier.
- No more web-framework wars. In Scala, there are many web frameworks, such as Finatra, Finch, Akka HTTP, and our own airframe-http, etc. Each of them has its own pros and cons, and choosing one of them has been a hard choice for us. Now, we can just start from Airframe RPC using plain Scala interfaces. If necessary, we can use airframe-http for adding custom HTTP endpoints.
- Seamless integration with Scala.js. Writing web browser applications in JavaScript that interact with servers is not easy. You may need to learn about existing frameworks like React.js, Vue.js, etc., and a lot of techniques for using them. By using Scala both for server and client code, an engineer just joined the company could write an RPC application using Scala and Scala.js in a few days.
Airframe RPC: Overview
For using Airframe RPC, first, define your RPC service interface using regular Scala functions by adding @RPC annotation. All public methods in this class will be your RPC endpoints. For the method arguments and return types, you can use arbitrary types (See Object Serialization for the list of available types). To pass complex messages, you can use case classes as well.
Here is an example RPC interface:
Next, implement this service interface in Scala:
To start an RPC web server, Airframe RPC provides Finagle-based web server implementation. The following code starts an RPC web server at http://localhost:8080/:
To access the RPC server, we need to generate an RPC client from the RPC interface definition. We can use an RPC client hello.api.v1.ServiceSyncClient generated by sbt-airframe, which reads an RPC interface code and generates HTTP client code for calling RPC methods.
Now, you are ready to call remote Scala methods:
That’s it! Now you can call remote Scala methods as if they were regular Scala functions. Airframe RPC also supports asynchronous clients using Future.
See also the documentation for more details:
What Makes Airframe RPC Unique?
The primary goal of Airframe RPC is using plain Scala functions for RPC as much as possible. In the above examples, only @RPC annotation is the special syntax in Airframe RPC. Other than that, everything can be written in plain Scala without any weird syntax nor DSL. That means if you know how to write Scala, it will be straightforward to build RPC services with Airframe RPC.
There are existing RPC frameworks for Scala, such as Sloth (pure-Scala approach) or gRPC wrappers like muScala, ScalaPB, etc., but none of them provides the same level of simplicity with Airframe RPC because they are trying to create DSLs over Scala’s type system or building bridges for gRPC.
To support natural RPC interface definitions, Airframe RPC uses various techniques behind the scene, notably, runtime/compile-time object-shape inspection with airframe-surface, schema-on-read codec of airframe-codec to support arbitrary data serialization, dynamic class loading in sbt-airframe plugins, etc. Providing a simple RPC interface for users actually needed significant amount of work. “Think Simple”, a slogan of Apple, doesn’t mean we can create a simple interface at ease.
Here, I’d like to thank my colleagues for helping Airframe RPC development. They used this RPC framework even before any documentation existed and helped me for fixing weird bugs and improving the usability until we became comfortable to use it in production.
Future Work
Here are a couple of items we are planning for Airframe RPC:
- Generating Open API specification from RPC interfaces. This will be useful for generating and implementing RPC clients for other than Scala.
- Documentation of airframe-http-rx library. Along with Airframe RPC, we also have built a reactive DOM rendering library (airframe-http-rx) for Scala.js. This resolves several shortcomings of existing ScalaTags and monadic-html and interacts nicely with Airframe RPC servers.
- Leveraging HTTP/2. The first milestone of Airframe RPC is completed as we can call remote Scala functions from Scala and Scala.js. We need to think about how to leverage HTTP/2’s advanced features for supporting bi-directional streaming like gRPC.
Related Article
Airframe RPC is built on top of Airframe HTTP:
