Designing Library API In Rust

Source: https://analyticsindiamag.com/unleash-the-power-of-rust-with-these-frameworks/

Table of Contents

History
The Story
— JSON-RPC API
— Libraries
The JSON RPC Library
— The Base Objects
— — The Request Object
— — The Response Object
— — The Error
— The RPC Processor
— The Library API
Outro

History

It’s been two weeks since I started coding using Rust. I’ve published my first story using Rust too:

Beginner Journey Learning Rust

Since then, I’ve never stopped coding in Rust and learning many things through my real coding experiences, and I have to admit, that I’m starting to fall in love with this language.

The Story

Based on my experiences, the most effective way to learn a new language is by building a real project using a real use case (not just Hello World). So last week I decided to build a library for JSON-RPC API.

JSON-RPC API

What is the JSON-RPC API?

A light weight remote procedure call protocol.
It is designed to be simple!

What is RPC?

In distributed computing, a remote procedure call (RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is written as if it were a normal (local) procedure call, without the programmer explicitly writing the details for the remote interaction.

Source:

Remote procedure call - Wikipedia

Example request-response of JSON-RPC:

--> data sent to Server
<-- data sent to Client

--> {"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}
<-- {"jsonrpc": "2.0", "result": 19, "id": 1}

--> {"jsonrpc": "2.0", "method": "subtract", "params": [23, 42], "id": 2}
<-- {"jsonrpc": "2.0", "result": -19, "id": 2}

--> {"jsonrpc": "2.0", "method": "subtract", "params": {"subtrahend": 23, "minuend": 42}, "id": 3}
<-- {"jsonrpc": "2.0", "result": 19, "id": 3}

--> {"jsonrpc": "2.0", "method": "subtract", "params": {"minuend": 42, "subtrahend": 23}, "id": 4}
<-- {"jsonrpc": "2.0", "result": 19, "id": 4}

For me, this API concept is simpler than REST API. The service provider provides many of their RPC methods, so the client is able to choose the method based on their needs.

Example of Ethereum RPC methods:

• web3_clientVersion
• web3_sha3
• eth_mining
• eth_coinbase
• eth_gasPrice

Source:

JSON-RPC API | ethereum.org

Libraries

I know that there are many of Rust’s libraries (or crates) that already provide a library for building the JSON-RPC API, such as:

GitHub - apoelstra/rust-jsonrpc: Rust JSONRPC library

GitHub - paritytech/jsonrpc: Rust JSON-RPC implementation

But my objective here is not just to work, but I want to learn more about the implementation using Rust, so I decided to try to build it.

The JSON-RPC Library

Now, I want to talk about my journey when building this library. There are three basic objects that I need to model from the spec:

• Request object
• Response object
• Error object

The Base Objects

The Request Object

#[derive(Debug, Serialize, Deserialize)]
pub struct RpcRequestObject {
    pub jsonrpc: String,
    pub method: String,
    pub params: Value,

    #[serde(skip_serializing_if="Option::is_none")]
    pub id: Option<RpcId> 
}

For the request object, it is just a simple struct that provides four object properties based on its formal specification. All requests must follow this structure.

The Response Object

#[derive(Debug, Serialize)]
pub struct RpcResponseObject<T, E> {
    pub jsonrpc: String,
    
    #[serde(skip_serializing_if="Option::is_none")]
    pub result: Option<T>,

    #[serde(skip_serializing_if="Option::is_none")]
    pub error: Option<RpcErrorObject<E>>,

    pub id: Option<RpcId>
}

impl<T, E> RpcResponseObject<T, E> {
    pub fn with_success(result: Option<T>, id: Option<RpcId>) -> Self {
        RpcResponseObject { jsonrpc: String::from("2.0"), result, error: None, id }
    }

    pub fn with_error(error: Option<RpcErrorObject<E>>, id: Option<RpcId>) -> Self {
        RpcResponseObject { jsonrpc: String::from("2.0"), result: None, error, id }
    }
}

This object is used when the provider tries to give the response back to the caller or client. On this object, I’ve decided to add some static methods to help construct the object itself. Rather than construct the struct and fill its fields manually, we will be able to construct the object using these helpers.

Example of usage

let response: RpcResponseObject<FakeParam, String>= RpcResponseObject::with_success(Some(result), None);
let jsonstr = serde_json::to_string(&response);
assert!(!jsonstr.is_err());
assert_eq!(jsonstr.unwrap(), r#"{"jsonrpc":"2.0","result":{"key":"testkey","value":"testvalue"},"id":null}"#)

The Error

I’m using Rust’s enum to group error types, also based on JSON-RPC formal specification

#[derive(Debug, Clone, Copy)]
pub enum RpcError {
    ParseError,
    InvalidRequest,
    MethodNotFound,
    InvalidParams,
    InternalError
}

impl RpcError {
    pub fn build(&self) -> (RpcErrorCode, RpcErrorMessage) {
        match self {
           RpcError::ParseError => (PARSE_ERROR_CODE, PARSE_ERROR_MESSAGE),
           RpcError::MethodNotFound => (METHOD_NOT_FOUND_CODE, METHOD_NOT_FOUND_MESSAGE),
           RpcError::InvalidRequest => (INVALID_REQUEST_CODE, INVALID_REQUEST_MESSAGE),
           RpcError::InvalidParams => (INVALID_PARAMS_CODE, INVALID_PARAMS_MESSAGE),
           RpcError::InternalError => (INTERNAL_ERROR_CODE, INTERNAL_ERROR_MESSAGE)
        }
    }
}

These enums will be used when construct the RpcErrorObject :

#[derive(Debug, Serialize, Deserialize)]
pub struct RpcErrorObject<T> {
    pub code: RpcErrorCode,
    pub message: String,
    
    #[serde(skip_serializing_if="Option::is_none")]
    pub data: Option<T> 
}

impl<T> RpcErrorObject<T> {
    pub fn build(err: RpcError, data: Option<T>) -> Self {
        let (code, message) = err.build();
        RpcErrorObject { code, message: message.to_string(), data }
    }
}

Examples:

#[test]
fn test_serialize_error_object() {
    let table = vec![
        (RpcError::ParseError, String::from(r#"{"code":-32700,"message":"Parse error"}"#)),
        (RpcError::InvalidRequest, String::from(r#"{"code":-32600,"message":"Invalid request"}"#)),
        (RpcError::MethodNotFound, String::from(r#"{"code":-32601,"message":"Method not found"}"#)),
        (RpcError::InvalidParams, String::from(r#"{"code":-32602,"message":"Invalid params"}"#)),
        (RpcError::InternalError, String::from(r#"{"code":-32603,"message":"Internal error"}"#)),
    ];
    
    for (validator, input, expected) in table_test!(table) {
        let err: RpcErrorObject<String> = RpcErrorObject::build(input, None);
        let errobj = serde_json::to_string(&err);
        assert!(!errobj.is_err());

        validator.
            given(&format!("{:?}", input)).
            when("build error").
            then(&format!("it should be: {:?}", expected)).
            assert_eq(expected, format!("{}", errobj.unwrap()));
    }
}

The RPC Processor

I’m using the term of processor , since this object is an object that has responsibilities:

• Register multiple RPC method handlers
• Parse incoming request payload
• Match and find the RPC method handler
• Give back the response

The flow will be like this

The implementation:

pub struct RpcProcessorObject {
    pub handlers: HashMap<RpcMethod, Box<dyn RpcHandler>>
}

impl RpcProcessorObject {
    pub fn build() -> Self {
        let mut handlers: HashMap<String, Box<dyn RpcHandler>> = HashMap::new();
        handlers.insert("prople.agent.ping".to_string(), Box::new(AgentPingHandler));

        RpcProcessorObject { handlers }
    }

    pub fn register_handler(&mut self, method: String, handler: Box<dyn RpcHandler>) -> () {
        self.handlers.insert(method, handler);
    }

    pub async fn execute(&self, request: RpcRequestObject) -> Result<RpcResponseObject<Box<dyn ErasedSerialized>, String>> {
        let method = request.method.clone();
        let params = request.params.clone();

        let handler = match self.handlers.get(&method) {
            Some(caller) => caller,
            None => {
                let err_obj: RpcErrorObject<String> = RpcErrorObject::build(RpcError::InternalError, None);
                let response = RpcResponseObject::with_error(Some(err_obj), request.id);
                return Ok(response)
            } 
        };

        match handler.call(params).await {
            Ok(success) => {
                let response = RpcResponseObject::with_success(success, request.id);
                Ok(response) 
            },
            Err(_err) => {
                let err_obj: RpcErrorObject<String> = RpcErrorObject::build(RpcError::InternalError, None);
                let response = RpcResponseObject::with_error(Some(err_obj), request.id);
                Ok(response)
            } 
        }
    }
}

The RPC method handler is any object that is able to implement the base trait of Handler :

#[async_trait]
pub trait Handler {
    async fn call(&self, params: Value) -> Result<Option<Box<dyn ErasedSerialized>>>;
}

It is just a simple trait that must be implemented by all available method handlers. The main point of this trait is, that any handler must accept the serde_json::Value as their main parameter, and must return the output that is able to be serialized by serde too.

To map between the RPC method and its handler, I’m using Rust HashMap

HashMap in std::collections - Rust

Example of this processor object usage:

let processor = RpcProcessorObject::build();
let request = RpcRequestObject{
    id: Some(RpcId::IntegerVal(1)),
    jsonrpc: String::from("2.0"),
    method: String::from("prople.agent.ping"),
    params: Value::Null
};

let response = processor.execute(request).await;
assert!(!response.is_err());

let jsonstr = serde_json::to_string(&response.unwrap());
assert!(!jsonstr.is_err());
assert_eq!(r#"{"jsonrpc":"2.0","result":{"message":"pong!"},"id":1}"#, jsonstr.unwrap())

The Library API

Now, this is my favorite part and one of the reasons why I’m starting to enjoy this language. It is about the library visibility.

The Rust Reference

My library will be look like this

mod errors;
mod id;
mod processor;
mod handler;
mod request;
mod response;

pub mod objects {
    use super::*;

    pub use errors::RpcErrorObject;
    pub use request::RpcRequestObject;
    pub use response::RpcResponseObject;
    pub use processor::RpcProcessorObject;
}

pub mod handlers {
    use super::*;

    pub use handler::AgentPingHandler;
}

pub mod types {
    use super::*;

    pub use processor::types::{RpcHandler, RpcMethod};
    pub use id::RpcId;
    pub use errors::{RpcError, RpcErrorCode, RpcErrorMessage};
}

pub mod prelude {
    use super::*;

    pub use types::*;
    pub use objects::*;
    pub use handlers::*;
}

I really love it. When we are designing a library, we as library authors able to decide which objects, and types should be consumed from the library caller. We can choose which of them that need to be exposed to the external world. And for me this is important. By providing solid library API that can be consumed publicly, we can help our library callers ensure that they are consuming the “safe” object or procedures.

Imagine if there were no public or private, where our callers were able to import and consume anything from our library, they (callers) would not know which object, type, or procedure to work with our library, and try to random pick anything, the next thing will have happened is just chaos and totally mess.

Our caller should not need to know any unnecessary internal details inside our library, what they need to know is just the public API of objects, functions, or methods that they need to consume based on their needs. Rust gives us this ability by providing good visibility management to separate between public and private things.

Outro

The more I learn from this language, the more confident I am in using this language on the production level. I know many people use Rust for low-level needs, but I think that it is also possible to use this language for high-level applications.

For now, I’m not sharing my codes yet, because I’m still working on this library too, once it’s ready I’ll tell you another story 👌