Statically Typed Client-Server Communication with F#: Proof of Concept

Introduction

Fable, the F# to Javascript compiler, has recently added support for a subset of the FSharp.Reflection namespace. Which is really great, allowing us to inspect type information at run-time in the browser.

Today, we will be exploring an application of meta-programming using FSharp.Reflection to abstract a very common task in web developement: client-server communication. The client in this case is a javascript application, written in F#, compiled with Fable to javascript to run in the browser. The server, is a Suave web server, written in F# and runs on .NET.

In this article I will walk you through a minimal implementation of this abtraction that I had been working on and besides showing you the results, I also will show you how and why I got these results explaining most the code along the way.

Abstraction

Let me explain to you with code what I mean by abstracting client-server communication. Imagine you have this async function on the server:

Pretty simple, it has the type string -> Async<int> . The goal is to be able to call it from a client Fable application on the browser and getting the results in a type-safe manner:

Wouldn’t that be nice? The benefits is that all types will be statically resolved at compile-time. We don’t think about serialization and deserialization. We don’t think about coming up with an http end-point on the Suave server and we do not have to send an http request from the client to that end point. It would be a great addition to F#’s happiness stack.

We will for now restrict ourselves with functions of type 'A -> Async<'B> and try to figure out how to automate the communication.

Approach

Our implementation will have two parts, one for the server and one for client. Both implementations will have to follow some sort of a protocol, therefore this protocol must be something I can share between the client and the server. Yeah you guessed it! It is either an interface or a record type referenced from both client and server where the user will be providing the actual functionality of the methods on the server and the client will only call a proxy that delegates the arguments to the server and gets the results back. I will go with the record type because I don’t want to deal with overloaded methods on interfaces.

The server will be responsible for:

• Mapping methods in the record type to http end points on the server.
• Deserializing incoming data.
• Passing the deserialized data as arguments to the correct method, invoking it dynamically at run-time.
• Compute results and return them serialized back to the client.

The client will be responsible for

• Creating a proxy from the shared record type.
• Computing what url to call based on the method name.
• Serializing the arguments of the called method and send them to the mapped url.
• When (serialized) results are back from server, deserialize them on the client.

Next, I encourage you to follow along with the implementation. using an fsharp script (.fsx) or a fsharp project.

Example Shared Record

Consider the record:

This record represents what the server can do. It will live in a seperate file or project referenced by both client and server.

On the server, A user (of this implementation) will have to provide functionality for the record, i.e.

Then I want to able to call the getLength method dynamically at run-time. Well, actually getLength is not a method, Although I will keep refering to it as such. Is it a property of type FSharpFunc<string, FSharpAsync<int>> . So I have to get it from the properties of the record, and pass some argument to it to get the async results back and some how extract (unwrap) the 'B from the Async<'B> because I want to serialize the result when sending it back to the client.

Server: Dynamic Invocation

The server must be able to invoke functions of a record at run-time. Our focus is to deal with functions of type 'A -> Async<'B> . But I think it is a good idea to start with just 'A -> 'B and work our way to handle async functions too. So first we simplify our record type to:

Then provide the functionality:

My first try for writing a function to do the invocation dynamically was straightforward:

You can see that type 'A will be infered from the type of methodArg but I still have to provide the return type 'B when calling the function, otherwise F# will complain about the results being of an undetermined generic type:

Nice! it worked. But I don’t want to provide a return type explicitly at compile-time. That is because what function gets executed on the server is determined by the incoming http request and it’s url that maps to that function at run-time. Therefore I only want to annotate my return type to be any obj. That will be a problem because if I just replace int with obj , then the operation unbox<'A -> 'B> is reduced to unbox<'A -> obj> and it will try to cast the actual property of type FSharpFunc<string, int> to FSharpFunc<string, obj> and that throws a run-time exception:

System.InvalidCastException: Unable to cast object of type ‘simpleSyncServer@6’ to type ‘Microsoft.FSharp.Core.FSharpFunc`2[System.String,System.Object]’.
at Microsoft.FSharp.Core.LanguagePrimitives.IntrinsicFunctions.UnboxGeneric[](Object source)

Where type “simpleSyncServer@6” is a class deriving from FSharpFunc<string, int> . I realized then I don’t want obj but ratherdynamic and let the DLR decide how and what to cast. F# doesn’t support the dynamic keyword, C# does!

C# to the rescue! A little bit of it won’t hurt, especially if it gets the job done :)

If you are wondering about what fsFunc.Invoke() is, it is just an instance method of FSharpFunc<T, U> that I am calling dynamically. Let us see if this works:

Yes it does! This is very convenient, we return the result of the invoked function only it is boxed!

Keep this in mind because next, we will take this up a notch and handle functions of 'A -> Async<'B> .

I had been fighting with this for a while to get it working. For example getLength has return type Async<int> . When I call getLength and box the result to obj and unbox it back (I need to unbox the value from the async for serialization before sending it back to client), it throws an exception because it does not know that 'T in Async<'T> must be an int. It seems that the type information of the 'T must be encoded somewhere and extracted back when unboxing the async.

The technique starts with this helper class AsyncBoxer<'T> , it unboxes an Async<'T> to an Async<obj> correctly:

This way, when unboxing obj to Async<'T> and then box again to Async<obj> it will now know what 'T is , provided that I give that information when creating an instance of AsyncBoxer<'T> . But where do I get that information from? Well, it just the 'B from 'A -> Async<'B> . I can extract that from the property info at run-time on the given record method. Our dynamicallyInvoke becomes a little bit involved:

Notice the part:

Here is where I put the type information of B into the AsyncBoxer<_> that makes unboxing to the correct type possible at run-time. At this point IT WORKED and no more run-time execptions! (for now)

Server: Automatic Route Generation

For every method on the record type, there must a corresponding route with an url such that when the server recieves an http request on that route, the data is extracted and deserialized using FableConverter from the body of the request, passed to the corresponding method on the record and returns the results serialized.

FableConverter is a specialized json (de)serializer that works with Json.NET such that the serialization and deserialization of fsharp values become compatible with serialization and deserialization on the client side with Fable.

One way of generating the routes is mapping every method in record type to the url /type-name-of-record/method-name. For example, our IServer above will one have one route, namely /IServer/getLength .

Generating the urls from a type is quite simple, assuming for now all properties are functions/methods:

However, I don’t want to just generate urls, I also want to handle passing the data to the correct method, it goes as follows, starting with these helper functions:

And now the function that generates the routes returns a WebPart:

Quite simple really, just creates a POST request handler per method and combines them with choose to get one WebPart .

Let’s try it out!

I created a console project along with the library I am writing. Here is the Program.fs :

Run

Send a POST request to http://localhost:8080/IServer/getLength with body “test-input”. I am using Postman.

POST Request to /IServer/getLength

And the response

Correct Response

Perfect! the server side is “finished”. Now we turn to the client

Fable Client: Proxy Template Function

Lets recap. A proxy is just an object that will mimic the type of the server (shared) record such that the user will use it as if it was calling actual functionality but actually the proxy will only delegate the arguments to the server and come back with results. To make this more concrete, a function call to obj.method args will be translated to a POST request with url = /obj-typa-name/method-name and data = toJson args . This applies to all methods of the record. We will create a template for every proxy method. I am using Fable.PowerPack to make the post requests with promises and turn them into asyncs:

Now we generate the proxy, it is just an empty or fake object literal with the same fields as with our record type. For example, a proxy for IServerin javascript is something like this:

var Server = {
    getLength: function(data) {
          return proxyFetch("IServer", "getLength")(data);    
    }
}

this is excatly what we will generate, first a couple of helper functions:

Thanks to Fable’s support for Reflection, I can call FShapType.GetRecordFields and get the names of the fields, such as getLength .

setProp just sets a property of a javascript object to some value. for example setProp "getLength" f obj becomes obj["getLength"] = f when compiled. Now combining these functions to write the function that creates the proxy:

Client + Server: Putting it all together

Almost done, I created a test application called MetaApp.Client (see github repo at the end of the article) it references a file (Models.fs) from MetaApp.Shared where the type of IServer is defined. At the same time, MetaApp.Shared is refenced (as an assembly) from MetaApp.Server where the suave web server is providing the functionality. Lets go through the code:

MetaApp.Shared has one file called Models.fs

MetaApp.Server has only one file Program.fs and it serves an index.html

As you can see, the server is also serving some static files, index.html and javascript files. Here is where it went a bit complicated with the files: the compiled client application went to the public folder relative to the assembly file of the server, that is inside /bin/Debug/public . I am sure there are better ways to do this.

Index.html is a minimal html page:

MetaApp.Client has one file called Main.fsx . I had problems using the fsproj so I just used a simple script and referenced everything I needed in there:

Run MetaApp.Server and open http://localhost:8080

YES!! that worked like a charm.

End of the journey

This had been quite a journey for me and I enjoyed it a lot! I hope you enjoyed it too and also learned something from it. This implementation is far from complete, but it gives you an idea on how to implement such feature or at the very least, if you will ever use it or use something similar, have an understanding of the magic going on behind the scenes.

If you liked this article, make sure to recommend it (by clicking the heart icon below) and share it. Thank you!

Sources

I published all the code represented above to my repo here:

Zaid-Ajaj/Fable.Remoting

Make sure to check it out.