Passing Dynamic Queries Between Client and Backend Server
To get the most out of this article, you will need a good understanding of creating Expressions with the .NET Framework. If you don’t already, then check out this article[^] before going any further.
During the development of our current ASP.NET Core web application, I had a particular set of forms where the client needed to dynamically query one particular dataset. The data being returned related to permission data. The data could be queried in a number of different ways e.g. the permissions for a particular user or the permissions for a particular service. Although the same data was being queried and returned in both cases, they would need to be implemented as two completely separate GET Requests.
All our queries are RESTful GET commands which invoke our ASP.NET Web API backend services. Each new query would involve creating a new controller, service-layer code, data-layer code etc. Much of this code would be very similar as it was effectively querying the same data and returning the same data structures (models).
This got me thinking. Instead of implementing each of these queries separately, that I instead create a dynamically queryable service instead. The client application passes a query to the service. The service executes this query against the data and returns the result back to the client. This would give the client the flexibility to query the data in any way as required.
I wasn’t even sure if this was possible. After much investigation, I came across some posts on Stackoverflow confirming that it was indeed possible. The client application would create an Expression tree. This would be serialised and sent to the ASP.NET Web API service where it would be de-serialised and executed.
The first problem would be serialising the Expression. It turns out that .NET Expression trees cannot be serialised / de-serialised. An Expression is not based on a static structure in the same way as a class, and therefore does not contain any definition for its structure. A class can be serialised because it contains type and structure meta data that can used by the serialiser. An Expression tree contains none of this meta data.
It turns out that there is a nuget package called Remote.Linq[^] that is able to handle the serialisation of Expressions. This handles all the serialisation / de-serialisation allowing your Expression to be passed to your backend service from the client application.
The first step is to add two package references to your project in Visual Studio.
1. Remote.Linq
2. Remote.Linq.Newtonsoft.Json
These will add the necessary extension methods and functionality needed to serialise / de-serialise your Expression trees.
You may need to create some helper functions similar to the ones below. These encapsulate the logic involved with serialising / de-serialising your Expression trees.
Hide Expand
Copy Code
/// <summary>
/// Deserialise a LINQ expression tree
/// </summary>
public Remote.Linq.Expressions.Expression DeserialiseRemoteExpression<TExpression>(string json) where TExpression : Remote.Linq.Expressions.Expression
{
JsonSerializerSettings serializerSettings = new JsonSerializerSettings().ConfigureRemoteLinq();
Remote.Linq.Expressions.Expression result = JsonConvert.DeserializeObject<TExpression>(json, serializerSettings);
return result;
}/// <summary>
/// Serialise a remote LINQ expression tree
/// </summary>
public string SerialiseRemoteExpression<TExpression>(TExpression expression) where TExpression : Remote.Linq.Expressions.Expression
{
JsonSerializerSettings serializerSettings = new JsonSerializerSettings().ConfigureRemoteLinq();
string json = JsonConvert.SerializeObject(expression, serializerSettings);
return json;
}/// <summary>
/// Convert the specified Remote.Linq Expression to a .NET Expression
/// </summary>
public System.Linq.Expressions.Expression<Func<T, TResult>> ToLinqExpression<T, TResult>(Remote.Linq.Expressions.LambdaExpression expression)
{
var exp = expression.ToLinqExpression();
var lambdaExpression = System.Linq.Expressions.Expression.Lambda<Func<T, TResult>>(exp.Body, exp.Parameters);
return lambdaExpression;
}
With those created, the first thing you will need to do is create your Expression and serialise it.
The examples I will use all relate to the scenario I described at the beginning of the article i.e. the ability to dynamically query permissions data.
Here’s a simple Expression that returns a list of permissions for the specified permission ID (in reality there would only ever be one permission returned for any given permission ID but for the purposes of this example let’s assume that one or more permissions will be returned by the Expression).
Hide Copy Code
const int permissionid = 1;
Expression<Func<PermissionEntities, List<PermissionEntity>>> expr1 = m => m.Permissions.FindAll(q => q.PermissionId == permissionid);Next we need to convert the Expression into a Remote.Linq expression and serialise it.
Hide Copy Code
var serialised = SerializerManager().SerialiseRemoteExpression(expr1.ToRemoteLinqExpression());The extension method ToRemoteLinqExpression() is provided by Remote.Linq and converts a .NET Expression into a Remote.Linq expression.
With our Expression now serialised into a string, we can pass it into a function to execute against our permission data. The function will need to perform the following actions.
1. De-serialise the Remote.Linq expression
2. Convert the Remote.Linq Expression into a .NET Expression
3. Invoke and execute the Expression against permissions data
Here’s an example of a function that accepts a serialised Remote.Linq expression and executes it against a permissions dataset.
Hide Expand
Copy Code
/// <summary>
/// Return the specified permissions from the remote expression
/// </summary>
public PermissionModels GetPermissionsDynamic(string payload)
{
if (string.IsNullOrEmpty(payload)) return null;
//create an empty default permission model
PermissionModels result = new PermissionModels();
//de-serialise back into a Remote.Linq Expression
Remote.Linq.Expressions.LambdaExpression expression =
SerializerManager().DeserialiseRemoteExpression<Remote.Linq.Expressions.LambdaExpression>(payload) as LambdaExpression; //convert the Remote.Linq Expression into a .NET Expression
var localexpression = SerializerManager().ToLinqExpression<PermissionEntities, List<PermissionEntity>>(expression); //grab all the permissions from the DB
PermissionEntities permissions = this.Data.GetPermissions();
//compile and invoke the expression
var compiled = localexpression.Compile();
var matches = compiled.Invoke(permissions); //if no matches were found then just return the default object with no items
if (matches == null || !matches.Any()) return result; return matches
}
Putting all the pieces together, here’s a simple unit test that demonstrates how to create an Expression and pass this to the above function to execute against actual data.
Hide Copy Code
[TestMethod]
public void GetPermissionsDynamicTests()
{
//Arrange
const int permissionid = 1;
PermissionsService service = new PermissionsService();
Expression<Func<PermissionEntities, List<PermissionEntity>>> expr1 = m => m.Permissions.FindAll(q => q.PermissionId == permissionid); var serialised = SerializerManager().SerialiseRemoteExpression(expr1.ToRemoteLinqExpression()); //Act
var results = service.GetPermissionsDynamic(serialised); //Assert
Assert.IsNotNull(results);
Assert.IsNotNull(results.Permissions);
Assert.IsTrue(results.Permissions.Any());
Assert.IsNotNull(results.Permissions.Find(q => q.PermissionId == permissionid));
}
To complete this we would need to write a controller method that invoked the function GetPermissionsDynamic(). It should be noted that although we are creating dynamic queries over HTTP, they will need to be implemented as POST rather than GET. The reason for this (as I found out) is because a GET querystring is limited in length. A serialised Expression will almost certainly break that limit. Therefore place the serialised Expression in the POST body of your Request. It is also more secure to pass your Expressions this way as they are less visible to prying eyes. You may want to consider encoding / encrypting the Expressions you pass from your client to your service for added security.
I wouldn’t use this pattern for every query. I would still create a single GET query for each Request for the majority of the queries I implement. However, when you have several queries all acting on the same data and returning the same data structures (models), then this pattern allows you to simply and easily implement those as dynamic queries. Like all patterns, it can solve a very specific problem if used as intended and its usage is clearly understood by the developer.
