Boost Your Manifest: Add Conditional Logic To Your App’s JSON Manifest
Manifest files are used in many applications to provide configuration, metadata, and other information for components in an application. We often use manifest files to tell our application how it should behave under certain conditions, to provide some external context that isn’t available in the source code or from the UI, or to define settings and functionality for a module that will load them at runtime.
Traditionally, manifest files were represented as XML, YAML or .INI files (the horror!), but most modern applications use JSON files, so this post will focus on that. Everything described here can be applied to any JSON document even if it is not used purely as a manifest (including JavaScript objects carrying settings and metadata).
As an example, JSON manifest files are used in several different places in the Sisense web application. We use manifest files to describe connector metadata and settings, and so these files can also be edited and customized by the end user, if needed. We also use JSON files to describe entire user interfaces (UIs) which are rendered at runtime and to define API routes and endpoints dynamically for our internal API generator module. Manifest files are mainly used internally but some are exposed to the user for editing. In some cases, a user with the role of Administrator may provide their own manifest file to define connector settings and other configuration.
What’s in a Manifest File?
JSON manifest files are just a collection of key-value pairs, where each value can also be an object or an array of values, recursively. This means you can express a pretty elaborate configuration via JSON, but it is still a static one.
One way to express a range of possible values is using regular expressions. This is especially common when defining validation rules (e.g., for valid IP addresses, URLs, emails, etc.), however these validation rules are still static.
What if we need the value of some key to be determined dynamically at runtime, based on some rules?
Or, what if it happens that the value of some key is dependent upon the values of other keys?
We need a way to express conditional logic and value dependencies in JSON format.
JSON as a Language
JSON is only a data representation format and was not designed to express logic or conditions. One possible solution is to make our own language notation for conditions in JSON and then parse it accordingly in our code. Besides the coding and testing overhead involved, this solution makes it harder to use for developers and external users who would need to learn a proprietary “conditional language” to express rules in.
There must be a simpler way. It would be great if we could use some standard, well-known notation like SQL, which also has ready-made parsing libraries available. But expressing SQL in JSON is not so elegant or straightforward.
We need an alternative. What is a widespread, well known conditional logic language that is compatible with JSON?
MongoDB query language!
Indeed, MongoDB uses JSON to represent document structure, and its query language is natively represented as JSON. We could use a library like Mingo, a JavaScript implementation of the MongoDB query language, to easily parse and evaluate elaborate conditions and resolve values at runtime. Mingo can run on the client side or the server side, so it’s suitable for many use cases.
This way, the value of one key can be dependent on the value of another key (or multiple keys) in the manifest, which itself is manipulated at runtime by the application reading the JSON document. A value of one key can also be entirely dependent on runtime context, which is only known at execution time.
For example, a dynamic value for myKey below can be expressed like this:
myKey: {
conditionalValue: {
value: “testWasSatisfied”,
condition: {
someOtherKey.value:
{
<comparison operator ($eq, $in, $gt etc)>: [..values]
}
},
else: “someDefaultStringValue”
}
}It’s up to the program serializing the JSON document to determine how to do the condition evaluation and value assignment. Here, we declare myKey as a conditionalValue. So our theoretical program assigns the value “testWasSatisfied” to myKey if the value of someOtherKey satisfies the logical test expressed in the query — the comparison clause (e.g., {$eq: “something”}). Otherwise, the value “someDefaultStringValue” is assigned.
This is just one schematic example of how conditions can be expressed, and the actual value assigned could be a boolean, a specific value (“testWasSatisfied”), or a set of values returned by a mongo find() query. The virtual MongoDB “collection” on which to run the query specified in the JSON manifest is determined by us in the source code of the program reading the JSON document. That collection could be the entire manifest (or some JSON block), a user input, or a separate data structure available to the executing program at runtime.
Real World Usage
At Sisense, we use MongoDB query language along with MingoJS to express conditional logic and dependencies in JSON manifest files.
User Interface
For example, the UI which comprises our “Add Data From Connector” wizard is automatically rendered at runtime according to a manifest file which describes the connector’s inputs. Each connector has its own manifest.json file, describing the necessary data required to connect to the data source, and thus the necessary input fields (and their types) to display to the user in the UI. This creates a single source of truth for generating both the connection data (resulting in a connection string) and the UI inputs required to produce it.
The manifest file is loaded when Sisense services start. When the user selects a data connector, React components are rendered dynamically at runtime based on the manifest to build the form displayed to the user.
Some forms are more complex than others, and whether a group of inputs is visible or enabled is dependent upon the value of some other input, provided during user interaction. In some cases, the entire contents of a drop down menu changes based on user selection (since different dynamic settings suggest different user options). This logic is not something we want to hard code or even store in a database — it is part of the logic of the connector input, and so it should be defined in the connector’s manifest.
Here’s one way to express a dependency of the “visibleIf” property for a rendered element:
{
"name": "authMechanism",
"title": "Authentication Mechanism",
"type": "list",
"description": "Select Authentication Mechanism",
"visibleIf": {
"$and": [
{
"subprotocol.value": "Hive Server 2"
},
{
"isZooKeeperDiscovery.value": true
}
]
}
(...list data defined here...)
}In the example above, the Authentication Mechanism drop-down menu will be visible to the user only if the value of “subprotocol” (another drop down menu) is set to “Hive Server 2”, AND the checkbox named “isZooKeeperDiscovery” is checked. Otherwise, the drop-down menu described by this JSON object will not be rendered in the UI.
{
"name": "isZookeeperDiscovery",
"title": "Use ZooKeeper Discovery",
"type": "boolean",
"enabledIf": {
"discoveryMethod.value": {
"$in": [
"IgniteCluster",
"ClusterNode",
"Local",
"TestMode"
]
}
},
"visibleIf": {
"isAdvancedMode.value": true
}
}In another snippet from the connector’s manifest.json file above, we see that the object associated with the “isZookeeperDiscovery” checkbox will only be visible if the value of another component, isAdvancedMode, is true (a radio button in this case), and it will be enabled in the UI (not grayed out) only if the value of another input, discoveryMethod, is one of IgniteCluster, ClusterNode, Local,or TestMode.
API Auto-Generation
Another manifest file in which we use conditional logic is our GraphQL2REST module. It is a Node.js library I developed which automatically generates a REST API from an existing GraphQL schema. A manifest JSON file is used to define REST routes and map them to the appropriate GraphQL queries or mutations.
GraphQL2REST lets you map a single REST API endpoint to several GraphQL operations, where the actual operation to be invoked is determined at runtime based on the contents of the REST payload.
For example, here we determine which operation to invoke based on the value (or existence) of a query parameter in the REST request:
"endpoints": {
"/elasticubes/schema": {
"get": {
"operations": [
{
"operation": "getElasticubeByTitle",
"description": "Find Elasticube by its title",
"condition": {
"params.title": {
"$exists": true
}
}
},
{
"operation": "listElasticubes",
"description": "List all Elasticubes",
"condition": {
"params.title": {
"$exists": false
}
}
}
]
}
}
}In this example, a simple “switch” logical statement is expressed for all /elasticubes/schema “GET” requests. If the request includes a “title” parameter, the GraphQL query “getElasticubeByTitle” will be invoked, and the value of the “title” parameter will be passed to it as is. If the GET request does not include a “title” parameter, the GraphQL query “listElasticubes” will be invoked instead (this query doesn’t accept any arguments).
Using simple conditional logic expressed in JSON we were able to map one HTTP GET endpoint to two different GraphQL operations based on the payload:
GET /elasticubes/schema?title=myElasticubeName => getElasticubeByTitle(‘myElasticubeName’)GET /elasticubes/schema => listElasticubes()
In this case, the MongoDB “evaluation” is done on the HTTP request parameters object req.params (that is the virtual “collection” on which the criteria {“title”: { “$exists”: true }} is tested).
MingoJS
On the source code side, what allows the straightforward evaluation and resolution of values based on logical rules is mingo, an open source JavaScript library which implements the MongoDB query language. Mingo supports all MongoDB query and projection operators, and even aggregation framework operators and custom operators. All this power could be overkill for just basic conditional logic in JSON manifests, but mingo is mainly used for other use cases:
It allows treating any JavaScript object (or JSON document) as if it was a MongoDB collection and run queries on it. This can be used for both testing a criteria to get a true or false result, as well as for running search and filter queries.
Mingo is also a good alternative to writing lots of custom code for transforming collections of objects and can also be used for testing, mocking, and for quick validation of MongoDB queries without the need for a database.
Alternatives
There are some alternatives to using MongoDB query language notation in JSON:
- The JSON Schema Draft 7 specification includes basic support for “if”, “then”, and “else” keywords for conditional schema evaluation, but it is limited and intended only for JSON schemas.
- James Snell from IETF authored JSON Predicate which allows testing whether a block of JSON meets the criteria defined by another block of JSON (and there’s a json-predicate JavaScript library, available only for the front-end).
- JsonLogic is an interesting library that allows building complex rules and serializing them as JSON.
- JSL is a JSON-based logic programming library, meant to be used as an embedded rules engine in JS applications.
- JSONata is a JSON query and transformation language with a rich function library.
While most of these libraries can be used to express conditional logic in JSON, each has their own limitations. Above all, they use proprietary language and concepts, whereas we wanted to provide basic conditional logic capabilities that could be written and understood by anyone. MongoDB query language is prevalent nowadays and is well documented and familiar to many developers and technical users. It is also among the most powerful query languages in the market, which is why we chose to use it.
Conclusion
By using dependencies and conditional logic you can add real expressive power to your JSON manifests, and make your configurations dynamic and context driven.
Two words of caution:
1. Dynamic inputs can be a security threat for injection attacks, so you should always limit and protect untrusted inputs. This means you should always prefer to use logic that assigns a boolean or that selects a value out of a limited number of known options in the manifest (rather than running a find() query). Even if the JSON file is only used internally by developers, always be sure to sanitize and validate values which are not static.
2. Manifests (and JSON documents in general) are not intended for describing business logic. If you find yourself trying to express complex logic and workflows in JSON, this probably means that logic should be expressed in source code, not in your manifest! :)
Happy coding!
