An Experiment with JSON Schema

We had earlier talked about how Carousell tackled the complexity of listing variety with our UI system. Siaw Young, one of the engineers behind the project, introduced the concept of fieldsets, documents written in Carousell specific markup to communicate a screen’s layout to clients.

Fieldsets are made up of fields, each represented by a component type on client. A listing’s description and price can both be represented by a paragraph component, albeit with different content. When writing a fieldset, you decide which components to use, what data they correspond to and how to order the fields.

Currently, Carousell’s fieldsets are managed as JSON documents, it’s simple for a new team member to pick up and the written document can be production ready without the need for further conversion.

JSON, a staple on web services, is a textual data exchange format that closely resembles a JavaScript object. It’s human-readable, easy to understand and highly flexible, which makes it ideal for communicating data.

🌾 An example of a field markup. 🌾

💻Title component with an error message rendered on client. 💻

However, over time we started running into a couple of problems, namely:

1. Growing number of fieldset documents to maintain.
2. Changes in fieldsets are validated through custom Python tests which is not scalable and opaque to a person whilst editing a fieldset.

Furthermore, we wanted to empower non-engineers to create fieldsets, shortening the time to create a new category. Editing large JSON documents can be daunting and doesn’t instil confidence. We started looking for a solution that could be more user friendly, like a form for instance.

We can create a form for each component type, enabling the creation of fields. Once these fields are created, they can be composed in an interface that allows ordering of the selected fields. These forms can provide both validation and structure to the user input values.

However creating component specific forms is not scalable nor easily translatable to another programming language.

🗺 JSON schema for the text component. 🗺

One convincing solution we ended up adopting was JSON schema. JSON schema is itself written in JSON, a declarative format that describes the structure of data and can be used to automate validation.

For example in the code snippet above, we have indicated that the value of label has to be of type string while the field’s visibility is of type boolean, with a default value of true.

Working with JSON schema allowed us to annotate type, similar to existing message encodings that depend on schemas¹. In addition, it supports validation specific annotations such as, required properties, pattern matching and conditional subschema, which means we no longer need handwritten tests² to validate fieldsets.

But the strongest feature of JSON schema is that the overall richness of it lends itself well to generated forms³. We are then able to generate the field creation forms described above in a scalable manner.

🙌 Text component schema generated form 🙌

Overall we treated our foray into JSON schema as an experiment, and we continue to look for ways to improve internal tools for fieldsets. We’ve seen how JSON schema can provide rich annotations to data, and tools in the growing ecosystem can simplify workflows with validation and generated forms.

I hope this writeup can provide inspiration to the problems you are solving. As always, feel free to reach out.

Footnotes

1. We are also looking at protobuf messages as a way to define components. Generators for both client and server are bountiful and it, too, supports required properties and pattern matching.
2. We have also started representing these components as Go structs, removing the need for custom server-side validation. If a requesting JSON body can be unmarshalled, it is a valid field.
3. react-jsonschema-form is a popular library. Support for nullable properties using multiple types "type": ["string", "null"], additionalItems, oneOf, anyOf can be rough.