Full Stack Type Safety: An Enum’s Journey
Software Engineering is a balancing act.
Attempting to balance competing concerns such as time vs. level of abstraction, or budget vs. feature wish list, is a never-ending struggle. One of these classic battles is the competing concerns of Security and Reliability vs. Speed and Agility. This battle has been tipped in the favor of agility and speed in many tech organizations as made famous by the quote “Move fast and break things.” While this has been shown to be very effective for consumer social media applications, at Forge the scales lean in favor of Security and Reliability.
There is a reason why our first and foremost concern is towards Security and Reliability. Simply put, we are handling large volumes of transactions ($500M+ in H1 2019) and large transaction sizes ($3M+ Average transaction size). We cannot afford to lose the trust of our clients, so we need to prioritize this above other concerns. This does not mean that we just sit on our hands and hide all our transactions under a mattress. We need to be able to build products at the speed of a consumer tech company and with the agility of a startup.
There are a variety of tools and processes we use to help us improve speed while also maintaining reliability, but one tool that we lean on aggressively is Full Stack Type Safety (FSTS).
Full Stack Type Safety is the ability to take a type that is defined at one layer of our application and generate that type at every other layer, keeping our types consistent across the entire application stack.
Why Type Safety?
Before we dive headlong into FSTS, we should briefly touch on why we want type safety at any layer. Type systems have been rising in popularity in many classically dynamic languages (see Typescript or Typings) because the communities have been recognizing the benefits of having type systems. One of the main benefits of type safety is nicely paraphrased by Mathias Verraes:
The concept is that using a type system you can reduce a whole class of errors in code without resorting to excessive tests.
Since we can reduce the number of tests we need to write to catch errors, we can be more confident at compile time that our code will work as intended. This leads to confidence when refactoring code for maintainability, iteration speed when implementing new features, and overall security.
Following this reasoning (and many others, of which entire books have been written) for why type-safety is important at each level, how then do we extend the power of type safety at every level with FSTS?
The Journey of an Enum
In order to show the power of FSTS, let’s start with a simple yet powerful example, the journey of an enum from the database all the way to our front-end applications.
Let’s start with a very simple example of a PSQL enum in our database, Order Asset Type. I’ll use this example because it provides limited options, but also allows me to explain a bit about our business and how the private market works along the way.
An order is a transaction in our system, the basic unit of the private market. If you are going to buy or sell shares in a company such as Nextdoor or Robinhood, you are doing so as an order.
The asset type is the type of shares that you are selling. If you are a classic employee in Robinhood for example, the asset type for your order would likely be common stock. If you were an early investor or venture capital fund, you might want to trade your preferred stock.
The First Layer: Postgres => Scala
We now have defined the options for an order in the database, but we want the backend to know and understand those options as native Scala types. We accomplish this by using the jOOQ library.
jOOQ generates Java code from your database and lets you build type-safe SQL queries through its fluent API. — jooq.org
How nice of them to succinctly recognize our goals of generating types across layers and provide a solution. With one simple command (and a bit of custom configuration), jOOQ inspects our postgres instance and generates the classes, enum, and objects needed to interface with our DB in a type-safe manner based off our database schema.
sbt jooqCodegenThis generates a fluent API, which we can use in a type-safe fashion to generate SQL queries, rather than using untyped strings as queries and mutations.
With this object we now have our Order Asset Type enum successfully type-checked in our backend code base. We can now write a query that selects order by Order Asset Type, knowing the range of values that we can use.
The Second Layer: Scala => GraphQL => Typescript
Having database types and our Order Asset Type enum on our backend is great, but that doesn’t get our type across the final layer of our stack, the API interface.
To expose this interface and GraphQL schema on the backend, we rely on Sangria. This allows us to turn the query we wrote above directly into a GraphQL endpoint that any frontend can use. On the frontend, we have benefited from two more recent developments in the community, the widespread adoption of Typescript, and the rising popularity of GraphQL. With both of these, we can provide a significantly stronger type interface than would have been possible only a couple of years ago.
To generate types across our GraphQL interface we use both GraphQL Code Generator (which has some of the best documentation I have seen) and React Apollo. GraphQL Code Generator produces typed queries, mutations, and interfaces, and combined with React Apollo produces strongly typed query components and hooks, saving us the effort of manually typing data that comes out of a GraphQL query.
That means when we can query against our previous example endpoint:
ordersByAsset(assetType: $assetType) {
id
asset_type
}and GraphQL Code Generator will generate this code in return:
Type Safely
Full Stack Type Safety (FSTS) helps us extend the benefits of type safety to every level of our application enabling increased developer focus, quicker feedback loops, and more confidence in our code. This allows us to increase our speed and agility while still maintaining our balance in favor of security and reliability.
Each of these layers of type safety would not be as strong as they are without the layer that they build upon. Our typescript bindings would not be real representations of our database fields if we didn’t have type safety across the entire stack.
There is one glaring hole in our type safety stack and that’s our lack of type-safe end users. If there is a library that someone is developing to help fix that, please consider sending it our way so we can support you on Patreon or Open Collective.
