Checking Types Against the Real World in TypeScript
A follow-up to Type-Driven Development with TypeScript.
By Jesse Hallett — Originally posted April 12th, 2018
The shape of data defines a program. There are important benefits to writing out types for your data.
Let’s consider a Hacker News client, which consumes stories and other items from the Hacker News API. This is a TypeScript type that describes the format for stories:
Story. The shape of the data is implied in the code that manipulates the data. But that means anyone reading the code has to mentally reconstruct that shape from context, or refer to documentation outside of the program itself.
TypeScript provides the option of documenting data structures in the form of types. An obvious advantage is that the type checker can identify mistakes like typos in property names when accessing data. The Hacker News API uses the property name
descendants; for some reason every time I try to type
descendants I end up typing
descendents by mistake. If I did not have a type checker to point out that
Story does not have a property named
descendents I could end up wasting a lot of time debugging!
List all changes
But this just scratches the surface. Types for data structures help keep programmers oriented. When a data structure is suddenly required to change, all you need to do is to update that particular type and the type checker will list all of the changes that need to be made to work with this new type.
Reducing cognitive load
When you come back to a program after you have been away from it long enough to forget how everything works, having descriptions of data structures right there in the code makes it much easier to understand what is going on. The same is true if more than one person is working on the project. Every detail that can be captured in types is one less detail that programmers have to carry in their heads. Reduced cognitive load leaves programmers with more energy for writing important business logic.
Bridging the gap with validators
But what if you make a mistake when you write the type? I mentioned that I had problems mixing up
descendents. I actually made the same mistake the first time I wrote the
Story type. The type checker cannot help me if I give it bad information from the start! Unfortunately, a static type checker cannot check types against data from an external API. But what you can do is to write a validator that will check at runtime that incoming data has the shape that you expect. Then you can extract a static type from the validator that is guaranteed to match any values that pass validation.
There is a nifty library called io-ts that works like magic. Instead of the
Story type above We can define a validator using io-ts combinators:
This looks similar to the
Story type from the beginning of the post.
StoryV expresses the properties of objects coming from the Hacker News API with a type for each property. (The
t.type() combinator produces a validator that expects an object with the given property names and types.) But this time the “types” for each property are actually values supplied by io-ts:
t.boolean, etc. Values can be referenced at runtime, types cannot. With
StoryV.decode(whateverValue). If the given value is not an object with the expected properties then
decode will return an error value.
From validator to type
What makes io-ts uniquely valuable is that it simultaneously defines a runtime validator and a static type.
StoreV.decode() returns a success result, then TypeScript knows that the resulting value has a
descendants property and does not have a
If a value passes validation, then it is guaranteed to match that static type, and we can use it to check the correctness of the rest of the program. If a value does not pass, then you will get a failure with a clear point in the program where it should be handled.
Validating incoming data at runtime allows the program to fail fast if there is a mismatch between the data and the program’s expectations. In development, that makes it easy to catch bugs early: any mismatch is identified immediately at the point where you call
decodeToPromise. You don’t need fixtures or unit tests to check data ingress. Yes, the validation step could lead to failures in production that you would not have seen otherwise if data comes in some unexpected shape under some condition - but the alternative is for the program to limp along with unknown data leading to possibly-undefined behavior. Failing fast is better!
To minimize unnecessary validation errors it is a good idea to make your validators permissive in what they accept. For example, err on the side of marking properties as optional if there is any possibility that those properties will be absent in some cases. And you can exclude properties from the validator that you are not going to use in your program.
Referencing types produced using io-ts
StoryV replaces the hand-written
Story type - so we no longer have a way to refer to the type of story objects. But we can get that type back! Io-ts provides a type operator called
t.TypeOf that extracts a static type from a validator. We can define a new
Story type like this:
Every TypeScript value has a type. You can reference and manipulate the value at runtime. Likewise, you can reference and manipulate the type at type check time. The expression
typeof StoryV uses TypeScript’s built-in
typeof operator to get the typecheck-time representation of
StoryV which conveniently holds a complete description of the shape of story objects. That description is wrapped in a validator type;
t.TypeOf pulls the shape description out into an independent type.
You can use the computed
Story type in annotations in the rest of your program:
When data comes in different shapes
The Hacker News API publishes more than just stories. The
/v0/item/ endpoint alone also provides comments, job postings, polls, and poll options, which all have different shapes. We want to be able to fetch an item from that endpoint and use a runtime check on the
type property in the returned object to determine what type of item it is. And we want the type checker to verify the correctness of the whole process.
Let’s use io-ts to create some more item definitions. These will be similar to the definition of
StoryV. Here are abbreviated definitions (see the accompanying code for complete definitions):
The Hacker News item API could return a story or any of these types, which means that the type of values from the item API is a union of all five types. More specifically the type is a tagged union: the
type property in API responses is a tag that we can use to distinguish between types within the union. A tagged union validator looks like this:
This is why it was important to use the
t.literal() combinator instead of
t.string for the type of the
type property in each item validator: using
t.literal() with a literal string makes the exact string value available to the type checker. With that information, TypeScript can use type guards to narrow the type of an
item to a specific item type based on the value of
item.type. For example:
By the way, io-ts also supports intersections, untagged unions, and other fun combinators. Oh, and io-ts supports Flow too — not just TypeScript!
This was just a quick introduction to what io-ts is capable of, and techniques for applying type-checking to external data. The concepts here are not limited to consuming API data: I recommend similar use of io-ts validators when working with data loaded from a database, serialized messages between micro-services, user input, or any other case where data can come in from outside the program.
The best way to cement your understanding of a pattern is to experiment with it. I encourage you to check out the accompanying code and try adding some features. One idea is to display ID numbers with story titles and add an option so that if the user passes an ID as a command-line argument when running the script it displays a link and some comments on the corresponding story.
Appendix A: definition for
In a hand-written definition for an object type you can use a question mark to indicate that a property might be absent:
There is no easy way to do that with io-ts because the argument to
t.type() is an actual object, and object properties are either present or not present. There is another combinator,
t.partial(), that describes an object where all properties optional. The idiomatic way to represent an object where some properties are optional is to use an intersection of
t.type() for required properties, and
t.partial() for optional properties:
I used a different pattern in this article. I didn’t want to introduce too many concepts all at once; so I didn’t introduce intersections and nested definitions right away.
optional() combinator is a union of the given type with
undefined. Technically this implies that we expect the given property to be present in every case, but that the value might be
undefined. In practice, that distinction often does not matter, and io-ts will validate an object that is missing a required property if the type of that property is allowed to be
undefined. But note that io-ts might make object validation more strict in the future!
This is the definition of
That is adapted from the
maybe combinator given in the io-ts README. It is pretty dense for readers who do not have much experience with advanced TypeScript use cases. This is the sort of function that should be put into a library, and I might do that in the future.
Appendix B: definition for
The built-in io-ts method
StoryV.decode() returns an
Either value, which is a type from the package fp-ts that can hold either an error or a successful result. It is similar to a promise except that it represents an immediate result, not an asynchronous one. The examples in this article use promises; so I wrote a function,
decodeToPromise to put validation results into the more familiar
Promise type. Here is the definition:
fold() is a method on the
Either type. It is used to collapse a possibility of success and a possibility of error into one definite value. TypeScript checks that the error-case callback and the value-case callback have compatible return types. One callback or the other will run depending on whether the
result is an error or a success value.
decodeToPromise also invokes an io-ts reporter to translate a set of validation errors into a readable message.
Originally published at www.olioapps.com.