Creating Custom io-ts Decoders for Runtime Parsing

Published in

agiledigital

6 min readNov 7, 2022

The Problem

io-ts introduces runtime type validation to TypeScript, providing type safety at the boundaries of our application. This blog assumes a basic understanding of fp-ts and io-ts, particularly the concept of an Either type.

Overview of fp-ts: https://gcanti.github.io/fp-ts/
Introduction to io-ts: https://medium.com/swlh/typescript-runtime-validation-with-io-ts-456f095b7f86
The Either type: https://dev.to/waynevanson/either-fp-ts-meets-if-else-and-switch-part-2-3-893

While io-ts has a good amount of built in types and combinators (see see io-ts-types for some additional common types), some scenarios call for the creation of custom codecs.

Consider the case where we want to take a comma separated list of values from an environment variable as input. We can validate that an unknown input is indeed a string:

(s: unknown /* input */) =>
  Either<string /* validation errors */, string /* unchanged data */>;

So our data is validated. We’re all good right? Not quite — our data isn’t in a form we can use yet. We’re yet to split the string into an array that our application is expecting. Fine, lets do the following instead:

(s: string /* input data */) =>
  string[] /* transformed data */

But now we’re assuming our input has already been validated. That’s a lot of trust… it’d be better to rely on our type system instead to ensure we don’t make that mistake. We also run the risk of mixing up our input validation and transformation with the actual code where we process this data. To avoid this problem, let’s combine our data validation and transformation into a single step. In other words, let’s parse our data:

(s: string /* input data */) =>
  Either<string /* validation errors */, string[] /* parsed data */>;

This doesn’t just help us write better code — it also keeps our application secure since we don’t make any assumptions about the integrity of external data sources. For example, we can sanitize user-supplied data to combat injection vulnerabilities.

Now we know our data is validated, and it is in a form ready to be used by our application. Alexis King says it best in this blog:

Get your data into the most precise representation you need as quickly as you can. Ideally, this should happen at the boundary of your system, before any of the data is acted upon.

Thankfully, io-ts allows us to achieve exactly this via custom codecs.

Creating Our Codec

A codec is at the heart of io-ts - it gives us a means to decode input coming into our application and encode outputs being sent by our application, all while maintaining type safety. We can introduce transformations into a codec in order to parse our data during decoding/encoding.

You can follow along at StackBlitz.

Let’s continue with the example of accepting an environment variable of comma separated strings.

First, we need to define the type of our codec:

import * as T from "io-ts";T.Type<
  string[], // runtime static type
  string, // encode this type
  unknown // decode this type
>;

When decoding, we’ll be taking an unknown input from our process.env, doing some transformations, then returning an array of strings (type string[]).

Now, let’s define our decoder.

We can leverage the existing io-ts primitives io-ts has to validate that our input is a string:

T.string.validate(input, context);

Then we need to transform the input. You could take things further by cleaning or formatting data further as needed, but in this case we are simply splitting a comma separated string into an array:

str.split(",");

Next, we need to return an fp-ts either to represent any validation errors. io-ts provides us with some utilities for success and failure. We could use this for some custom validation, such as checking the length of our array is within a certain range. In our case, we’ll simply return a success with our split string:

T.success(str.split(","));

Using pipe - an fp-ts function - we can compose these two steps together as:

// Only succeeds if our value can be decoded to our runtime target - string[]
(input: unknown, context: T.Context) =>
  pipe(
    T.string.validate(input, context), // unknown -> string
    E.chain((str) => T.success(str.split(","))) // string -> string[]
  ),

Finally, we can put everything together into a complete codec. Recall that codecs are composed of both decoding and encoding functions, and a type guard. This gives us:

import * as E from "fp-ts/lib/Either";
import { pipe } from "fp-ts/lib/function";
import * as T from "io-ts";const CsvCodec: T.Type<string[], string, unknown> = new T.Type<
  string[], // runtime static type
  string, // encode this type
  unknown // decode this type
>(
  "CommaSeparatedValueCodec",
  // Type guard
  (input: unknown): input is string[] =>
    Array.isArray(input) && input.every((value) => typeof value === "string"),
  // Parsing - both our validation and transformations contained together.
  // Only succeeds if our value can be decoded to our runtime target - string[].
  (input: unknown, context: T.Context) =>
    pipe(
      T.string.validate(input, context),
      // Note - this doesn't account for the case where the comma is escaped
      E.chain((str) => T.success(str.split(",")))
    ),
  // Custom Encoding.
  // Converts a value of type string[] -> a value of type string
  (output: string[]) => output.join(",")
);

If we wanted to encode to the same type as our static type without any transformations, we could simply use T.identity instead and modify our encoding type accordingly:

T.Type<string[], string[], unknown>;

I should note here that io-ts allows you to define custom decoders independently rather than an entire codec. You might also be able to get away with primitives and the combinators refine or parse. A codec might be overkill for some use cases where you don’t need to encode or use type guards. Though here codecs allows us to easily compose larger types which we’ll use later on.

Using our Custom Codec

Here is a simple wrapper for our codec we can use for testing:

import * as E from "fp-ts/Either";
import { pipe } from "fp-ts/lib/function";
import * as T from "io-ts";
import { formatValidationErrors } from "io-ts-reporters";// Failure handler
const onLeft = (errors: T.Errors): string =>
  `${errors.length} error(s) found: ${JSON.stringify(
    formatValidationErrors(errors)
  )}`;// Success handler
const onRight = (s) => `No errors: ${JSON.stringify(s)}`;// Parse the validation the decode returns
const simpleDecode = (input: unknown) =>
  pipe(input, CsvCodec.decode, E.fold(onLeft, onRight));

Let’s see what decoding a few example inputs give us:

// String input comma separated values
simpleDecode("A,B,C");
// => No errors: ["A","B","C"]// Input type isn't a string
simpleDecode(123);
// => 1 error(s) found: ["Expecting CommaSeparatedValueCodec but instead got: 123"]// String input with a single value
simpleDecode("something");
// => No errors: ["something"]// The codec outputs a value of type string[], but the input is of type string
simpleDecode(["A", "B", "C"]);
// => 1 error(s) found: ["Expecting CommaSeparatedValueCodec but instead got: [\"A\",\"B\",\"C\"]"]

Again, this ensures the data coming in at our application boundaries is safe to be processed.

Our initial use case was to parse environment variables, so let’s see that in action by composing our codec together with some other primitives:

const EnvironmentVariables = T.intersection([
  T.type({
    NAMES: CsvCodec,
    LOCATION: T.string,
  }),
  // Optional
  T.partial({
    COLOURS: CsvCodec,
  }),
]);
// Convert to TypeScript type
type EnvironmentVariables = T.TypeOf<typeof EnvironmentVariables>;

Some examples:

// All values present
envVarDecode({
  NAMES: "John,Bob,Harry",
  LOCATION: "Space",
  COLOURS: "Red,Green,Blue",
});
// => No errors: {"NAMES":["John","Bob","Harry"],"LOCATION":"Space","COLOURS":["Red","Green","Blue"]}// Invalid type for NAMES
envVarDecode({
  NAMES: 1,
  LOCATION: "Space",
});
// => 1 error(s) found: ["Expecting CommaSeparatedValue at 0.NAMES but instead got: 1"]// Invalid type for NAMES and required values missing
envVarDecode({
  NAMES: 1,
});
// => 2 error(s) found: ["Expecting string at 0.LOCATION but instead got: undefined","Expecting CommaSeparatedValueCodec at 0.NAMES but instead got: 1"]

You can find a complete code example on StackBlitz.

Conclusion

As illustrated by this example, custom codecs are a powerful way to parse data at runtime. This approach can be applied more generally to bring flexible type safety to the boundaries of your application, while decoupling our validation and parsing from application logic.

io-ts does require the conceptual overhead of fp-ts, which might not be suited to your particular style and could be difficult to integrate with an existing codebase. There are a few alternatives to io-ts for runtime validation in the TypeScript ecosystem to consider:

zod: more concise than io-ts, especially when it comes to defining partial types. Though looses the benefits of using Eithers and integrating with other fp-ts constructs.
yup: similar to zod, but missing a few features such as partials, unions and intersections as of writing.
envalid: small library specialized for validation of environment variables with support for middleware.

Regardless of what you choose, the key takeaway is:

Parse, don’t validate

Creating Custom io-ts Decoders for Runtime Parsing

The Problem

Creating Our Codec

Using our Custom Codec

Conclusion

Written by Timothy James