Handling Real-World JSON Data in Elm

Union Types to the Rescue

Beyond the Basics

As anyone who has dabbled in Elm knows, one of the biggest early challenges we all run into is writing JSON decoders. There are lots of great resources on this and the elm-decode-pipeline makes pipelines a lot easier to write. But just when I thought I had it all figured out, I came across some interesting real-world situations that made me have to dig deeper. I’m using this blog post to walk through a recent challenge I ran into as I tried to build a real REST API and consume it with an Elm application.

Dealing with API Errors

I’m in the process of designing an API that can return either valid data or a set of validation errors. So if I do a request like the following.

POST /ap1/v1/users

with payload

{ "user" : "Jack Jones",
  "email" : "jack@jones.com"
  "age" : 40
}

If the post succeeds, I will return the data that was actually written to the database. For instance.

{
  "data": {
    "name": "Jack Jones",
    "id": 16,
    "email": "jack@jones.com",
    "age": 40
  }
}

But if the post request fails, I respond with validation errors such as the following.

{
  "errors": [
    {
      "email": "has already been taken"
    },
    { 
      "age": "must be greater than 17"
    }
  ]
}

I chose this formats it matches the spirit of the JSON API specification. You’ll note that this specification specifically sets this constraint.

The members data and errors MUST NOT coexist in the same document.

This requirement complicates our JSON decoding significantly. We now have to decode what is essentially two different response formats with a single JSON decoder. So how do we go about do this? Let’s dive in.

The Elm Model

Let’s first take a look at the model we want to use to store this data on the client side. This was my first thinking on how to structure the model. Note that I used the Elm Maybe type to conditionally store data or error.

type alias User =
    { name : String
    , email : String
    , age : Int
    }
type alias Error =
    { key : String
    , value : String
    }
type alias Model =
    { user : Maybe (List User)
    , errors : Maybe (List Error)
    }

Upon further thought, this model doesn’t seem to make the best use of the Elm type system as it allows an illegal state (users and errors both exist) to be possible. Making use of union types, we can refine this model as follows.

type alias User =
    { name : String
    , email : String
    , age : Int
    }
type alias Error =
    { key : String
    , value : String
    }
type UserHttpResponse 
    = ValidUserResponse User
    | ErrorUserResponse (List Error)
type alias Model =
    { httpData : UserHttpResponse
    }

This avoids the illegal state and will hopefully make it easier to decode the two distinct flavors of JSON into a single result.

In order to write the decoders we need, we also need to define two more types that represent the response data for both correct and error data.

type alias userValidResponse =
    { data : User
    }
type alias userErrorResponse =
    { errors : List Error
    }

Creating the Decoders

With the model we described above, we are looking for one of two different patterns to decode from. This looks like a perfect situation for the Json.Decode function, oneOf. The documentation describes the function this way.

So we will write a decoder for each of the two possibilities and use oneOf to choose between them depending upon the data that was retrieved.

First, we need to create decoders for each of the possible response lists. I used the elm-decode-pipeline package to make these decoders more readable.

Here are the decoders for the users. Note that we first created a decoder for a single user and also a decoder to decode the JSON object mapping data.

userDecoder : Json.Decode.Decoder User
userDecoder =
   Json.Decode.Pipeline.decode User
       |> Json.Decode.Pipeline.required "id" Json.Decode.int
       |> Json.Decode.Pipeline.required "name" Json.Decode.string
       |> Json.Decode.Pipeline.required "email" Json.Decode.string
       |> Json.Decode.Pipeline.required "age" Json.Decode.int
userResponseDecoder : Json.Decode.Decoder UserValidResponse
userResponseDecoder =
    Json.Decode.Pipeline.decode UserValidResponse
        |> Json.Decode.Pipeline.required "data" userDecoder

These are the decoders for the errors. This uses same approach as the user decoder. The only complication is the need to support a list of Error records.

errorDecoder : Json.Decode.Decoder Error
errorDecoder =
    Json.Decode.Pipeline.decode Error
        |> Json.Decode.Pipeline.required "key" Json.Decode.string
        |> Json.Decode.Pipeline.required "value" Json.Decode.string
errorListDecoder : Json.Decode.Decoder (List Error)
errorListDecoder =
    Json.Decode.list errorDecoder
errorResponseDecoder : Json.Decode.Decoder UserErrorResponse
errorResponseDecoder =
    Json.Decode.Pipeline.decode UserErrorResponse
        |> Json.Decode.Pipeline.required "errors" errorListDecoder

With these in place, we can then create decoders to decode each of the two union types. For each of these decoders, we use Json.Decode.map to take a constructor for one of the union types and “unwrap” the data portion of the record returned from the API.

validUserDecoder : Json.Decode.Decoder UserHttpResponse
validUserDecoder =
    Json.Decode.map
        (\response -> ValidUserResponse response.data)
        userResponseDecoder

errorUserDecoder : Json.Decode.Decoder UserHttpResponse
errorUserDecoder =
    Json.Decode.map
    (\response ErrorUserResponse response.errors)
    errorResponseDecoder

Finally, we use the JSON decoder oneOf function to decode a JSON string containing one of these formats.

userHttpResponseDecoder : Json.Decode.Decoder UserHttpResponse
userHttpResponseDecoder =
    Json.Decode.oneOf
        [ validUserDecoder
        , errorUserDecoder
        ]

Putting it All Together

With all of this in place, we can now decode both valid and error JSON strings into the same Union type. For instance, given this JSON input representing an error,

{ "errors" :
    [
      { "key" : "age", "value" : "must be greater than 17" },
      { "key" : "name", "value" : "must be at least 12 characters" }
    ]
}

we use userHttpResponseDecoder to decode this into this Elm Result value.

(Ok
    (ErrorUserResponse
        [ Error "age" "must be greater than 17"
        , Error "name" "must be at least 12 characters"
        ]
    )
)

Similarly, given JSON representing a valid response,

{ "data" :
  {
    "name": "Joe",
    "id": 1,
    "email": "joe@example.com",
    "age": 40
  }
}

we can use the same decoder to produce this Elm Result data.

(Ok
    (ValidUserResponse
        { "name": "Joe"
        , "id": 1
        , "email": "joe@example.com"
        , "age": 40
        }
    )
)

Summary

No matter how many times I work with JSON decoders in Elm, it takes some effort effort to get them working. They can be a bit overwhelming unless you break them down into small pieces. But just as you build the Elm data out of nested combinations of lists and records, you build the corresponding JSON decoders using the same hierarchy of decoders. Brian Hicks compares this process to composing something out of Lego blocks.

Using the oneOf decoder allows us to decode our data into the richer set of types offered in the Elm ecosystem. The end result is a clear and easy to maintain application that more than makes up for the effort involved.