Strict JSON Deserialization with Enum Validation

It’s important to validate data sent to your APIs. Silently ignoring unrecognised fields or not detecting and rejecting values outside of the allowable range is a dangerous way for errors to go undetected.



Clients will be calling your API, getting a 200 response and blissfully assuming everything is wonderfully integrated. Enraged customers will eventually find the problem and it won’t result in you getting a pay rise.



These risks are especially inherent to JSON APIs. With XML we have XSDs to enforce the schema. We haven’t quite got there with JSON yet, so it’s vital to be extra vigilant in your code.



I recently found out that it’s actually not as easy as it should be to reject unrecognised JSON fields and out-of-range enum values in Scala (using the Play Framework).



Eventually I found a solution that I am delighted with, though it is slightly contentious. In this post I’ll share it with you. Essentially you need to do this:

• Use the Scala Jackson library with strict deserialization enabled
• Use Java enums

Minimum Possible Demonstration

I will demonstrate my solution to this problem by creating a very basic API. It accepts two parameters — your email address and your preferred time of contact.

Here’s an example of the JSON being supplied for a valid request.

I want this API to completely reject any request that either contains an unknown field in the json or an invalid value for preferred time of contact. I do not want clients to incorrectly think that I have received some information they have sent because I silently swallowed it.



Note: Setting the content type as text/plain is a bit of hack to prevent the play framework from parsing the response as a JSON object. There are trivial solutions for circumventing this.

Jackson Scala Mapper

Play Json is the Play Framework’s own JSON library. Overall it’s not bad, but if you want strict JSON deserialization, as far as I can tell, you have to create a whitelist of allowed properties and some custom infrastructure — ideally I don’t want to carry out that time-consuming activity, especially not for every endpoint. It’s laborious, tedious and extremely brittle.

Fortunately I don’t have to do that, because Jackson and the Jackson Scala Module will do it for me.

As a bonus, and one of the reasons I love Jackson, is that it is conventional by default — you don’t have to create and pass around Formats/Reads/Writes for all of your case classes. Simplicity wins.

Firstly you add Jackson as a dependency in your build file. Then you create an instance of the mapper, register the Scala module, and add set FAIL_ON_UKNOWN_PROPERTIES to true. True is the default, but I like to be explicit.

val mapper = {
  val m = new ObjectMapper()
  m.registerModule(DefaultScalaModule)
  m.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true)
  m.configure(DeserializationFeature.READ_UNKNOWN_ENUM_VALUES_AS_NULL , false)
  m
}

Java Enums… whaaat?

Scala has it’s own enums, but their usability with Jackson is ugly and confusing (it’s not great in general, either). I choose to avoid.

Alternatively you can also create enums using sealed traits. I had a look at getting them working with Jackson but it’s not a trivial task and I let it go for now. So I use Java enums instead.

Jackson works out of the box with Java enums, and you can use Java enums in Scala projects. I’m sure someone will read that and shudder. Yet I found it the easiest, most maintainable solution for the least amount of effort.



Curse me if you must Scala purists. I’ll be over here getting stuff done.

To demonstrate, in my API, the preferred time of contact is a Java enum with two values — morning and evening. If you pass anything else into the API, it will reject your request and tell you precisely what you did wrong.

And here’s the code model — a scala case class with a Java enum:

case class Registration(email: String, preferredTimeOfContact: PreferToBeContacted)
public enum PreferToBeContacted { morning, evening }

Unrecognised Fields

Let me try sending a request with an unrecognised field in the json (the “invalidProperty” field):

Again, precisely the kind of behaviour and diagnostics I want. Garbage in — a detailed error message telling you how to fix the problem back out.

Minimal Code

For this comprehensive JSON validation and diagnostic behaviour, the implementation is incredibly-minimal. Here’s the controller method that handles those requests.

def register = Action { request =>
  try {
    val registration = mapper.readValue(request.body.asText.get, classOf[Registration])
    save(registration)
    Ok("""{ "message": "thanks for registering" }""")  
  } catch {
    case i: InvalidFormatException =>
      BadRequest(error(s"${i.getValue} is not a valid value for: ${i.getPath.asScala.head.getFieldName}"))
    case u: UnrecognizedPropertyException =>
      BadRequest(error(s"Invalid field: ${u.getPath.asScala.head.getFieldName}")) 
    case e: Exception =>
      InternalServerError("")
  }
}

When parsing fails, Jackson will throw an exception with the precise information you need to identify what went wrong and communicate that in your API response.



Above you can see that the InvalidFormationException allows you to identify the field and the value that were incorrect.



Admittedly you don’t want that boiler plate in all of your controllers. However, the logic is definitely reusable across your API. By resuing you will have invalid property validation and incorrect enum value validation for all of your endpoints without having to faff with custom validators and parsers.



Here’s a quick example:

object JsonAction {
def apply[A](body: A => Result)(implicit m: Manifest[A]) = {
    Action { request => 
      try {
        val js = request.body.asText.get
        val a = mapper.readValue(js, m.erasure.asInstanceOf[Class[A]])  
        body(a)
      } catch {
        case i: InvalidFormatException =>
          BadRequest(error(s"${i.getValue} is not a valid value for: ${i.getPath.asScala.head.getFieldName}"))
        case u: UnrecognizedPropertyException =>
          BadRequest(error(s"Invalid field: ${u.getPath.asScala.head.getFieldName}")) 
      }
    }
  }
val mapper = {
    val m = new ObjectMapper()
    m.registerModule(DefaultScalaModule)
    m.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true)
    m.configure(DeserializationFeature.READ_UNKNOWN_ENUM_VALUES_AS_NULL , false)
    m
  }
private def error(m: String) = s"""{ "error": "$m" }"""
}


That let's you write concise code like the following with all of the benefits of the previous example:
def register = JsonAction[Registration] { registration =>
  save(registration)
  Ok("Thanks for registering")
}
To me that's about as concise, elegant, robust and maintainable as it gets. Those Java enums weren't such a bad compromise after all.

Do I Recommend this Approach

I wholeheartedly recommend this approach. Play Json is a robust library with touches of elegance, but it lacks strict deserialisation. I don’t have the time or motivation to write fragile deserialization rules for every endpoint, so I’ll be using Jackson and the Scala module as much as I possibly can.

What about Java enums, surely I was joking? Absolutely not. Using Java enums has barely any impact on the readability, maintainability or expressiveness of my code. Sure, I’d like to use sealed-traits, however it’s imeasurably easier to write a conversion from a Java enum than to write custom parsing or validation logic for each endpoint.

If you know a better way, please let me know. Happy parsing.