Finagle Filter for Thrift Exceptions

Hey friendo. If you are reading this article, you probably use Finagle and Thrift for internal services like we do at ActionIQ. The mixture of thrift and finagle makes writing typed RPC services in Scala a breeze. However, you likely have had to make a decision about how to propagate errors to the clients of your services and are maybe finding that hard / running into some issues. This article provides insight into how we solved for this and why.

We made a decision early on to use a single “wrapping” exception type for our services. Since our naming game is so strong, we named this exception ServiceException. Using a single exception works if every engineer remembers to throw only that exception. So how do you handle unexpected exceptions? That’s where finagle filters come to the rescue!

Before getting into the details of the filter, I want to share our ServiceException. It looks similar to most JVM exceptions with a message and cause (which allows us to wrap any other exception). We added in a code and level for easier parsing on the receiving end. For traceability, the source file and line are also included.

exception ServiceException {
  1: string message
  2: optional string cause  // If converted from another exception, store that exception's class here
  3: ServiceExceptionType level  = ServiceExceptionType.MINOR
  4: optional string code
  5: optional string file
  6: optional i32 line
  7: optional string userVisibleMessage
  8: optional map<string, string> metadata
}

If you have never dealt with finagle filters before, you should read the docs here. In short, a filter is code that can intercept requests and responses for all of your services. People tend to use filters for logging, auditing, etc… Filters are easy to write for HTTP requests because you get access to headers, uri, body, etc.. However, with thrift you only get access to an array of bytes for requests and responses. Somehow you have to look at a response, determine if it represents an unexpected exception, and then create a byte array for a different exception (in our case ServiceException).

Luckily, the thrift protocol is pretty predictable. The result/response of any service is generally two fields wrapped in a structure. The structure field is named by functionName_result. The first field is the “success” field, which optionally contains the return type. The second field is the “ex” exception field, which optionally contains the exception. Thrift client code knows if the success field isn’t present and the exception field is, then the response is an exception and it gets parsed correctly. Another interesting tidbit, a known exception in a filter looks like a successful future, but an unknown exception looks like a failed future.

Here is the filter I whipped up to handle unexpected exceptions. Basically, it looks for a failed future, wraps the exception in a ServiceException, uses the thrift protocol to make a proper thrift message, and then encodes into a byte array.

With this handy dandy filter in place, our engineers do not have to worry about wrapping all of the code we write in services in some sort of try catch block. Hopefully this article helps you in your quest to be exceptional. Feel free to leave any feedback, except if your feedback is to use GoLang instead. Nobody wants to hear that feedback.

Larry (or sometimes Lawrence) Finn is a software engineer focusing on core libraries and infrastructure in the full stack team at ActionIQ . He has spent many years building APIs and working with Scala, Java, PHP (shhh don’t tell anyone), and MySQL. Larry enjoys cats, salads, coffee, whiskey, and solving technical problems pragmatically. Sometimes he is grumpy but coffee usually helps.