Making Newtonsoft.Json and Protocol Buffers play nicely together.

I hit an issue trying to serialize some protocol messages to json. When Newtonsoft.Json deserialized my protocol message, it did not match the original. I found a simple work around that may work for you too.

Protocol Buffers

Protocol Buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data — think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use generated source code to easily write and read your structured data. Protocol buffers also know how to read and write themselves as json. Here’s a sample protocol message definition:

I was writing a sample for a new Google Stackdriver API. Nearly all of Google’s Cloud APIs use protocol buffers as their wire format.

I had a protocol message called AlertPolicy. I wanted to serialize it to disk as json, so it would be easy to back up and restore all my alert policies. The protocol buffer library could easily serialize an AlertPolicy for me, but it could not serialize an IEnumerable<AlertPolicy>. More generally, the protocol buffer library knows how to serialize protocol messages to json, but the root object being serialized must be a protocol message.

So, I looked to Newtonsoft.Json.

Newtonsoft.Json

Newtonsoft.Json is the most popular library for serializing and deserializing json with .NET. And it’s the most popular for good reason: the library makes it trivial to convert objects to and from json. One line of code, JsonConvert.SerializeObject(yourObject) is all it takes to convert your object to a json string:

So, I tried serializing my IEnumerable<AlertPolicy> with Newtonsoft.Json. The problem, I learned, is when I deserialized the IEnumerable<AlertPolicy> and compared it to my original IEnumerable<AlertPolicy>, a big chunk of data was missing.

Before:

After:

Fortunately, Newtonsoft.Json allowed me to specify a custom JsonConverter. I implemented a custom JsonConverter that uses Protocol Buffers’ json serialization function, to avoid losing data. So, 30 lines of code later, I had a solution for serializing all my protocol messages:

And here’s how I invoked it:

So, yay! I can use this ProtoMessageConverter for any object that contains Protocol Message objects, and everything will serialize and deserialize correctly. You can too. The full sample code lives on github.com.