How I Improved JSON Parser Performance Twofold

Introduction

During game development, the amount of data processed by the game increases. Artists create new assets, programmers write code, and game designers add more configurations and settings to the game. When the size of these configurations exceeds tens of megabytes, the performance of even the most trivial components, such as the JSON config parser, becomes crucial. In this article, I share my experience optimizing parsers for our game data management tool called Charon.

About Code Optimization

The first step in any code optimization is profiling. Any assumptions about what needs optimization without objective profiling data are likely to be incorrect. Many, including myself, have fallen into the trap of thinking “I know better how it will execute.” Even with a profiler, performance assumptions about specific parts of your program are only valid for your hardware and OS version. However, it’s better than making blind changes.

Switching from Parsing char[] to Raw byte[]

When writing parsers for text-based data formats/protocols, there are two ways to handle the input data: treat it as strings or as numbers. JSON is a text format that supports UTF-8 encoded strings, but all significant characters are within the ASCII range. This means it can be parsed without decoding the characters, though multi-byte characters must be skipped. The previous tokenizer implementation was:

1. Read bytes from the stream into a byte buffer.
2. Decode this byte buffer into another char buffer.
3. Tokenize this char buffer.
4. Interpret these tokens.

The optimization idea was to tokenize the byte buffer directly and decode only string tokens:

1. Read bytes from the stream into a byte buffer.
2. Tokenize this byte buffer.
3. Decode string tokens.
4. Interpret these tokens.

This required a significant overhaul of the tokenizer and provided minimal performance improvement by itself. However, this approach enabled several other optimizations, such as short string caching and using number parsers from byte buffers, which will be discussed below.

Millions of Calls: The Smallest Changes Impact Performance

A typical data parser consists of a tokenizer and a lexer. The tokenizer processes each byte/character in the data and splits the data stream into tokens, while the lexer provides context to these tokens (this token is a number, this token is a string, etc.). The code in the tokenization loop is called the most frequently, and even small optimizations can significantly boost performance.

NextLexeme() doing heavy lifting for a parser

Replacing if Chains with Single Array Lookup

For example, determining whether a character is whitespace can be done with an if chain, but the longer the chain, the more operations are needed per character. In the case of JSON, all whitespace characters are within the ASCII range (0 to 127), so this check can be done with a single array lookup. Another optimization is replacing such an if chain with a switch, and if the values are contiguous numbers, they turn into a jump table, which is cheaper to execute than multiple if statements.

// OLD
if (charCode == SPACE_CHAR_CODE ||
     (charCode >= TAB_CHAR_CODE && charCode <= RETURN_CHAR_CODE) ||
     charCode == NO_BREAK_SPACE_CHAR_CODE ||
     charCode == IDENTIFIER_SEPARATOR_CHAR_CODE ||
     charCode == VALUE_SEPARATOR_CHAR_CODE ||
     charCode == END_ARRAY_CHAR_CODE ||
     charCode == END_OBJECT_CHAR_CODE ||
     charCode == BEGIN_OBJECT_CHAR_CODE ||
     charCode == BEGIN_ARRAY_CHAR_CODE) {
  // ...
}

// NEW
static bool[] LexemeTerminatorChars;

if (charCode < LexemeTerminatorChars.Length && 
    LexemeTerminatorChars[charCode]) {
  // ...
}

Unfolding ArraySegment Structure from Field into Local Variables

The tokenizer does not read the file byte-by-byte but works with a data buffer. In C#, the ArraySegment structure is used for chunks of data and is initially stored in the reader class, occupying 16 bytes. The tokenizer accesses it many times in the loop while parsing the buffer into tokens. Since the compiler cannot assume the state of this field in the class between accesses, each access generates instructions for reading, null-checking, and boundary-checking. By moving the array and boundaries into the method and out of the loop, the compiler can optimize these checks. This simple modification provides a noticeable performance boost.

private ArraySegment<byte> rawJsonValue;

// OLD
bool NextToken()
{
  if (this.rawJsonValue.Count == 1)
  {
    var oneCharNotation = this.rawJsonValue[this.rawJsonValue.Offset];
    // ...
  }
  else if (this.rawJsonValue.Count == 4 && 
           SequenceEqual(this.rawJsonValue, JsonNotation.Null)) 
  {
    // ...
  }
}


// NEW
bool NextToken()
{
  var rawJsonArray = this.rawJsonValue.Array;
  var rawJsonOffset = this.rawJsonValue.Offset;
  var rawJsonLength = this.rawJsonValue.Count;

  if (rawJsonLength == 1)
  {
    var oneCharNotation = rawJsonArray[rawJsonOffset];
    // ...
  }
  else if (rawJsonLength == 4 && 
           SequenceEqual(rawJsonArray, rawJsonOffset, JsonNotation.Null)) 
  {
    // ...
  }
}

Using Utf8Parser Class for Byte to Number Conversion from Byte Arrays

.NET has long supported parsing dates, times, and numbers directly from byte arrays, bypassing the byte-to-character decoding stage, using the Utf8Parser class. Utilizing this reduces string allocations and improves number parsing performance in text.

// OLD
double ReadJsonAsNumber()
{
  string tokenAsString = this.ReadJsonAsString(); // costly and wasteful
  return double.Parse(tokenAsString, CultureInfo.InvariantCulture);
}

// NEW
double ReadJsonAsNumber()
{
   if (Utf8Parser.TryParse(this.rawJsonValue.AsSpan(), out double value, out _))
   {
      return value;
   }
   else 
   {
      throw new FormatException(/* error message */);
   }
}

Unfolding Complex ReaderNode Structure into Separate Fields: From Abstraction to Concrete Implementation

“Any problem in computer science can be solved with another layer of indirection, except for the problem of too many indirections.” — David Wheeler

My previous parser implementation had an elegant interface for the parser’s current state, where the ReaderNode structure held the current token and its value. The token value was also abstracted to support storing numbers, strings, and complex objects. According to the profiler, this flexibility came at a significant performance cost. The optimization was to simplify everything into properties/fields and forget about abstractions.

// FROM
class JsonGameDataReader {
  public ReaderNode Node { get;}
}

public readonly struct ReaderNode
{
  private readonly object value;

  public readonly ReaderToken Token;
  public readonly Type ValueType;

  public bool ValueAsBoolean => /* ... */
  /* ... */
}

// TO
class JsonGameDataReader {
  private ReaderToken token;
  private bool boolValue;
  private bool stringValue;
  /* ... other types */
 
  public ReaderToken Token => this.token;
  public bool ValueAsBoolean => this.boolValue;
  public bool ValueAsString => this.stringValue;
}

Using String Caching to Reduce Memory Allocation

“There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.” — Leon Bambrick

Another ancient trick is caching the results of heavy algorithms. The most memory and performance-intensive part of parsing JSON is string handling. Often, the result of this processing is used only once and discarded. I’m talking about field names, which frequently repeat and are used only to bind data to objects. Their allocations can be reduced. I implemented a simple cache using Dictionary<Int64, string>, where the key is the first 8 characters of the string interpreted as a 64-bit integer. This key can lead to collisions between strings like "A\0\0\0\0\0\0\0" and "A," but this was not an issue in my case.

if (this.rawJsonValue.Count <= 8)
{
  var stringCacheKey = 0UL;

  // create key from rawJsonValue bytes
  var rawJsonArray = this.rawJsonValue.Array;
  var end = this.rawJsonValue.Offset + this.rawJsonValue.Count;
  for (var offset = this.rawJsonValue.Offset; offset < end; offset++)
  {
    var charCode = rawJsonArray[offset];
    stringCacheKey = stringCacheKey << 8 | charCode;
  }
  //

  if (this.stringPool.TryGetValue(stringCacheKey, out var stringValue))
  {
    return stringValue;
  }
  else
  {
    var chars = this.ReadJsonAsChars();
    return this.stringPool[stringCacheKey] = stringValue = new string(chars.Array ?? this.charBuffer, chars.Offset, chars.Count);
  }
}

Conclusion

In this article, I detailed several optimization strategies that significantly improved the performance of JSON and Message Pack parsers for my game development tool — Charon. These changes resulted in a substantial performance boost, demonstrating the impact of even small optimizations in high-frequency code paths.

Here is comparison benchmark of updated parser vs System.Text.Json one on 10MiB chunk of real JSON data.

| Method                         | Mean     | Error    | StdDev   | Ratio | RatioSD | Gen0      | Gen1     | Gen2     | Allocated | Alloc Ratio |
|------------------------------- |---------:|---------:|---------:|------:|--------:|----------:|---------:|---------:|----------:|------------:|
| JsonFormatter                  | 74.45 ms | 1.163 ms | 1.384 ms |  1.03 |    0.03 | 3714.2857 | 142.8571 |        - |  23.82 MB |        1.05 |
| SystemTextJsonFormatter        | 72.40 ms | 0.607 ms | 0.538 ms |  1.00 |    0.00 | 3714.2857 |        - |        - |  22.64 MB |        1.00 |
| JsonFormatterWithStringPooling | 65.76 ms | 0.743 ms | 0.695 ms |  0.91 |    0.01 | 2500.0000 | 750.0000 | 250.0000 |  14.98 MB |        0.66 |

I’m eager to hear your thoughts and experiences with similar optimizations. Are there other techniques or strategies you have found effective in enhancing parser performance? Write me your suggestion in comments below.