Query hashing to reduce GraphQL Payload Size

Varun Sharma
Feb 15 · 3 min read

The Challenge

We at Tokopedia use GraphQL for interacting with services from our front-end clients Android, iOS and Web. Using GraphQL we send POST requests to the server and get back JSON response. But as the feature grows, size of the query POST body also grows. We wanted to reduce the payload size for our GraphQL queries due to following reasons :

  • We can save client bandwidth by sending less data to the server.
  • Network speeds < Memory speeds, CPU speeds.
  • Client can benefit from Request Body compression in terms of responsiveness.

Proposed Solutions

  • Using utilities like gunzip Drawback here was we needed to check system wide compatibility.
  • Creating a custom solution to support POST body compression i.e. Query Hashing.

Adopted Solution

  • Using Query Hashing to provide client an option to reduce the payload that is being sent as POST body. We adopted this solution because it allowed modular control. We can use Firebase configs to control which queries to compress. This also prevented overall system testing overhead that would have come with gunzip implementation.

Implementation Challenges

  • We needed to come up with hashing algorithm that is performant. So that we do not incur server overhead.
  • We have to formulate fallback strategy in case hash resolution fails. For this case we decided to send normal POST request once we get hash resolution error code from server.

How it works - Client Perspective

  • For the very first request client will send the GQL POST body with a header queryhash set to true.
  • Server will receive the request with POST body and queryhash set to true. Server will create the Query hash for the client using POST body. After that server will send it back in the same header like : ‘queryhash:hash-value;false’. Where ; is the delimiter.
  • Client will store the queryhash against the POST request in a database indexed to Hash of POST request body. Index is calculated at run time.
  • For the next request client will check if the queryhash against the POST body is available. If it’s available then client will not sent the POST body and send the header ‘queryhash:hash-value;false’. Where false tells server not to calculate the queryhash again and use header value.
  • Server will get the ‘hash-value’ from queryhash header. Server resolve it into POST body using queryhash vs POST body map which is stored at the serve end.
  • In case server fails to resolve the Query Hash to a POST body it will send the client status code ‘244’. In that case client will follow Step 1. again.
Client GQL interaction

Sample Payload POST body Without Query Hash

{
“meta_data_1”: “meta_data”,
“meta_data_2”: “”,
“query”: “query sampleQuery($queryParams:ParamType){query body}”,
“variables”: {
“variable_1”: “”
}
}

Sample Payload POST body With Query Hash

{
“meta_data_1”: “meta_data”,
“meta_data_2”: “”,
“query”: “”,
“variables”: {
“variable_1”: “”
}
}

Benefits

For the network speeds < 512kbps and GQL query of size > 2KB, we will save between 20–40 ms at for each network request. Also payload size reduced by > 70 percent for each such request.

Tokopedia Engineering

Story from people who build Tokopedia

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store