Choosing GraphQL to build Drift’s messaging platform
Drift is building the customer messaging platform for businesses to communicate with prospects and customers from their own website or application.
Like most internet companies today, we built our platform starting from a RESTful JSON API. But even with a clean slate, we still ran into familiar challenges that all API developers face: schema definition and introspection, versioning, paging, data aggregation and response shaping.
We’d been there before. At our previous company, HubSpot, I led the engineering team through a complete rewrite of the product which saw the company grow through an IPO. We built an engineering organization from a handful of engineers to over a hundred, all managing hundreds of REST APIs, powering just as many single-page desktop web applications and mobile clients.
I am a backend engineer, and like most backend engineers I’m both lazy and easily bored. When I’m writing a simple CRUD API I want it to be easy to write, operate and scale.
The one thing I like even less than building these APIs by hand are the endless tweaks and changes that have to be made as the frontend is being built.
At HubSpot, as soon as a front-end engineer would ask for another piece of data, especially one that required either calculation or aggregation, I would suggest they make one or more extra calls on the client.
Unfortunately, they obliged.
Our applications began experiencing load times well into the tens of seconds and our UX was forced to continually add more loading screens everywhere.
Many of our front-end engineers decided to tackle the problem head-on and began deploying lightweight node.js proxy services to combine multiple requests from our APIs into simpler responses containing exactly what they needed while reducing network latency.
Why did they do that?
Because our customers demanded speed, and our API wasn’t delivering.
It didn’t make sense to force our front-end engineers to make up for weaknesses in our backend services, so we shifted the responsibility back to the API engineers to build richer endpoints to better shape the data and reduce the number of requests.
Once we realized at Drift that we were going down the same path, it was time to evaluate alternative solutions. We discovered a new query language specification designed to help us with many of the challenges we’d seen before called GraphQL. Those frontend engineers at HubSpot were on the right path in the first place.
What is GraphQL?
GraphQL is a query language developed by Facebook over the past several years. A GraphQL API is centered around a schema that provides a complete, machine and human-understandable description of the data in your API. This schema is used by the server to provide clients exactly what they need and nothing more. Since it focuses on the schema or data model, it makes it easier to evolve APIs over time; you can easily add new properties or entirely new models without much overhead.
GraphQL allows you to create a schema that exposes your entire organization’s data model via read and write operations. This creates a single point of entry into your API and enables clients to ask in a single call what most REST JSON APIs take many calls to fulfill.
As the complexity of our app grew, we found ourselves in need of a more scalable solution to building our API. We quickly realized there was more to GraphQL than request chaining or collapsing and response shaping. By adhering to a strict schema it would help us build more stable and self-documenting APIs without us having to invent our own versioning schemes.
We are now also leveraging best practices in paging through Relay connections and can’t wait to start using GraphQL Subscriptions to keep our single-page application running entirely in the browser and mobile clients up to date.
GraphQL at Drift
As we learned during our explosive growth at HubSpot, our goal is to enable our web and mobile developers to easily build out our customers user experience as quickly as possible. Let’s see how GraphQL changes how our teams have normally worked in the past versus how fast they can move today.
Our application’s main purpose is to manage information and conversations about our customers’ customers.
Our desktop web application lets our customers do many things, like browse through their customer database, create user segments, schedule one-to-many messages and respond to one-to-one conversations via email or chat.
Meanwhile, our first mobile client in iOS is quickly growing in functionality and has similar demands as our desktop app.
The real-time nature of our business requires our clients to fetch many different data models and keep them in sync through update notifications sent via websocket frames.
A typical feature, such as the ability for a mobile client to render a customer profile, will require lengthy conversations between frontend and backend engineers to agree upon and verify the large number of API requests necessary to fulfill the requirements outlined by designers and product managers for an optimal user experience.
Let’s take a look at this simple screen of a user profile in our mobile app.
The screen requires information from two of our core RESTful services: Customer and Conversation APIs.
Customer attributes come from a single request that returns name, title, location, time zone and avatar. But in order to display the account owner, we’ll need to query the organization’s team endpoint to fetch that name and avatar. To render those colorful tags, we need to fetch the tag definitions from another endpoint. The list of all contact avatars requires a search for all customers in the same company against our ElasticSearch backend. The chat count and last contact requires multiple calls to our Conversation API and one last call to fetch that user’s online status or last active timestamp.
Whoa. I’m exhausted, and now am beginning to understand the pain of frontend engineers every time they take on an implementation of a new feature or screen in our application.
It’s not only difficult to keep track of all of these moving parts for each team, but it’s even harder to communicate how all of these endpoints can come together. Both teams are creating almost identical code to manage these data structures and are creating analogous clients for each API.
Now let’s imagine the GraphQL query necessary to power this view.
Wait. Wut.
What just happened?
We were able to expand our query based on the needs of the client and request a ton of information that usually would have taken multiple requests, a lot of boilerplate and unnecessary code written on both the client and the server.
The payload now conforms to exactly what the customer wanted, and it gives the server the ability to optimize the resources necessary to compute the answer.
Best of all, this was all done in a single request.
Unbelievable.
Welcome to the future.
How to get started with GraphQL?
We’re extremely excited to share the progress we’ve made as well as some of things we’ve learned along the way. For the API side of things (Java-based microservices) we’re using the excellent graphql-java library. In our app we’re excited to dive into Relay and Apollo and learn the best practices they provide.
We love GraphQL so much we are exposing it to you.
If you want to see it in action, sign up for our Drift (it’s free!) and visit https://app.drift.com/graphql to explore our schema and try out some queries for yourself.
At Drift, we believe that every company needs to keep the customer central to everything they do. For us that means creating a channel that helps customers and companies to connect with each other in real time through messaging.
If you are interested in GraphQL and developing real-time messaging applications, you know where to find us. We’re hiring in Boston and San Francisco, but remote folks are welcome as well.
Special thanks to those in the GraphQL community that helped shape this post especially Thea Lamkin, Jonas Helfer, Zack Bloom, Vignesh Mohankumar, Mike Isman, Wout Mertens, and Bryan Kane.
PS. Please consider hitting the ❤ or sharing this with this tweet. 🙏
