Your GraphQL is Turning into a Bottleneck — Time to Federate it

Monolithic development isn’t all it’s cracked up to be until massive traffic and new features start to creep in.

Here at Kitalulus, we provide career development targeted at fresh graduates and blue-collar workers in Indonesia. With Indonesia’s youth population ranking up to the 4th largest in the world, the demand for user-friendly job hunter career apps increased over time. And so the new features.

We’re big fans of GraphQL. With its easily customizable query to fetch necessary data, GraphQL makes the development of new APIs easier for the front end and back end. A GraphQL endpoint acts as a gateway between client and server.

Our initial implementation of GraphQL

But with it comes one big question: How does a GraphQL service scale?

Initially, our development APIs only consist of a few domains: Vacancies, Company data, and User data. They’re all combined into one service. But then came Community APIs related to Membership APIs, Feed APIs, among other domains.

Imagine how big the GraphQL codebase will be!

Due to the increasing number of services handling business logic, the GraphQL API has to keep up by adding more queries. This comes with several disadvantages:

1. Double workload. For every new feature added, the team has to work on GraphQL API to resolve schema and solve business logic on the service back-end.
2. Bloated GraphQL API. Despite our efforts to scale services by splitting them, wouldn’t it mean we’re using the GraphQL API as yet another giant monolithic service?

The way things were going, it became obvious that the GraphQL API became our single point of failure. So we started researching how other companies scale their services using GraphQL:

Case Study: Netflix

Netflix went through a similar development hurdle. They were a prominent users of GraphQL. Using the One Graph principle as a gateway for their client to access data. The simplicity of using one GraphQL API (called “Studio API”) to resolve data across multiple services reliably; was a success.

Netflix initial implementation of GraphQL (Source)

But as mentioned earlier, the number of services kept growing. The Studio API became so big that it overwhelmed the engineering team working on it. As they quote it:

“We knew that there had to be a better way — unified but decoupled, curated but fast moving.”

No Longer One Graph

As seen in the documentation for GraphQL, the supporting principle to One Graph is the Federated Graph. It keeps the notion of a single source of truth for GraphQL schema but allows it to be maintained separately.

Instead of implementing an organization’s entire graph layer in a single codebase, responsibility for defining and implementing the graph should be divided across multiple teams.

Though there is only one graph, the implementation of that graph should be federated across multiple teams.

GraphQL Federation serves to split the schema across multiple services. So how does one break down a giant GraphQL schema into several codebases? By using a GraphQL Gateway as an entryway and compiling all those separate schemas into one.

The GraphQL Gateway might look something like this:

import { ApolloGateway } from '@apollo/gateway';
import { ApolloServer } from 'apollo-server';
import dotenv from 'dotenv-safe';
dotenv.config({
  allowEmptyValues: true,
});
const startServer = () => {
  let gateway: ApolloGateway;
  if (process.env.NODE_ENV === 'development') {
    gateway = new ApolloGateway({
      serviceList: [
        {
          name: 'jobs',
          url: process.env.JOBS_SERVICE_URL,
        },
        {
          name: 'user',
          url: process.env.USER_SERVICE_URL,
        },
				// insert other services here
      ],
    });
  } else {
    gateway = new ApolloGateway();
  }
  const server = new ApolloServer({
    gateway,
    playground: process.env.NODE_ENV !== 'production',
    subscriptions: false,
  });
  server
    .listen({
      port: 4000,
    })
    .then(({ url }) => {
      console.log(`🚀  Server ready at ${url}graphql`);
    })
    .catch(err => console.log('Error launching server', err));
};
startServer();

This is the entire content of GraphQL Router. An index.js file to start the server, while listening to other servers containing federated schemas, called Subgraph. The router does not need to scale alongside other services. It will remain as entry point, adding new services and subgraphs to pay attention to inside the serviceList array.

So how does one split up an existing schema into several subgraphs?

For example, you might want to split a GraphQL project called graphql-api to be split into smaller subgraphs:

The graphql-api contains your existing GraphQL schema and resolvers. While the graphql-gateway contains the GraphQL Gateway code. We’ll design the graphql-api to contain a basic examples of schemas, queries, and resolvers.

const { ApolloServer, gql } = require('apollo-server');
const typeDefs = gql`
  
  type Query {
        helloQuery: String
        authors: [Author]
	books: [Book]
  }
  type Mutation {
	helloMutation: String
   }
  type Author {
        authorId: ID!
        name: String
        books: [Book]
  }
  type Book {
	bookId: ID!
        title: String
        author: Author
  }
`;
const authors = [
  {
    authorId: '1',
    name: 'Kate Chopin'
  },
  {
    authorId: '2'
    name: 'Paul Auster'
  }
]
const books = [
  {
    title: 'The Awakening',
    author: 'Kate Chopin'
  },
  {
    title: 'City of Glass',
    author: 'Paul Auster'
  },
];
const resolvers = {
  Query: {
    helloQuery: () => 'Query: hello world!'
    books: () => books,
    authors: () => authors,
  },
  Mutation: {
    helloMutation: () => 'Mutation: hello world!'
  }
};
const server = new ApolloServer({ typeDefs, resolvers });
server.listen().then(({ url }) => {
  console.log(`🚀  Server ready at ${url}`);
});

Duplicate your graphql-api into two, and name them differently. Let’s say you’re splitting them into authors-service and books-service.

authors-service and books-service currently are an exact duplicate of, but soon we will make differences on each of them as a way to begin making the split for the services.

Create a .env file for your graphql-gateway directory. This will pinpoint which servers containing the GraphQL schema and resolver to listen to:

AUTHORS_SERVICE_URL=http://localhost:4001/graphql
BOOKS_SERVICE_URL=http://localhost:4002/graphql
NODE_ENV=development

Make sure to set each service to run on the correct port. After that, run each server. A successful nodemon run in your graphql-gateway would look something like this:

A successful Apollo Federation GraphQL Router run

But this is not going to happen yet! Since both authors-service and books-service have the same schema. You will encounter an error message instead:

Error launching server Error: A valid schema couldn't be composed.
The following composition errors were found:
  Field "Query.helloQuery" can only be defined once.
  Field "Query.books" can only be defined once.
  Field "Query.authors" can only be defined once.
  Field "Mutation.helloMutation" can only be defined once.

This is because the router is listening to several services and detected multiple definitions in your schema. With Apollo Federation, we now have a process where all federated DGS schemas are aggregated into one unified schema. The result is a generated schema ready to be consumed by our client.

In other words, Apollo Federation gives the illusion of a single giant schema.

To see your schema working as intended, you can comment/disable the schema authors-service and books-service to make sure there are no duplicates.

Your authors-service schema:

extend type Query {
  authors: [Author]
}
extend type Mutation {
}
type Author @key(fields: "authorId") {
  authorId: ID!
  username: String
}

Your books-service schema:

extend type Query {
    books: [Book]
}
type Book @key(fields: "bookId") {
  bookId: ID!
  title: String
  author: Author
}
extend type Author @key(fields: "authorId") {
  authorId: ID! @external
  books: [Book]
}

Streamlining Development

To summarize our scaling process in Kitalulus back-end, this is what we started out with. A GraphQL-based service where every request is funneled through the GraphQL API, creating a bottleneck:

With Apollo Federation, the GraphQL API will be handled across different teams. This would result in a request flow like this:

But this still leaves room for improvement. Remember how in the previous One Graph approach where each GraphQL API is wired to the corresponding service containing the business logic? While this approach worked nicely for the mono GraphQL service, it seems we’re now able to combine each subgraph and resolvers alongside the business logic into one complete service.

This is our current work plan. There really is no reason to keep the extra GraphQL API layer when the business logic can immediately be combined into it.

The engineering team at Kitalulus is looking deeper into the subject matter.

Read further what possible languages can be used, along with the pros and cons in this article by one of our back-end engineers.

We’ll keep an eye on this transition phase and report back with an update!