Scaling GraphQL development at Meetup
At Meetup we’re invested a great deal of our products technical strategy in GraphQL. For our core product API’s we lean heavily on GraphQL Java for exactly that.
Scaling can be defined in several ways. Typically people talk about scaling out servers to handle request load. In this post I’ll take about scaling (in) development time, through the lens of reducing load placed on developers so they can make better use of their time. Over the years we’ve evolved our practices to optimize for reducing maintenance and faster turn around time for common GraphQL-oriented changes. This post outlines our current practices based on what we’ve learned.
To start off, it’s useful to introduce the vocabulary of GraphQL Java. A DataFetcher is the interface through which you fetch data for a given field or set of fields that a GraphQL client selects. GraphQL Java invokes these when a client selects their associated field based on a TypeRuntimeWiring which is what binds your code to your GraphQL schema. To some degree you can think of this like AWS Lambda where the trigger integration is GraphQL Java and your functions are DataFetchers. In fact we’re not the first to think of that analogy .
In our humble beginnings we started out with an approach that looked similar to the hello world example in the GraphQL docs: a single class with multiple runtime type wirings for various fields. This quickly grew unwieldy so we split that into multiple files which represented types and all of their respective fields. This worked for a while, but not before a few less than obvious issues started to creep in. Exposing a field meant writing the code that fetched the field’s data and also the code required to wire that into the schema, and that became onerous. The latter was very repetitive, sometimes error prone, and the file that contained that grew slowly but surely into the same shape as our initial approach over time. It surfaced as nonessential complexity to new engineers. We are believers in well tested code and what we found was that this design also impacted testing experience because our tests effectively mapped to these files which contained many unrelated functions each with their own set of test cases so our test sources themselves became unwieldy and hard to maintain.
Our web team has adopted Next.js as its web framework which comes with a handy feature: to add a new page you just drop a file in a directory called “pages” and you are mostly done. You don’t have to think about how that page then get “wired” into the rest of the system. We essentially wanted an experience like that but for GraphQL fields. At this point we already started to create conventions for directories named after types and separating DataFetchers into separate files. So we started with an approach of resolving at runtime a list of DataFetchers to wire into the schema. To do that we used an exceptional library called Classgraph to do that, creating classpath filters based on classes with a simple annotation. While that worked extremely well it was not without its own drawbacks.
As the number of GraphQL fields grew so did our startup times, almost linearly.
At this point we liked what we had but couldn’t help be haunted by a bit of old and still relevant conventional wisdom.
Code is ran more often that is written and also more often than it is compiled.
This startup time delay also costed productivity time for running tests as well as production deployment time. Our measurements concluded about 1–2 seconds of start up time was spent just on resolving these DataFetchersat runtime 🐢.
The rest of this post is how we have our cake and eat it too with a system we call “Self Wired” fields.
There is a feature of Java that’s been around for ages but likely often goes unnoticed to most engineers called annotation processing and one of the nicer features of it is that you can use it to generate code at compile time. It’s a builtin compiler feature that let’s you effectively run code that can generate code as its being compiled 🤯. If you are familiar with Rust, you might be familiar with the idea because it has feature called procedural macros which are similar in spirit.
To write an annotation processor you need two things: an annotation and a processor for it. Yes, obvious, but we had to get that out of the way first.
Below is what our annotation for GraphQL fields looks like. In effect all our GraphQL DataFetchers declare an annotation like this @GraphQLField(parent = "Event", name = "duration")
package com.meetup.graphql.annotation;
import java.lang.annotation.*;
@Target(ElementType.TYPE)
// 👇 this annotation doesn't persist past source code compilation
@Retention(RetentionPolicy.SOURCE)
// 👇 this annotation may be repeated on a give type, allowing for aliased fields
@Repeatable(GraphQLFields.class)
public @interface GraphQLField {
// 👇 the field's parent type name
String parent();
// 👇 the field's name
String name();
}Bellow is the example code we use to process this annotation.
package com.meetup.graphql.annotation;
import static java.util.stream.Collectors.joining;
import static javax.tools.Diagnostic.Kind;
import java.io.IOException;
import java.nio.file.*;
import java.util.*;
import java.util.function.*;
import java.util.stream.Stream;
import javax.annotation.processing.*;
import javax.lang.model.SourceVersion;
import javax.lang.model.element.TypeElement;
import javax.lang.model.type.TypeMirror;
import javax.tools.StandardLocation;
@SupportedAnnotationTypes({
"com.meetup.graphql.annotation.GraphQLFields",
"com.meetup.graphql.annotation.GraphQLField"
})
public class Processor extends AbstractProcessor {
// 👇 some enforced naming conventions
private static final Set<String> NAME_SUFFIXES = Set.of("Query", "Mutation");
private static final Predicate<String> UNCONVENTIONAL_NAME =
name -> NAME_SUFFIXES.stream().noneMatch(name::endsWith);
@Override
public SourceVersion getSupportedSourceVersion() {
return SourceVersion.latest();
}
@Override
public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment env) {
annotations.stream()
.forEach(
annotation -> {
var log = this.processingEnv.getMessager();
var lines =
env.getElementsAnnotatedWith(annotation).stream()
.map(TypeElement.class::cast)
// 👇 check assumption that this is a DataFetcher impl
.filter(isDataFetcher(log))
.map(line(log));
write(log, lines);
});
return true;
}
Predicate<TypeElement> isDataFetcher(Messager log) {
var dataFetcher = dataFetcher();
return elem -> {
var is = isAssignable(elem, dataFetcher);
if (!is) {
// 👇 compile time error if annotated type is not a DataFetcher
log.printMessage(
Kind.ERROR, "GraphQLField impl must implement graphql.schema.DataFetcher", elem);
}
return is;
};
}
Function<TypeElement, String> line(Messager log) {
return elem -> {
var clsName = elem.getQualifiedName().toString();
return Arrays.stream(elem.getAnnotationsByType(GraphQLField.class))
.map(
field -> {
if (UNCONVENTIONAL_NAME.test(clsName)) {
// 👇 compile time error if annoted type doesn't follow our naming conventions
log.printMessage(
Kind.ERROR,
"GraphQLField impl class name should end in Mutation or Query for"
+ " consistency",
elem);
}
return String.join(",", clsName, field.parent(), field.name(), field.alias());
})
.collect(joining("\n"));
};
}
// 👇 write data out to CLASS_OUTPUT directory as a compile time artifact
void write(Messager log, Stream<String> lines) {
try {
// 👇 create a file which represents a preindexed list of comma-delimited className,parentName,fieldName)
var resource =
Path.of(
processingEnv
.getFiler()
.getResource(
StandardLocation.CLASS_OUTPUT,
"",
"META-INF/services/graphql.schema.DataFetcher")
.toUri());
resource.getParent().toFile().mkdirs();
var contents = lines.collect(joining("\n")) + "\n";
Files.writeString(resource, contents, StandardOpenOption.CREATE, StandardOpenOption.APPEND);
} catch (IOException e) {
log.printMessage(Kind.ERROR, "unable to save GraphQLField info " + e);
}
}
TypeMirror dataFetcher() {
return this.processingEnv
.getTypeUtils()
.erasure(
this.processingEnv
.getElementUtils()
.getTypeElement("graphql.schema.DataFetcher")
.asType());
}
boolean isAssignable(TypeElement a, TypeMirror b) {
return this.processingEnv.getTypeUtils().isAssignable(a.asType(), b);
}
}The trick to exposing an annotation processor is simply to follow standard Java ServiceLoader conventions create a file called src/main/resources/META-INF/services/javax.annotation.processing.Processor including the contents com.meetup.graphql.annotation.Processor
In addition you’ll need to register this with your build system. In the case of Gradle, we do that by adding our graphql-annotation build as an annotation processor dependency of the build which contains our DataFetchers
dependencies {
annotationProcessor(project(":graphql-annotation"))
}At this point we've only setup a system which generates an index of DataFetchers for GraphQL fields and metadata about what parent type name they are attached to as well as their field name.
At runtime we are able to use this to remove maintenance task of manually wiring fields into the schema with the following code, imports wildcarded and logging removed for brevity.
package com.meetup.graphql.wiring;
import com.meetup.graphql.base.ServiceRegistry;
import graphql.schema.DataFetcher;
import graphql.schema.idl.TypeRuntimeWiring;
import java.io.IOException;
import java.lang.reflect.*;
import java.util.*;
import java.util.function.*;
import java.util.stream.Stream;
public class SelfWiring {
private final ServiceRegistry serviceRegistry;
private final Supplier<List<Creator>> creators;
// 👇 define a way to create a class instance, more on this below
static class Creator {
enum Flavor {
NO_ARG,
SERVICE_REGISTRY;
}
public final String parent;
public final String name;
public final Flavor flavor;
public final Constructor<?> ctor;
public Creator(String parent, String name, Flavor flavor, Constructor<?> ctor) {
this.parent = parent;
this.name = name;
this.flavor = flavor;
this.ctor = ctor;
}
DataFetcher<?> create(ServiceRegistry registry)
throws InstantiationException, IllegalAccessException, InvocationTargetException {
return (DataFetcher)
switch (flavor) {
case SERVICE_REGISTRY -> ctor.newInstance(registry);
default -> ctor.newInstance();
};
}
}
public SelfWiring() {
this(new DefaultServiceRegistry().get(), SelfWiring::creators);
}
SelfWiring(ServiceRegistry serviceRegistry, Supplier<List<Creator>> creators) {
this.serviceRegistry = serviceRegistry;
this.creators = creators;
}
// 👇 resolves a list of runtime wiring dynamically instantiated objects for a set of schema types.
public List<TypeRuntimeWiring> resolve() {
return creators.get().stream()
.map(
creator -> {
try {
// 👇 this is effectively what we used to do by hand for every new field
return TypeRuntimeWiring.newTypeWiring(creator.parent)
.dataFetcher(creator.name, creator.create(serviceRegistry))
.build();
} catch (InstantiationException
| InvocationTargetException
| IllegalAccessException e) {
throw new RuntimeException(e);
}
})
.toList();
}
static List<Creator> creators() {
return Thread.currentThread()
.getContextClassLoader()
// 👇 this is there our compile time artifact lives on the class path
.resources("META-INF/services/graphql.schema.DataFetcher")
.map(
url -> {
try (var inputStream = url.openStream()) {
return new String(inputStream.readAllBytes());
} catch (IOException e) {
return null;
}
})
.filter(Objects::nonNull)
.flatMap(
fields ->
fields
.lines()
.flatMap(
line -> {
var data = line.split(",");
try {
var cls = Class.forName(data[0]);
try {
// 👇 assume single constructor that accepts a ServiceRegistry
return Stream.of(
new Creator(
data[1],
data[2],
Creator.Flavor.SERVICE_REGISTRY,
cls.getConstructor(ServiceRegistry.class)));
} catch (NoSuchMethodException e) {
// 👇 fallback no no-arg constructor
try {
return Stream.of(
new Creator(
data[1],
data[2],
Creator.Flavor.NO_ARG,
cls.getConstructor()));
} catch (NoSuchMethodException e2) {
}
return Stream.empty();
}
} catch (Exception e) {
return Stream.empty();
}
}))
.toList();
}
}The essence of this is code instantiates a class, and registers it with the schema based on its index metadata which is an approach that dramatically brought down our startup times by an order of magnitude.
Within this solution there was problem introduced which we addressed in the simplest way possible. Because we are dynamically creating instances of classes, we needed a way to have them opt into accepting their dependencies in a uniform shape for DataFetchers that have them. The simplest way to do that was to allow these classes to look up their dependencies by type name from a type which we call our ServiceRegistry
In simplest terms this is literally light weight dependency injection which is appropriate because we’ve no need for the costs that come from a heavier weight framework. An alternative might be Guice, Dagger, Spring or similar if you are already using one of those.
package com.meetup.graphql.base;
import java.util.*;
import java.util.concurrent.*;
public class ServiceRegistry {
private final ConcurrentMap<Class<?>, Object> map;
private ServiceRegistry(ConcurrentMap<Class<?>, Object> map) {
this.map = map;
}
// 👇 get an instance of type by its class either pre-registered or service loaded
@SuppressWarnings("unchecked")
public <T> T get(Class<T> cls) {
return (T)
Optional.ofNullable(map.get(cls))
.map(Optional::of)
.orElseGet(
() -> {
Iterator<T> iter = ServiceLoader.load(cls).iterator();
return iter.hasNext() ? Optional.of(iter.next()) : Optional.empty();
})
.orElseThrow(
() ->
new NoSuchElementException(
"No service of type " + cls.getName() + " was defined"));
}
// 👇 make it easy to construct one of these correctly
public static class Builder {
private final ConcurrentMap<Class<?>, Object> map = new ConcurrentHashMap<>();
public Builder register(Object obj) {
this.map.put(obj.getClass(), obj);
return this;
}
public Builder register(Class<?> iface, Object obj) {
if (!iface.isInstance(obj)) {
throw new RuntimeException(
"class "
+ obj.getClass().getSimpleName()
+ " is not an instance of "
+ iface.getSimpleName());
}
this.map.put(iface, obj);
return this;
}
public ServiceRegistry build() {
return new ServiceRegistry(map);
}
}
}The outcome of our self wiring system has the following benefits over our initial approaches
- The activity of adding new fields is scoped expressly to just the writing of logic for those fields. One only needs to annotate that with
@GraphQLField(...)and you’re off to the type wiring races - This system reduced the pain involved in maintaining large test files. We now have a clear mapping between a
DataFetcherand its associated test file. As a result, when making changes, it’s clear and unambiguous where to go looking for and updating tests and with them being focused test files they are also easier to run in isolation. We heavily leverage the latter in our build system to only run tests impacted in a changeset. - Our system is naturally built with testing in mind. It’s now much easier to “inject” dependencies into tests in through a single, easy to explain and coherent interface.
- We’ve made no tradeoffs in startup time. The index of fields generated at compile time completely removes the cost of traditional runtime annotation resolving. Previously this costed 1–2 seconds of startup time and is now down to 2–4 milliseconds.
- We now have a system we can wire in convention rules at compile time. As mentioned above we now have a few naming conventions for DataFetchers. Rather that running a linter at runtime we can enforce that at compile time.
