REST API to OpenApi to GraphQL
The journey from undocumented untyped API to typed GraphQL using GraphQL-Mesh
I love GraphQL. I really believe it is the next generation of API implementation. It is widely adopted by the industry and many use cases can benefit from it.
My current client builds a platform for connecting Doctors and Patients in real time and we develop it using React Native for mobiles and tablets and Vue for the web application, both talk to a NodeJS API layer implemented over REST.
With this type of stack, different devices require different scopes of data.
Often, users in different areas of the world use slow internet connection and this just screams:
Use GraphQL!!!
Why GraphQL?
GraphQL allows (among other things) to only query for what you need and that helps reduce network load on small devices connected to slow networks and minimizes the number of network round trips (AKA N+1 problem).
APIs defined over GraphQL are typed, predictable and self documenting.
The Challenge
Our NodeJS backend is very rich, we have a lot of routes and logic behind them. Unfortunately, we don’t use TypeScript and the API is not documented over OpenApi. Our Models and Mongoose Schemas are complex with tons of refs and virtual values. This means that implementing GraphQL from scratch was a huge effort with many points of potential failure.
So, what do you do? On the one hand, you know that the product can benefit greatly from using GraphQL, on the other hand implementing it is a huge effort.
It was apparent we needed a tool that generates the GraphQL Schema automagically. We had a couple of options:
- Refactor the code to use TypeScript. There are tools out there to create a GraphQL Schema from TypeScript classes — TypeGraphQL to name one.
- Document the entire API with OpenAPI and use GraphQL Mesh OpenApi handler to map it to a GraphQL Schema
- Use GraphQL Mesh Mongoose handler to map Mongoose Schemas to GraphQL Schema
- Give up
Never Give Up
GraphQL-Mesh is really a wonderful tool. I urge you to try it out.
It has a long list of handlers that transform different technologies (like OpenAPI, Mongoose, gRPC, SOAP, and more) into a GraphQL Schema.
It is open source, for the community by the community and the maintainer team is awesome. Throughout the process I am going to describe below, I chatted with Uri, Dotan and Arda almost daily, with lightning fast response time and issues that arose from my trying out the platform were handled in a professional manner. This really boosts the confidence when you choose a library to use.
First Try
For us, since our Mongoose Schemas are really robust, it seemed that using GraphQL-Mesh Mongoose handler we could generate a GraphQL Schema with zero lines of code and a minimal YAML configuration. So this is the first thing we tried.
However, at the time of writing this, the Mongoose handler wasn’t mature enough. It didn’t support refs and virtual values and the schema generation failed for our complex models (no worries, Arda is already working on adding support).
Plus, creating a GraphQL Schema from the Mongoose layer means that you loose a lot of logic implemented higher up in the Controller/Service layer and that leads to a less than efficient GraphQL Schema.
So, we were left with the huge effort of refactoring the APIs to TypeScript or the huge effort of documenting the APIs with OpenApi.
The Dilemma
How can you choose from 2 bad options?
Well, I lied. We had another option. a hail mary option.
What if we could, programatically, intercept API calls, analyze the request and response, and generate an OpenApi definition file that can later be used to generate the GraphQL Schema using GraphQL-Mesh OpenApi handler?
This is something that takes a couple of days to implement, will update and enrich the OpenApi definitions as APIs are being used, without us having to do almost any manual work.
Do or Die
The Hail Mary
We could not afford to invest in any of the huge efforts. It was the Hail Mary or nothing.
We opted for using a middleware. a middleware is a piece of code that all API calls go through, it is exposed to the request and the response, and it can easily be limited to run only for dev environments so as not to introduce unwanted overhead in production. Moreover, we only allowed it to run and analyze the request/response if a specific query param was passed on the request. This allowed us to control when it runs.
The goal was to create a valid OpenAPI document and for that we had to achieve 3 main requirements
1. Analyze the request and the response.
We did this by simply overriding res.json. All our controllers use it, so it was easy to leverage it. Plus, it was the only way to get both request and response data.
2. Update and append or extend previously mapped API calls
Some of our APIs return slightly different responses under different circumstances. This basically means we treat all our response parameters as optional. We want to be able to update and append or extend previously mapped API calls with richer set of response properties to support that.
Request parameters are easily derived from req.params and req.query,
request body (when applicable) is derived from req.body
and the response structure can be derived from the returned data.
Our code determines the data types from the values and creates the OpenAPI compatible structure, and then checks if the [path][method] already exists in the OpenApi definitions and adds/updates it accordingly.
3. Reuse schema types
We want the generated GraphQL Schema to reuse types.
Multiple APIs sometimes return the same structure, for example getAllItemsList and getItemById will both return [Item] and Item respectively. Had those been typed to begin with, they would have returned the same type.
However, GraphQL-Mesh can’t know (guess) to reuse the same structure as a Type, it can only create a new type.
It was important for us that the fact that these APIs return the same type will be reflected in the OpenApi definition and by extension in the GraphQL Schema.
It is important not only for keeping the OpenApi Definition and GraphQL Schema smaller in size, but it will also allow efficient use of GraphQL Fragments on all Queries/Mutations that return the same type.
This was relatively simple to achieve.
After building the definition in the previous step, all we have to do is check the existing OpenApi [components][schemas] and look for a schema component with the same structure and inject
$ref: #/components/schemas/[component] instead of the structure we built.
This ensures that we reuse the same component schema for all occurrences of the same type across multiple APIs and this in turn ensures that
GraphQL-Mesh reuses those types when it creates the corresponding Queries/Mutations.
Further, it does not break point 2 above, since the code can check for a $ref definition and update/extend the component it points to when needed.
Important Note To Consider
We decided we only apply this logic to response types. We will not apply this to an API that happens to have the same structure in the request parameters or the request body (i.e. input types). This is mainly because:
- It is good practice to avoid reusing input types between GraphQL Queries and Mutations, so that when one changes, it will not affect or break the other.
- Even if we wanted to also apply this logic to input types, it would have required the middleware to do a lot more heavy lifting, analyzing the structure against the $ref and making decisions which should change and under which circumstances.
So, with as little as 250 lines of code and 11 lines of GraphQL-mesh YAML configuration file, we can now generate REST 2 OpenAPI 2 GraphQL in a semi-automatic process.
Now, all that is left is use Apollo-Vue for the web application and
Apollo-Client for the React Native application to call the GraphQL Schema Queries and Mutations.
Conclusion and Next Steps
I can’t recommend TheGuild’s set of tools enough. They are robust, production ready, and well maintained and supported (which is very important when choosing open source for you high-scale production products).
Tools like GraphQL-Modules and GraphQL-Inspector really go a long way for successfully maintaining a large-scale application in the GraphQL space.
Inspector integrates with your CI/CD process and alerts when it detects potential breaks. It validates schemas and detect changes, integrates with Slack for sending schema change notifications and keeps Operations and Fragments consistent.
GraphQL-ESLint helps you keep your GraphQL code aligned with the standards and GraphQL-Scalars adds a long list of scalars to your arsenal that are not provided out of the box.
Lastly, GraphQL-Tools is a great resource for common patters and best practices based on the GraphQL-First approach, that you will benefit greatly from using.
Next Steps
- We opted for adding the generated OpenApi Definitions JSON file to the repository. This allows to detect changes in the OpenApi in Pull Requests and verify nothing breaks the process.
- GraphQL-Mesh allows to dump the generated GraphQL Schema to a file. This is useful, especially with GraphQL Inspector described above.
- We also used GraphQL-codegen to take code generation to the next level and generate TS types and React Apollo hooks for running queries and mutations based on the generated GraphQL schema in the frontend.
Thanks for reading up to this point, I hope you find this post useful.
I am more than happy to answer questions in comments here or you can reach out to me in person!
See you on the next post :)
Sefi Ninio
