Migrating from MongoDB to GraphQL
Hey!
Today I’m going to demonstrate how I migrated a MongoDB collection to GraphQL. With Parse shutting down at the end of the month, there are a lot of applications are looking for a new home for their data and since GraphQL is the future, it’s a great time to migrate! I was genuinely surprised by how easy the process was so here is a little guide so you can migrate your data yourself.
Step 1: Define the schema
The first step in migrating your data is to prepare your GraphQL schema. I’ll be using MongoDB’s sample restaurants collection found here. The restaurants collection is defined by a pretty simple Schema.
An single restaurant from mongoexport output looks something like this:
The first thing we are going to do is define our GraphQL schema to mirror that of the original dataset. For consistency, we’re going to change a few field names, but don’t worry, the migration script will handle transforming each data item. Here is our final schema:
Thats it! Now all we need is a server to host our API.
Notice that our
Grade
andAddress
types do not implement Node. It is unlikely thatGrades
orAddresses
are shared by more than one restaurant so there is no need to create full connections and thus we don't need to implementNode
.
Deploy the schema on Scaphold.io
To get up and running quickly, we’ll deploy an API defined by our new schema on Scaphold.io! It takes about 3 minutes.
- Go to Scaphold.io and create an app.
- You will be taken to the schema designer where you can create the 3 types listed above. Make sure that your
Address
andGrade
types do not implement theNode
interface. - You’re done! Don’t worry if you made a mistake, the GraphQL type system will let you know exactly where you went wrong when you start pushing data.
Tip! Watch this video to learn more about the Scaphold Schema Designer
The Migration Script
A migration task like this is referred to as ETL (Extract, Transform, Load). We’re going to write a simple node.js script that streams data from our restaurants.json MongoDB dump into our GraphQL API. In real world situations, our data might be too large to put in memory so we are going to read it in line by line and then queue our API calls.
Here is our script! We are using two packages that are not available to a basic node.js installation. Before running this script make sure you install async
and request-promise
via npm install async request-promise
from your project directory.
Take a look at our script.
The $date syntax from the mongodump exists because mongodb stores its documents using BSON not JSON. BSON is a binary superset of JSON that adds some additional functionality and thus needs the extra annotations in order serialize itself to JSON.
If you’re following along all you should have to do now is run your migration script via node ./migrator.js
. If everything is setup correctly, your terminal will start printing out success messages for each item it uploads!
Test the migration
As soon as it is done, you can immediately start querying your deployed GraphQL API. Try this one in the GraphiQL tab in the Scaphold portal and/or from your application:
Migrating more complicated data
You can use this same process to easily migrate any data to GraphQL. Scaphold offers a couple features that can make it a lot easier to migrate more complicated data as well. If you have native relations in your datasets already, take a look at the nested create operators in your Scaphold API. They allow you create and associate Node implementing types in a single API call.
For example, assume we had the following types.
If we had a dataset with a lot of post nodes that were already associated with a category then we could create our posts as well as associate them with the category with a query like this:
Thanks for reading!
If you have any questions please let me know below or Join us on Slack!
We’d love to hear what you think and are even more excited to see what you build!
Originally posted on scaphold.io