Migrating from MongoDB to GraphQL

Move your MongoDB data into the future with GraphQL

Hey!

Today I’m going to demonstrate how I migrated a MongoDB collection to GraphQL. With Parse shutting down at the end of the month, there are a lot of applications are looking for a new home for their data and since GraphQL is the future, it’s a great time to migrate! I was genuinely surprised by how easy the process was so here is a little guide so you can migrate your data yourself.

Step 1: Define the schema

The first step in migrating your data is to prepare your GraphQL schema. I’ll be using MongoDB’s sample restaurants collection found here. The restaurants collection is defined by a pretty simple Schema.

An single restaurant from mongoexport output looks something like this:

The first thing we are going to do is define our GraphQL schema to mirror that of the original dataset. For consistency, we’re going to change a few field names, but don’t worry, the migration script will handle transforming each data item. Here is our final schema:

Change restaurant_id to mongoId in the Restaurant type

Thats it! Now all we need is a server to host our API.

Notice that our Grade and Address types do not implement Node. It is unlikely that Grades or Addresses are shared by more than one restaurant so there is no need to create full connections and thus we don't need to implement Node.

Deploy the schema on Scaphold.io

To get up and running quickly, we’ll deploy an API defined by our new schema on Scaphold.io! It takes about 3 minutes.

1. Go to Scaphold.io and create an app.
2. You will be taken to the schema designer where you can create the 3 types listed above. Make sure that yourAddress and Grade types do not implement the Node interface.
3. You’re done! Don’t worry if you made a mistake, the GraphQL type system will let you know exactly where you went wrong when you start pushing data.

Tip! Watch this video to learn more about the Scaphold Schema Designer

The Migration Script

A migration task like this is referred to as ETL (Extract, Transform, Load). We’re going to write a simple node.js script that streams data from our restaurants.json MongoDB dump into our GraphQL API. In real world situations, our data might be too large to put in memory so we are going to read it in line by line and then queue our API calls.

Here is our script! We are using two packages that are not available to a basic node.js installation. Before running this script make sure you install async and request-promise via npm install async request-promise from your project directory.

Take a look at our script.

The $date syntax from the mongodump exists because mongodb stores its documents using BSON not JSON. BSON is a binary superset of JSON that adds some additional functionality and thus needs the extra annotations in order serialize itself to JSON.

If you’re following along all you should have to do now is run your migration script via node ./migrator.js. If everything is setup correctly, your terminal will start printing out success messages for each item it uploads!

Test the migration

As soon as it is done, you can immediately start querying your deployed GraphQL API. Try this one in the GraphiQL tab in the Scaphold portal and/or from your application:

Migrating more complicated data

You can use this same process to easily migrate any data to GraphQL. Scaphold offers a couple features that can make it a lot easier to migrate more complicated data as well. If you have native relations in your datasets already, take a look at the nested create operators in your Scaphold API. They allow you create and associate Node implementing types in a single API call.

For example, assume we had the following types.

If we had a dataset with a lot of post nodes that were already associated with a category then we could create our posts as well as associate them with the category with a query like this:

Thanks for reading!

If you have any questions please let me know below or Join us on Slack!

We’d love to hear what you think and are even more excited to see what you build!

Originally posted on scaphold.io