How to Write a Gatsby Source Plugin (featuring Cat Facts)

There are many great plugins already written for Gatsby. You should be able to pick pretty much any data source, from local Markdown files to Wordpress to Contentful, and have something running pretty quickly. The way you pull in data from an external source and turn it into GraphQL’able stuff is actually pretty interesting, so today I’d like to take you from zero-to-cat-facts in Gatsby.

“shallow focus photography of brown cat” by James Sutton on Unsplash

Picking an API

There are many free APIs for you to query for example projects, or you may have your own that you want to consume in Gatsby. You can find a list of them on Todd Motto’s GitHub if you want an easy place to start. I am going to make the obvious choice from that list: Cat Facts.

Plan of Attack

Before we get started, I like to have an idea of the tasks I need to accomplish. I like to do everything on GitHub and use its features to my advantage. I’ve written about this before, if you would like more details about leveraging GitHub for personal projects. Taking the new repository approach we can begin to structure our steps.

  1. Create a repo named gatsby-source-*
  2. Basic project setup
  3. Fetch data in gatsby-node.js
  4. Create nodes from data
  5. Test!

You can install your plugin from GitHub, link it with npm, or test your gatsby-node.js directly in an existing project.

Keeping it Simple

There are a lot of fancy things you can do in source plugins. Checkout some of the popular ones in the Gatsby repo like Contentful or Wordpress if you want to see the advanced tactics. Today, since I want Cat Facts in Gatsby as soon as I can, I am going to do the minimum viable approach. I think this is a good way to learn!

Project Setup

Consider your plugin as its own npm package. It will have its own package.json and you can put pretty much whatever you need in there, including your own dependencies and helper scripts. If you plan on submitting your plugin to the official Gatsby repo, it is best to follow their guidelines, but for now we can just live our dreams in a separate repo.

Source Nodes in Gatsby

Before we go and get our data, we need to know where we can put this code. In our gatsby-node.js file, whether it is in your project's or plugin's root folder, we can export a named variable sourceNodes. It actually takes a function, which gives us some nifty parameters to work with. These functions will magically tell Gatsby how to turn our mysterious Cat Facts into queryable data. We're getting a little ahead, so for now, we can just use a blank async function. If you want to pause and do a bit of reading on async/await, check out this article.

Time to Talk to the ☁️

We will need a library to help the Node environment fetch data. Pick your favourite, I will go with node-fetch myself.

The simplest way to see some results is to just console.log in your function, which will show in the gatsby develop command. So let's see some Cat Facts in our console with this code in gatsby-node.js.

// gatsby-node.js
const fetch = require('node-fetch');
exports.sourceNodes = async () => {
const catFacts = await fetch('<https://cat-fact.herokuapp.com/facts>');
// The .all is the data from the cat facts API
// This would be a good spot to do some error handling
catFacts.all.forEach(catFact => {
// .text is the cat fact
console.log('\\n\\n', catFact.text);
});
}

Make sure to name your variables accordingly to the data you’re using!

To test your function, run gatsby develop and you should see your terminal spit out some data. All we've done is show ourselves that we can get data, we haven't passed any of it to Gatsby yet. This is where we get back into the Gatsby node business!

It may be a bit confusing that we’re in the file gatsby-node.js using node-fetch and making nodes of data. Actually, it's definitely confusing. Let's clarify.

  • gatsby-node.js is a file that Gatsby runs in Node.js. It is optional, but you can do stuff like go get your own data.
  • node-fetch is a way to use the Fetch API you may have seen in browsers, but in Node.js.
  • A Gatsby node is a piece of data that you will be able to query via GraphQL in your app.

If you’re using a different API and getting errors, make sure you are using the right keys. I have a .all and .text that may not work for your non-cat-facts-api.

Turn Data into Gatsby Nodes

Now we’re ready to use those parameters we talked about on the sourceNodes function a while ago. We can now change our function signature like this:

- exports.sourceNodes = async () => {
+ exports.sourceNodes = async ({ actions, createNodeId, createContentDigest }) => {

We will use each of these new parameters to help us create data that Gatsby can understand. While we’re creating these new pieces of data, it’s important to note that Gatsby understands a certain type of data. We’ve talked about the Gatsby node, so now lets look at the structure. Using our parameters and Cat Facts data, we can now do this:

const query = await fetch('<https://cat-fact.herokuapp.com/facts>');
// Process data into nodes.
catFacts.all.forEach(catFact => {
const nodeContent = JSON.stringify(catFact);
const nodeMeta = {
// the cat fact unique id is in _id
id: createNodeId(`cat-facts-${catFact._id}`),
parent: null,
children: [],
internal: {
// this will be important in finding the node
type: `CatFacts`,
content: nodeContent,
contentDigest: createContentDigest(catFact),
},
};
const node = Object.assign({}, catFact, nodeMeta);
// remove this once it works!
console.log('\\n\\n', catFact.text);
createNode(node);
});

With this code, we should be able to get the Cat Facts into our app. I’ve left the console log in there so you can see some feedback in the console.

Testing in a Gatsby App

There are a few ways we can pull in this data to make sure it works. We can put our Cat Facts code in the gatsby-node.js file in our test, project, but it's best to use npm link with our plugin repo and test project, so we get a better idea of how it will work in production. To do that, you can simply run one command from your test app's root directory:

npm link ../gatsby-source-cat-facts

In your gatsby-config.js file, add gatsby-source-cat-facts to your plugins array. Don't have one yet? Here's a simple version to get you started:

// gatsby-config.js
module.exports = {
plugins: ['gatsby-source-cat-facts']
};

The easiest way to see if our Cat Facts were loaded properly is to run gatsby develop and head over to your GraphiQL tool, located by default at http://localhost:8000/___graphql. If all has gone according to plan, you should be able to run this query:

{
allCatFacts {
catFacts: edges {
catFact: node {
text
upvotes {
user
}
}
}
}
}

Edges and nodes are the way GraphQL describes the items that are more commonly thought of as array items. The JSON we get back will actually have edges as an array and nodes as the items, but I like to rename them in the query to make the React code more readable.

With that query you should see your fresh facts, and they’re ready to load into your Gatsby app!

Final Touches

I’ve added a few things and published my gatsby-source-cat-facts to npm, so you can play with it yourself! Checkout the repo for the additions I made to get it npm ready, if that’s a step you want to take with your own plugin.

Let me know on Twitter @_cwlsn how the process was and if you have any questions!