How to Build Substack on IPFS

Substack has become the go-to resource for newsletters and blogging. For people who want to put up a membership wall and a paywall, Substack provides a simple solution. Today, we’re going to build a version of this using IPFS.

To be clear, this will be an implementation that lacks the newsletter functionality of Substack. We will be adding email capabilities to send out authentication emails, so it shouldn’t be too hard to extend that to include newsletter functionality.

What We’ll Need

Since we will need a little bit of a server implementation to go along with our client-side application, let’s use Next.js. It sets us up nicely for both the front-end and the back-end. Next.js is built and open-sourced by the company Vercel. Vercel also provides free Next.js hosting for small projects and developers who are just getting started, so let’s go ahead and sign up for an account with them.

We want to make sure our content is pinned properly and highly-available. We also want to be able to limit who can see our pinned content. For that, we’ll use Pinata. Sign up for a free Pinata account and then we can get down to the business of making sure you have the right dependencies installed on your machine.

Ready?

Ok, you’ll need the following:

• NodeJS
• NPM
• Text Editor

Getting Started

Next.js uses React for its front-end so getting started is as simple as running a command-line script that will look very simple to you React developers:

npx create-next-app substack-clone

This will create a new Next.js project called “substack-clone”. When the install process is done, you’ll switch into the directory that was just created: cd substack-clone.

If you open your project, you’ll see that while React is in there somewhere, the layout of the entire project is very different. And the reason for that is Next.js contains your front-end code AND your back-end code. In fact, Next.js provides you with server-side-generated React front-ends. Fortunately, the directory structure is very natural.

There is a pagesdirectory that includes an api sub-directory as well as your _app.js and your index.js files. A confusing thing for me at first is that the _app.js and the index.js is essentially a reversing of the paradigm you’re used to with React apps. You register and mount your code for React in the index.js file, and the root component for React is normally App.js. Here, the _app.js file mounts your root component, and the index.js file acts as your root component.

The cool thing is your API code is ready to go. Open up the api folder and you’ll see a hello.js file in there as an example. When you start your app, you’ll have the front-end code available, and you’ll have a running API out of the box. Don’t believe me? Let’s prove it.

Run npm run dev from within the substack-clone directory. Your front-end app will be running on localhost:3000. We will also be able to make API requests to the API. Go ahead and open up the developer console and enter the following code. If you have Postman, open it open and make a GET request to http://localhost:3000/api/hello. If you don’t have Postman, you can do the same from a new terminal window with cURL like this:

curl --location --request GET 'http://localhost:3000/api/hello'

You’ll see the response is:

{
"name": "John Doe"
}

Pretty cool, and it will come in handy later.

Creating Our Front-End

Let’s build out a VERY simple front-end for our Substack clone. We need three things:

• Home Page
• Registration/Sign In Page
• Posts Page

The Home Page and the Posts Page will be the same but one will show all the posts and the other will only show posts that don’t require membership.

In your pages/index.jsfile, let’s create our entry point. Clear out everything inside the opening and closing divs within that file except the Head tags. Let’s make the file look like this:

Obviously, you can call your Substack clone blog whatever you’d like. I’m using “Pinnie For Your Thoughts” as a nice reference to the Pinata mascot. Now, let’s take a look at our main entry point for the app. We have a header with a login link, logo, a headline, a subheadline, a subscribe option, and a link to read a sample post.

This is all pretty straightforward and not much is wired up to do anything yet. The one exception is the Login link and the read a sample link both use the built-in Next.js routing component Link. Those link components are already pointing to pages we’ll wire up soon, so there’s literally nothing else you need to do to make them work.

The Pinnie logo I’m using is in the public folder of the project. You’ll want to put all your images in that folder to make them accessible by Next.js.

As for styling, I’m going to keep it simple and we’ll write style objects in our component files. Feel free to style this however you’d like, but if you want to follow along exactly, here’s the style object for this component:

With all of this done, your entry page should be looking pretty nice. Here’s an example of my page:

Looks a lot like a Substack page, huh?

Ok, what should we work on next? We can start the login page. We can wire up the subscribe functionality. However, I think we should create our sample post since it’s going to introduce us to some of the Pinata functionality.

Let’s create the page first, and then we’ll work on getting the content that will be displayed. In your pages directory, create a new directory called samples. This is where any sample content will be displayed. We can extend this to allow for multiple posts, but we only need one component. So within that directory, create a file that Next.js specifically uses for handling routing with parameters. So, let’s say we are sending the user to a route like this /samples/89eff-5763-s1sdf7. If we create a file called [sid].js we will be able to use the sid variable to read in the identifier in the route. So, go ahead and create a file just like that, called [sid].js. You set the name in the brackets to whatever you’d like. I’m using sid to represent “Sample ID”.

In that file, we’re going to do a few things:

1. We’re going to read the id parameter in the URL path and fetch the appropriate content from IPFS.
2. We’re going to convert that content (which will be stored in markdown format) to JSX.
3. We’re going to render the content.

To help us out here, we’re going to install react-markdown, but we’ll also install some dependencies we’ll use later: jsonwebtoken, sendgrid, and axios. You can install those by running npm i react-markdown axios jsonwebtoken @sendgrid/mail. Once all that is installed, import axios and react-markdownat the top of your [sid].js file. Now, let’s take care of fetching our content, converting it, and rendering it.

You would surely refactor this if you were building a production app, but I think it gets the point across. We are using the route parameter to fetch content from the Pinata IPFS Gateway. We are then converting that content to JSX using react-markdown. I’ve added a bit of styling and I added a class called blog-post that can be used to style the content that is rendered after the markdown is converted. I’m not going to spend time on that in this tutorial though. You all are styling wizards, I can tell.

So, now the question is, how do we get the sample content onto IPFS and available for us to display? First, we need to write it. Go ahead and write up a blog post in markdown.

Once you’ve finished that, we need to upload that to IPFS. Log into your Pinata account and go to the Pinata Upload page. Choose the upload file option and upload your markdown file. You will want to give your file a name there. To make loading posts easier later, let’s prefix all posts we upload with blogPosts — . So your name when uploading might look like: blogPost — MyPost Title. You’ll see why we do that later. When it’s done, you’ll get a content hash. Copy that hash as we’ll need it for our sample.

Go back to your code and open the main index.js file again. Remember, this is where we added our link to read a free sample. Update that link to include your content hash. It should look something like this:

<Link href="/samples/QmPEEogbDouMH8bVSt9K2Tyzg98ACzhZ3cszYhwh7bfWXN">Read a Sample First</Link>

Now, when you go to your home page and click on the “Read a Sample First” link, you’ll be taken to your sample page. If all goes well, the sample post will be fetched from the Pinata IPFS gateway and rendered on screen!

You’ll want to style this page in a way that makes it look nice, but at the very least you can see that we are definitely able to render content from IPFS. Now, the next question is, how do we only render content that is accessible by uses who have subscribed?

Let’s go back to our main entry page, index.js. We need to wire up the subscribe button first. We previously had a console.log in the onSubmit function for our subscribe form. We’re going to connect this to an actual subscription.

Our form element’s onSubmit handler will need to be updated to call a function named handleSubscribe like this:

<form onSubmit={handleSubscribe}>...

While we’re there, go ahead and update the input element to include a value property and an onChange handler. It should look like this:

<input style={styles.input} onChange={(e) => setEmail(e.target.value)} value={email} type=”email” placeholder=”Type your email” />

You’ll note we set the value to equal an email variable and the onChange handler calls setEmail. We’re going to be using React Hooks, so let’s set that up and stub out the handleSubmit function.

At the top of the file, let’s import useState from React. You can do that like this:

import { useState } from "react";

Then, under your main Home function, we can create our state variables using hooks. We’re going to set ourselves up for capturing the email address, setting an error message, and setting a success message.

const [email, setEmail] = useState("");
const [errorMessage, setError] = useState("");
const [successMessage, setSuccessMessage] = useState("");

Now, we can set up our handleSubmit function. To do this, I want to start breaking out some of our code into a separate file. So, within your pages directory, add a new directory called actions. Within that directory, add auth.js. And finally, inside that file, let’s stub out the function to handle subscriptions:

export const subscribe = async (email) => {
  try {
    console.log(email);
  } catch (error) {
    throw error;
  }
}

Now, back in our main index.js file, import that function like this:

import { subscribe } from “./actions/auth”;

Now, we can connect our handleSubscribe function.

Now, we should probably display the error or success message on the screen. Below the subscription form, add the following:

If you try to enter your email and hit subscribe, you should get the success message on the screen. I styled my success message to be green and my error message to be red, but you can do whatever you’d like.

We are powering through this tutorial now! It’s time for some serverside action. We need to create an API route to subscribe. Before we do, you’ll need to sign up for a Sendgrid account. Don’t worry, anything we do here will fall under their free tier. Once you’ve created that account, get yourself an API key and store it somewhere safe.

While you’re at it, go ahead and create a new API key through Pinata as well. Log into your Pinata account, click the avatar dropdown in the top-right, then click Keys. There, you’ll click New Key, and create an “Admin” key. Note: Pinata has great scoped key features that we will make use of later. When the key is generated, save the JWT.

Now, inside your pages/api directory, add a new file called subscribe.js. I’m going to drop the entire code example below, but I’ll walk you through it:

Let’s look at our import statements and then we’re going to work bottom up on this file.

We are importing our required dependencies, but we are also assigning private variables to things. You normally would not want to store those variables in a file like this. You’d make use of something like environment variables to help protect them. Vercel and Next.js have a nice solution for this that’s outside of the scope of this tutorial.

Scroll back to the bottom of the file and find the first function. The first function you see is actually the handler function for your API route. When a user hits the YOUR_DOMAIN/api/subscribe this is the function that handles everything. As we step through this, I’ll walk you through everything happening.

We first check to see if the user trying to subscribe has an existing subscription. The checkSubscriptions function handles this for us. You’ll see that function is querying our Pinata account for pins with metadata including the user’s email. If any are found, we know that the user has subscribed before. If not, we are safe to create a new subscription.

Assuming the user isn’t already a subscriber, we create a new subscription by creating a JSON object that can be stored on IPFS with our Pinata account. That is handled in the createNewSubscription function.

Next, we need to create a JSON Web Token (JWT)that will act as our user’s authentication mechanism very soon. It’s a very simple token with a payload that includes an expiration stamp and the user’s email address. We sign this with our SECRET. Again, this is a secret that should be complex and should likely not be stored in this file. All of this happens in the createJWT function.

Finally, with the JWT created, we can send an email to our user with that token embeded in the link. This happens in the sendEmail function. You’ll note in that function that we are setting the URL to localhost. Obviously, we wouldn’t do this in production, but I think you can identify how this is being used. The link in the email directs the user to our app’s /verify route with a query parameter of the token.

Ok, that was a lot. Take a breath. Get a glass of water or a pint of beer, and come on back for more.

We have two things left to do:

1. Logging in existing subscribers
2. Displaying posts for subscribers

Let’s start with the first one because we’ll re-use a lot of the code we already wrote. In your pages/api directory, create a login.js file. Add the following:

You’ll note we are importing functions from our subscribe.js file. To make that work, flip back over to that file and add the export keyword in front of the checkSubscriptions, createJWT, and sendEmail functions. This is all we need for our login route. Go ahead and try it, and just like when you subscribed, you should get a login link.

Now, let’s make sure our front-end code works for both subscribing and logging in. Since we’ve already created a pin representing a test subscriber, you can unpin that directly through the Pinata Pin Explorer interface. Find the pin in the explorer table and click the menu button to the far-right of the pin. Click Remove Pin.

Now, if you remember, we created an actions folder for our front-end code, and we already created an auth.js file in there. Return to that file, and let’s finish up the subscribe function and create a login function.

Your file should look like this:

We could have easily made this one function with a if/then to decide if we’re logging in or subscribing, but I wanted to be as explicit as possible for this tutorial. We have a subscribe function that calls the api/subscribe endpoint, and we have a login function that calls the api/login endpoint.

We’ve already wired up the subscribe function. Now, we just need to wire up login. If you remember, early on, we created a link to go to the login page. Now, we just need to create that page. Go ahead and create a file called pages/login.js. We’re going to borrow a lot of what we used for the main entry page:

Just like our subscribe form, we’re creating a simple login form with a little bit of styling. We have a back button to take us back to the home page. We also have an error container and a success container. Because we’ve already created our login function, this should just work.

Try logging in with an email you used to subscribe previously. You should get a message indicating that an email was sent.

Perfect! Now we just need to handle the email token verification link and rendering posts. Almost done, folks!

Create a verification component at pages/verify.js like this:

This is a very simple component that has one job: verify an auth token that was emailed to a user. If the token is valid, we are redirecting the user back to the home page (we’ll take care of rendering a new home page for a logged in user in just a moment). If the token is bad, we display an error message on the screen.

Now, what about verifying the token. Well, we can’t really do that client-side because a user could just send any token through and force a validation. So, we need to create a new function in our actions/auth.js file that will make a request to the API and ask the API to verify the token. Add this function to that file:

As you can see that is simply taking the user’s token and making an API request to validate it. If the token is verified, we are storing it to localStorage. You can choose to store this as a session cookie or you can manage the user session some other way. There are a lot of tutorials around session management, and this post is getting super long, so I’ll leave it up to you to manage this in a different way if you so choose. So let’s create a new API endpoint called validate. In the api directory add a validate.js file that looks like this:

We are making use, again, of the jsonwebtokens library to validate the JWT. It’s important that the SECRET variable is the same one you used to sign the token originally. Again, this variable should probably be stored as an environment variable A) for consistency and B) for security.

Now, if you go to http://localhost:3000 and log in, you should get an email. If you structured your email like I did previously, there will be a link that you can click on and your new verify component will just work.

However, we have a problem. When the token is validated, we are redirected to the home page which shows us…

The subscribe option.

Let’s fix that. In the main index.js entry file, we need to do a check to see if our user is logged in or not. The first thing we should do is add useEffect to our imports at the top of the file.

import { useState, useEffect } from “react”;

Next, let’s add a state variable at the beginning of our function component to hold our logged in state:

const [loggedIn, setUserSession] = useState(false);

Then, we’ll use useEffect to check the use session state on page load:

useEffect(() => {
    checkSession();
}, []);
const checkSession = async () => {
    const isUserLoggedIn = await checkUserSession();
    setUserSession(isUserLoggedIn);
}

Next, we’ll add some logic to our navigation (the login button) and our main page. Find the div that contains our Login button and change it up to look like this:

<div style={styles.header}>
    {
        !loggedIn ?
        <Link href="/login">Login</Link> :
        <a onClick={logout}>Logout</a>
    }
</div>

We created a Logout link that references a logout function. That function is very simple. In your component, outside the return statement, add this:

const logout = () => {
    localStorage.removeItem("pinnie-token");
    window.location.reload();
}

We are simply removing the token and refreshing the page. You can choose some other paradigm besides window.location.reload, but I want to keep it simple here.

Now, we need to conditionally render our main page content. If we’re logged in, we should not show the subscribe form.

Find the div that contains our main content and change it to look like this:

You can see, we are simply rendering a component called PostList if the user is logged in. Otherwise, we are rendering the subscribe form.

You probably noticed we have two things left to handle. One, we are checking for a user session but we haven’t built the checkUserSession function, and two, we need to create our PostList component.

Let’s create the checkUserSession function first. In actions/auth.js add this function:

We are grabbing the token from local storage. If it doesn’t exist, the user is obviously not logged in. If it exists, we need to validate the token. We use the existing validateToken function to check on the token we found in localStorage.

Remember to import hat function into our index.js file like this:

import { subscribe, checkUserSession } from “./actions/auth”;

Now, we need to render the PostList component. In reality, you probably wouldn’t create this component in the main index.js file, but for the sake of time, we’re going to. Outside of the main return statement in your component, create a new function like this:

You can see here we are using useState to set a loading state variable as well as our posts variable. We are using useEffect to load the posts when the component function mounts. Then, we are rendering the posts on the screen once they are loaded.

Of course, we have to actually fetch the posts. At the top of the file, let’s import the loadAllPosts function:

import { subscribe, checkUserSession, loadAllPosts } from “./actions/auth”;

Now, we need to write the loadAllPosts function. Go to your actions/auth.js file and add the following function:

We are grabbing our stored token and making a request to the API to fetch posts. For simplicity, we are setting a token header for our request, but you could set an Authorization header or something else.

Let’s write our /api/posts API endpoint function now. In your api directory, add a file called posts.js. It should look like this:

This endpoint is doing a couple things. First, we validate the token the user has provided. If it’s valid, we then fetch all pins on Pinata with a name that includes blogPost. Remember, this is the indicator I set, but you may have chosen something else.

Once we’ve fetched those pins, we shape the data to make it easier to display in the UI (name, hash, date). Then, we return that to the client.

If you save this and go to the home page of your app now, you should see any posts (including the same post we created at the beginning show up in the posts list like this:

It’s a little plain, but you can make this look prettier. The point is, it’s working!

The last thing we need to do (I promise, it really is the last thing) is create a page to render posts for logged in users. We don’t want to re-use the sample post page exactly because it doesn’t check if a user is logged in at all.

Just like with our sample post page, we need to create a new folder in the root of our project. We’ll call this folder posts. Inside that folder add a file called [id].js. Inside that file, we’re going to mostly copy and paste from the samples/[sid].js file.

The only real difference is we are verifying that the user is logged in before trying to fetch the content.

If you test this out, your posts should now load! We’ve come a long way, but there are a ton of improvements that can be made.

We should probably deploy this thing. You can do that for free with Vercel. Caching content would be important. Refactoring, better styling, and more can all be achieved. What will you do with this new knowledge? Maybe build a commenting system, also hosted on IPFS? Maybe add reactions? Create an engine to write the posts and programmatically upload them to IPFS? Something else? Whatever it is, I hope you’ve found this post valuable.