How To Upload To IPFS From The Frontend With Signed JWTs
Using Next.js and Pinata
Managing file uploads in an application can quickly become unreasonably complex. It can be a source of frustration and bugs if not handled properly. Depending on the application, developers may also have to figure out how to manage large file uploads, which adds an additional layer of complexity. In web3, there’s a further level of complexity as developers often want to do as much as possible on the client, rather on a server. This gives their users a credible exit. However, protecting API keys and secrets becomes a challenge.
Let’s explore how we can solve for all of this complexity by building a simple app that uploads and pins files to the Interplanetary File System (IPFS) directly from the frontend. But first, let’s talk about some of the challenges in more detail.
Challenges
Let’s say you’re building the next hot photo sharing app. Your users have photos on their device that they need to upload to your service (which is hopefully using IPFS and open data). How do you allow these users to upload?
- Authenticated upload to a backend server
- Authenticated upload to a serverless function
- Authenticated upload from the client
Each of these options has challenges.
Authenticated upload to a backend server
Running a backend server is one of the most standard paradigms, especially when it comes to uploading media. However, it creates a significant amount of overhead, and it can create additional latency if you don’t carefully architect your client-server relationship. The cost of running and maintaining a dedicated server can be an especially high burden to overcome for new startups and independent app developers.
Authenticated upload to a serverless function
Serverless functions, of course, use a server. However, it’s not a server you run and maintain. These functions are on-demand servers that spin up when you need them and spin down when you don’t. There are many services that offer built-in serverless functions, like Netlify and Vercel. You can also create your own serverless function through cloud platforms like AWS and Google Cloud Platform. They reduce costs and complexity in the early stages, and can do so even as you scale. However, they are ill-equipped to handle media uploads. Most serverless functions have a payload limit of about 4MB. That only enough for text files and small photos.
Authenticated upload from the client
The seemingly easiest solution is to upload files from the client. You don’t have to worry about how long it takes because you can use the UI to inform users of what’s happening. You don’t have to worry about payload limits. However, there’s a critical downside. When you’re using a pinning service like Pinata to upload files to IPFS (or even if you’re running your own IPFS nodes and need to authenticate the requests), you would need to expose your API keys in the client-side code. This means anyone with a bit of technical skill can find those keys and use them to abuse your service.
Solutions
Uploading from the frontend sounds like the best solution, but how can we get around the API key problem? One method might be to use Scoped Keys from Pinata. When doing this, you reduce the surface area of an attack by limiting what the keys can do. But even then, let’s explore what might happen.
If you scope a key to only have upload capabilities, a malicious actor can run a script that uses that key to upload tons of files, costing you significant money or locking out your account. Fortunately, there’s a solution, and we’re going to show it off through code today. Let’s get to building.
Getting started
For this project, we’ll be making use of Next.js because it has built-in serverless functions. To create Next.js applications and to use Pinata’s SDK, you’ll need to be running Node.js version 16 or greater.
You’ll also need to have a code editor and a Pinata account. I’ll let you handle the code editor, but let’s get you set up with a Pinata account. Head to Pinata and sign up for a free account here. When you’ve signed up, you’ll need to create an API key to use with our project. To do that, click on the API keys link on the left side. Then click the button to create a new API key.
Once you have the keys save it all somewhere. We’ll be using the JWT soon.
Creating the app
We’re going to make this as simple as possible. We’re going to use the Pinata Next.js Starter Template. So, fire up your terminal and navigate to the folder where you keep all your projects. In the terminal, run the following:
npx create-pinata-appFollow the prompts to create your project. I’ll be building my example in plain JavaScript, but feel free to use TypeScript if you prefer. I will be using the Tailwind option, but the design of the app is not a huge focus for this tutorial.
Now that we have our project, be sure to change into the directory that was just created and open the code in your code editor. You’ll see an .env.sample file. Copy that and create a .env.local file. In this new file, paste in your API JWT you saved from before. You’ll notice both a NEXT_PUBLIC_GATEWAY_URL variable and a NEXT_PUBLIC_GATEWAY_TOKEN variable. We’ll use the NEXT_PUBLIC_GATEWAY_URL but we won’t use the other on in this tutorial.*
*Learn more about Dedicated IPFS Gateways and Gateway Access Controls
For this tutorial, we’re not going to modify the default content and styling of the app that comes out of the box with the Pinata Next.js Starter Template. Instead, we’re going to focus on updating the upload method to allow for uploads directly from the frontend. That means we will also have to update our API routes so that we can fetch signed JWTs for uploads.
Let’s dive in!
Updating the upload function
Let’s start with the upload functionality. If you take a look at the project in your code editor, you’ll see an uploadFile function in the pages/index.js file. This is what we’re going to modify, but before we do, let’s take a look at what it’s doing now. If you’ve followed my previous tutorial on building a simple IPFS-powered file-sharing app, you’ll already know that this function takes the file to upload and sends it to our serverless API endpoint. That endpoint is used to protect our Pinata API key which should be kept a secret and should not be shared with the frontend.
The current function looks like this:
const uploadFile = async (fileToUpload) => {
try {
setUploading(true);
const formData = new FormData();
formData.append("file", fileToUpload, { filename: fileToUpload.name });
const res = await fetch("/api/files", {
method: "POST",
body: formData,
});
const ipfsHash = await res.text();
setCid(ipfsHash);
setUploading(false);
} catch (e) {
console.log(e);
setUploading(false);
alert("Trouble uploading file");
}
};Now, let’s take a look at what the serverless function looks like. Open up pages/api/files.js and you’ll see the POST route looks like this:
if (req.method === "POST") {
try {
const form = new formidable.IncomingForm();
form.parse(req, async function (err, fields, files) {
if (err) {
console.log({ err });
return res.status(500).send("Upload Error");
}
const response = await saveFile(files.file);
const { IpfsHash } = response;
return res.send(IpfsHash);
});
} catch (e) {
console.log(e);
res.status(500).send("Server Error");
}
}And the saveFile function looks like this:
const saveFile = async (file) => {
try {
const stream = fs.createReadStream(file.filepath);
const options = {
pinataMetadata: {
name: file.originalFilename,
},
};
const response = await pinata.pinFileToIPFS(stream, options);
fs.unlinkSync(file.filepath);
return response;
} catch (error) {
throw error;
}
};Fortunately for us, we can nearly just copy and paste that saveFile function and bring it into the frontend code. We’ll need to make a couple of minor tweaks, and there’s still the problem of getting a signed JWT rather than using the secret Pinata API key.
Head back to your pages/index.js file and let’s update the uploadFile function to look like this:
const uploadFile = async (fileToUpload) => {
try {
setUploading(true);
const formData = new FormData();
formData.append("file", fileToUpload, { filename: fileToUpload.name });
const res = await fetch(
"https://api.pinata.cloud/pinning/pinFileToIPFS",
{
method: "POST",
headers: {
Authorization: `Bearer ${JWT}`,
},
body: formData,
}
);
const json = await res.json();
const { IpfsHash } = json;
setCid(IpfsHash);
setUploading(false);
} catch (e) {
console.log(e);
setUploading(false);
alert("Trouble uploading file");
}
};The first thing to note is that we are not using the Pinata SDK to upload. The SDK (at the time of this article) is designed to work in Node.js environments and will throw errors when used on the client. So, we’re using the built in fetch HTTP request function to send our file. You’ll also notice our JWT variable is not defined.
Let’s define it.
Generating signed JWTs
Pinata is unique in that it allows you to generate signed JWTs that have limits on their functionality. In this case, the signed JWT we will use is a one-time JWT with upload functionality. To generate this we need to make an update to our serverless code.
Right now, our pages/api/files.js file handles POST and GET requests. But remember we don’t need the POST request in its current form. Let’s use that request method to generate our signed JWT (in a product app, you’d probably want to create a more descriptive endpoint for this, but we’re keeping things simple). Let’s remove the saveFile function and replace it with a variable we’ll use for generating our signed JWT:
const keyRestrictions = {
keyName: 'Signed Upload JWT',
maxUses: 1,
permissions: {
endpoints: {
data: {
pinList: false,
userPinnedDataTotal: false
},
pinning: {
pinFileToIPFS: true,
pinJSONToIPFS: false,
pinJobs: false,
unpin: false,
userPinPolicy: false
}
}
}
}You can read more about generating keys and JWTs through the Pinata API here.
Now, let’s update the POST request function to look like this:
if (req.method === "POST") {
try {
const options = {
method: 'POST',
headers: {
accept: 'application/json',
'content-type': 'application/json',
authorization: `Bearer ${process.env.PINATA_JWT}`
},
body: JSON.stringify(keyRestrictions)
};
const jwtRepsonse = await fetch('https://api.pinata.cloud/users/generateApiKey', options);
const json = await jwtRepsonse.json();
const { JWT } = json;
res.send(JWT);
} catch (e) {
console.log(e);
res.status(500).send("Server Error");
}
}Here, we are taking the key settings variable and passing it to the Pinata API to request a signed JWT that can only allow uploads and only be used once. Note: in your production app, you’d want to do some sort of authentication verification of your own. That depends on the type of app you’re building so it’s outside the scope of this tutorial.
Now, let’s go back to our frontend code in the pages/indes.js file and add one more thing to the top of the uploadFile function:
const jwtRes = await fetch("/api/files", {method: "POST});
const JWT = await jwtRes.text();You can place that request and response right above the formData variable. With this, you’ll now have a one-time use signed JWT to use for uploading directly from the frontend. Let’s test it to make sure it all works.
Run your app with the following command:
npm run devIf you open localhost:3000 in your browser, you should see this:
Again, this is the same styling and look to the base starter app, but the upload function is completely different. Rather than find yourself limited to 4mb payloads, you can upload files of almost any size, and you can do it safely without revealing your main Pinata secret key.
Conclusion
Signed JWTs are nothing new. Many platforms offer this as a solution to taking actions on the frontend. However, in the world of IPFS and web3, this paradigm is uncommon. Without it, developers sometimes result to unsafe practices like passing their API keys to the client.
With signed JWTs, you can now upload to IPFS from the frontend without fear.
Happy pinning!
