How to Include Metadata with Your IPFS Uploads
A deep dive into how Pinata’s IPFS metadata functionality allows you to easily index and search for your content on IPFS.
The Problem
Currently, when uploading and pinning content to IPFS, it’s hard to keep track of your data. While IPFS is a great tool for storing your content in a verifiable and tamper-proof way, it doesn’t provide much in terms of discoverability.
The IPFS network doesn’t offer the querying functionality we’ve become accustomed to with modern databases. Developers can’t query the IPFS network and say “find me all of the legal agreements that I’ve signed” or “find me all the JSON objects my service has uploaded pertaining to this userID”.
So, this leaves us in a scenario where decentralized products have all of this data being pinned on IPFS, but at the same time have a very limited view into what’s actually being stored.
Introducing Pinata metadata support
With the newest release of Pinata Cloud, we’ve given developers the option to associate metadata with their IPFS uploads.
From this point onward, when adding content to IPFS through Pinata’s API, developers can optionally include:
- A custom name for the content being added
- Up to 10 custom key / value pairs
If included in an API request, Pinata will associate these pieces of metadata with whatever you’re uploading to allow for easy querying of your IPFS content through our userPinList API endpoint.
What’s an API request look like?
Please see our pinFileToIPFS documentation for more thorough explanations and code examples.
When uploading files to Pinata via our pinFileToIPFS endpoint, if you wish to include metadata about your upload, you’ll need to include the following “pinataMetadata” object with your multipart/form-data body.
pinataMetadata
The value for this object will take the following format:
{
"name": "aCustomNameForYourUpload",
"keyvalues": {
"customKey": "customValue",
"customKey2": "customValue2"
}
}- Please note that while both “name” and “keyvalues” are optional, if you include the “pinataMetadata” object in your API request, you need to include at least ONE of the two pieces of metadata inside the “pinataMetadata” object*
- Please also note that if you do not provide a custom “name” for your file, the “name” value will automatically be set to the original name of the file being uploaded
The custom “keyvalue” entries can be any of the following types:
- A string of alphanumeric characters
- An number (in the form of an integer or decimal)
- An ISO_8601 date (this is still a string, but we can actually query on dates in this format)
An Example
Let’s pretend you’re building a service for lawyers and want to tag every IPFS upload with the Layer’s ID, the ID for the Lawyer’s client, a charge code, and the cost of the service being provided.
Your metadata object may look like this:
{
name: 'ExampleNameOfDocument.pdf'
keyvalues: {
LawyerName: 'Lawyer001',
ClientID: 'Client002',
ChargeCode: 'Charge003'
Cost: 100.00
}
}How to add metadata for JSON objects?
Please see our pinJSONToIPFS documentation for more thorough explanations and code examples.
We’ve also added new functionality to the pinJSONToIPFS endpoint that allows you to include metadata with your JSON uploads. If you wish to have metadata associated with your JSON object, you’ll need to format your request body like this:
{
pinataMetadata: {
name: 'customNameForYourJSONObject',
keyvalues: {
customKey: customValue,
customKey2: customValue2
}
},
pinataContent: {
Here is where you'll put your original JSON object
}
}Let’s break down what’s happening here. When the Pinata API receives a request via the pinJSONToIPFS endpoint, it parses through it to see if there’s a “pinataMetadata” object inside of it. If this object is found, it then looks for a “pinataContent” object. The JSON content inside of “pinataContent” is what will actually be uploaded to IPFS. The content inside of “pinataMetadata” will be stored separately as metadata that you can query on.
A JSON example
Let’s pretend you’re building a system to verifiably track items through a supply chain and every time an item reaches a new point in the supply chain, data is uploaded in the form of JSON about the item’s current status.
In this example, the body of our JSON upload with metadata may look like this:
{
pinataMetadata: {
name: 'ItemStatus'
keyvalues: {
ItemID: 'Item001',
CheckpointID: 'Checkpoint002',
Source: 'CompanyA'
WeightInKilos: 5.25
}
},
pinataContent: {
itemName: 'exampleItemName',
inspectedBy: 'Inspector001',
dataValues: [
{...}
]
}
}Remember, only the contents of “pinataContent” will actually be stored on IPFS. Don’t take content out of your “pinataContent” body just because you’re adding those key / values to the “pinataMetadata” object.
Querying the data
Please see our userPinList documentation for more thorough explanations and code examples.
To make use of our new metadata uploading powers, we’ll need to be able to query it. New metadata query functionality has been added to the userPinList endpoint which allows developers to specifically search for IPFS uploads that match custom metadata parameters.
Query parameters should be added onto the end of the userPinList request URL in the following formats:
To query on the metadata file attribute:
?metadata[name]=exampleNameTo query on the metadata keyvalue attributes:
?metadata[keyvalues]={"exampleKey":{"value":"exampleValue","op":"exampleOp"}}OR
?metadata[keyvalues][exampleKey]={"value":"exampleValue","op":"exampleOp"}To query on both the metadata name and multiple keyvalue attributes in the same query:
?metadata[name]=exampleName&metadata[keyvalues]={"exampleKey":{"value":"exampleValue","op":"exampleOp"},"exampleKey2":{"value":"exampleValue2","op":"exampleOp2"}}Explaining the "value" and "op" key / values
As seen above, each query on custom values takes the form of an object with a "value" key, and an "op" key.
The "value" is fairly straightforward. This is simply the value that you wish your query operation to be applied to. These values can be:
• Strings
• Numbers (integers or decimals)
• Dates (Provided in ISO_8601 format)
The "op" is what query operation will be applied to the value you provided. The following op codes are available to query with:
• "gt" - (greater than)
• "gte" - (greater than or equal)
• "lt" - (less than)
• "lte" - (less than or equal)
• "ne" - (not equal to)
• "eq" - (equal to)
• “between" - (When querying with the 'between' operation, you need to supply a 'secondValue' to be consumed by the query operation)
For Example:
?metadata[keyvalues]={"exampleKey":{"value":"2018-01-01 00:00:00.000+00","secondValue":"2018-02-01 00:00:00.000+00","op":"between"}}?metadata[keyvalues]={"exampleKey":{"value":4.00,"secondValue":5.50,"op":"notBetween"}}
• “like” — (you can use this to find values that are similar to what you’ve passed in)
For example, this query below would find all entries that begin with “testValue”. A “%” before your value means anything can come before it, and a % sign after means any characters can come after it. So %testValue% would contain all entries containing the characters “testValue”.
?metadata[keyvalues]={"exampleKey":{"value":"testValue%","op":"like"}}• “notLike” — (you can use this to find values that are do not contain the character string you you’ve passed in)
• “iLike” — (The case insensitive version of the “like” op code)
• “notILike” — (The case insensitive version of the “notLike” op code)
• “regexp” — (Regular expression matching)
• “iRegexp” — (Case insensitive regular expression matching)
In Conclusion
We hope you enjoy utilizing the new metadata functionality in Pinata. We hope developers enjoy the added insight into what they’re pinning on IPFS and we can’t wait to see what exciting things our community will build!
As always, please send us an email at team@pinata.cloud if you have any questions! We’d love to hear from you!
