Using IPFS Cluster Service for Global IPFS Data Persistence

How to install and configure IPFS’s cluster service across your IPFS network

Ross Bulat
Jan 3 · 8 min read

Automatic replication and pinning across your IPFS network

An IPFS cluster is a powerful tool that gives you a simple means of distributing content globally, via IPFS, with an easy to use CLI. The project homepage can be found at https://cluster.ipfs.io, and the Github project here.

Now, there are two methods of deploying an IPFS Cluster Service. Which method you favour will depend on your deployment methods. They are as follows:

  • Method 1: To configure your entire cluster peerset and list them in the configuration file of your initial cluster node. Deploying a cluster service on this node will then sync the remaining peerset nodes to the cluster. This method is favourable for automated solutions with tools such as Ansible, whereby your deployment is run as an automated process. It is documented in more detail here.
  • Method 2: To deploy the first peer of the cluster and introduce more as they are deployed. This method is a more favourable approach for manually introducing IPFS nodes to a cluster.

This article will adopt Method 2, and will focus on the higher level required configurations over the more technical.

Prerequisites - IPFS

With that, let’s get to how to install and configure an IPFS cluster service.

Installing IPFS Cluster Packages

Installing ipfs-cluster-service

cd ~
wget https://dist.ipfs.io/ipfs-cluster-service/v0.7.0/ipfs-cluster-service_v0.7.0_linux-amd64.tar.gz
tar xvfz ipfs-cluster-service_v0.7.0_linux-amd64.tar.gz
cd ipfs-cluster-service
sudo cp ipfs-cluster-service /usr/local/bin

What we are doing here is downloading and unpacking the ipfs-cluster-service package inside your home directory. Then we are copying the unpacked binary file into a known $PATH location. I have opted for /usr/local/bin.

To test whether the package installed correctly, run ipfs-cluster-service help to bring up the corresponding output.

Installing ipfs-cluster-ctl

cd ~
wget https://dist.ipfs.io/ipfs-cluster-ctl/v0.7.0/ipfs-cluster-ctl_v0.7.0_linux-amd64.tar.gz
tar xvfz ipfs-cluster-ctl_v0.7.0_linux-amd64.tar.gz
cd ipfs-cluster-ctl
sudo cp ipfs-cluster-ctl /usr/local/bin

Again, run ipfs-cluster-ctl help to verify that the binaries are working as expected.

Before moving on, take a read of the overview page of IPFS cluster service to familiarise yourself with the features and limitations of the project:

Initiating a Cluster Service

If you prefer an alternative directory, it can be changed by either using the -c flag: ipfs-cluster-service -c <path>, or setting the IPFS_CLUSTER_PATH environment variable.

The structure of an .ipfs-cluser folder looks like the following:

.ipfs-cluster/
service.json
raft/
peerstore

Let’s briefly explore what these files and folder consist of:

  • service.json: The configuration file of our cluster. A full example of this file can be found here.
  • raft/: A folder containing the consensus data, as well as snapshots, of our cluster. Raft is the name of the consensus protocol that Protocol Labs have developed for the service.
  • peerstore: Simply a list of IPFS multiaddresses of each peer in the cluster. This file however will not be present upon initiating a cluster service, and instead will be introduced / updated as more peers are added to your cluster.

Note: If you are adhering to Method 1 of cluster setup, the peerstore file is required (among other things) before running the IPFS Cluster daemon.

Initiating the ~/.ipfs-cluster folder is done with theinit command:

ipfs-cluster-service init

However, before we hastily run this command there are a couple of things we need to know.

One of those things is the Secret Key of the cluster.

The Cluster Secret Key

By default, initiating a cluster will generate a secret key which can then be obtained in service.json. This secret key will then need to be applied to all other peers that make our cluster.

The automatic generation of a secret key is convenient for our initial cluster peer, but successive peers of the cluster will require that identical secret key in their service.json files.

A secret key can be generated and predefined in the CLUSTER_SECRET environment variable, and will subsequently be used upon running ipfs-cluster-service init.

Run the following to generate a secret key and display it in your terminal:

export CLUSTER_SECRET=$(od  -vN 32 -An -tx1 /dev/urandom | tr -d ' \n')echo $CLUSTER_SECRET

Now, with a known secret key being stored asCLUSTER_SECRET, we can now initiate the ~/.ipfs-cluster folder:

ipfs-cluster-serivice init

service.json Configuration

Opening Required Ports

sudo firewall-cmd --zone=public --add-port=9094/tcp --permanent#repeat for other ports#reload firewalld
sudo firewall-cmd --reload

This is all that is required here.

Running IPFS Cluster Service in the Background

Provided you already have installed supervisord, amend your configuration file with this additional process. I have provided the IPFS_CLUSTER_PATH environment variable, as well as the full path of the .ipfs-cluster folder:

[program:ipfs-cluster-service]
environment=IPFS_CLUSTER_PATH=/home/<user>/.ipfs-cluster
command=/usr/local/bin/ipfs-cluster-service daemon

Note: Supervisord runs as root, so using ~/.ipfs-cluster as the directory will actually point to /home/root/.ipfs-cluster. Not what we want here.

I have also provided the full path to the ipfs-cluster-service binaries in the command value.

Note: I have had inconsistent experiences when simply providing ipfs-cluster-service daemon , with some VPS systems not recognising the program. This may be due to the /usr/local/bin path I opted for in the installation process. But suffice to say, providing the absolute path fixed this issue.

Now, updating your supervisord processes will start the daemon for your cluster peer. We will want to test things in the foreground firstly; update and then stop the new process thereafter:

sudo supervisorctl rereadsudo supervisorctl updatesudo supervisorctl stop ipfs-cluster-service

Testing IPFS Cluster Service

When you have confirmed there are no errors, start your supervisord process again:

sudo supervisorctl start ipfs-cluster-service

At this point the initial cluster peer is up and running, and we have our secret key ready for additional peers to be added. Let’s now visit bootstrapping an additional peer to the cluster.

Bootstrapping Additional Peers

  • Install ipfs-cluster-service and ipfs-cluster-ctl packages (identical to initial peer).
  • Export the CLUSTER_SECRET environment variable, setting the secret key you gathered from the initial peer before initiating the cluster service:
export CLUSTER_SECRET=<initial_peer_secret>ipfs-cluster-service init
  • Open the required firewall ports (identical to initial peer).
  • Run the deamon in the foreground, this time with the --bootstrap flag containing the multiaddress of the initial cluster peer. The format of this address is as follows:
/ip4/<initial_peer_ip_address>/tcp/9096/ipfs/<initial_peer_identity_hash>

Replace the 2 values above with the public IP address the node is running on and the IPFS peer identity hash respectively:

ipfs-cluster-service daemon --bootstrap /ip4/<initial_peer_ip_address>/tcp/9096/ipfs/<initial_peer_identity_hash>
  • Add the daemon command within your supervisord configuration. Since your cluster peer has already been bootstrapped, we no longer need to include the--bootstrap flag for subsequent daemon starts.
[program:ipfs-cluster-service]
environment=IPFS_CLUSTER_PATH=/home/ross/.ipfs-cluster
command=/usr/local/bin/ipfs-cluster-service daemon

Note: If restarting a cluster peer, the --bootstrap flag is not necessary, unless the peer has been removed from the peerset and needs to be added again.

To verify your peers are being added to the cluster successfully, run:

ipfs-cluster-ctl peers ls

Managing Content in the Cluster

For clarity, the add and pin commands add content to your cluster. Pinning content will ensure their persistence across the cluster, and in my opinion is the best use case of IPFS Cluster Service thus far, to globally persist content.

# adds content to the cluster
ipfs-cluster-ctl add myfile.txt http://domain.com/file.txt
# pins a CID in the cluster
ipfs-cluster-ctl pin add Qma4Lid2T1F68E3Xa3CpE6vVJDLwxXLD8RfiB9g1Tmqp58

Production Deployments

Closing Comments

Finally, the roadmap acts a valuable resource to plan your deployments to coincide with vital upgrades that your organisation may require.