IPFS Cluster Peer Installer for EC2

Here’s what I wanted when we first started using IPFS and IPFS Cluster: A one-line installer script:

IPFS Cluster Peer installer

NOTE: You’ll need the second gen of Amazon Linux for the SystemD Services.

Run this on each box you want to be a cluster peer. It will install ipfs, ipfs-cluster-service, and ipfs-cluster-ctl … initialize ipfs and ipfs-cluster… write some auto-restarting SystemD Services for each, and then finally start them both.

Usage

First node (node_0) setup

$ export CLUSTER_SECRET=$(od -vN 32 -An -tx1 /dev/urandom | tr -d ‘ \n’)
$ echo $CLUSTER_SECRET
<secret> ← other nodes must also use this secret

Other nodes (node_n>0) setup

On node_0 after running the installer,

$ journalctl -u ipfs-cluster -n10

In the above log output, look under the line INFO cluster: IPFS Cluster listening on: cluster.go and make a note of the full non-loopback ip4 cluster multiaddress (cluster.listen_multiaddress). This will reference your instance’s private IP address and will be used to bootstrap other nodes.

Back to other nodes (node_n>0),

$ export CLUSTER_SECRET=<node_0 secret>
$ export CLUSTER_BOOTSTRAP=<node_0 cluster.listen_multiaddress w/ instance private IP>

Run the installer

$ wget https://gist.github.com/sanderpick/8660d93abd7cef3c8372565081e280fe/raw/5f190e578a6c480feda8ba210b5fd80b4583ac47/install.sh && bash install.sh

Sanity check

$ sudo systemctl status ipfs
$ sudo systemctl status ipfs-cluster

Tail Cluster Logs

$ journalctl -u ipfs-cluster --follow

Load balance the gateway

Running a public gateway? You could put each peer box behind an EC2 Application Load Balancer by creating a Target Group that maps the LB’s port 80 to instance port 8080 (or whatever your instance gateway ports are).

Firewalls

Lastly, configure a security group for cluster peers and use it on each box. Swarm traffic should most likely be open to the world, but all other ports can be limited to security groups within your VPC. However, this all depends on your needs. Something like this…

IPFS Cluster Peer security group

… where the gateway port is limited to a load balancer SG, the cluster REST API proxy is limited to some other SG in your VPC (possible an application API or wherever you plan to be able to pin files from), and inter-cluster communication is limited to the peers’ SG.