Nodes w/ Built-In Replication: High Performance & Security + Consensus Free

After our breakthrough with S3X which lets you turn any S3 application into an IPFS application with a single line change to your code base and no change in application design. We decided to kick thing up a notch with our nodes to ensure support and performance for big data IPFS work loads.

TemporalX nodes ship with a consensus-less built-in replication protocol that can be enabled via the configuration file. (Benchmarks Below)

The whole reason for TemporalX’s existence is needing to support production scale IPFS workloads, and provide production scale performance, which means we want to remove possible problems like those that occur with Raft, and that the speed + size concerns for CRDT consensus are undesirable.

The solution was quite simple and is done with a replication protocol using gRPC as the transport layer, LibP2P PeerIDs to be be used for TLS encryption, and client-based authentication. This allows us to reuse as much of our existing code and not have to deal with third-party libraries, or maintaining a new codebase.

Replication in TemporalX is controlled by a replication file. Allowing us to specify things like:

  • The “author” (identity responsible for signing the replication files),
  • The servers to replicate to
  • Their corresponding PeerIDs to derive the certificates used for TLS,
  • The CIDs to replicate, and the minimum number of servers to replicate with for a replication to be healthy.

The act of having signed replications provides extra security to replication updates. For example, in IPFS Cluster, to update the “pinset” (list of pins to replicate) it’s guarded by a single swarm key stored in plaintext, and available to everyone. If you have this swarm key, you have access to the entire cluster and can alter the pinset at will.

Contrast to TemporalX’s replication, which requires the use of signed updates to new replications — providing extra security to replications, so you can ensure that the incoming update is being executed by an authorized entity.

Now, this is somewhat similar to IPFS Cluster, which requires a swarm key, however, there is two big difference.

  • 1st: The swarm key must always be available in the config file stored on disk in plaintext, which means if someone gets access to a participating server, they can join your cluster.
  • 2nd: The swarm key is a symmetric key, where replication uses a public key. Even if a replicating server is hacked, the private key is still safe.

TemporalX allows you to broadcast your replication update to the participating nodes and then you no longer need access to that replication file until you want to broadcast another update.

TemporalX can perform replication updates under very high-security measures.

So lets talk performance!

For the full detailed benchmarks: https://github.com/RTradeLtd/xreplb

What is TemporalX ?

(Give us a shout to try it today)

Join Temporal’s online community on Twitter or Telegram. We also have some great IPFS tutorials on our medium.

Written by Kevin Vanstone. Follow him online @KevinVanstone for the latest from RTrade Technologies.

Temporal.cloud

Enterprise Solution For Distributed Data Storage