Spartan PoC v2 now on Substrate

Published in

Autonomys Network

7 min readJun 14, 2021

We are pleased to announce that Subspace Labs has delivered the second milestone for our Web 3 Foundation Open Grant to bring Proof-of-Capacity (PoC) consensus to Substrate! For more information on the purpose of this grant and what PoC consensus is, please refer to our previous article.

In the previous milestone, we demonstrated the ability to run a single node development chain using our new Spartan PoC consensus module, which only supported a single farmer with a constant sized plot. In this milestone, we have upgraded Spartan to support multiple farmers with variable sized plots, who are able to cooperatively farm new blocks over a local test network. In addition, Spartan also supports synchronization with other full and light client nodes, including a browser app that displays the current amount of disk space pledged to the network by all farmers.

Dynamic Solution Range

The key technical challenge for this milestone was to implement a dynamic solution range adjustment algorithm to ensure a constant rate of block production, similar to the dynamic work difficulty adjustments in Proof-of-Work (PoW) blockchains, such as Bitcoin.

Spartan, like BABE, is based on Ouroboros Praos, which divides time into discrete timeslots and larger epochs. Currently we have one-second timeslots and 32 x 6 = 216 timeslots per epoch. Ideally we would like to see one block every six seconds, in order to ensure that (on average) any block is able to fully propagate across the network before the next block is found, reducing the likelihood of honest forks, which degrades the security of the network.

In Spartan PoC consensus, for each new timeslot we issue a new challenge. Each farmer then queries their plot to find a commitment to an encoding which satisfies the current solution range. Any farmer who finds a commitment within a specified range (the solution range) of the challenge may then generate a PoC and produce a new block. For more details, please refer to the design document. To ensure that we maintain an average of six seconds (six timeslots) between blocks we must ensure that the protocol automatically adjusts the solution range based on the observed rate of block production, which will change as the disk space pledged to the network grows and shrinks over time.

To accomplish this we further group timeslots into eras of 6 x 2016 = 12,096 slots or roughly 3.4 hours. This mimics the work difficulty resets of Bitcoin which take place every 2016 blocks. Each era we count the number of blocks produced and then compute an adjustment factor based on the target of 2016 blocks. If the adjustment factor is less than one, it means the space pledged has decreased and blocks are too far apart, requiring the solution range to be increased. If the adjustment factor is greater than one, it means the space pledged has increased and blocks are too close together, requiring the solution range to be decreased.

To observe this in action you can spin up a new Spartan node from genesis and just double the size of its plot from the default 1GB to 2GB, which will cause one block to be produced roughly every three seconds until the first era change is enacted. To make this easier to observe for testing we have adjusted the default settings such that era changes are enacted every 32 blocks.

Note: These instructions assume you run the farmer in one terminal and the client in a second terminal.

First, install Docker.

Initialize Farmer (Terminal 1)

Create volume for plot, pull latest image and initialize 2 GiB plot (should take a few minutes):

Note — if you have previously run a spartan farmer you will need to first erase the plot before creating a larger plot, or change the farmer name.

docker volume create spartan-farmer
docker pull subspacelabs/spartan-farmer
docker run --rm -it \
  --name spartan-farmer \
  --mount source=spartan-farmer,target=/var/spartan \
  subspacelabs/spartan-farmer plot 512000 spartan

Run the Client (Terminal 2)

This will create a virtual network, pull the latest image from docker and start a single node (client only) development chain:

docker network create spartan
docker pull subspacelabs/node-template-spartan
docker run --rm --init -it \
  --net spartan \
  --name node-template-spartan \
  --publish 127.0.0.1:30333:30333 \
  --publish 127.0.0.1:9944:9944 \
  --publish 127.0.0.1:9933:9933 \
  subspacelabs/node-template-spartan \
    --dev \
    --tmp \
    --ws-external \
    --node-key 0000000000000000000000000000000000000000000000000000000000000001

Run the Farmer (Terminal 1)

Once the client is running, you can connect a farmer to it by running the following in the first terminal:

docker run --rm --init -it \
  --net spartan \
  --name spartan-farmer \
  --mount source=spartan-farmer,target=/var/spartan \
  subspacelabs/spartan-farmer \
    farm \
    --ws-server ws://node-template-spartan:9944

You should now see block production occurring in Terminal 2, first every three seconds, then eventually every six seconds (once the era change is enacted).

Full Client Sync

A full client, also known as a full node, maintains the full account database or state of the blockchain, as well as the last several blocks. It does not participate in consensus, unless it also has an attached farmer.

Run the full client (Terminal 3)

We can now run another full client and sync the chain from the first client we started earlier:

BOOTSTRAP_CLIENT_IP=$(docker inspect -f "{{.NetworkSettings.Networks.spartan.IPAddress}}" node-template-spartan)
docker run --rm --init -it \
  --net spartan \
  --name node-template-spartan-full \
  subspacelabs/node-template-spartan \
    --dev \
    --tmp \
    --ws-external \
    --bootnodes /ip4/$BOOTSTRAP_CLIENT_IP/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp

Light Client Sync

A light client does not maintain the account database or state of the blockchain. Instead, it only tracks the block headers in order to be assured it is actually on the longest chain, or in our case, the chain with the most space.

Run the light client (Terminal 4)

We can also run a light client and sync the chain from the client we started earlier:

BOOTSTRAP_CLIENT_IP=$(docker inspect -f "{{.NetworkSettings.Networks.spartan.IPAddress}}" node-template-spartan)
docker run --rm --init -it \
  --net spartan \
  --name node-template-spartan-light \
  subspacelabs/node-template-spartan \
    --dev \
    --tmp \
    --light \
    --ws-external \
    --bootnodes /ip4/$BOOTSTRAP_CLIENT_IP/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp

Browser Based Client

We have also extended the substrate-frontend-template to compute and display the total space pledged to the network by all farmers, which can be derived from the quality of the proof-of-space itself.

Install and Run the Browser Client (Terminal 5)

The codebase is installed using git and yarn. This step assumes you have installed yarn globally prior to installing it within the subdirectories. For the most recent version and how to install yarn, please refer to yarn documentation and installation guides.

git clone -b w3f-spartan-ms-2 https://github.com/subspace/substrate-front-end-template.git
cd substrate-front-end-template
yarn install
yarn start

This should automatically open a new browser tab pointed to: http://localhost:8000/substrate-front-end-template

You should now see the current blockchain stats, including the amount of space pledged to the network, which will update with each era change, or every 32 blocks.

Running a Local Test Network

We can now add more farmers to create a local test network and the solution range will automatically adjust itself every 32 blocks.

You can use the script ./ run-node-farmer-pair.sh testto run more full nodes on the network, each with its own farmer. Where test is the name of the pair. You can create as many pairs as needed, they will all join the same local test network.

# You must first clone or pull down the latest from our fork of 
# substrate:git clone -b w3f-spartan-ms-2 https://github.com/subspace/substrate.git# or if you have already cloned locallycd ../substrate
git pull 
git checkout w3f-spartan-ms-2# Now spin up as many farmers as you like (three in this example)# Terminal 6 (second farmer)
cd substrate/bin/node-template-spartan
./run-node-farmer-pair.sh farmer-1# Terminal 7 (third farmer)
cd substrate/bin/node-template-spartan
./run-node-farmer-pair.sh farmer-2# Terminal 8 (fourth farmer)
cd substrate/bin/node-template-spartan
./run-node-farmer-pair.sh farmer-3

Use Ctrl+C to stop each pair, everything will be stopped and cleaned up automatically.

Next Steps for Spartan

For our next and final milestone, we will extend Spartan to be secure against all known attacks against proof-of-space consensus protocols, such that we can guarantee safety and liveness for any economically rational adversary who controls less than one-half of total network storage resources. These include:

Equivocation, or building on both branches of a fork.
Simulation, better known as the nothing-at-stake attack.
Sybill Farming, or using multiple identities for the same disk plot.
Compression Farming, the worst type of space-time trade-off attack for Spartan.
History Rewriting, also known as the long-range attack.

We are also preparing to launch an initial un-incentivized test network so that we can begin building or community of storage farmers and test the robustness of our protocol in the wild. Later, once we are able to extend Spartan into Subspace, with consensus based on proofs-of-useful-storage, we will transition to an incentivized test network in preparation for our mainnet launch.