How to build a custom Polkadot-compatible chain with Gossamer

Published in

ChainSafe

8 min readJun 4, 2021

The following blog post was originally shared as a HackMD document, which was itself shared during Gossamer Tech Lead Elizabeth Binks’s Polkadot Decoded talk on May 20th.

In this post, Elizabeth briefly introduces Gossamer, before walking through how to build a custom chain with Gossamer, as well as how to use Gossamer to run a Polkadot or Kusama syncing node. We invite you to follow along the way with this workshop!

For those who are unfamiliar, Gossamer is an implementation of the Polkadot Host, formerly known as the Polkadot Runtime Environment. To learn more about what exactly that is, watch our introductory talk from DotCon during Berlin Blockchain Week. These blog posts here, here, and here also serve as great supplementary learning material.

0. Introduction

What is Gossamer?

Gossamer is a Go implementation of the Polkadot Host. You can think of it as an equivalent to Substrate.

A Polkadot Host provides two main functions:

Import a runtime and easily launch a chain with custom logic
Create a Polkadot-compatible chain that could potentially become a parachain

Current status of Gossamer:

It is interoperable with Substrate nodes
It is able to act as a Polkadot/Kusama full node
It is compatible with existing tooling such as Polkadot.js

Future plans for gossamer:

Interoperate with Polkadot as an authority node
Currently running internal Gossamer devnet
Afterwards, we plan to run cross-client devnet to ensure interoperability between authority functionality

A Polkadot Host cannot become a parachain on its own; it requires an additional library which connects the chain to the relay chain:

Cumulus: https://github.com/paritytech/cumulus

We have been given a W3F grant to build Go-Cumulus, which will be the library to use in the future:

Go-cumulus! https://medium.com/chainsafe-systems/building-parachains-with-go-cumulus-9998735fa5a4

Agenda

We will run through a few things today in this tutorial:

How to import a runtime into gossamer
How to launch a gossamer development chain with a custom runtime
How to interact with the chain using polkadot.js
How to run a local non-authority node to connect with authority dev node
How to run a Polkadot/Kusama full node

Workshop prerequisities

Go 1.15
Node.js

Building gossamer

git clone https://github.com/ChainSafe/gossamer
cd gossamer
make gossamer

This places the Gossamer binary in ./bin/ .

1. Importing a runtime

Obtaining a runtime

Custom runtimes are built using FRAME, a domain-specific language for creating runtimes that satisfy the runtime API. They are then compiled into wasm for use with a Polkadot Host.

The runtime API looks like this in Go:

Version() (Version, error)
Metadata() ([]byte, error)
BabeConfiguration() (*types.BabeConfiguration, error)
GrandpaAuthorities() ([]*types.Authority, error)
ValidateTransaction(e types.Extrinsic) (*transaction.Validity, error)
InitializeBlock(header *types.Header) error
InherentExtrinsics(data []byte) ([]byte, error)
ApplyExtrinsic(data types.Extrinsic) ([]byte, error)
FinalizeBlock() (*types.Header, error)
ExecuteBlock(block *types.Block) ([]byte, error)

Creating a custom runtime is out of the scope of this workshop; I will be using a slightly modified substrate node runtime: https://github.com/paritytech/substrate/tree/master/bin/node/runtime

Note: gossamer is compatible with the Polkadot v0.8 runtime, so we will need to checkout that branch/tag.

git clone substrate
cd substrate
git checkout sub-v.0.8.24.patch

git checkout 56205634082f7f64ae2eb1f09a57d79d04dc8597

Modifications made to the substrate node runtime:

Changed to impl_name to gossamer-node
Update BabeGenesisConfiguration c value to be (1,1)

The c value is a tuple (a,b) where the probability of a slot being empty is 1-(a/b)
therefore c = (1,1) means the probability of an empty slot is 0, thus a block is produced every slot
This is nice for a development chain!

pub const VERSION: RuntimeVersion = RuntimeVersion {
 spec_name: create_runtime_str!(“node”),
 impl_name: create_runtime_str!(“gossamer-node”),
 authoring_version: 10,
 // Per convention: if the runtime behavior changes, increment spec_version
 // and set impl_version to 0. If only runtime
 // implementation changes and behavior does not, then leave spec_version as
 // is and increment impl_version.
 spec_version: 261,
 impl_version: 0,
 apis: RUNTIME_API_VERSIONS,
 transaction_version: 2,
};fn configuration() -> sp_consensus_babe::BabeGenesisConfiguration {
 // The choice of `c` parameter (where `1 — c` represents the
 // probability of a slot being empty), is done in accordance to the
 // slot duration and expected target block time, for safely
 // resisting network delays of maximum two seconds.
 // <https://research.web3.foundation/en/latest/polkadot/BABE/Babe/#6-practical-results>
 sp_consensus_babe::BabeGenesisConfiguration {
 slot_duration: Babe::slot_duration(),
 epoch_length: EpochDuration::get(),
 c: (1, 1),
 genesis_authorities: Babe::authorities(),
 randomness: Babe::randomness(),
 allowed_slots: sp_consensus_babe::AllowedSlots::PrimaryAndSecondaryPlainSlots,
 }
}

See the modified source code here: https://github.com/noot/substrate/blob/noot/v0.8-dev-runtime/bin/node/runtime/src/lib.rs

To compile the modified runtime, you must have rust installed:

rustup update
rustup default nightly-2020–10–01
rustup target add wasm32-unknown-unknown
cargo build --target wasm32-unknown-unknown --release

This places runtime in the substrate/target directory. Note: if there is an error compiling, do cargo clean

For this workshop, you can download the pre-compiled runtime here: https://github.com/noot/substrate/blob/noot/v0.8-dev-runtime/target/wasm32-unknown-unknown/release/wbuild/node-runtime/node_runtime.compact.wasm

Creating the genesis file

The compiled runtime bytecode of a Polkadot chain is placed into the state trie at the key :code .

Importing a runtime into gossamer is as simple as placing the wasm bytecode into the genesis state at the :code key.

Gossamer has a nice sub-command to do this for you: https://chainsafe.github.io/gossamer/usage/import-runtime/

./bin/gossamer import-runtime <custom-runtime.wasm> > genesis.json

eg.

./bin/gossamer import-runtime ~/substrate/target/debug/wbuild/node-runtime/node_runtime.compact.wasm > genesis.json

Note: this uses the genesis parameters from chain/gssmr/genesis.json

This places the wasm bytecode into the genesis file at :code for you.

If you open up the genesis file, you will also see other information such as the token symbol, authorities, and genesis balances:

json
“properties”: {
 “ss58Format”: 17,
 “tokenDecimals”: 12,
 “tokenSymbol”: “GOSS”
},json
“Babe”: {
 “Authorities”: [
 [
 “5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY”,
 1
 ],
 [
 “5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty”,
 1
 ],
 [
 “5FLSigC9HGRKVhB9FiEo4Y3koPsNmBmLJbpXg2mp1hXcS59Y”,
 1
 ],
 [
 “5DAAnrj7VHTznn2AWBemMuyBwZWs6FNFjdyVXUeYum3PTXFy”,
 1
 ],
 [
 “5HGjWAeFDfFCWPsjFQdVV2Msvz2XtMktvgocEZcCj68kUMaw”,
 1
 ],
 [
 “5CiPPseXPECbkjWCa6MnjNokrgYjMqmKndv2rSnekmSK2DjL”,
 1
 ],
 [
 “5DeWA2u2f8Bb1DXBEohUHWXW3NUzKNg75jYSHaEZWKdE6ZuR”,
 1
 ],
 [
 “5GMykc9TjNsmDftompy44gdWtJHnHxZ5wo7h7mJRKdowjz4z”,
 1
 ],
 [
 “5CULg8vw1SMeCsUHcXR6YZP43eMxjdMLqMriZZe5q9HA3V3c”,
 1
 ]
 ]
},
“grandpa”: {
 “authorities”: [
 [
 “5DFNv4Txc4b88qHqQ6GG4D646QcT4fN3jjS2G3r1PyZkfDut”,
 1
 ]
 ]
},
“palletBalances”: {
 “balances”: [
 [
 “5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY”,
 1152921504606847000
 ],
 [
 “5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty”,
 1152921504606847000
 ],
 [
 “5FLSigC9HGRKVhB9FiEo4Y3koPsNmBmLJbpXg2mp1hXcS59Y”,
 1152921504606847000
 ],
 [
 “5DAAnrj7VHTznn2AWBemMuyBwZWs6FNFjdyVXUeYum3PTXFy”,
 1152921504606847000
 ],
 [
 “5HGjWAeFDfFCWPsjFQdVV2Msvz2XtMktvgocEZcCj68kUMaw”,
 1152921504606847000
 ],
 [
 “5CiPPseXPECbkjWCa6MnjNokrgYjMqmKndv2rSnekmSK2DjL”,
 1152921504606847000
 ],
 [
 “5DeWA2u2f8Bb1DXBEohUHWXW3NUzKNg75jYSHaEZWKdE6ZuR”,
 1152921504606847000
 ],
 [
 “5GMykc9TjNsmDftompy44gdWtJHnHxZ5wo7h7mJRKdowjz4z”,
 1152921504606847000
 ],
 [
 “5CULg8vw1SMeCsUHcXR6YZP43eMxjdMLqMriZZe5q9HA3V3c”,
 1152921504606847000
 ]
 ]
},

These accounts correspond to the built-in keys (which are the same as substrate and subkey). The built in keys are named alice, bob, charlie, dave, eve, ferdie, george, and ian.

Tip: if you have subkey installed, you can show the keys using subkey inspect //Alice:

$ subkey inspect //Alice
Secret Key URI `//Alice` is account:
 Secret seed: 0xe5be9a5092b81bca64be81d212e7f2f9eba183bb7a90954f7b76361f6edb5c0a
 Public key (hex): 0xd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d
 Account ID: 0xd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d
 SS58 Address: 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY

2. Initializing and starting the node

Let’s modify the genesis file to have 1 grandpa authority so that blocks get finalized each round. We can also remove the other BABE authorities, but if we are just running one dev node, it doesn’t make a difference.

Now that we have the genesis file, we can initialize and start the node as an authority:

./bin/gossamer init --genesis genesis.json
./bin/gossamer --key alice

The node is now running and should be producing a block each slot!

using --chain dev

The previous steps have been condensed for you into a--chain dev option which uses the genesis previously generated:

./bin/gossamer init --chain dev --force
./bin/gossamer --chain dev

The--chain option uses the genesis and config files found in the chain/ directory for the specified chain.

Tip: if you wish you use your own config file, you can use--config path-to-config.toml

3. Interacting with the node

Using polkadot.js in node.js

Tested using node 14.16.1.

Also see documentation here: https://polkadot.js.org/docs/api/examples/promise

mkdir gossamer-polkadotjs
cd gossamer-polkadotjs
npm init -y
npm i @polkadot/api@4.6.1

create a file transfer.js:


const { ApiPromise, WsProvider, Keyring } = require(‘@polkadot/api’);async function main() {
 const wsProvider = new WsProvider(‘ws://127.0.0.1:8546’);
 const api = await ApiPromise.create({ provider: wsProvider });const keyring = new Keyring({ type: ‘sr25519’ });
 const aliceKey = keyring.addFromUri(‘//Alice’);
 const addrAlice = aliceKey.publicKey 
 const addrBob = keyring.addFromUri(‘//Bob’).publicKey;let account = await api.query.system.account(addrAlice);
 console.log(`alice balance before: ${account.data.free}`)
 account = await api.query.system.account(addrBob);
 console.log(`bob balance before: ${account.data.free}`)const txHash = await api.tx.balances.transfer(addrBob, 12345)
 .signAndSend(aliceKey);console.log(`submitted with hash ${txHash}`);const unsub = await api.rpc.chain.subscribeNewHeads(async (lastHeader) => {
 console.log(`latest block: #${lastHeader.number} `);
 let {block} = await api.rpc.chain.getBlock();
 if (block.extrinsics.length > 1) {
 console.log(`found extrinsic in block ${lastHeader.number}`)
 unsub()
 }account = await api.query.system.account(addrAlice);
 console.log(`alice balance after: ${account.data.free}`)
 account = await api.query.system.account(addrBob);
 console.log(`bob balance after: ${account.data.free}`)
 });
}main();

Then, do
node transfer.js

You should see the transfer get submitted, get the tx hash, see what block it was included in, and the balance update:

$ node transfer.js 
balance: 19342813113832646479343115
submitted with hash 0xf75b1aa7417e08f7b1339664030ab688f90bcb0fd33abc69741323566eb527b3
all done
latest block: #41 
balance: 19342813113832646479343115
latest block: #42 
found extrinsic in block 42
balance: 19342813113831226163401053

Using polkadot.js apps UI

See documentation here: https://chainsafe.github.io/gossamer/integrate/connect-to-polkadot-js/

Note: currently the UI works fully with api v4.6.x; we are hosting our own UI here: http://gossamer-dev.chainsafe.io:3000/#/explorer or http://gossamer-dev1.chainsafe.io:3000/#/explorer

4. Running a non-authority node

The node we previously started is an authority, ie. it authors blocks and participates in finality voting.

Let’s now see how to run a non-authority node that will sync with our existing node.

We need to firstly specify the node is not an authority with —- roles 1:

roles:
0: no network
1: non-authority (full node)
4: authority node

./bin/gossamer --chain dev --roles 1 --port 7002 --basepath /tmp/gossamer_node --rpc=false --ws=false

Can also add --bootnodes <multiaddr-of-first-node> for the nodes to directly connect as opposed to discovery over mDNS.

The newly launched node should connect to the first node and begin to sync!

5. Running a Polkadot/Kusama full node

Running a Polkadot/Kusama node is as simple as specifying the chain name with --chain:

./bin/gossamer --chain polkadot./bin/gossamer --chain kusama

Once the node is running, you can see it on Telemetry: https://telemetry.polkadot.io/#/Polkadot

Get Involved

Want to work with ChainSafe? COME JOIN US!!! Check out the new Careers section of our website and our open positions, and get in touch with us at careers@chainsafe.io if you are interested!

If you are interested in getting involved and contributing to the project, check out our Github. If you would like to get in contact with one of the Gossamer team members, feel free to drop by on Chainsafe’s Discord, or email info@chainsafe.io. We would love to know more about you, your team and your project!

For more details on Gossamer, please head to our *new* documentation site.

Learn more about ChainSafe by visiting our website, through our Medium, via Twitter, or by visiting the ChainSafe GitHub.