Juno’s experimental peer-to-peer launch

TL;DR:

Juno’s experimental peer-to-peer (P2P) implementation is available on the Sepolia testnet!

Table of Contents
· Introduction
· How Does a Peer-to-Peer Network Work?
· Juno P2P Sync Process
· How Do Nodes Trust Each Other?
· Running a Juno P2P Node: A Step-by-Step Guide
· Running Your Own Juno P2P network

Introduction

As Starknet transitions from relying on the feeder gateway to access the blockchain state to a decentralized network structure, anyone running a node on Starknet will soon become a crucial part of its network!

This shift demands a detailed P2P (peer-to-peer) specification, a feature the Juno team is diving into alongside the teams behind the Pathfinder and Papyrus full nodes. In this blog post, we’ll discuss why a peer-to-peer network is important, and provide a step-by-step guide for setting up your own Juno node to join the P2P network.

💡 The P2P feature is currently under active development and being tested on smaller Juno networks, so syncing with non-Juno nodes may be unstable!

How Does a Peer-to-Peer Network Work?

When a new Juno node joins the network, it begins with no blocks, effectively starting at a block height of zero. To update itself, the node first connects with another node in the network, chosen at random, to find out the current highest block number, which could be, for example, 3312. With this information, the new node embarks on syncing blocks from its peers, doing so in chunks of 100 blocks at a time.

This process continues in 100-block increments until the node catches up to the most recent block. For instance, if the node has synced up to block 3300 and the network’s current block height is 3345, the node will then sync the last 45 blocks to be fully up-to-date. This method ensures the new node gradually and efficiently synchronizes with the rest of the network, block by block until it reaches the current block height.

Every time the node begins syncing a new set of blocks, it triggers a process that unfolds across five distinct stages. These stages are designed to systematically integrate the blocks into the node’s blockchain, ensuring that each block is properly verified and that the node’s ledger remains accurate and in sync with the network.

An overview of these five stages is given below.

Juno P2P Sync Process

Juno p2p sync process (right to left)

Stage 1 — Establishing Peer Connections and Requesting Block Components

Stage 1 is where the peer connections are established, and the requests for block information are made.

To aid synchronization, the P2P spec requires the block to be split into five different components so that each piece can be requested from a different peer. The five different block components are block bodies, block headers, transactions, receipts, and events. For each block component, Juno will open a connection stream with a peer, and request that specific block part for the entire range of blocks being queried (e.g., up to 100). For example, the Juno node will ask peer “A” for the events data for the first 100 blocks, and at the same time, ask peer “B” for the block header data for those first 100 blocks, etc.

Each process of requesting a specific block part is shown as a distinct process in Stage 1 of the diagram. These requests are made concurrently, and a given process like the query “Spec Events” can move onto stage 2 before the other block parts have been received from the peers.

Stage 2 — Populating the Intermediate Block Type with Available Block Components

When the Juno node receives the specific block part from a peer, it moves on to Stage 2, which involves populating an intermediate (Spec) Block type with whatever information is currently available. For example, if the event information has been fully received from a peer, then this event data will be used to partially populate the Block type. All other fields like block headers will remain empty at this stage, which is why a separate stage is needed to tie all the block parts together.

Stage 3 — Assembling Block Components

When all the different block parts have been received from peers, they will be pieced together at stage 3. The Juno node will then be ready to convert the entire (Spec) Block into a Juno Block type and perform some sanity checks, which is what stage 4 is responsible for.

Stage 4 — Creating the Juno Block

At stage 4, the Juno node will have all the information required to create the entire Juno block. The block information provided from the process at stage 3 is used to create a Juno block. The Juno database expects the block to be in the correct format to commit it to the database, which is what this process is responsible for. This stage also performs a set of sanity checks on the block, such as verifying the block and transaction hashes.

Note that at this stage in the process, Juno may only have a subset of the 100 blocks it queried, and importantly, it may not have the blocks in the correct order. For example, imagine the node has block 42, but no other block. In this scenario, Juno will be unable to store block 42 because the previous blocks don’t exist in its blockchain. Since the blocks need to be stored in order, Juno requires a process to feed blocks into its database in the correct order, which is the responsibility of stage 5.

Stage 5 — Ordering and Committing Blocks to the Blockchain

When the next block is ready to be stored in the blockchain (e.g., we have stored blocks 0–41, and now want to store block 42), this process will take the next block when it’s available (e.g., block 42), and perform sanity checks at the blockchain level. For example, block hashes need to be correctly linked together. If these sanity checks are passed, then the stage 5 process passes the latest block onto another process to commit the block to the blockchain. Following this, stage 5 will wait for the next block (e.g., block 43), and repeat the same process.

This process will continue until the Juno node has fully caught up to the head of the blockchain. At any point in this process, the node can serve whatever data it has to its peers.

How Do Nodes Trust Each Other? 🤝

For those closely following, a valid question arises: How can new nodes trust the blocks received from their peers? The integrity of the blocks is ensured through a cryptographic verification process. The block data is used to generate a blockhash, and similarly, a hash is also produced from the state-diff. These two commitments are then used to generate a signature for the block.

To validate this signature — and by extension, the block data and state-diff — a node compares it against a public key it receives from the feeder-gateway (a query is made to the feeder-gateway when the node starts). This mechanism ensures that if any block data or state-diff were altered by a malicious node, the resulting commitments and therefore the signatures would differ from the expected values. This cryptographic process effectively safeguards against tampering, ensuring that nodes can confidently trust the blocks they receive and maintain the network’s integrity.

Running a Juno P2P Node: A Step-by-Step Guide

Prerequisites and Requirements

We highly recommend using Docker to run a Juno node, which is our only software requirement.

Regarding hardware, the minimum requirements are:

• 2+ CPU Cores
• 4Gb of RAM
• 50Gb+ SSD
• 10 Mbps/sec

Installing Docker

Go to the Docker webpage and select the relevant download option for your system. For example, selecting “Download for Windows” will download an executable file to install Docker on your system, and the same applies for Mac. For Linux-based operating systems, you will also need to select the relevant platform (e.g., Ubuntu). Then, simply open the relevant executable and follow the install instructions.

To verify that Docker has been downloaded, open the Command Prompt / Terminal and run:

docker --version

You should see the version of Docker installed on your system. Note: you may need to open up Docker Desktop on Windows before running this command.

Downloading Juno

With all prerequisites in place, you’re now ready to download, install, and run a Juno node. To do this we will first need to download one of Juno’s images from Docker hub using the docker pull command. For this tutorial, we will pull the image with Tag 0.10.0

1. Open the command prompt / terminal.
2. Pull the Docker image by executing:
   docker pull nethermind/juno:v0.10.0
3. Verify the image has been pulled by executing:
   docker images
You should see something like this:
   REPOSITORY TAG IMAGE ID CREATED SIZE
   nethermind/juno v0.10.0 98bc124d8dbc 13 hours ago 215MB

Running Juno

Now that we have the Docker image downloaded, all we need to do to run a Juno node in p2p mode is execute the following command if using Windows:

docker run -d --name juno_p2p ^
-p 7777:7777 ^
-p 6060:6060 ^
-v %USERPROFILE%/juno_sepolia_p2p:/var/lib/juno ^
nethermind/juno:v0.10.0 ^
--db-path "/var/lib/juno" ^
--network "sepolia" ^
--log-level "debug" ^
--http ^
--http-host "0.0.0.0" ^
--http-port "6060" ^
--p2p ^
--p2p-addr /ip4/0.0.0.0/tcp/7777 ^
--p2p-peers=/ip4/35.237.92.52/tcp/7777/p2p/12D3KooWLdURCjbp1D7hkXWk6ZVfcMDPtsNnPHuxoTcWXFtvrxGG

or the following if using Linux:

docker run -d --name juno_p2p \
-p 7777:7777 \
-p 6060:6060 \
-v $HOME/juno_sepolia_p2p:/var/lib/juno \
nethermind/juno:v0.10.0 \
--db-path "/var/lib/juno" \
--network "sepolia" \
--log-level "debug" \
--http \
--http-host "0.0.0.0" \
--http-port "6060" \
--p2p \
--p2p-addr /ip4/0.0.0.0/tcp/7777 \
--p2p-peers=/ip4/35.237.92.52/tcp/7777/p2p/12D3KooWLdURCjbp1D7hkXWk6ZVfcMDPtsNnPHuxoTcWXFtvrxGG

And voilà! You are now running a Juno node over the P2P network!

Executing the above command sets your Juno node into P2P mode, linking it to the Juno node at the address provided in the --p2p-peers flag. This particular node is currently managed by the Nethermind Juno team, but you have the flexibility to connect your node to any other Juno node in the network. If you're interested in hosting a more extensive network setup on your own, refer to the section below about setting up your own Juno P2P network.

Monitoring your Juno node in Docker

To view the logs of your P2P Juno node running inside a Docker container, you indeed have two main options: using docker logs for a snapshot of the container's log output, and using docker attach for a live feed of the container's standard output. Here's a bit more detail on each method:

Viewing the logs

To see the logs output by your Juno node container since it started, you can use the docker logs command followed by the name or ID of your container. This is useful for getting a historical view of what has happened with your node up to the current moment. For example, if your container is named juno_p2p, you would use: docker logs juno_p2p. You should see something like this:

       _                    
      | |                   
      | |_   _ _ __   ___   
  _   | | | | |  _ \ / _ \  
 | |__| | |_| | | | | (_) |  
  \____/ \__,_|_| |_|\___/ v0.10.0

Juno is a Go implementation of a Starknet full-node client created by Nethermind.

20:09:26.946 22/02/2024 +00:00 WARN node/node.go:154 P2P features enabled. Please note P2P is in experimental stage
...
...
20:09:26.954 22/02/2024 +00:00 INFO p2p/p2p.go:206 Listening on {"addr": "/ip4/127.0.0.1/tcp/7777/p2p/12D3KooWQyCMEhpUFahNvfqBYqaUbcMM62hHArBvyyBfU1jKovrN"}
20:09:26.954 22/02/2024 +00:00 INFO p2p/p2p.go:206 Listening on {"addr": "/ip4/172.17.0.2/tcp/7777/p2p/12D3KooWQyCMEhpUFahNvfqBYqaUbcMM62hHArBvyyBfU1jKovrN"}
20:09:26.954 22/02/2024 +00:00 DEBUG p2p/sync.go:86 Continuous iteration {"i": 0}
20:09:26.954 22/02/2024 +00:00 DEBUG p2p/sync.go:630 Number of peers {"len": 1}
20:09:26.954 22/02/2024 +00:00 DEBUG p2p/sync.go:631 Random chosen peers info {"peerInfo": "{12D3KooWLdURCjbp1D7hkXWk6ZVfcMDPtsNnPHuxoTcWXFtvrxGG: [/ip4/35.237.92.52/tcp/7777]}"}
20:09:27.308 22/02/2024 +00:00 DEBUG upgrader/upgrader.go:81 Application is up-to-date.
...

Attaching to the container

If you want to view the logs in real-time, you can attach your terminal’s standard input, output, and error streams to the container using docker attach : docker attach juno_p2p.

When you’re attached to a container using docker attach, you can detach without stopping the container by typing the escape sequence CTRL-p CTRL-q.

Some helpful Docker commands

This section outlines some useful commands for working with Docker. If you are already familiar with Docker feel free to skip it.

To see all available Docker images on your system, you can use the following command: docker images

If you need to remove a specific image, you can do so using: docker rmi INSERT_IMAGE_ID

To see all Docker containers, including those that are currently stopped, the following command comes in handy:docker ps -a

To remove a specific container use:docker rm INSERT_CONTAINER_NAME

To stop a docker container: docker stop INSERT_CONTAINER_NAME

To start a docker container: docker start INSERT_CONTAINER_NAME

Shutting Down Your Juno Node and Removing Saved Data

If you want to completely remove the Juno node, and any of the blockchain data it saved from your system, then follow these steps. Note however, that deleting the saved blockchain data is irreversible, so you’ll have to sync from scratch if you decide to run the P2P node again.

To shut down your Juno node run: docker stop juno_p2p

After stopping the container, you might want to remove it to clean up your Docker environment: docker rm juno_p2p

To remove the image: docker rmi 98bc124d8dbc

Note that if the docker rmi command above doesn’t remove the image, then simply replace the image id (98bc124d8dbc)above with the corresponding one on your machine. You can find it by running docker images.

You may also want to delete the blockchain data that the Juno node downloaded (assuming you used persistent storage). To do this in Windows open the Command Prompt and execute:

rmdir /s /q "%USERPROFILE%\juno_sepolia_p2p”

or if you’re using Linux, then execute:

rm -fr $HOME/juno_sepolia_p2p

These steps ensure your Juno node is properly shut down, and all associated data is removed, freeing up valuable space on your system.

Making Sense of the P2P Flags

For more information on how addresses are defined in a P2P setting, check out the “addressing” section in the libp2p docs.

• --p2p: this flag is used to enable the p2p features of the Juno node. Example usage: --p2p
• --p2p-addr : this flag should reflect the actual IP address or hostname of the machine where your node is running, along with the port designated for P2P communications. For a node on a machine with an IP of 192.168.1.100 listening on port 7777, you'd set:
  --p2p-addr=/ip4/192.168.1.100/tcp/7777
• --p2p-peers: this flag is used to connect your node to peers. To do this use their external IP addresses or hostnames, the ports they listen on, and their PeerIDs. For example, to connect to a peer at 192.168.1.123 on port 7778, with a PEERS_PEER_ID PeerID, the flag would look like:
  --p2p-peers=/ip4/192.168.1.123/tcp/7778/p2p/PEERS_PEER_ID
• --p2p-feeder-node: this flag designates your Juno node as a feeder-node, meaning it syncs blocks directly from the Starknet feeder-gateway and supplies them to other nodes in the P2P network. This flag is crucial for creating a feeder-node, a necessary component of any Juno P2P network to ensure all nodes receive the latest blocks. Avoid using this flag if you do not wish to operate your node as a feeder-node.
  Example usage: --p2p-feeder-node
• --p2p-private-key: this sets the private key of the node. If no key is provided, one is generated at random at boot. It should be a hexadecimal representation of a private key on the Ed25519 elliptic curve. u
  Example usage: --p2p-private-key "INSERT_PRIVATE_KEY"

Juno offers a simple sub-command to generate a new set of P2P keys and a PeerID. To generate these, simply run:

docker run --rm -it nethermind/juno:v0.10.0 genp2pkeypair

which will output something like this:

P2P Private Key: 57b17e8de8……e1c6ef0585d5c84
P2P Public Key: e79d6b5a4……1c6ef0585d5c84
P2P PeerID: 12D3KooWRQVZyvS11F2CrMP32RnVYW9fvvXTDVeF3fkcYqpLgsL7

The private key should be supplied to the node via the --p2p-private-key flag. You can then share your PeerID with other network participants. They will use it in conjunction with your node’s IP address and port, forming a complete address that’s added to their node’s --p2p-peers flag. This enables direct P2P connections between your node and others in the network.

Running Your Own Juno P2P network

To create your own peer-to-peer (P2P) network with Juno, begin by configuring a feeder node to sync with the feeder gateway. After this initial step, you can launch more nodes that are not feeder nodes. These will receive block data from your feeder node, either directly or via other nodes in the network. Below is an easy-to-follow guide to initiate this process:

1. Establishing a Docker Network

Simply run the following command:

docker network create juno_network

2. Generating a Private-Key and Peer-ID for a Node

Before setting up your feeder-node within the Juno P2P network, it’s advisable to generate a stable P2P peer address. This ensures your feeder-node retains a consistent address, facilitating reliable connections within the network, as opposed to relying on a randomly generated address each time the node starts. The P2P address is a key element for networking, as it is used by other nodes to establish connections.

docker run --rm -it nethermind/juno:v0.10.0 genp2pkeypair

After executing this command, you’ll receive a P2P Peer ID and P2P Private Key as mentioned above. These values are essential for configuring your feeder-node with a fixed P2P address. The P2P Private Key will be passed into the feeder nodes p2p-private-key flag, whereas the P2P Peer ID should be given to any node that needs to connect to your feeder node, typically as part of a full address in their p2p-peers flag.

3. Setting Up the Feeder-Node

To start your feeder-node on Windows, use the following placeholder command, where you will need to replace FEEDER_NODE_PRIVATE_KEY with the actual P2P Private Key you generated above:

docker run -d --name juno_feeder_node ^
--network juno_network ^
-p 6060:6060 ^
-p 7777:7777 ^
-v %USERPROFILE%/juno_sepolia_feeder:/var/lib/junoFeeder ^
nethermind/juno:v0.10.0 ^
--db-path "/var/lib/junoFeeder" ^
--network "sepolia" ^
--log-level "debug" ^
--http ^
--http-host "0.0.0.0" ^
--http-port "6060" ^
--p2p ^
--p2p-feeder-node ^
--p2p-private-key=FEEDER_NODE_PRIVATE_KEY ^
--p2p-addr="/ip4/0.0.0.0/tcp/7777"

For Linux users, the command is similar, again replacing FEEDER_NODE_PRIVATE_KEY with the actual P2P Private Key you generated above:

docker run -d --name juno_feeder_node \
--network juno_network \
-p 6060:6060 \
-p 7777:7777 \
-v $HOME/juno_sepolia_feeder:/var/lib/junoFeeder \
nethermind/juno:v0.10.0 \
--db-path "/var/lib/junoFeeder" \
--network "sepolia" \
--log-level "debug" \
--http \
--http-host "0.0.0.0" \
--http-port "6060" \
--p2p \
--p2p-feeder-node \
--p2p-private-key=FEEDER_NODE_PRIVATE_KEY \
--p2p-addr="/ip4/0.0.0.0/tcp/7777"

If you don’t specify the --p2p-addr flag when starting your feeder-node, Juno will automatically generate a random P2P address each time the node is launched. This means that the feeder-node will have a new address (specifically PeerID) on every startup, which could complicate connections within the P2P network.

To locate the dynamically generated P2P address of your node, you should examine the startup logs of the node by running docker logs juno_feeder_node. Search for a log entry near the top that looks like this:

"addr": "/ip4/127.0.0.1/tcp/39421 /p2p/12D3KooWEa......qh26GZbddFvNzJNKzKG”

This string represents the feeder nodes address for the current session, which you can provide to other nodes that need to connect to your feeder-node. Nonetheless, for a more stable network setup and simpler management, it’s advisable to generate a fixed P2P peer address as outlined earlier.

4. Launching Additional Juno Nodes

To expand your Juno network by adding more nodes, you’ll first need to generate a unique private key and peer ID for each new node. Let’s refer to these as NODE1_P2P_PEER_ID and NODE1_P2P_PRIVATE_KEY for the first additional node.

After generating these values, you can launch an additional Juno node by substituting the actual values for FEEDER_NODE_P2P_ID (the P2P Peer ID of your feeder node) and NODE1_P2P_PRIVATE_KEY (the private key you just generated for the new node).

For Windows, use the following placeholder command, making sure to replace the placeholders with the actual values:

docker run -d --name juno_node1 ^
--network juno_network ^
-p 6061:6061 ^
-p 7778:7778 ^
-v %USERPROFILE%/juno_sepolia_node1:/var/lib/junoNode1 ^
nethermind/juno:v0.10.0 ^
--db-path "/var/lib/junoNode1" ^
--network "sepolia" ^
--log-level "debug" ^
--http ^
--http-host "0.0.0.0" ^
--http-port "6061" ^
--p2p ^
--p2p-addr=/ip4/0.0.0.0/tcp/7778 ^
--p2p-peers=/dns4/juno_feeder_node/tcp/7777/p2p/FEEDER_NODE_P2P_PEER_ID ^
--p2p-private-key=NODE1_P2P_PRIVATE_KEY

And for Linux, the command is structured similarly:

docker run -d --name juno_node1 \
--network juno_network \
-p 6061:6061 \
-p 7778:7778 \
-v $HOME/juno_sepolia_node1:/var/lib/junoNode1 \
nethermind/juno:v0.10.0 \
--db-path "/var/lib/junoNode1" \
--network "sepolia" \
--log-level "debug" \
--http \
--http-host "0.0.0.0" \
--http-port "6061" \
--p2p \
--p2p-addr=/ip4/0.0.0.0/tcp/7778 \
--p2p-peers=/dns4/juno_feeder_node/tcp/7777/p2p/FEEDER_NODE_P2P_ID \
--p2p-private-key=NODE1_P2P_PRIVATE_KEY

By following these steps, you’ll have initiated a Juno P2P network that consists of a feeder-node and at least one non-feeder node, with the option to expand by adding more nodes into the network as needed. This setup allows for a more distributed network architecture, improving both the scalability and resilience of your blockchain operations.

This scalable approach ensures that as your network grows, it remains robust and capable of handling increased load, making it a powerful solution for developing decentralized applications and services on the Starknet ecosystem.

Share Your Feedback

As this is an experimental phase, your feedback and questions are very welcome! Chat with Juno devs in:

Telegram: Juno — Starknet Full Node

#Juno channel on Discord: Discord — A New Way to Chat with Friends & Communities

Nethermind Starknet’s X (Twitter): https://twitter.com/NethermindStark

About Nethermind

Nethermind is a team of world-class builders and researchers. We empower enterprises and developers worldwide to access and build upon the decentralized web. Our work touches every part of the web3 ecosystem, from our Nethermind node to fundamental cryptography research and infrastructure for the Starknet ecosystem.

If you’re interested in solving some of blockchain’s most difficult problems, visit our job board!