Playing with decentralized p2p network & Rust Libp2p Stacks

Source: https://blog.libp2p.io/2023-01-12-rust-libp2p-in-2022/

Overview

I have a personal project that has a use case that relates to decentralized networks, specifically on P2P networks. I’ve been exploring, researching, and learning about P2P networks for a long time, including following all interesting projects that already implement this concept, such as:

• Any cryptocurrency networks (L1/L2), such as Ethereum, Polkadot, Solana, NEAR, etc
• IPFS
• BitTorrent

I’m just curious about how to build and manage decentralized P2P networks.

The Journey

About libp2p

I need two requirements for my personal project

• It should be a decentralized P2P networks
• It should provide a secure channel of communication

In the beginning, I just thought that I needed to create the networks and the communication channel in two separate things, and for the networks, I just thought that I needed to develop from scratch.

Until I found this networking stack, called libp2p. This project, as far as I know, is a part of IPFS, and fortunately, this library also supports the Noice Protocol, so I’ve got an advantage here, from a single library, I’m able to fulfill my two requirements above.

The libp2p itself actually is a standard of specification to build modular networking stacks.

The main goal of this repository is to provide accurate reference documentation for the aspects of libp2p that are independent of language or implementation. This includes wire protocols, addressing conventions, and other “network level” concerns.

Source: https://github.com/libp2p/specs

Rust Libp2p

As I’ve said before this project ( libp2p ) is a standard specification, and it means, there is an implementer of this spec. You can find the implementations from multiple languages here: https://libp2p.io/implementations/

And fortunately, there is an implementation using Rust, called rust-libp2p .

GitHub - libp2p/rust-libp2p: The Rust Implementation of the libp2p networking stack.

crates.io: Rust Package Registry

libp2p - Rust

Since all of my needs are possible through this library, I decided will integrate this library into my personal project. Short story, I need at least two weeks to learn the libp2p basic concepts first and the rust-libp2p implementations, including for its APIs.

My journey begins with, its tutorials, which you can find here:

libp2p::tutorials - Rust

There are two basic concepts of rust-libp2p that we need to know

• Transport . It’s an abstraction that provides a set of abstract methods (interface methods), that take responsibility for how to send bytes.
• Network Behavior . It’s an abstraction that provides a set of abstract methods (interface methods), that take responsibility for what bytes to send, and to whom.

An interesting part, this library also implements an event-driven mechanism, which I think is a good approach too, because it will be natural with networking behaviors too. You’ll find a lot of event message exchanges when using this library, for example:

The best part is, there are modules that already implement Transport and Network Behavior too. For the Transport , it already has a Tcp , Websocket even for the Memory too, although I think that the MemoryTransport will be used for testing purposes only.

For the Network Behavior , it already has

• floodsub
• gossipsub
• identify
• ping

The most interesting part is, that we are able to create wrapper behaviors, that use more than one of the available behaviors, for example:

My suggestion is before you decide to create your own behavior from scratch, think twice, that maybe we can use currently available implementations from this library.

For the peering discovery and routing, there are ready-to-use modules:

• identify
• kademlia (or kad )
• mdns

On my project, I’m using identify and kademlia for peering discovery purposes, like the screenshot above. And for the transporters, I’m using tcp::Transport and also using noice to support Noice Protocol.

Using this setup, I was able to communicate between 4 nodes (in my local environment/laptop). The network flows itself will be like this:

The Node 1, will act as a bootstrap node, it’s the first node that needs to be live in the network. The other nodes, are able to dial to other nodes through the address in the format of MultiAddr , something like this:

/ip4/127.0.0.1/tcp/8000

All of these nodes will keep updated through the identify module that will always exchange the node’s information with each other.

When the node runs the first time, if we do not specify the bootstrap-node, it means the node itself will act as a bootstrap-node. If we give some address, the node will dialling to that node.

Each time a node is connected to another node, these two nodes will keep synchronizing their peer information and updating the routing table (DHT). Example:

For now, I’m not adding the ability to explore the node through the bootstrap node, because to do that, I need to implement the message exchange through the floodsub or gossipsub .

What makes me happy is, that all of these node’s communications, automagically already run through the secure channel communications that already implement the Noice Protocol with zero efforts, although I’m still using the default implementation of it (in Noice, there are multiple patterns too, like XX, NN, etc). Example:

The Project

I’ve already stored my codes here:

GitHub - hiraqdev/poc-rust-libp2p-kademlia: poc rust-libp2p-kademlia-dht

Please remember, that this repo is just for my PoC (Prove Of Concept), to learn how to assemble the rust-libp2p modules.

And I hope this little project can help others to learn the libp2p through the rust-libp2p library implementation.