Introduction to libp2p

A tool in our decentralized toolbox

Photo by Fleur Treurniet on Unsplash

This is the third article in a series exploring the concepts and tools behind the Keep network and other decentralized systems.

Software engineering is a mix of craft and science. In contrast to a traditional engineering discipline, like civil engineering, the problems in software engineering are poorly defined. Often, they are a mix of technology (“How does it work?”) and people (“How should it work? How can we make it work?”).

Compared to building a bridge, building a software system is difficult to estimate, plan, and execute reliably. The construction of decentralized software systems is an even less mature study. In the “decentralized” way of thinking we need to get out of the client/server model used often in centralized services and shift to libp2p’s dialer-listener interaction. Libp2p abstracts the mess of protocols we need to deal with in a decentralized context into a single protocol and handles the punching out of unfriendly networks, negotiating with a variety of gateways, and then effectively connecting with many, many peers.

That history makes libp2p noteworthy. In a space without well understood rules and best practices, Protocol Labs has found a way to build a powerful tool for the next generation of decentralized systems.

We’re using libp2p in the development of the Keep client. It’s pretty neat, and we wanted to share a bit of our experience for other developers.

Our test app

Discovery is the first step in many peer-to-peer networks — participants need to know how to connect to each other before they can solve a problem together. In this tutorial we’ll build a simple example of peer-to-peer discovery.

We’ll use a few pieces of the libp2p JS library to create nodes that discover and communicate with each other. This is a simplified setup suitable for demo purposes. In practice, you’ll need additional functionality, such as encryption and multiplexing to create secure connections and avoid handshake overhead.

Prerequisite

Install the essential libp2p packages

npm install libp2p peer-info

Picking libp2p modules

libp2p offers a number of modules to support different network architectures.

This app requires the Transport interface, Discovery and the Distributed Hash Table (DHT) lookup service. Let’s look at them in more detail.

Transport

libp2p is transport agnostic. Many popular protocols are available, including TCP, WebRTC, Web Sockets, and UDP. For the sake of simplicity in our example we’ll use WebRTC and the signalling-star discovery mechanism.

Install the WebRTC transport with a signalling server.

npm install libp2p-webrtc-star

Discovery

Nodes must have a way to find each other. libp2poffers a few methods to achieve that. A common discovery mechanism for employed by decentralized applications is too hard code each regular node with a list of bootstrap nodes. Bootstrap nodes are the initial place that the new node goes to find peers. The list of peers is updated as new nodes are added. For local network usage, there are more efficient discovery methods, including the MulticastDNS module. libp2penables many other discovery mechanisms — you can even write and plug in your own!

The WebRTC transport includes a built-in discovery mechanism called a rendezvous or signalling server, a known point in the network where nodes can handshake their signalling data and establish a connection.

The signalling server must be up and running when a node node boots. For demo purposes, we can use a hosted server at star-signal.cloud.ipfs.team provided by IPFS team. The full address for a peer should look like /dns4/star-signal.cloud.ipfs.team/wss/p2p-webrtc-star/ipfs/<your-peer-id>. We’ll come back to this at the node configuration step.

Distributed Hash Tables (DHTs)

One of the first questions you might have is how do we locate data offered by the peers? Historically there have been various implementations for this task by p2p projects including Napster, Freenet, CoralCDN to name a few. Those approaches eventually evolved into what is now called a distributed hash table.

It is an efficient, highly scalable protocol that provides decentralized lookup services, which we recently wrote about in detail. In libp2p, this concept is called content routing. For our example we’ll use the libp2p-kad-dht module, which is an implementation of the Kademlia DHT. We’ll also need the content identifier (CID) service to help us reference content in our p2p network.

Install the the following

npm install cids
npm install libp2p-kad-dht

A quick note on content routing. In order to find providers of specific content we first need to inform the network that the content is available, this can be simply done as node1.contentRouting.provider(cid) where cidis a unique content identifier. Now the content can be easily resolved by other peers of the network by calling the findProviders method, i.e. node2.findProviders(cid).

Our first bundle

Let’s define a bundle to enable libp2p to find peers to connect to:

Configuring your node

As mentioned in the discovery section, we’ll use the signalling server hosted by IPFS.

Running the nodes

Here’s a basic example how to run a lib2p2 node by calling createNode() function described above:

To see all the code written above in action run the following commands in your terminal:

git clone https://github.com/ngrinkevich/libp2p_example.git && cd libp2p_example && npm install && npm start

When we see the following message - serving “dist” at http://127.0.0.1:9090 we can open our web browser to http://127.0.0.1:9090

Next, if we open our web browser’s console we should be able to see log messages like this:

Node Qmb7BgjCc2k2s5UqN4Dxy54npkLL6toyBTE has just started
Discovered a peer: QmbLdCE7RFGwL7NcVMHyjCZ4JQhcB5Vj25mWv
Discovered a peer: QmXo71NbeSFN9g43RWtrUcENp7pwgRBoboTsY
...

By the way, you can run multiple nodes by simply opening new browser windows at http://127.0.0.1:9090.

That’s it! The rest of the example code can be found on Github.

While this is a simple use of libp2p, it should give you an idea of the power of easy node discovery and content routing. If you’re interested in learning more about building decentralized systems, follow us on Twitter, join the discussion on Slack, or read up on the Keep network.

Special thanks to Lex Sheehan, Raghav Gulati, Antonio Salazar Cardozo, and Matt Luongo for collaborating and reviewing.

Learn More

For more information about the Keep Network:

• Join us on Reddit.
• Check out our whitepaper.
• Read our business primer.
• Subscribe for email updates.
• Follow us on Twitter.
• Join our Slack.
• Join our Telegram.