a fast, scalable, and easy-to-use p2p overlay network stack (used by NKN)
nnet is a fast, scalable, and easy-to-use network stack for decentralized and distributed systems. This is one of the four core components of NKN, now modularized and released as a separate top level repository. It handles message delivery, topology maintenance, etc in a highly efficient and scalable way, while providing enough control and flexibility through developer-friendly messaging interface and powerful middleware architecture.
The new nnet repository is now open source at:
https://github.com/nknorg/nnet
Please don’t forget to leave your star if you like it. :-)
First of all, nnet features a modular and layered overlay architecture which makes it extremely versatile. By default, nnet uses Chord Distributed Hash Table as an overlay technology, but other topologies like Kademlia can be added by implementing a few interfaces. nnet provides extremely easy to use message sending and handling interface with built-in message routing, featured with a novel and highly efficient message broadcasting algorithm that achieves optimal throughput and near optimal latency. The protocol also offers a flexible transport-aware address scheme which allows each node to choose its own transport protocol to communicate such as TCP and KCP which are offered by default. Finally, nnet offers a powerful and flexible middleware architecture that allows the application to interact with node/network lifecycle and events.
With a powerful and flexible protocol such as nnet, we believe there are many applications that can benefit from this network stack. Any application that needs to connect multiple users across a wide variety of networks could take advantage of this interface. As an example, NKN uses nnet to efficiently handle peer discovery and topology maintenance, user data transmission, transaction and block propagation, and voting in consensus algorithm. Any other large scale peer to peer projects can benefit from nnet by focusing on its own logic without worrying about the network layer.
nnet is truly a fast, scalable, p2p networking stack for the next generation of applications on decentralized infrastructure. It handles message delivery and topology maintenance while providing control and flexibility through a developer-friendly messaging interface and powerful middleware architecture.
Feature highlights
- nnet uses a modular and layered overlay architecture. By default an improved and much more reliable version of Chord DHT protocol is used to maintain a scalable overlay topology, while other topologies like Kademlia can be easily added by implementing a few overlay interfaces.
- Extremely easy to use message sending interface with both async and sync message flow (block until reply). Message reply in sync mode are handled efficiently and automatically, you just need to provide the content.
- Deliver messages to any node in the network (not just the nodes you are directly connected to) reliably and efficiently in at most log_2(N) hops (w.h.p) where N is the total number of nodes in the network.
- Novel and highly efficient message broadcasting algorithm with exactly once (or K-times where K is something adjustable) message sendingthat achieves optimal throughput and near-optimal latency. This is done by sending a message through the spanning tree constructed by utilizing the Chord topology.
- Powerful and flexible middleware architecture that allows you to easily interact with node/network lifecycle and events such as topology change, routing, message sending and delivery. Applying a middleware is as simple as providing a function.
- Flexible transport-aware address scheme. Each node can choose its own transport protocol to listen to, and nodes with different transport protocol can communicate with each other transparently. nnet supports TCP and KCP transport layer by default. Other transport layers can be easily supported by implementing a few interfaces.
- Modular and extensible router architecture. Implementing a new routing algorithm is as simple as adding a router that implements a few router interfaces.
- Only a fixed number of goroutines will be created given network size, and the number can be changed easily by changing the number of concurrent workers.
- NAT traversal (UPnP and NAT-PMP) using middleware.
- Use protocol buffers for message serialization/deserialization to support cross platform and backward/forward compatibility.
- Provide your own logger for logging integration with your application.
Coming soon:
- Latency measurement
- Proximity routing
- Push-pull broadcasting for large messages
- Efficient Pub/Sub
- Test cases
Call for developers
Please check out our GitHub (https://github.com/nknorg/nnet)and download the code today to use in your next great application! Don’t forget to leave your star if you like it.
We welcome other projects to use nnet as their p2p networking layer, as NKN has already successfully done.
We invite all developers to submit a bug, suggestion or feature request, and contribute patches to nnet repository.
Github: https://github.com/nknorg
Discord: https://discord.gg/yVCWmkC
