Noise v1.1.2: Fearless, decentralized p2p networking in Go.
https://github.com/perlin-network/noise/releases/tag/v1.1.2
The latest release, Noise v1.1.2, finally arrives 9 months after the initial release of Noise.
The release involves a complete refactor that has allowed for Noise to adopt significant performance, security, privacy, and developer ergonomic improvements.
Click here to access Noise’s Github, or here to access Noise’s godocs for example code and documentation.
What is Noise?
noise is an opinionated, easy-to-use P2P network stack for decentralized applications, and cryptographic protocols written in Go.
noise is made to be minimal, robust, developer-friendly, performant, secure, and cross-platform across multitudes of devices by making use of a small amount of well-tested, production-grade dependencies.
Now, let’s dig through what’s new.
Noise is connectionless.
A bounded connection pool was introduced to alleviate developers from having to arduously maintain the lifecycle of connections and resources associated with a single peer.
The connection pool by default is bounded such that a single node may only have, at most, a specified number of inbound and outbound connections open at any time.
Should the pool be full and a new connection needs to be established, the pool will gracefully disconnect the last-inserted connection (block the current goroutine until the last-inserted connections entire read/write buffer is emptied) to make room for the new connection.
This change brings together a much simpler user experience for developers using Noise.
To send/receive messages, and send requests/responses across peers requires no management over connections whatsoever from a developer.
Network protocols implemented on top of Noise therefore no longer need to worry about assumptions behind the state or lifecycle of peer connections.
Noise builds networks.
With the introduction of connection pooling comes a decoupling of peer identities from long-lived connections to external nodes.
Peer identities are now optionally persisted away from the instance of connection in a decouplable Kademlia routing table that may be persisted on-disk/in-memory.
Additionally, the Kademlia API for Noise now supports sending/receiving messages to/from peers, or connecting to peers by their Ed25519 public key.
The peer discovery module has been generalized to support searching for peer IDs (comprised of their public IP/port and Ed25519 public key) should you have their Ed25519 public key.
Noise is quiescent.
To improve the anonymity, privacy, and security of nodes, Noise now strictly assumes all connections are to be short-lived.
Connections by default timeout after a specified time duration should no read/write activity occur.
Having an idle timeout allows for nodes to gracefully close unused connections and release extraneous resources.
Frequently recycling connections also allow for sensitive cryptographic primitives associated with encrypting/decrypting messages over-the-wire across peers to have their keys be frequently recycled.
Noise uses the best of best practices.
Peer connections by default are now encrypted using AES 256-bit Galois Counter Mode (GCM) with a Curve25519 shared key established by an Elliptic-Curve Diffie-Hellman Handshake.
In detail, the handshake protocol executed upon establishing a connection with a peer goes as follows:
- peers send each other their ephemeral Ed25519 public keys,
- peers convert the public keys they received into Curve25519 public keys,
- peers convert their ephemeral Ed25519 private keys into Curve25519 private keys,
- peers establish a shared secret by performing ECDH with their private Curve25519 private key and their peers Curve25519 public key,
- peers use the shared secret as a symmetric key and communicate from then-on with messages encrypted/decrypted via. AES 256-bit GCM with a randomly-generated 12-byte nonce.
The process is akin to how handshakes are performed over TLS 1.2 without the use of key derivation function (KDF), but without requiring importing and maintaining the whole TLS suite into your project.
Peer IDs are represented by an IPv4/IPv6 address and an unsigned 16-bit integer port, and an Ed25519 public key. IDs are exchanged after an encrypted session is established over a connection across peers.
In a later release, options will become available to disable/customize the handshake protocol used for nodes.
The idea for customizing the handshake protocol comes from the following Github issue submitted by a community member.
Noise is extensible.
New methods were introduced for creating plugins that may hook onto events emitted throughout a node's lifecycle.
Kademlia is now fully-decoupled and is now an optional plugin that may be implemented into your application.
It may be integrated into your application simply by calling the Bind method to your node before you have your node start listening for new peers.
A plethora of functional option APIs is also provided to customize timeouts, settings, and logging as well.
Noise provides sane defaults.
All timeouts, connection pool settings, and logging are provided sane defaults so that you can use Noise from the get-go. More explicitly:
- No logs are printed by default.
- A random Ed25519 key pair is generated for a new node.
- Peers attempt to be dialed at most three times.
- A total of 128 outbound connections are allowed at any time.
- A total of 128 inbound connections are allowed at any time.
- Connections timeout after 10 seconds if no reads/writes occur.
Noise is minimal.
A lot of dependencies have been cut out to their bare minimum. In total, Noise v1.1.2 only comprises of approximately 4000 lines of code.
The biggest change is a shift in logging libraries from rs/zerolog to uber-go/zap after performance benchmarks in applications we have made internally with Noise.
A lot of our applications are performance-critical and need to trace large amounts of logs. The built-in buffering of logs with uber-go/zap has been extremely helpful.
More explicitly:
- Logging is handled by uber-go/zap.
- Byte buffer pooling is handled by valyala/bytebufferpool.
- Unit tests are handled by stretchr/testify.
- Ed25519 signatures are handled by oasislabs/ed25519.
- Elliptic-curve Diffie Hellman Key Exchange (ECDH) over Curve25519 is handled by agl/ed25519.
Noise simplifies formatting messages.
A serialization and deserialization function may optionally be specified for Go types that are to be sent to/from peers.
This allows you to specify the wire format of messages sent between your peers to be in a wide variety of formats such as protobuf, msgpack, json, or even raw binary (click here for an example!).
All methods that end with a suffix of *Message in the API for Node, such as RegisterMessage for example automatically handle on-the-wire serialization/deserialization.
Refer to the examples and documentation here to learn more.
Noise embraces concurrency.
Several methods (wait until handshake finished, wait until connection closed) were introduced to maintain and track the lifecycle of goroutines, node instances, peer instances, and even connection resources.
This has helped us a lot in creating unit tests and integration tests that check networking edge cases for highly-concurrent applications we have implemented in-house.
In total, a single connection spawns 4 goroutines:
- one for handling protocol logic,
- one for recycling a connection should it timeout,
- one for reading and buffering raw bytes from a connection, and
- one for sending and buffering raw bytes to a connection.
A single node spawns only 1 goroutine for listening for new connections.
Noise is fast.
You can also run your own separate benchmarks locally on a single core by running either one of these commands:
go run github.com/perlin-network/noise/cmd/benchmark_send
go run github.com/perlin-network/noise/cmd/benchmark_rpcNoise finally adopts semantic versioning.
Releases from now on are marked with a version number formatted as MAJOR.MINOR.PATCH.
Major breaking changes involve a bump in MAJOR, minor backward-compatible changes involve a bump in MINOR, and patches and bug fixes involve a bump in PATCH starting from v2.0.0.
Therefore, Noise from now on mostly respects semantic versioning. Additionally, Noise will remain with MAJOR at v1 as it still should be considered to be in its initial development phase.
It mostly respects it because, due to an unfortunate incident where prior releases of Noise were incorrectly tagged (v0.1.0, v1.0.0, v1.1.0, and v1.1.1), Go module information of prior releases was improperly cached on proxy.golang.org and sum.golang.org.
This had caused several issues with users looking to adopt Noise into their projects; as showcased here, and here.
As a result…
Noise from now on will have future releases starting from v1.1.2.
Until Noise’s API is stable, subsequent releases will only comprise of bumps in MINOR and PATCH.
Integrating noise into your Go project support Go modules is now as simple as running go get github.com/perlin-network/noise.
Noise is now well-documented with examples.
One pain point that strongly resonates with me from the community is that prior releases of Noise were not well-documented.
Starting from this release, all methods and structs in Noise are now well-documented, with testable Go example code provided all in Noise’s godoc located here.
As a matter of fact, using the latest Go, run these two commands and you can try out a decentralized chat example built with Noise!
[terminal 1] go run github.com/perlin-network/noise/cmd/chat -p 9000
[terminal 2] go run github.com/perlin-network/noise/cmd/chat -p 9001 127.0.0.1:9000We highly welcome more examples provided from the community and are happy to answer any further questions on the documentation for Noise.
Open up a PR, or ask your questions in a new Github issue.
Better yet, come join us on our Discord server and talk to us directly if you have questions, or are looking to contribute any code.
The roadmap moving forward.
Now that Noise is mostly complete, a few features come off the top of my head that I feel we should prioritize on will be coming in the next few versions.
- The option to customize handshake protocols/disable handshaking.
- Reintroduction of NAT-PMP/UPnP port forwarding.
- A transition to an alternative options API for configuring options.
- A more extensive API for registering Go types that are to be automatically serialized/deserialize over-the-wire.
- Packages for gossiping protocols implemented on top of Noise.
- Packages for consensus protocols implemented on top of Noise.
- More examples and documentation.
Contributions are highly welcome and appreciated, with support from us always available should you wish to have one of these features come out sooner.
Otherwise, we hope you try out Noise and that it significantly helps you make your next decentralized application.
Minor breaking changes will prevail at times, but we have made it a mantra that they should be highly unlikely in forthcoming updates from now on.
P.S. In case you are wondering about updates on Perlin, a post will be released on what’s coming up next for Perlin in the near future.
’Til next time,
Kenta Iwasaki.
