The Actual Networking behind the Ethereum Network: How It Works
Most Ethereum users don’t understand the underlying peer-to-peer communication network that makes it tick. The Ethereum blockchain receives all of its power only because it is, essentially, an enormous decentralized computer. In other words, a network of computers can communicate without using a central gateway to traffic all information through it.
A central gateway of information, such as a server, strictly conflicts with Ethereum’s biggest strength. If a server were to handle all of Ethereum’s communication, any admin would be able to alter all information going in and out of the blockchain, thus making the blockchain unfair and not secure. Another problem is that once the server shuts down due to technical issues or power outage, the whole network can’t communicate and is disabled completely until the server is back online.
In peer-to-peer systems, each peer or client can directly send and receive data directly from any other client. Each of the peers in the network behaves both as a server and as a client. Each node can request data to be saved in some nodes database, or it can request to read from a node’s database. In Ethereum, all nodes can request from another node some information, about Ethereum’s current state (smart contract, account balance, last block etc.). The node knows that some information is correct and valid because it can be verified it with Ethereum’s consensus protocol.
Technical Difficulties, Please Stand By: The Rise and Limits of IP Addresses
The Internet isn’t built for such networks. Due to some historic mistakes — which will I get into shortly — and other security risks, Internet service providers limit such network traffic.
In 1980, the Internet Engineering Task Force (IETF) described in rfc 791 the standard for internet addresses. Internet Protocol Addresses (IP addresses) are a way for a computer to identify locations for network communication. The next year, the IETF decided that IP addresses would be 32-bits long, thus limiting the maximum nodes on the internet to 4,294,967,296 (due to some protocol specifications the actual value of possible nodes is lower). Throughout the years, it was observed that many more computers would need to be connected to the internet, and so in 1991 a solution was proposed: Network Address Translators or NATs.
NATs are usually a part of a home network WiFi router but can also be separate component the ISP uses to widen its IP address range. NATs use a single global IP address and can send data to a large amount of clients. NATs hold a table that states commands similar to this: “all incoming connections to my global IP on port X I will redirect to this internal IP Y on port Z”. This means that one NAT can handle as many clients as ports it manages (usually computers have 65,535 ports). NATs enable ISPs and infrastructure managers to create large local networks that only require one global IP address. This is amazing, but NATs are designed to allow traffic only from the inside of the local network, to a global IP address.
NATs are the main thing that blocks computers from communicating on the internet in a peer-to-peer manner :( . NAT devices did not allow clients from their local network to know the global IP nor port because they didn’t need to (remember NAT was invented for server-client communication and not peer-to-peer). Throughout the years, the support for such features has become more popular and now there are many protocols that can map local IP address to a global IP and port (UPnP, PCP, NAT-PMP to name a few). A HA, another catch: Some of them are vulnerable to security breaches and most NATs disable them by default.
You might have encountered this phenomenon when trying to setup a Bitcoin or Ethereum node… Your client might have not been able to connect to the p2p network because it couldn’t map itself to a global IP address (the go-ethereum client uses UPnP and NAT-PMP). Or maybe it couldn’t get its own global IP address from the NAT device. The solution is to configure your router manually to “add a new port mapping” to your local IP address.
Kademlia and RLPx Protocol Basics
To make a p2p network complete, there must be an implementation of node discovery, which allows peers to discover more nodes that support and run an ethereum client. Ethereum uses a node discovery protocol that is very similar to the Kademlia protocol. Kademlia is a Distributed Hash Table for p2p networks designed by Petar Maymounkov and David Mazieres. It can create a distributed server that all peers can store and read data in a decentralized way. Ethereum uses a Kademlia-like protocol for node discovery.
Protocol Specification
Every node has a specific ID that is unique (Ethereum uses the SHA3 hash of your public key, node-IDs also consist of global IP and port so a connection can be established). Each node stores a table of known active nodes each has K nodes in it. The node’s main goal is to maintain exactly K known nodes in each row. Row i represents all the known nodes that have the same first i bits in their address as the current node (row numbering starts at 0). Each node can request from another node to lookup a node. A lookup request consists of the requested node ID and the source node ID.
Example: Node A sends a lookup request to node B to look for node C. Node B sends back to node A all the nodes it has in the row that node C would fit in it’s table. If node A did not find out about all the nodes in node B’s response — in this case, node C — it will request from each of the nodes in node B’s response to lookup node C and so on until the some node finds node C and responds to node a with it’s address.
This protocol allows Ethereum nodes to discover nodes and connect to them efficiently on order to maintain the network’s decentralized factor.
But how can a new node discover the first node and join the network? When a new node tries to join the network, it needs to establish a list of some known peers that it can communicate with. To do so, Ethereum clients are hard-coded with three main peers that are maintained by the Ethereum Foundation. Upon joining the network for the first time, the new peer will ask one of these bootstrap peers for a list of active nodes. Take note, these three bootstrap peers are a form of centralization (the only one in decentralized peer-to-peer systems according to researchers Jochen Dinger and Oliver P. Waldhorst). So whenever a peer tries to contact a node besides one of these three, it won’t!
RLPx and Data Transfer
For data transfer, Ethereum uses a protocol named RLPx. RLPx enables nodes to transfer encrypted, serialized data. Firstly, RLPx’s node discovery protocol is a Kademlia-like protocol similar to the one mentioned above.
When two nodes want to communicate, they send each other some cryptographic data (public keys and such) to make sure all of the subsequent data transfer is encrypted (using ECDH, ECDHE, ECIES and more elliptic curve cryptography) and cryptographically signed (you can learn more about elliptic curve cryptography here). Then, both nodes send to each other which protocols and which versions of these protocols they support: The Ethereum protocol is “eth,” Ethereum’s Whisper protocol is “shh” (Get it?), and the Light Ethereum Node Subprotocol is “les.”
After all messages are encrypted and a protocol agreed upon, the subsequent messages are dependant on the protocol chosen.
As of the time of writing, the ethereum network consisted of 11,200 nodes all around the world. And none of them would be able to communicate securely without the suite of protocols and technologies developed to make sure the internet can be somewhat decentralized.
Join the Orbs community:
- GitHub: https://github.com/orbs-network
- Telegram: https://t.me/orbs_network
- Twitter: https://twitter.com/orbs_network
- Reddit: https://www.reddit.com/r/ORBS_Network/
- Read the Orbs white papers: https://www.orbs.com/white-papers
