The TezEdge Debugger —view all messages from the P2P and RPC layers (for both Ocaml and Rust-based nodes)

Whenever you are developing a node in OCaml or Rust and your software runs into a bug, an error or an otherwise undesired state, you want to be able to identify and locate the instance when and how the problem happened.

However, when you are working with an encrypted network, such as the Tezos peer to peer (P2P) network, locating the problem is not so easy. The messages must first be decrypted and deserialized before you can make sense of them.

The TezEdge Debugger

During the development of the TezEdge node, we wanted to have a debugger tool that could operate equally well with OCaml as well as Rust. In order to thoroughly debug and simulate any state in the node, we need to access the P2P network, RPC endpoints and the storage.

For this purpose, we’ve created the TezEdge Debugger, a tool that records messages sent and received by your Tezos node via the P2P and RPC layers, decrypts and (if necessary) deserializes them, and finally providing them in a human-readable format in its own user interface (UI), which is known as the TezEdge Explorer. While the Debugger forms the back end of this system, the Explorer constitutes its front end.

Through the TezEdge Explorer, users can view:

• Who sent the P2P message(s) to your node
• The content of the message(s)
• The time when they were made
• RPCs made in connection with your node (both requests and responses)

All of this information is recorded in the storage, allowing users to examine all of the aforementioned details whenever they want to.

The TezEdge Debugger is node-agnostic, meaning that it will run on any kind of Tezos node, whether it is the native OCaml version, the Rust-based TezEdge node or other alternatives.

As this is an early version, certain features, including full text search and pagination, have yet to be developed. We plan on adding them in future releases.

How it works

The core function of the debugger is to intercept all network communication made between the node (local) and the rest of the network (remote). We want to have a tool that is capable of capturing all of communication intended for the local node.

There are multiple methods of intercepting network communication. In our case, we chose to use a raw TCP socket.

The internet operates as a “layered” system, with multiple protocols on each layer. Internet protocol (IP) is protocol of the 3rd layer while user datagram protocol (UDP) and the transmission control protocol (TCP) belong to the 4th layer of the TCP/IP model.

Peer to peer (P2P) communication between Tezos nodes operates under the aforementioned TCP protocol. The internal node uses a TCP socket. From the point of view of the application, the TCP socket works as a channel, as you input data, they are released on the other end in the order that they were received. Everything else is handled by the operating system.

A Raw TCP socket works differently-it captures packets with headers, but you have to compose the message yourself and make sure it’s in the right order. However, it allows you to capture all of the communication, not just the one that is intended for your application.

Applications commonly use standard TCP sockets, with the TCP stack being held internally by the operating system. We, on the other hand, use a “raw” TCP socket, which is received from the operating system and operates on TCP packets (instead of already processed bytes) but does not contain a TCP stack.

This allows it to capture all TCP packets, including their TCP and IP headers. Using the raw socket, you can intercept all of the packets that constitute TCP communication. Based on the data in the packet headers, you can then filter the packets that are used in communication with the Tezos node.

The raw TCP socket

In addition to the message, a TCP header contains two more items: an IP header and the TCP header itself.

• The IP header contains the IP address, which informs us about the packet’s sender and receiver.
• The TCP header provides us with the TCP port, which informs us which application the packet belongs to.

Using Docker, you can create a shared network — there were two containers on the same network, with one running a Tezos node and the other running the Debugger. Since all of the communication is contained within this network, you can use the Debugger to analyze it. This removes the obligation to set up a tun device, which makes it a more user-friendly experience.

Simulating network traffic

In order to perform replay, we need to simulate a running node by literally connecting the node onto another node. Now that you have a local node running, you launch the Debugger

If you want to perform replay on the running node, you need to open another TCP socket locally. In this case, it is a common TCP socket (not raw). Through this socket, the Debugger communicates with the node as if it were another node, sending messages based on what is stored in its database.

In order to perform replay, we need to create a snapshot of communication we want to simulate (by using the debugger). Using the snapshot and plain TCP socket, we can use the debugger as if it is a node itself, but instead of actual P2P communication logic, it uses the created snapshot as a guide to drive the P2P communication.

Operating within the Tezos network

When you launch the TezEdge Debugger, it will run parallel to your Tezos node, recording the messages, packet and RPCs made in connection with your node. The TezEdge Debugger makes use of the identity.json which is generated by a node before it makes its first connection to the Tezos network. It then uses the private key from the identity.json in combination with the public keys and nonces it intercepts as they are being sent across the network

At a broad level, the P2P Debugger operates by setting up an interception layer between two peers before they exchange the public keys and nonces.

Let’s say Alice is running a node on the Tezos P2P network. She wants to run the TezEdge P2P Debugger to examine the messages sent between her node and Bob’s node. The communication between Alice and Bob is unencrypted since they haven’t exchanged the public keys and nonces necessary for encryption.

1. Alice is the local user. She launches the TezEdge debugger, creating an interception layer between Alice (local) and Bob (remote).
2. Alice sends a ConnectionMessage to Bob. This message contains (among other items) her public key and a nonce.
3. The debugger intercepts and records the message, creating a copy of it and storing it onto its storage, providing the data in a readable format through an exposed application programming interface (API). The original ConnectionMessage with Alice’s public key is delivered to Bob.
4. Bob sends his ConnectionMessage (including his public key and a nonce) to Alice, the debugger intercepts and records Bob’s message, copying it onto its storage, providing the data in a readable format through its API. The original message with Bob’s public key is delivered to Alice.
5. Now that both Alice and Bob have received the public keys and nonces (thus completing the handshake), their communication can now be encrypted.

Now that the TezEdge debugger has received the public keys and nonces, it is ready to intercept, decrypt and deserialize the messages that flow between the two peers:

6. Alice uses Bob’s public key, her own private key and her nonce to encrypt a message which she then sends to Bob.

7. The TezEdge debugger intercepts, decrypts (and, if necessary, deserializes) Alice’s message, providing the data into the TezEdge debugger which displays it via its UI.

8. Bob receives Alice’s original message.

9. Alice can access the UI to browse the data intercepted by the TezEdge debugger at any time

How to run the TezEdge Debugger

1. Open shell and type this code into the command line and then press Enter

git clone https://github.com/tezedge/tezedge-debugger.git
cd tezedge-debugger

2. Download and install Docker and Docker Compose

Open shell and type this code into the command line and then press Enter:

sudo ./docker.sh

3. Open the TezEdge Explorer in your browser

You can view the status of the debugger in your browser by entering this address into your browser’s URL bar:

http://localhost:8080

We thank you for your attention and hope you enjoyed today’s article. To read more about Tezos and the TezEdge node, see our documentation at tezedge.com. Feedback, comments or questions are always welcome — feel free to email me at jurajselep@viablesystems.io