The TezEdge Node — A deep dive into the mempool, part 1

Nodes are the pillars of a blockchain network. Each node plays its part in maintaining the network and ensuring consensus about the blockchain’s state. Managing the constant stream of unconfirmed operations that flow across the network is one of a node’s most important duties.

The purpose of the mempool, short for memory pool, is to temporarily store and manage the movement of new operations before they are validated and baked into blocks.

Without a mempool, it would be impossible to control the vast amount of operations that are being constantly made on the Tezos blockchain.

As the number of operations increases, the mempool must manage a greater volume of tasks. One of the most important duties of the mempool is to efficiently manage resources in order to prevent accidental overloading or intentional flooding by adversaries.

To improve overall resiliency, the node uses the actor model, a conceptual model for software architecture in which tasks are performed by individual actors. An actor is a primitive unit of computation that is capable of performing tasks, but is completely separated from other actors and can use various strategies to automatically recover from failure.

How operations are processed inside the mempool

To understand what happens inside the mempool, here is a scenario in which an operation enters and exits the mempool, starting when it is received from the network and ending when it has been baked into a block.

1. A P2P message arrives from the network

A P2P message of the CurrentHead type arrives from the network.

It is then processed by the ChainManager, an actor that is responsible for processing all of the messages that come in from the P2P network once a trusted connection has been established.

The CurrentHead contains only the hashes of operations. In order to get the full details of these operations, Alice’s node needs to send GetOperations, another type of P2P message. When the network receives the GetOperations message, it replies with Operation.

2. The ChainManager actor receives the Operation.

The operation is stored in the mempool’s storage. Please note that the mempool’s storage is different from the storage module that contains the current state.

3. Checking operations from peers

This triggers a CheckMempoolCompleteness which checks whether all mempool operations were received from peers. If they were not received, Alice’s node sends a GetOperations message.

4. Notifying peers of received operation

The ChainManager sends a MempoolOperationReceived message, which notifies all of the actors that are subscribed to the shell channel that a new operation was received

5. Messaging the shell channel

The MempoolPrevalidator receives the MempoolOperationReceived and sends a ValidateOperation message to the shell channel (since the MempoolPrevalidator is subscribed to the shell channel, it receives the message as well)

6. Inserting into pending operations

The MempoolPrevalidator processes the received ValidateOperation message and checks whether the operation is stored in the mempool storage and whether the operation was already validated. If the checks are successful, the operation is then inserted into the pending operations, thus modifying the current state of the mempool.

7. Moving the operation towards validation

The MempoolPrevalidator then handles the pending operations.

• The operation is sent to the protocol for validation.
• The result is then handled and, if needed, the current mempool state is changed, removing the operation from pending and moving it to applied, refused, branch_delayed or branch_refused. The applied operations are also known as “known_valid” operations (in the context of peer messages).

We will take a closer look at how operations and blocks are validated in a future article.

Ensuring consistency between the Rust and OCaml nodes’ mempool

We want to be able to test whether operations are successfully and correctly propagated from the native OCaml node’s mempool into the TezEdge node’s mempool. To do this, we run the following test:

First we run two nodes; the TezEdge node and the OCaml node, both are run in sandbox mode.

Using the Tezos-admin-client, we create a connection between the two nodes.

Using the Tezos-client, we activate a protocol inside the OCaml node, thus creating the first block.

We wait until TezEdge synchronizes with the OCaml node.

We call each node with the pending_operations RPC and compare the return values. We want both values to be empty, which means their mempools are empty.

Using a Tezos client, we inject a valid transaction into the OCaml node. We call each node with the pending_operations RPC again and compare the return values.

Again, we should see the same value from calling both nodes, but this time it will not be empty. We can see operation in the applied field. This means that the transaction has been successfully propagated from the OCaml node into the TezEdge node.

Click here to see the results of our own continuous integration (CI) test.

As we continue mastering our knowledge of the mempool, we are approaching our milestone of developing chain configurators, including a blockchain sandbox, a developer tool which allows you to run a node without an internet connection. This would simulate a live blockchain on a local device, permitting developers to test their smart contracts in a safe and stable setting.

We thank you for your time spent reading this article and learning something new about Tezos. More articles such as this will be published as we work on the various modules of the TezEdge node.

To read more about Tezos and the TezEdge node, please subscribe to our Medium, follow us on Twitter or visit our GitHub.