The Lifecycle of an Operation in Tezos
This piece covers the lifecycle of an operation in tezos, starting at its creation and following the operation until it is added to a block on the Tezos blockchain. The goal of this post is to create a better understanding of how tezos operations work by illustrating the different components that take part in the lifecycle of an operation. This post assumes familiarity with the tezos proof-of-stake mechanism.
Tezos’ modular design allows two major components to be distinguished: the network shell and the economic protocol. The network shell has a generic definition of an operation. However, actual operation types and their effects on the state of tezos are defined by the economic protocol, with the initial implementation referred to as “protocol Alpha”. Protocol Alpha is unrelated to the alphanet, a tezos test network.
This post discusses the lifecycle of operations governed by protocol Alpha only.
In protocol Alpha, operations may be thought of as transactions like those in other blockchains, but they also represent more than that. For example, a transaction in Ethereum is just one of the many operation types that is possible inside protocol Alpha.
If you think of tezos as a state machine, then a new block transitions the state machine from an old state to a new one. A block is a batch of operations in a given order. Each operation is executed against the current state of the system (i.e. the tezos blockchain), transitioning the system to a new state. In tezos, this state is known as the “context”.
Types of operations in protocol Alpha
The various types of operations in the current tezos protocol (protocol Alpha) are:
- Endorsement: The “endorsement” operation specifies the head of the chain as seen by the endorser of a given slot. The endorser is randomly selected to be included in the block that extends the head of the chain as specified in this operation. A block with more endorsements improves the weight of the chain and increases the likelihood of that chain being the canonical one.
2. Seed nonce revelation: Bakers (stakers in tezos) use this operation to reveal the nonce they committed in their baked blocks (“baking” is the term used to describe block production and validation in tezos) from previous cycles. All such revelations for nonces committed in a previous cycle are used to decide baking and endorsement rights for the next cycle. For more information on baking and endorsing in tezos, check out this piece by Arthur Breitman.
3. Double endorsement evidence: This operation is used by bakers to provide evidence of double endorsement (endorsing two different blocks at the same block height) by a baker.
4. Double baking evidence: This operation is used by bakers to provide evidence of double baking (baking two different blocks at the same height) by a baker.
5. Activate Account: This operation is used to activate accounts that were recommended allocations of tezos tokens (XTZ, “tez”) for donations to the Tezos Foundation’s fundraiser.
6. Proposal: This operation is used by bakers to submit and/or upvote proposals to amend the protocol.
7. Ballot: This operation is used to vote for a proposal in a given voting cycle.
8. Manager operations: This operation type contains a bundle of operations that require a signature from the signing key associated with an implicit account (tz1…).
- Reveal: This operation is used to reveal the public key associated with a tz1 address (implicit account/public key hash).
- Transaction: This is a standard operation used to transfer XTZ (“tez”) to an account (implicit or originated). If the receiver is an originated account (KT1…), then optional parameters may be passed. When an originated account contains parameters/code, it is referred to as a “contract”. A transaction must be signed by the private key associated with the manager account (a tz1 account) of the source account if it is an originated account.
- Origination: This operation is used to create originated accounts. Originated accounts have addresses starting with “KT1”, unlike implicit accounts with addresses that start with “tz1”. Originated accounts are used for two reasons: delegation and smart contracts (when an originated account has associated Michelson code to be executed when called). Funds in an originated account can be delegated to a delegate (implicit account registered as a delegate).
- Delegation: This is used to delegate funds to a delegate (an implicit account registered as a delegate). The source address for delegating XTZ to a delegate must be an originated account (KT1 address).
The order in which the above types are listed is the same order in which operations are applied when included in a block. Manager operations are applied in the order of the counter (lower 1st). The counter is an account nonce which is used to prevent transaction replay (as required in all account-based systems).
Operation Lifecycle
The diagram below provides an overview of an operation’s lifecycle from the moment it enters a block to the moment it is included on the tezos blockchain.
Before Mempool
Operations are received by nodes in two different ways:
1. From a client via RPC
2. From a peer via advertisement
All local operations (local baker, endorser daemon, and manager operations from a local client) are received by RPC services exposed by the shell. This injection service is defined in the lib_shell_services function in the network shell which is called by the RPC function inject_operation in protocol Alpha’s client code. Local operations then pass through the validation subsystem. Let us now take a look at this validation subsystem.
Inside Mempool
The node maintains a memory pool (“mempool”) to keep track of not-invalid-for-sure operations. The mempool keeps track of two different kinds of operations:
- Known_valid: This is a list of operations that are valid to apply to the current context (state) and may be included in a block if requested by a baker. The resulting context after applying these operations on the current context is called the “Pending context”.
- Pending: A bounded set of operations which are known to not be invalid for sure and are eligible to be broadcasted to peers. This set contains two kinds of operations:
i) branch_refused: an operation which could be valid in a different branch
ii) branch_delayed: an operation which has come too soon (i.e. there’s a gap in the account counter)
Operations in the mempool are broadcasted to peers whenever the mempool is updated. The Peer Node fetches the operation from the remote peer, using the operation hash in the advertisement. This is another way a new operation enters a node. All operations pass through the validation subsystem. When an operation is received by a peer, the peer node is notified of the validation pass result.
A valid operation lives in the mempool until it is added to a valid block of the chain which the node considers to be canonical. If the operation is not added for its time-to-live duration, the operation is removed from mempool.
Prevalidator and Pending context
The prevalidator in the validation subsystem is responsible for deciding which operations get added to the mempool. It also keeps a Pending context, which is the result of applying ‘valid for sure operations’ to the current context, ready to be applied when it is time to do so.
An operation passes through the steps pre_filter, apply, and post_filter inside prevalidator, failure at any step results in the transaction being refused (hence not added to the mempool and also not broadcasted).
Pre_filter
The Pre_filter function performs some basic checks for manager operations. Other operation types are excluded from this check. For manager operations, this check ensures that enough gas required for execution has passed and that the sender has enough of a balance (in XTZ terms) to pay for the gas.
Apply
At the apply stage, all of the operations that pass pre_filter, are applied to the pending context. If apply is successful, the operation receipt and new pending context will be passed back. In the event of failure, an error trace is passed back.
The types of errors thrown for various operation types fall into either permanent, temporary, or branch. Operations that result in permanent errors are refused, while others are put in the pending list of the mempool. The different error types that are possible during the apply stage of an operation and how those operations are applied can be seen in apply.ml. apply_operation of the registered protocol is called inside the prevalidator module.
Examples of a permanent error include: gas exhaustion (not enough gas to parse an operation), invalid activation, or inconsistent evidence in the event of a denunciation such as double baking evidence or double endorsing evidence.
Examples of a temporary error include: the voting period in a ballot is not the current voting period, or double baking evidence is received too early.
An example of a branch error type is unrequired double baking evidence when the delegate is already denunciated in the current cycle.
Post_filter
At the post_filter stage, a decision to add an operation to the mempool based on the result is made. This is done via the post_filter function. Added operations are propagated via advertising. The advertise function advertises by using distributed_db’s current_head function in the advertise module.
From Blocks to Blockchain
When a baker has to bake (produce) a block, it queries the mempool to get all valid operations. It then waits for endorsement operations of the current head of the chain. A hard limit for the waiting time is set to half of the next baking slot’s time. Currently, a baker waits for all the endorsements until 1/3rd of the limit has passed. Between 1/3rd and 2/3rd of the time limit, a baker waits for 2/3rd of the endorsements. If 2/3rd of the time limit has passed, a baker waits of 1/3rd of endorsements until the hard time limit has passed. Once the time limit is reached, a baker then bakes a block with the endorsements it has received. The baking code for this operation is available at client_baking_forge. Blocks are injected into nodes via an RPC call using the injection module in lib_shell_services.
When a new block is received by a node via an injection from a baker, the block is passed through the validator subsystem. The components of the validator subsystem that a block passes through are the block validator and chain validator. If a block is fetched from a peer, then it passes through the peer validator before passing through the block validator and chain validator.
For a block injected by a baker, the block validator is called directly as the data required for a block validation to start is already present locally. If a block is fetched from a peer, all the data required to validate the block (i.e. operations in the block and previous blocks in the chain extended by this block if the current node is bootstrapping) is fetched by Peer validator by using the distributed db.
The Block validator validates a block and calls the Chain validator if the current head of the chain can be updated. The validate function then validates the block by using apply_block. In apply_block, the block header validation is done by using protocol parameters, and the protocol’s apply_operation is used to validate and apply all operations inside the block.
The Header validation includes checks for errors like invalid level, non-increasing timestamp, non-increasing fitness, and unexpected number of validation passes.
The Block validation includes checks for errors such as too many operations, oversized operation, incorrect protocol version, unallowed validation pass, invalid fitness, unavailable protocol, and errors while applying the operation.
Details about errors encountered during block validation can be found here.
Once a block is validated and it is a candidate for the new head of the chain, it is passed to chain validator, which checks if the fitness score of new head is higher than the fitness score of the current head. If it is not, the block is ignored. If it is and this block replaces the current head, chain validator then calls prevalidator to flush the mempool and create a new pending context. Finally, the new block head is advertised to the peer using distributed_db’s Advertise module. This block becomes part of the canonical chain only if future bakers bake on top of it. All operations inside this block are then part of the tezos blockchain.
