BlockVigil Transaction Manager — Part 1: The Ethereum transaction life-cycle

Platforms providing support for decentralised application development, inevitably face the requirement of a management layer that can create and submit transactions to execute app business logic on the blockchain.

At BlockVigil we are working on implementing a transaction management layer, focussed on the Ethereum platform, which naturally requires a clear understanding of the different stages in the life-cycle of an Ethereum transaction. This article, the first from the “BlockVigil Transaction Manager” series, aims to provide that understanding, while taking a closer look at how transactions executing business logic can fail or in other curious cases “get stuck”.

Ethereum transactions in the DApp universe

In Ethereum, transactions have a larger role to play than just execute fund transfers between accounts. Transactions can also execute methods defined in a “smart contract” — a piece of code that implements the business logic of the DApp. Obviously, you’d also need transactions to deploy smart contracts.

Deploying a smart contract on Ethereum network

The Ethereum network is a decentralised peer-to-peer network, that maintain a record of transactions between accounts. There are two types of accounts in Ethereum:

1. Externally owned accounts
2. Contract accounts

Externally Owned Accounts(EOAs) are normal user accounts identified by addresses. The state of the blockchain can only be altered through a valid transaction asymmetrically signed by an EOA’s private key.

Contract accounts on the other hand come into existence when a new smart contract is “deployed” on the Ethereum blockchain. These contract deployments are initiated by EOAs through signed transactions.

Contract deployment initiated by an Externally Owned Account [Image source]

A transaction that deploys a smart contract on the Ethereum network is characterised by the following two features:

1. This transaction isn’t addressed to an existing EOA. On the other hand this transaction when executed will create a new contract account identified by an address.
2. The data field of the transaction contains the encoded bytecode data of the smart contract being deployed. This bytecode resides on the Ethereum Virtual Machine (EVM) and enables method call executions to the deployed smart contract.

Structure of an Ethereum transaction that deploys a smart contract

Method execution calls to deployed smart contracts

Once a smart contract is deployed successfully on the Ethereum network, methods can be executed on the deployed smart contract.

Write call to a contract [Image source]

Naturally, transactions that try to execute contract methods must —

1. be addressed to a contract account where the smart contract is deployed
2. contain encoded information regarding what method to execute and the corresponding arguments.

Transaction executing a method on contract deployed at 0xContractAddr

Transaction submission → validation → confirmation

So depending on whether you want to deploy a smart contract or execute a method on a deployed contract, you create a transaction. But this transaction needs to be executed and the state change be recorded on the Ethereum blockchain.

In order for that to happen a transaction after creation needs to go through a few stages:

1. Transaction submission to an Ethereum node running a client like geth/parity.
2. Transaction validation and broadcast by Ethereum node
3. Transaction confirmation by a miner node

Simple overview of the different stages in the life-cycle of a deployed smart contract

Transaction submission to an Ethereum node and validation

Once the difficult part of creating the transaction is over, the transaction is first submitted to an Ethereum node. An Ethereum node is any thing that runs an Ethereum client like Geth/Parity and tries to stay in sync with the blockchain state.

Transaction validation on Ethereum node and broadcast

Irrespective of what the purpose is, a transaction will always follow a basic structure with some specific data-fields —

Transaction structure [Image source]

1. nonce: This is a count of the number of transactions initiated by the externally owned sender account. It determines the order in which transactions from an account is executed and makes sure that a transaction execution happens only once.
2. gasLimit: This estimates the maximum amount of gas that the transaction execution would require.
3. gasPrice: This decides the fee that the sender is willing to pay per unit of gas required to execute the transaction.
4. to: Address of the recipient account of the transaction. As mentioned before, for contract deployments, this field is set as null.
5. value: funds to be transferred from transaction sender to recipient. For contract deployments, this determines the initial balance for the contract account.
6. data: Contains the contract bytecode for contract deployments or the encoded method execution call data.
7. signature: A 65 byte ECDSA signature in R || S || V format.

Every new submitted transaction goes through validation checks on the values held by these data-fields.

For example, a submitted transaction is rejected as invalid by a node running a Geth client if —

• the transaction nonce is too low.
• the size of transaction is not more than the maximum permissable limit to prevent DoS attacks on the Ethereum network.
• the transaction value isn’t negative. RLP encoded transactions aren’t negative, but a transaction created using RPC might have negative transaction value.
• the transaction has an estimated gas limit set to less than the current block gas limit.
• the transaction signature is consistent with the signer address.
• the signer account has enough funds to cover the cost(=value + gas_price * gas_limit)
• the transaction has been allotted enough gas to cover the basic transaction fee.

Transaction submission, validation and broadcast of processable transactions

But transactions despite being valid might not be ready for broadcast. Only transactions that are “processable” are broadcast to the Ethereum network.

Processable transactions?

Simply put, transactions that are in correct nonce order are processable. If the last transaction that is pending to be mined has a nonce of x, the next submitted transaction will be deemed processable if it has a nonce of x+1.

All submitted transactions with nonce > x+1 is put in the queued list of the transaction pool for the node. Queued transactions become processable only when a transaction with nonce x+1is submitted to the Ethereum node. We will discuss this in greater detail in a later section where we ponder over the curious case of “stuck transactions” in the Ethereum network.

Once transactions become processable, they are broadcast to the network, where other nodes also carry out their own respective validation checks.

Transaction confirmation

Any node in the Ethereum network that can confirm a set of transactions by mining a block can be termed as a miner node or simply a miner. When a miner picks up a pending transaction to include in a block, the transaction goes through the following stages:

• The miner node validates the transaction like all other node.
• The transaction is executed along with other transactions that have been picked to be included in a block.
• A new block is mined upon successful completion of the proof-of-work computation and is announced.
• Other nodes in the Ethereum network validates the block received and executes all transactions in the block.
• Eventually the node that had broadcast this transaction, receives the block and updates the contract account data locally.
• At this point, when a read call is made to fetch the data point that was updated by our write call, we will be able to get the updated data.

What happens if a transaction fails to execute at any point?

A transaction that has failed execution is still included in the block because gas is used to process the transaction. The state changes due to the transaction or contract call are reverted if the execution fails.

The curious case of stuck transactions

There are instances when Ethereum transactions seem to get stuck. This happens under two circumstances —

• Transactions stuck in the transaction pool’s queued list
  In a series of transactions sent, one or more of the transactions are rejected. This leads to a situation where all transactions with nonces greater than the failed transaction get stuck on the transaction pool’s queued list. When a new transaction fills up this “nonce-gap”, these stuck transactions move from the queued list to the pending list of transactions.

State of transaction pool, from a single signer perspective, with transactions stuck in “queued” state

• Transactions stuck in pending state
  Transactions that are ready to be processed/included in a block by miners are said to be in pending state and are stored in the transaction pool. However if one of these transactions have a low gas price set, then that transaction and all following transactions run the risk of staying stuck in the pending list till a miner decides to include the low gas transaction.

State of transaction pool, from a single signer perspective, with transactions stuck in “pending” state

There are two ways stuck pending transactions can be unstuck:

1. Cancelling these transactions:
   A transaction with a particular nonce stuck in a pending stuck can be cancelled by sending a higher priced transaction with toaddress same as the fromaddress.
2. Replacing these transactions with a higher gas price:
   The price bump needs to meet the minimum price bump required to replace the original transaction, failing which the replacement transaction would be discarded by the client’s transaction pool. This price bump requirement is also required to be met for cancellation of transactions.

Failed contract executions

A confirmed transaction on the Ethereum blockchain could still fail to execute a method call on a deployed smart contract or deploy a smart contract due to various reasons:

1. Bad instruction error: This could happen in case of wrong encoded ABI data as part of the transaction (incorrect method selector, invalid parameters, etc)
2. Out of gas: The contract execution might take more gas than what was allotted to the accompanying transaction. This is a result of a wrong gas estimation in the transaction creation stage.
3. Warning! Error encountered during contract execution [Reverted]: This error is also common when other errors occur during execution of a contract call. Eg: https://kovan.etherscan.io/tx/0x5f7628af46d9e72c6aaff2e606b38ee7debdeccb57b4efdcb11f6669aaa60eaa

Recall from the “Transaction confirmation” section that transactions that have failed to execute are still included in the blockchain

Need for a transaction manager

Its must be apparent by now that to be able to handle this life-cycle, there needs to be a transaction management layer on top of clients like Geth or Parity to

• handle failure responses from clients for rejected transactions,
• prevent transactions from getting stuck,
• identify and process stuck transactions
• support contract execution requests from different clients having control over separate signer accounts,
• while also optionally having support to track contract execution results.

The next article in this series will explain how at BlockVigil we are working on implementing a failure resistant, self-recovering state-machine manager for Ethereum transactions.

Further reading

[1] “How does Ethereum work, anyway?”

[2] This article on “Transactions in Ethereum”

[3] This other article on the “Life Cycle of an Ethereum Transaction”