Optimism Bedrock Wrap-Up Series 5 [ENG]

Roles and Behaviors of Essential Optimism Bedrock Components

Onther aims to deliver valuable information for current developers interested in Optimism and the evolution of the Ethereum ecosystem.

Special Thanks to Theo Lee, Austin Oh, and Justin Gee for their contributions to this post.

You can check the Korean version of this article here.

Fig. Illustration of Optimism (Source: OP LABS)

This post is the fifth installment of the ‘Optimism Bedrock Wrap-Up Series,’ comprising five planned articles by Onther. Here, we comprehensively examine the roles and operational logic of Op-Batcher and Op-Proposer.

Given the interconnected nature of the series, we recommend reading the articles sequentially for a cohesive understanding.

Series 1. Overview of the Bedrock Upgrade: It provides an overview of the Bedrock upgrade, its components, and the smart contracts deployed within its layers.

Optimism Bedrock Wrap-Up Series 1 [ENG]

Series 2. Key Changes Since the Bedrock Upgrade: In this section, we aim to unravel the substantial changes introduced by the Bedrock upgrade, laying the groundwork for a comprehensive understanding to navigate the upcoming segments of the series.

Optimism Bedrock Wrap-Up Series 2 [ENG]

Series 3. Analysis of the Deposit/Withdrawal Process: We will conduct a step-by-step analysis of the deposit/withdrawal process, unraveling the core code logic within its layers.

Optimism Bedrock Wrap Up Series 3 [ENG]

Series 4. Block Derivation: Once blocks are generated on the Layer 2(Optimism mainnet), the system initiates a process to rollup these blocks to Layer 1. Subsequently, in the Block Derivation phase, L2 blocks are reconstructed using exclusively the data that has been rolled up. We will provide detailed guidance through each step of the block derivation process, offering code examination along the way.

Optimism Bedrock Wrap-Up Series 4 [ENG]

Essential components of an Optimistic Rollup solution

Fig. Optimism prerequisite architecture

As you can see from the provided architecture, Optimism Rollup currently relies on four fundamental elements: op-geth, op-node(rollup-node), op-batcher, and op-proposer. Presently, all four operate in a centralized manner, appearing as a unified entity. However, understanding the overall mechanism necessitates an examination of each one independently. The degree to which they ought to be acknowledged as independent entities remains ambiguous and prompts numerous inquiries.

In decentralized systems such as Ethereum, a distinct separation of authority and roles exists among entities like geth, sequencer, proposer, verifier, and others. We anticipate that Optimism’s ultimate goal will align with this established decentralization of components, and they are in their ongoing efforts to achieve it. Consequently, we will consider and the aforementioned four elements as autonomous entities, taking into account their nuanced correlations and distribution of roles. However, the future division of each element remains uncertain as Optimism continues its journey toward greater decentralization.

First of all, let’s initiate with concise descriptions of each element before delving into the analysis.

op-geth(L2 Geth)

Its responsibility lies in incorporating transactions from both L1 and L2 users into the old state to generate the new state. This meticulous process safeguards the integrity of all state transitions by overseeing the storage and modification history of the state throughout the entire transaction processing journey.

op-node(Rollup Node)

Fig. Correlation between components by layers (Source:Optimism)

In the diagram above, the op-node centrally integrates the distinct elements (sequencer, verifier, proposer) cohesively. However, the individual roles of these elements in each process are clearly defined, and
breaking them down facilitates a better understanding of the overall mechanics of the solution.

To begin, there’s a sequencer responsible for generating L2 blocks. It achieves generating blocks by passing and converting the raw data of L2 transactions from L1 to the execution layer(op-geth) through the standard Ethereum Engine API. Moreover, sequencers undertake the tasks of both op-batcher and op-proposer, it is beneficial to generally regard them as a proposer.

Lastly, although not covered in this series, there is the role of the verifier, which validates the L2 chain with the batch and output submitted to L1. (We’ll proceed to further elucidate the roles of each element in the subsequent text.)

op-batcher

Op-batcher, also referred to as batch submitter or batcher, plays a crucial role in converting L2 transactions into batches and then writing them to L1’s BatchInbox. Throughout this process, Op-nodes compress L2 transactions to optimize data transfer and minimize memory usage. This compression facilitates the efficient derivation of the L2 chain.

op-proposer

Concerning op-proposer, its primary duty is to conclude state transitions taking place in L2 within L1. In practical terms, for a rollup solution to instill trust in the process, every transaction in L2 must undergo verification and attestation in L1. Any L2 transaction not acknowledged in L1 is essentially treated as though it never transpired.

Upon op-geth updating the state, the op-proposer captures the modified state by submitting a commission to L1 for that specific state. This submission goes beyond mere documentation; it concurrently suggests a new Merkle root for the state, to curtail transaction costs through the reduction of data written to L1. Subsequently, these proposals for state roots are posted to L1’s L2OutputOracle, and the validation process takes place following a 7-day finalization period.

Having delved into op-geth and op-node in the preceding four parts of this series, the subsequent focus will be on how op-proposer and op-batcher collaboratively submit L2 state and transactions to L1.

Role and behavior logic of op-proposer

As previously mentioned, the role of the proposer involves submitting the output root for the L2 state to L1’s L2OutputOracle contract. First, let’s examine the configuration of the output root.

Fig. specs/proposals.md (Source: github link)

• version_byte : Signifying the version of the output root, this is updated whenever there are structural changes.
• payload: A byte string of arbitrary length, configured as illustrated above.
• state_root: The Merkle-Patricia-Trie (MPT) root of all execution layer accounts.
• withdrawal_storage_root: The Merkle-Patricia-Trie (MPT) root of the MessagePasser contract storage.
• lastest_block_hash: The block hash of the most recent L2 block.

Following that, let’s explore how frequently L2 output is submitted to L1.

Fig. deploy-config/mainnet.json (Source: github link)

Fig. A section of the State Batch that collects the transactions of an op-proposer (Source: optimistic.etherscan.io)

The L2 account’s state transition occurs every 2 seconds, aligning with the block generation interval. While obtaining the output root every time it occurs would be ideal, the associated costs make it prohibitive. Therefore, it is crucial to strike a balance and receive it at a reasonable interval. This cycle is configured by the deploy-config/mainnet.json file during the initial deployment of a node, presently set to 1,800 blocks for mainnet, 120 blocks for Goerli, and 20 blocks for Devnet. With 1,800 blocks on the mainnet, the L2 output root is submitted to the L1 every 3,600 seconds, equivalent to 1 hour. (This is the reason why the withdrawal transaction takes over an hour to be submitted to the L1 L2OutputOracle contract in the first step of the withdrawal process.)

Examining the optimistic.etherscan screenshot above, you can observe transactions occurring approximately every hour. The details include the associated L1 block where they are stored, the L1 transaction hash, and the output root.

Now, let’s proceed to the analysis of the code.

CLIConfig

Fig. config.go/CLIConfig (Source: github link)

The above code is for configuring the op-proposer. There are several parameters for this configuration structure, which are described below.

Required parameters

• L1EthRpc: The L1 HTTP provider’s URL.
• RollupRpc: The Rollup-node HTTP provider’s URL.
• L2OOAddress: The address of the L2OutputOracle contract.

Fig. flags.go/PollIntervalFlag (Source: github link)

• PollInterval: The frequency at which L2 blocks (transactions) are queried to generate the output root is set to 6 seconds, as indicated above. Given that L2 blocks are created every 2 seconds, 3 blocks are queried simultaneously. This process accumulates 1,800 L2 blocks, which are subsequently submitted to L1.
• AllowNonFinalized: A bool flag set to true to allow proposals for the output of L2 blocks derived from L1 transactions that have not yet been finalized.
• TxMgrConfig: A structure for transaction management.

Optional parameters

• RPCConfig: A structure for Remote Procedure Call.
• LogConfig: A structure for configuring logging in Proposer.
• MetricsConfig: A structure for configuring the metrics of the Proposer.
• PprofConfig: pprof is a tool for profiling go application data, and you can track cpu, memory, trace, etc. of the desired target. This is easy to use because go tools have built-in support for pprof.

Main

Fig. L2_output_submitter.go/Main (Source: github link)

The Main function serves as the entry point for the L2 output submitter (proposer) service, accepting two parameters: version and cli.Context. The version parameter specifies the service version, and cli.Context represents the command-line arguments passed to the service during runtime. Now, let's delve into a more detailed examination of the function.

• flags.CheckRequired: Verifies if the required flags mentioned earlier are set, triggering an error if any are missing.
• NewConfig: Utilized to create a new Config object within cli.Context, encapsulating all the configuration (flag) information previously set.
• oplog.NewLogger: Initiates a new logger, using the oplog.SetGlobalLogHandler function to establish it as the global log handler.
• metrics.NewMetrics: Generates a new metrics object.
• NewL2OutputSubmitterConfigFromCLIConfig: Constructs a new L2OutputSubmitterConfig object from a Config object to initialize the L2 output proposal.
• NewL2OutputSubmitter: Creates a new L2OutputSubmitter object from an L2OutputSubmitterConfig object.
• L2OutputSubmitter: Initiates the L2 output submitter using the Start method of this object.
• defer: Ensures that, upon the function’s conclusion, the L2 output submitter also terminates.

With this, we conclude the creation of the new submitter. Let’s proceed to the code responsible for running the pprof, metrics, and rpc server.

Fig. L2_output_submitter.go/Main (Source: github link)

Initially, employ the prof.StartServer function to initiate the pprof server, capturing data such as CPU, memory, and trace information for the service. Additionally, utilize a defer statement to guarantee that when the function concludes, the pprof server also terminates.

Fig. L2_output_submitter.go/Main (Source: github link)

Subsequently, employ the m.Start function to launch the metrics server and log a message signifying its initiation. Similarly, use a defer statement to ensure the metrics server terminates along with the function. Lastly, invoke m.StartBalanceMetrics to track the balance of the op-proposer’s Ethereum Account. This is pivotal as posting output to the L2OutputOracle contract incurs a gas fee, necessitating the required balance for successful execution.

\Fig. L2_output_submitter.go/Main (Source: github link)

The concluding segment of the Main function involves creating a new RPC server through the oprpc.NewServer function and incorporating the admin API. Proceed to initiate the RPC server using the Start method and log a message confirming the server's commencement.

Start(loop) / Stop(loop)

Fig. L2_output_submitter.go/Start(), Stop() (Source: github link)

In the proposal process, the ‘loop’ method serves as an event loop that consistently queries for new L2 transactions, facilitating their transfer back to L1. The Start() method orchestrates the execution of this loop, receiving the output submitted by the L2OutputSubmitter.

Subsequently, the Stop() method takes charge of canceling the context used by the loop method, closing the ‘done’ channel, and awaiting the conclusion of the loop method through the wg.Wait() method, effectively stopping the L2 output submitter.

Now, let’s delve into the details of the loop method.

loop()

Fig. L2_output_submitter.go/loop (Source: github link)

The ‘loop’ method has the crucial task of continually retrieving the next output information. It evaluates whether to propose the output, invoking and recording the method responsible for transmitting the proposed output to L1.

Firstly, the ticker in the ‘pollInterval’ field of the L2OutputSubmitter structure triggers at regular intervals, executing the first case of the select statement. After the execution, the FetchNextOutputInfo method is invoked to acquire the next output for submission. This method returns a boolean indicating whether the fetched output is in a proposal-ready state. If it is, it returns an OutputInfo object denoting the error. If an error is encountered, the loop is interrupted and can only resume after the ticker is re-triggered.

Conversely, if the output is suitable for submission, a new context with a timeout of 10 minutes is created using context.WithTimeout. Subsequently, the sendTransaction method is called to submit the output to L1. If the transaction is successful, the RecordL2BlocksProposed method is invoked on the metrics object to log the proposed blocks.

While we have provided a brief overview of the ‘loop()’ method and its overall flow, let’s delve into more details regarding the specific calls to the FetchNextOutputInfo and sendTransaction methods.

FetchNextOutputInfo

Fig. L2_output_submitter.go/FetchNextOutputInfo (Source: github link)

The FetchNextOutputInfo method, invoked within the loop(), takes a context.Context object as a parameter, employing it to set a timeout for the performed request.

Commencing with the use of the context.WithTimeout method, a new context.Context object is created with a timeout set to l.networkTimeout. Subsequently, a bind.CallOpts object is generated with the From field configured to the From address of the txMgr object and the Context field set to this newly formed context.

The callOpts object is then employed to invoke the NextBlockNumber method of the L2 output contract, retrieving the next checkpoint block number. Following this, another context is created using the context.WithTimeout function, setting the timeout to networkTimeout, and this time requesting information about the L2 block head currently being processed.

The rollupClient.SyncStatus is called to obtain the current synchronization status with L2, with the context.Context as a parameter. At this juncture, the currentBlockNumber variable is assigned the latest L2 block number.

Subsequently, contingent on the L2OutputSubmitter’s allowNonFinalized field, the safe head is set. If true, a safe head is used; if false, a finalized head is employed. The L2OutputSubmitter then checks whether the current block number is lower than the next checkpoint block number. If it is, the determination is made that it’s too early to propose, and the method returns nil. Conversely, if the current block number is greater than or equal to the next checkpoint block number, the fetchOutput method is invoked to retrieve the output for submission.

In summary, this function can be comprehended as a sequence involving obtaining the block number for the next proposal, deciding whether to propose based on the current state of L2, and fetching the corresponding output.

sendTransaction

Fig. L2_output_submitter.go/sendTransaction (Source: github link)

The final step in the propose process involves the sendTransaction method, responsible for dispatching the output root to L1. This method takes a context.Context and an eth.OutputResponse object as parameters, with the eth.OutputResponse encapsulating the L2 output transaction information intended for submission.

Initially, it invokes the waitForL1Head method, ensuring the proper sequence for submitting the L2 output transaction to the specified L1 block. Then, it calls ProposeL2OutputTxData to generate the L2 output transaction data, converting it into a byte array, and returning it within the eth.OutputResponse.

Following the conversion, the txMgr.Send method is invoked to transmit the transaction data to L1. This method takes a txmgr.TxCandidate object as a parameter, containing the transaction data, the address of the contract for transaction submission (L2OutputOracle), and the gas limit for the transaction.

Upon a successful transaction, the status of the transaction receipt is examined to verify its success. Subsequently, the transaction hash, L1 block number, and L1 block hash are logged.

Now, let’s delve deeper into the proposeL2OutputTxData function called within this method.

proposeL2OutputTxData

Fig. L2_output_submitter.go/proposeL2OutputTxData (Source: github link), L2OutputOracle/proposeL2Output, emit outputProposed (Source: github link),

The proposeL2OutputTxData function initiates by utilizing the abi.Pack method packing(consolidating multiple values into a single variable) the arguments for the L2OutputOracle contract's proposeL2Output function. The necessary data for packing is extracted from the eth.OutputResponse provided.

Subsequently, the proposeL2Output function of L1’s L2OutputOracle contract is invoked, receiving the following four parameters:

• _outputRoot: The output root of the L2 block.
• _l2BlockNumber: The L2 block number generating the output root.
• _l1BlockHash: The L1 block hash presently being processed.
• _l1BlockNumber: The block number of the aforementioned block hash.

Examining the emit OutputProposed, it takes four parameters:

• nextOutputIndex(): The index of the next output.
• _l2BlockNumber: The L2 block number generating the output root.
• Block.timestamp: The timestamp of the current L1 block.

To store information about the proposed L2 output transaction, a new Types.OutputProposal structure is generated. This structure pushes the output root, the timestamp of that block, and the block number of the L2 output transaction.

So far, we’ve explored how the op-proposer submits L2 output to the L2OutputOracle on L1. We will now continue with the description of the op-batcher.

Role and behavioral logic of op-batcher

Fig. Data transformation process for L2 blocks to be submitted to L1’s BatchInbox

In the Legacy version before the Bedrock upgrade, one block was generated for each L2 transaction. Then, all L2 blocks were submitted to the CTC (Commitment To Compliance) contract, consolidating multiple transactions into a single batch at the batch-submitter’s polling interval. However, with the Bedrock upgrade, the approach has evolved to combine multiple batches into a unified channel for submission, to minimize data availability costs.

BatchInbox

The transaction created by the batcher is directed to a special EOA (Externally Owned Account) named BatchInbox on L1. As BatchInbox is an EOA address, the EVM code is not executed, resulting in savings on gas costs.

• Ethereum goerli: 0xff00000000000000000000000000000000000420
• Ethereum: 0xFF00000000000000000000000000000000000010

Now, let’s proceed to examine the implementation of the op-batcher’s code. We’ll provide an overview of the broader concept first, followed by an exploration of the specific logic within the detailed methods.

loop()

Fig. driver.go/loop() (Source: github link)

The provided code encapsulates the loop() method of the batcher, designed for continuous iteration to retrieve data for the conversion of L2 blocks into channels.

In this context, two crucial functions come into play: loadBlocksIntoState and publishStateToL1. The loadBlocksIntoState method is tasked with integrating newly detected L2 blocks into the local state, preparing them for submission. On the flip side, the publishStateToL1 method orchestrates the transformation of the loaded blocks into channel frames, ultimately crafting a batcher transaction for submission to the L1 BatchInbox.

This process is governed by a select statement, which waits to receive one of three events: a tick from the ticker, a receipt from receiptsCh, and a signal from shutdownCtx. Upon receiving a tick, the latest blocks from L2 are loaded into the state by calling the loadBlocksIntoState method and submitted to L1 using the publishStateToL1 function.

Let’s proceed to a more detailed examination of the loadBlocksIntoState and publishStateToL1 methods.

loadBlocksIntoState

Fig. driver.go/loadBlocksIntoState (Source: github link)

The loadBlocksIntoState method initiates by invoking the calculateL2BlockRangeToStore function, responsible for determining the range(start, end) of L2 blocks to be stored in the state. This function takes a context.Context object as a parameter and returns two eth.BlockID objects representing the range. It synchronizes all block information in the L2 chain since the last submitted batch.

Fig. driver.go/loadBlocksIntoState (Source: github link)

Then, a loop is initiated, calling the loadBlockIntoState method to load the state of the channel manager overseeing the channels of the op-batchers within the specified range. In the event of encountering an L2 re-org error, it gets logged, and as a precautionary measure, the lastStoredBlock variable is reset to an empty eth.BlockID. This action prevents the state from being submitted to L1, mitigating potential complications associated with the L2 re-org error. Conversely, in the absence of errors, the lastStoredBlock and latestBlock variables are updated to align with the loaded state.

Fig. driver.go/loadBlocksIntoState (Source: github link)

Following the loading of all corresponding blocks into the channel manager's state, the method invokes the derive.L2BlockToBlockRef function. This call serves to store references to the most recent block and the genesis block with which the rollup chain was initiated. To wrap up its operations, the method records the number of L2 blocks loaded into the state using Metr.RecordL2BlocksLoaded.

publishStateToL1

In summary, the publishStateToL1 method encapsulates the process of taking the block information stored in the channel manager’s state, converting it into a channel frame, and returning it to the loop() method.

Fig. driver.go/publishStateToL1 (Source: github link)

To initiate this process, a new channel named txDone is created to signal the completion of loading all state. Subsequently, a new goroutine(a lightweight thread managed by the Go runtime) is launched to send the state and wait for its transmission to L1. Within this goroutine, a loop structure is employed, invoking publishTxToL1 for each transaction in the queue. This function is responsible for converting the L2 block information into a batcher transaction and transmitting it to L1.

The logic for transforming from this method to a batcher transaction revolves around the TxData function of publishTxTo, which we’ll examine next.

publishTxToL1

Fig. driver.go/publishTxToL1 (Source: github link)

To begin, the l.l1Tip method is invoked to retrieve the latest L1 block header. The recordL1Tip method is then called to log the L1 tip, also known as the priority fee. Following this, the TxData method of the local state is utilized to obtain the next batcher transaction. If there is no transaction data to retrieve, a trace log is logged, and an io.EOF error is returned.

The TxData function plays a pivotal role in the overall process by orchestrating the creation of a batch that encompasses L2 block information. Its responsibilities include encoding and compressing this information, depositing it into a channel, and subsequently segmenting the generated channel into channel frames. This comprehensive procedure results in the formation of a batcher transaction, marking a critical step in the entire process. Further details on this crucial function will be provided below.

Subsequently, the sendTransaction method is called to dispatch the transaction data to L1. This method requires data, queue, and receiptsCh object as parameters. It plays a crucial role in sending a batch to L1 and will be elaborated on in greater detail.

TxData

Fig. channel_manager.go/TxData (Source: github link)

Initially, the TxData function scans the channelQueue to identify the first channel that has a frame ready for submission. If such a channel is found, the variable firstWithFrame is set accordingly. Following this, it checks for pending data or the closure of the channelManager object. If either condition is met, the nextTxData method is called to retrieve the subsequent transaction data. Conversely, in the absence of pending data, the function checks for available stored blocks to construct a frame. The ensureChannelWithSpace method is then invoked to ascertain if a channel with ample space for submission to L1 exists.

If a suitable channel is identified, the processBlocks method is called to populate the channel with stored blocks. Then, the registerL1Block method is invoked to register the current L1 head.

Once these steps are completed, the outputFrames method is used to generate frames, which are then added to the frame queue. Lastly, the nextTxData method is employed to gather the channel frames from the frame queue and form a batcher transaction for subsequent retrieval.

Next, we’ll delve into the processBlocks method to understand how it populates the channel with stored blocks.

processBlocks

Fig. channel_manager.go/processBlocks (Source: github link)

In this segment, a loop is instituted to process each individual block stored in the block slice separately, terminating when the channel reaches full capacity. The AddBlock method is then employed to append each stored block from the slice to the channel.

Upon successful addition of all blocks to the channel, the blocksAdded variable is incremented by one. Subsequently, the RecordL2BlockInChannel method is utilized to comprehensively record the pertinent details in the channel:

• blocksAdded: Records the number of blocks added.
• len(s.blocks): Records the number of pending blocks.
• s.currentChannel.InputBytes(): Counts and records the input bytes of the current channel.
• s.currentChannel.ReadyBytes(): Calculates and records the ready bytes of the current channel.

Now, having clarified the process of populating channels, let’s return to the final segment of the publishTxToL1 method: the invocation of sendTransaction and the subsequent submission to L1.

sendTransaction

Fig. driver.go/sendTransaction (Source: github link)

First, let’s begin by estimating the gas fee offline using the core.IntrinsicGas method. Once we have this estimate, we proceed to create a txmgr.TxCandidate object that encapsulates the transaction data, intrinsic gas limit, and the BatchInbox address. Finally, we conclude the process by employing the Send function to generate a transaction. This transaction will send the previously created batcher transaction to the Layer 1 (L1) BatchInbox.

Closing thoughts

With the conclusion of this fifth installment, the Bedrock Wrap Up series has reached its end. Navigating the intricacies of the introduction of novel terms and concepts, including op-stack, op-proposer, op-batcher, sequencer, verifier, derivation, sequencing window, and others, posed complexity with occasional variations in naming for the same objects. A methodical deconstruction of each term, combined with a lucid explanation of the logic and relationships at the code level, proved to be essential. Despite occasional terminology overlaps, documenting relationships clarified the research landscape, fostering a more coherent comprehension.

We expect this five-part research to be valuable for blockchain researchers globally, and we pledge to continue exploring and sharing discoveries in the field of Layer 2 technologies. Thank you.

Increasing Confidence in the OP Mainnet Bridge with Two-Step Withdrawals

optimism/specs/overview.md at develop · ethereum-optimism/optimism

optimism/specs/proposals.md at master · ethereum-optimism/optimism

optimism/packages/contracts-bedrock/deploy-config/mainnet.json at develop ·…

OP Mainnet Block #109880905 | Optimism

optimism/op-proposer/proposer/config.go at develop · ethereum-optimism/optimism

optimism/op-proposer/flags/flags.go at develop · ethereum-optimism/optimism

GitHub - google/pprof: pprof is a tool for visualization and analysis of profiling data

optimism/op-proposer/proposer/l2_output_submitter.go at develop · ethereum-optimism/optimism

optimism/packages/contracts-bedrock/src/L1/L2OutputOracle.sol at develop ·…

optimism/op-batcher/batcher/driver.go at develop · ethereum-optimism/optimism

Address 0xff00000000000000000000000000000000000420 | Etherscan

Null: 0xFF0...010 | Address 0xFF00000000000000000000000000000000000010 | Etherscan

optimism/op-batcher/batcher/channel_manager.go at develop · ethereum-optimism/optimism