Why Must ERC-4337 Bundlers Work with Block Builders?

In our previous article, we mentioned several issues that are currently unclear in the Account Abstraction (AA) ecosystem, including a risk in the way Bundler processes UserOperations. The risk is that if a user sends UserOperations with the same nonce to two Bundlers at the same time and the two Bundlers submit the transactions in the same block, one transaction will fail. In this case, the failed Bundler will have paid a small amount of gas cost for nothing. We call this a nonce collision. Even if it’s not a testing behavior, this situation has been happening on the blockchain all along.

In the official documentation of ERC-4337, it mentions that the Bundler is either a block builder itself or collaborates with block builders. In theory, a Bundler collaborating with a block builder can completely avoid transaction failures caused by nonce collisions. However, due to the early stage of AA development, this is not always the case in reality.

In this article, I will talk about this issue in detail as well as the current state of the Bundler ecosystem, and provide some insights into the potential emergence of the public mempool and the future of the Bundler ecosystem after widespread adoption of AA.

What Happens Exactly

To best showcase the problem, we did a test on Polygon Mumbai following these steps,

1. Prepare an already deployed contract wallet
2. Deposit enough fund for gas of multiple UserOperations
3. Construct a UserOperation with a short calldata and no Paymaster or Aggregator, to keep it simple.
4. Send this UserOperation with the same nonce to two different Bundlers at the same time. Check the EntryPoint contract history to see the transactions sent by these two Bundlers.

The table below lists some details of the two transactions. (Transaction 1; Transaction 2)

Table 1. Two bundlers send transaction with a same UserOperation

Here is what happened for both bundlers. After receiving the UserOperation, each bundler simulated it based on the current blockchain state. Each was successful because there is enough gas in the wallet contract, calldata is fine, nonce is correct. Based on the gas calculation, this UserOperation is profitable. So they submitted it on-chain in a transaction respectively.

Next, a problem occurred because the EVM processes transactions in a linear manner, so these two transactions must have a sequence, with one transaction being processed before the other. When the first transaction was completed, the EVM found that the nonce of the UserOperation was incorrect when processing the second transaction, so the second transaction was reverted. Consequently, the two Bundlers obtained completely different results. As shown in the table above, the first Bundler earned 0.00028 $Matic, while the second Bundler lost 0.0000728 $Matic. The following chart shows the workflow.

Figure 1. Flowchart of a same UserOperation processed by two bundlers at the same time.

This is just a gas loss from the failed transaction (a bundle) with one UserOperation. What if the transaction includes more UserOperation? Here is another test’s result. Following the same steps of the previous test, except each bundler sent a transaction including two UserOperations instead of one. (Transaction 1; Transaction 2). The first UserOperation of Bundler A is identical from the first one of Bundler B. To be specific, as long as they have the same sender and nonce, the problem happens. We found that the entire transaction failed, which caused the bundler to pay all the gas for nothing. Compared to the transaction with a single UserOperation, the gas increased by 5408, and the other executable UserOperation in the same transaction was not executed on-chain either. So, this transaction not only caused more economic losses to the bundler, but also caused time losses to another user.

Table 2. Two bundlers send a transaction with two UserOperations, one of which is same

The calldata is the same and pretty simple in the tests here. If those UserOperations were more complex, involved more interacting contracts or had more content themselves, then the gas loss would be even higher. On the other hand, imagine that when AA becomes more popular, Bundlers will bundle more UserOperations together. If a revert occurs, the gas loss will be even greater. The table below shows the gas of the failed transaction containing one duplicated UserOperation of increasing numbers of UserOperations in which the calldata is as same as the tests done above.

Table 3. Gas of the failed transaction containing one duplicated UserOperation

Dark Forest or Open Field

Apart from intentional testing or attackers sending transactions to multiple Bundlers, the above situation occasionally occurs on the chain. The reason is that a considerable number of Bundlers send Bundle transactions to the Ethereum mempool without MEV protection. As the UserOperation is in plaintext, when a MEV bot discovers a profitable UserOperation in the Ethereum mempool, it will directly package the UserOperation, submitting a new Bundle transaction in the same block with a higher gas price. The block-producing nodes naturally prioritize transactions with higher gas prices, making the original sender of the transaction a victim of nonce collision. In this process, the MEV bot not only increases the gas price but also consumes more gas in the end, as they have their own MEV bot contracts, making the race to submit transactions more complex. These Bundle transactions in the Ethereum mempool are like people holding torches in the dark forest, making themselves prey to predators.

Whether it is intentional by attackers or MEV bots front-running, before the public mempool of ERC4337, the above situation can be considered a UserOperation nonce collision attack. And this must be prevented. Here, we have come up with some temporary solutions when there is no public mempool yet.

Pre Public Mempool

For a more direct approach, Bundlers could proactively block the sender or IP address of UserOperation from the attackers after the revert occurs. However, there are two issues with this approach. First, banning IP addresses can lead to false positives, and attackers can change their sender and IP address. Second, the user may have unintentionally committed this action.

Regarding MEV bot, one solution is to use MEV protection against front-running, such as Flashbots, but this will undoubtedly incur some cost or sacrifice some efficiency. Another solution is to calculate the difference between the gas submitted for the Bundle transaction and the gas returned by the EntryPoint contract when submitting the Bundle transaction, keeping it at a low profitable level, making it unprofitable for MEV bots to front-run the transaction. This should be the simplest and the most direct solution in the early adoption of AA, as currently, most Bundle transactions contain only one UserOperation and there are few AA transactions in one block.

Another solution that we can think of is to introduce some randomness into the bundling process. Instead of immediately bundling and submitting the received UserOperation to the blockchain, the Bundler could wait for a random number of blocks (e.g., between 0 and 5 blocks) before simulating and bundling it. This makes the attack much less effective, as the attacker cannot guarantee that the Bundlers will bundle the UserOperation in the same block. The cost of the attack increases as the range of the random number increases. This solution may be feasible on high-speed chains, such as Arbitrum, where the block time is relatively short. However, on low-speed chains like Ethereum, where each block takes 12 seconds, the user experience may not be as good.

The above solutions are temporary before the public mempool is available. While there is a public mempool, the situation will change.

Post Public Mempool

After the public mempool is introduced, the most UserOperations are from this pool. There must be a mechanism to prevent multiple Bundlers from bundling the same UserOperation in the mempool. Etherspot, who developed the Skandha bundler client, is developing the p2p network of the mempool. According to Etherspot’s information, mempool will be available in October. The UserOperations to be bundled will be transmitted in this p2p network, and once bundled and processed on-chain, they will be marked and delisted. It is meaningless to send duplicate UserOperations to the mempool because it can be easily detected by the p2p network.

At this point, multiple Bundlers will access the same set of unprocessed UserOperations. Bundlers will start packaging more UserOperations into the same Bundle transaction. If the transaction is still sent to the Ethereum mempool, as long as one UserOperation is front-run, the entire transaction will revert. The probability of failure increases significantly. This is harmful to both the Bundler community and the AA ecosystem.

As commented from Parthasarathy, the core developer from Etherspot, the p2p interface will be piloted in a chain that supports mev-boost/block builders. This will ensure the elimination of duplicate submissions and reduce online rejections. Marc from Candide team also told us that collaborating with block builders is by design and not optional, one cannot and should not run a bundler in the p2p mempool without a service like mevboost on a public mempool on Ethereum.

In an ideal state, Bundlers after the launch of a public mempool will avoid getting caught in the chaotic competitive state of the dark forest. By combining their private mempool, Bundlers can process UserOperations more efficiently while earning maximum profits.

We believe that the development of AA will definitely go through a process of discovering and solving problems, and what we are discussing here is just one of them. After going through these developments, AA will eventually be widely adopted. As an innovative web3 infrastructure, we will continue to conduct more research and testing in the field of AA, and hope that more developers will participate in our AA community.