I don’t care if I get sandwiched, bigger things are coming to MEV
The inner workings of MEV and how the industry is shifting
Over the past couple of months, extracting MEV has become increasingly challenging for searchers.
And, yes, the tooling has gotten better, so it seems like the technical hurdle is finally gone, and that we can all compete as equals. But, as easy as it has become for anyone to jump on the bandwagon, the competition to stay on top is getting fiercer than ever.
You’ll probably have experienced a situation where you spend a couple of nights without sleep, and the EVM simulation results start showing multiple profitable sandwich/arbitrage bundles on your console:
🥪🥪🥪🥪🥪🥪🥪🥪 SANDWICH OPPORTUNITY SPOTTED 🥪🥪🥪🥪🥪🥪🥪🥪
- Optimized amount in: 1.27463 WETH
- Potential profit: 💎💎💎💎💎💎💎💎
🥪🥪🥪🥪🥪🥪🥪🥪 SANDWICH OPPORTUNITY SPOTTED 🥪🥪🥪🥪🥪🥪🥪🥪
- Optimized amount in: 1.27463 WETH
- Potential profit: 💎💎💎💎💎💎💎💎
🥪🥪🥪🥪🥪🥪🥪🥪 SANDWICH OPPORTUNITY SPOTTED 🥪🥪🥪🥪🥪🥪🥪🥪
- Optimized amount in: 1.27463 WETH
- Potential profit: 💎💎💎💎💎💎💎💎And you’re thinking,
‘Ah, all those nights just for this. I’m the king now.’
But, soon after, you’ll find out that all those bundles are sandwiched by the EigenPhi top searchers. And whichever way you look at your bundle and compare it to theirs, you’ll realize that you just can’t win: speed-wise, gas-wise, bribe-wise. If everything worked in reverse, you’d win all the time.
And now, you’re thinking,
‘Ah, all those nights just for this? I’m in pain now.’
(Of course, in this case, you’re probably looking at a short-tail opportunity where competition is very fierce, so you need to look further out to less competitive markets. We’ll cover that topic in a later post.)
Today, I would like to talk about how the MEV industry is shifting, and what we can do to stay on top as searchers in the forthcoming bull market. By understanding how the tides are shifting, we can find a way to compete with the top searchers and be a gentleman (quoted directly from the industry star libevm):
Keep reading if you don’t want to miss out on the big wave 🌊🌊🌊
Table of contents
In this two part article series, we’re going to cover the following topics:
📍Part #1: How the MEV world is evolving
- A look inside the public mempool
- Understanding what block builders do
- Private order flows: What they are, and how we can use POFs to win more
📍Part #2: How to win as a searcher now
- Neo-Subway-rs: Building a decent sandwich bot
- Theorizing how we can win more
- IDCMEV: A novel way of submitting your bundles/private transactions
To truly grasp the trajectory the MEV space is taking, let’s take a step back and delve into the origins and growth of the MEV industry, understanding how it has evolved into its current state.
After reading this article, you may find yourself thinking:
‘I don’t really care if I get sandwiched either. There’re bigger problems now.’
In the next article, I’ll provide everyone with a highly efficient toy sandwich bot, offering a hands-on experience of how all the pieces fit together. Additionally, I’ll introduce a new service currently in the beta phase — testing internally with selected MEV searchers — called “I Don’t Care About MEV" (https://idcmev.oopy.io/). This service aims to assist both users and searchers in submitting their bundles like seasoned pros.
Please feel free to join our community and discuss the future of MEV together:
I know we’re all excited. So let’s go! 🔥
1. A look inside the public mempool
It all began with a pending transaction.
🌐 1st gen MEV: Mempool monitoring
It could have been a Uniswap order transaction or any other - who knows. But people quickly found out that these orders can be monitored through the public mempool, and that they can play around with gas fees to land their transactions either in front of or right behind the target transaction.
How is this possible? Because block building was largely dependent on transaction ordering based on gas prices. Suppose there are two different transactions with the set gas prices as outlined below:
In this case, since TX #2 is paying 1 gwei more than TX #1, it will be preferred over TX #1 when block building. And since there’s something close to 12 seconds before a new block is built, you have enough time to fix gas prices to ensure you can land in front of (frontrun) or in back of (backrun) the target transaction.
The first generation MEV searchers played around like this under the radar for a bit, but word got out, and a lot more people wanted to join the fun.
⚔️ 2nd gen MEV: Gas bid wars
This is when gas bid wars emerged.
Going back to the example above, let’s assume now that another party joins the space who also has access to the same mempool data.
This time, the two MEV searchers are looking to frontrun the same target transaction (TX #1) that’s paying 20 gwei.
The two searchers submit TX #2-a, TX #2-b respectively with different gas prices (21 gwei, 22 gwei), because they initially don’t know about the other party.
Then the transaction ordering would look something like this:
However, this causes a gas bid war among multiple different searchers.
Searchers will constantly monitor the mempool for competitor’s transactions and bid higher gas prices so that they can land on top of the competition.
🤖 3rd gen MEV: Flashbots and the private mempool
Enter Flashbots.
Flashbots took the initiative and started providing two essential services to MEV searchers:
- Private mempool
- Bundles
These two concepts mark the beginning of a new era, so it is worth going over.
Private mempool
Unlike the public mempool, people could now submit private transactions to Flashbots. The difference was that the public mempool was on the blockchain network, so anyone could access the data. This accessibility posed a risk of frontrunning or backrunning.
However, the private mempool operated on a separate server managed by Flashbots, ensuring that the data remained hidden from the public eye. Those who sent private transactions to Flashbots no longer had to worry about MEV.
And the solution was quite ingenious except for the fact that it didn’t really solve MEV.
The emergence of Flashbots was supposed to remove sensitive public transactions flowing through the mempool, but it certainly did not.
A lot of transactions are still sent through the public mempool as you can see below:
❗Quick quiz❗
Q: Why do you think that people are still submitting to public mempools? Is it because they don’t know about MEV and Flashbots?
I don’t believe that’s the case anymore. I’m fairly certain that people are now aware of what MEV is. But, of course, this doesn’t imply that all of the public transactions are being submitted intentionally to the public mempool.
If you monitor the mempool data for a while, you’ll notice that most if not all public transactions are being picked up by MEV searchers and are thus confirmed within the next block.
What if these people are intentionally sending public transactions so that they can land safely in the next block. If you’re willing to take on the slippage costs, doing so would make sense.
Bundles
Another ingenious feature we could get with Flashbots was something called the bundle.
Bundles ensure that the ordering of certain transactions are fixed.
What does it mean to ensure transaction ordering?
Let’s go back to our example where transactions are ordered as such:
In this scenario, as a result of gas bid wars, the two frontrunning transactions will both land in front of TX #1. However, this can be problematic, because the two transactions can shift the token price too much and make TX #1 revert. Searchers need to make sure that the transaction they are frontrunning, backrunning does not revert, so that they can profit from this MEV.
Bundling will help with that and make the competition look like this now:
Now the two different searchers batch the transactions up into a bundle and submit the bundle to Flashbots.
The bundles will revert if the ordering isn’t kept. But as you can see, TX #1 can only be used once, so only one bundle will survive and land in the next block.
This new type of transaction processing mechanism introduces another war called the: bribe war, which we’ll look into in the next section.
2. Understanding what block builders do
⛓️ 4th gen MEV: Block building and bribe wars
A word that comes to mind when considering private transactions is: centralization.
Indeed, removing public order flow from the blockchain is beneficial to a certain extent as it can help mitigate MEV by ensuring user privacy. However, this comes at the cost of centralization, creating a catch-22 situation, which is quite interesting.
How can we determine if our private transactions or bundles are being handled fairly once they’ve been sent to a private mempool? The answer is: we don’t know.
So, the next obvious step was to open up the block building space to anyone willing to participate. And that’s exactly what happened.
Right now, validators are responsible for both:
- block building,
- block proposing.
But most validators are not capable of building very profitable blocks, thus, they outsource this task to block builders.
Block builders will use a series of optimization techniques to build better blocks and hope that the next validator will propose their block as the next block.
I know all this is quite vague, so we’ll try to visualize what block building is by building one ourselves.
🚀 Let’s call our service: Solid Builder. We want to directly compete with Flashbots and build better (more profitable) blocks.
Let’s go back to the original example using the pending transaction TX #1 from the public mempool. Again, there are two searchers competing to use the same pending transaction (TX #1) within their bundles:
Searcher #1 submits Bundle #1, and this searcher wants to frontrun TX #1 and pays out 0.01 ETH as bribe to the block builder.
Searcher #2 submits Bundle #2, and this searcher wants to sandwich TX #1 using TX #3-a, TX #3-b and pays out 0.011 ETH as bribe to the block builder.
With this setup, we can understand that Searcher #2 is probably a bit more profitable so is able to pay the block builder more bribe. So builders will select Bundle #2 for that reason.
In a world where only one block builder exists, Bundle #2 will most likely win.
The power of searcher + builder
However, the dynamics have shifted a little as a result of the new player in town, Solid Builder. Now searchers have the option to send bundles to either one or both of the existing builders.
Let’s say for example that Searcher #1 only sends to Flashbots, and Searcher #2 only sends to Solid Builder, because Searcher #2 is actually Solid Builder.
In that case, Flashbots will build a block with Bundle #1.
And Solid Builder will build a block with Bundle #2.
The end result is that the second block is more profitable, so Bundle #1 will fail to get included and Flashbots will lose the block bid.
Now, the searcher + builder identity, Solid Builder, can earn profits on both ends:
- MEV searching,
- block building.
These identities won’t use a simple sort by bribe method to select the most profitable bundles. For them, the profit is calculated as: MEV profit + block bribe. So it’s likely that their own bundles will get selected over other searchers’ bundles.
Searcher reputation
Another way to maximize block value is to score MEV searchers by historical profitability and prioritizing their bundles over others. This is done by using what’s known as the searcher reputation:
This is quite necessary, because if reputation didn’t matter, searchers would blindly submit bundles and clog the network with their requests. Builders are bound by computational power so they can only simulate a certain amount of bundles prior to building their blocks. And they will penalize searchers that harm their networks in any way for that reason.
The implementation logic will slightly differ for all builders, but builders will essentially keep a queue connected from the RPC endpoint to an in-memory pool of yet-to-be simulated bundles like so:
From here, they will sort the bundles by:
- Time: first come first served,
- Priority: higher priority (reputation)
and start simulating them to see if they’re profitable.
Some people say time isn’t a factor, but I think that’s not the case. The fact that builders are bound by computational power is an indication that the faster you submit your bundles, the more likely it is to be considered a “candidate” into the next block.
To construct the most profitable block, builders have to take a step further and solve a knapsack problem: how should we select and order these bundles so that it can maximize our profits.
By generating numerous combinations of transactions and bundles, builders will run simulations on the entire combination to see which one will increase their ETH balance the most.
As you can see, block building is a strategy all on its own.
Hopefully, all this made sense and helped shed some light on what block builders are doing.
There’re some really good resources to help you get started if you’re willing to start a builder business:
Also, I recommend this read for anyone interested in block building:
3. Private order flows: What they are, and how we can use POFs to win more
🤝 5th gen MEV: Private-Order-Flow wars
Reading about block builders will leave you thinking that it’s important for builders to:
- build a big community to gain access to more bundles used in block building,
- know powerful MEV searchers so that their bundles are worth more.
This implies that “access to private order flow” is the most crucial aspect of block building. Builders should either have access to a large pool of unique searchers or to competent searchers.
If every builder had access to the same transactions (public order flow) and the same searchers with the same optimization techniques, then builders would win blocks in rotation. This would put a halt to the evolution of the entire MEV supply chain. Fortunately for us, though, the landscape is very dynamic in nature.
So the fight for private order flow is an ongoing trend that can be seen everywhere.
There are:
Semi-private mempools
This is a method of providing users with MEV protection by returning certain amounts of MEV opportunities that they generate. Users can submit their transactions to semi-private mempools, where they can disclose some information about their transaction and permit searchers to backrun their transactions.
If a searcher successfully backruns the transaction, that searcher is obligated to refund the user for facilitating that opportunity. By structuring semi-private mempools in this manner, it creates a win-win scenario for all parties involved.
Below are some popular semi-private services that users and searchers alike can utilize:
- MEV-Share (Flashbots)
- MEVBlocker (CoW DAO)
- BackRunMe (bloXroute)
Intent-based swaps
Another effective method for eliminating MEV is to establish an open marketplace for matching orders. These orders, which come with objectives such as:
- “I want to buy token A with x amount of WETH,”
- “I want to sell token B and get y amount of ETH,”
are referred to as ‘intents’ as they represent the users’ intentions.
This concept is akin to limit orders on centralized exchanges (CEX), but with a slight difference — there is competition among searchers or ‘solvers’ to fulfill the intents in the most optimal way possible. The individual who accomplishes this task most effectively will execute the users’ orders.
Below are some popular intent-based swaps that offer MEV protection by utilizing private order flows:
- CoW Swap
- Uniswap X
- Limit order protocols (some people won’t consider limit order protocols as intent-based swaps, but I don’t know, they all seem alike to me)
Feeling dizzy? Overwhelmed by the amount of information to study? Don’t worry, because you’re not alone. We’ll delve deeper into semi-private services and intent-based swaps in later articles because they’re quite interesting. But for now, just know that there’s a whole community of product builders working towards achieving the holy trinity of MEV:
A world where everyone wins: users, searchers, builders.
Merry Christmas everyone! 🎄
I’ve been away for a couple of months, working on a product I really love with inspiring people, and I was able to learn more about MEV in the process.
I hope to keep creating quality content so that we can all learn a thing or two about MEV and truly have fun together.
Feel free to contact me through Twitter, or X… for anything:
