Arbitrum Stylus vs. Solidity: Performance Benchmark
I — Context
We aim to evaluate how Arbitrum Stylus improves the smart contract performance compared to the Ethereum Virtual Machine (EVM). To achieve this, we propose using the case of a Central Limit Order Book (CLOB) product. This is a real-world example that requires a high number of transactions for market making and is therefore very performance-sensitive.
Arbitrum is a layer 2 chain built on top of Ethereum and is recognized as one of the leading chains in the first stage of rollup by l2beat.com. This advanced rollup stage makes Arbitrum the most suitable L2 solution for hosting CLOB applications that are both performant and cost-effective, while maintaining EVM compatibility (Ethereum virtual machine).
The Arbitrum foundation has recently introduced Arbitrum Stylus, an SDK that enables developers to write EVM-compatible smart contracts using Rust. By leveraging Rust’s performance capabilities within the EVM environment, we can potentially achieve faster and more cost-effective results compared to Solidity Smart Contracts, as Rust operates at a lower level than Solidity.
II-Premise
Arbitrum Stylus claims significant gas cost savings in several areas, making it an attractive option for developing high-performance and low-cost smart contracts.
With memory costs being 100x-500x cheaper, compute costs being 10x-50x cheaper and potential storage costs being also cheaper (according to the official documentation), Arbitrum Stylus presents a substantial advantage over Solidity smart contracts. While there are minor overheads when calling the contract functions, ranging between 128–2048 gas units, the overall reduction in fee consumption makes Stylus an interesting choice for developing an ERC20-compatible CLOB that outperforms Solidity-based CLOBs.
III — Naive Translation Benchmark
A — Introduction
In this section, we will perform a naive translation of the Dexalot Solidity smart contract into a Stylus smart contract without any optimization. Our goal is to determine whether Stylus can achieve the performance benefits indicated without the need for any specific optimization.
B — Red Black Tree Benchmark
The core of a Central Limit Order Book (CLOB) lies in its data structure, which is often based on a Red Black Binary Tree. This self-balancing tree ensures that orders are always maintained in the correct sequence, with the lowest price as its leftmost child and the highest price as its rightmost child.
To accurately compare the performance of Stylus Smart Contracts and Solidity Smart Contracts, we will conduct a benchmark test. This test will involve using both types of contracts to implement and interact with a Red-Black Binary Tree. Both contracts will be deployed on Arbitrum Stylus. This approach will allow us to evaluate the performance differences between Stylus and Solidity in the context of a Central Limit Order Book (CLOB).
Since this is primarily a storage-based contract, we don’t anticipate significant performance gains from using Stylus. However, we still expect to see some reduction in gas fees or improvement in execution speed.
Upon conducting the first benchmark test, we observed that the Stylus Smart Contract was both slower and more expensive than the Solidity Smart Contract in terms of gas consumption, with an average of 25% higher gas consumption when inserting 2000 nodes in batches of 150 nodes at a time.
To better understand the reasons behind this performance gap, we will now take a closer look at the code of both contracts and analyze potential causes for the discrepancy.
One key difference is the use of maps in both languages, with read/write methods to these maps being very different in Solidity and Stylus.
C — Map insertion benchmark
To validate our assumption regarding the potential cause of the discrepancy in performance, we need to conduct a test just on the insertion of elements in the map.
We will create a simple function in both Stylus and Solidity that adds an element into a map, and then compare the gas consumption of both implementations. This will help us determine whether the difference in the map access methods is indeed responsible for the observed performance gap.
fn batch_insert(&mut self, limit: U256) {
let mut start = U256::from(0)
while start < limit {
let mut ref_index: stylus_sdk::storage::StorageGuardMut<'_, stylus_sdk::storage::StorageUint<256, 4>> = self.order_book_map.setter(start);
ref_index.set(start)
start = start + U256::from(1);
}
}In Stylus we cannot directly modify value within a map without using the methods .setter() and .set()
function batchInsert(uint256 limit) external {
uint256 start = 0;
while (start < limit) {
test[start] = start;
start = start + 1;
}
}After conducting the test with the simple function to add an element into a map, we once again observed that the Stylus contract was more expensive than the Solidity contract in terms of gas consumption. At first glance, we could think that this difference is just due to the overhead gas fees that are spent at each function call.
But with a closer look, the difference in gas consumption between the Stylus and Solidity contracts is not only attributed to the overhead gas price. The observed divergence in gas consumption suggests that there is another factor contributing to the higher gas consumption in the Stylus contract, causing the difference to exceed the maximum overhead gas price of 2048 units. This indicates that there is a more significant discrepancy between the two contracts than just the overhead gas price.
Based on the observations from our tests, it appears that the issue is indeed related to the way we access the map in the Stylus contract. The use of the setter() and set() methods in Stylus seems to be contributing to the higher gas consumption compared to the Solidity contract.
III — Optimized version Benchmark
A — Introduction
Now that we have determined that the naive translation from Solidity to Stylus does not provide gas savings or performance improvements, it’s time to leverage the power of Rust to optimize the implementation of the Red Black Binary Tree in the Stylus contract. By using Rust’s low-level control and performance capabilities, we can potentially achieve significant improvements in both gas consumption and execution speed.
B — Zero Copy implementation
The goal of implementing a zero-copy approach for the Red Black Binary Tree is to load the tree only once, work with references, and then save the changes back to the tree. This approach ensures that the data being inserted into the tree is not copied or duplicated during the insertion process. Instead, the original data is directly referenced and inserted into the tree without creating any intermediate copies.
The zero-copy approach has been implemented by well-known CLOBs such as OpenBookV2 and Phoenix, which operate on the Solana blockchain and utilize Rust smart contracts. By avoiding unnecessary data copies, the zero-copy strategy can help reduce the amount of memory and processing required for inserting new nodes into the tree, resulting in improved performance and efficiency.
After implementing a zero-copy approach for the Red Black Binary tree based on the simple version of the Phoenix order book, we encountered a limiting issue during the orderbook initialization phase.
We found that initializing a tree of size 20 max resulted in excessive gas consumption, with a spike of 1,700,000 units of gas consumed during the initialization process. A bigger tree size led to the contract crashing.
#[external]
impl ClobTree {
pub fn init_market(&mut self) {
let bids = RedBlackTree::<FIFOOrderId, FIFORestingOrder, ORDERS_LEN>::new();
let market = FIFOMarket { bids };
let bytes = bytemuck::bytes_of(&market);
self.market_bytes.set_bytes(bytes);
}
}The issue resides in the .set_bytes() function provided by Stylus that can process only 32 bytes at a time due to the EVM nature that can process 256-bit words at a time, which results in multiple calls to the .set() function and that is causing this spike in gas fees, whereas on the SVM ( Solana virtual machine ) we can load all of the tree at once.
The zero-copy strategy using the simple version of the Phoenix order book was found to be unfeasible for implementing the red-black binary tree in Stylus due to excessive gas consumption during tree initialization. The maximum tree size that could be initialized in Stylus was only 20, whereas Phoenix’s tree has a size of 1048.
IV — Conclusion
Based on the results of our benchmark, we can conclude that Stylus smart contracts, in their current state, may not be the best fit for storage-heavy contracts due to the way they handle memory reads and writes. Additionally, we found that Rust-native approaches, such as the zero-copy strategy, are also limited in the context of Stylus because of the need to comply with the EVM environment.
Our findings were confirmed by the developers of the Stylus SDK, who acknowledged that the project is still in its early stages. They informed us that further development is planned to address the gas consumption issues with storage-only smart contracts.
