Dolomite leverages Alchemy for data integrity

Alongside the launch of Dolomite, we would like to highlight our infrastructure provider Alchemy. They have been pivotal in our development and made it possible to launch our fast, secure, and liquid decentralized exchange this week.

Since our users’ trade decisions and orders are affected by the latest blockchain transactions, having the latest data is important. However, getting the latest data correctly is challenging, even when queried directly from an ETH node. When making requests from the blockchain, not all queries are sent to identical copies of the chain. Plus, reorganizations of the chain can occur. This is problematic if one node sees a transaction as mined, but a different node does not. If our queries hit different nodes, traders may see their orders matched and filled, removed from order books, then reappear on order books.

Data integrity is critical for Dolomite, but difficult to achieve without Alchemy

Today, we have Alchemy to help us always get the right data, but the path here was bumpy. At first, we used a well known free blockchain node provider but encountered problems with data integrity and instability. Then we tried running our own nodes, but our own nodes took forever to sync and crashed all the time.

When we ran into problems on the free node service, we asked for help in their support channel only to have our question buried by other customers’ questions. We would post our question repeatedly for better visibility, but their support team never answered.

Workarounds for the free infrastructure option work meant long waiting periods for users

We spent weeks dealing with inconsistencies in the data. Re-orgs have always caused issues for us, but the extent of the problem varied. Re-orgs would result in a delay of events being fired by two seconds or longer. In some cases, re-orgs would undo events like trades, only to see them re-added a few seconds later. For users, this situation would cause data to flash. The data would appear, get deleted in the undo event, and reappear later.

In our efforts to make the free option work, we wrote a system of fail-safes to check the integrity of the data at least nine times, iterating through repeatedly to ensure the blocks did not reorganize. Each time we detected a reorganization, we would wait for more blocks and check again. These sequential checks succeeded in verifying the data but took a significant amount of time, which delivered a poor user experience. Still, waiting was better than reverting trades and putting orders back on order books.

Better data from Alchemy at 270x faster speeds

When we first tried Alchemy, we tested the service stringently. We defined false positives to be any data errors due to blockchain re-orgs, fake re-orgs, and inconsistencies between nodes. Over the course of two weeks of running two identical servers side-by-side, we found that Alchemy had 91% fewer false-positives compared to the free option.

On top of better data, our queries with Alchemy now take 10ms total for us to run and check, since the data is more reliable and the performance is faster. With the free service, our queries used to take at least 2.7 seconds to process, since we ran nine or more checks that took 300ms each. We can listen to the chain much more closely, and we just lag one block behind for far less false positives. The Dolomite user experience is significantly improved in the process too.

Write code in modular and flexible ways, and avoid losing momentum from context switching

Besides finding a super reliable infrastructure solution, we want to share best practices in blockchain development. One of the best practices we’ve learned in the past year is to write code in a modular way. We can make changes once and have it propagate, rather than replicating manually everywhere.

At Alchemy’s recommendation, we now use different API keys for different parts of our application. This partitioning makes it cleaner to understand usage data from different parts of Dolomite. Instead of having all of our API calls bury one another, we can easily check up on how polling for the latest block data or submitting trades to the chain for settlement are working.

Lastly, avoid context switching as much as possible. We have already sunk hundreds of hours over the past year into checking infrastructure and detecting bad data, which equates to weeks building workarounds. That doesn’t account for the opportunity cost of time we could have spent building Dolomite. On top of that, we lost momentum whenever we had to switch back and forth from Dolomite development to blockchain infrastructure troubleshooting.

In summary, we now have a lot more confidence in our system with Alchemy. Data consistency was a critical problem that Alchemy solved for us. We can now show the right information to our users when they are looking to trade, and are excited to open up invites to our platform.

Trade on Dolomite now to be one of our earliest users, and get started here with Alchemy!

Trade on Dolomite: https://dolomite.io/

Try Alchemy’s Infrastructure: https://alchemyapi.io/

You can find us at:

• Website: dolomite.io
• Twitter: twitter.com/Dolomite_io
• Instagram: instagram.com/dolomite_io
• Facebook: facebook.com/DolomiteOfficial
• LinkedIn: linkedin.com/company/dolomiteio
• Reddit: reddit.com/user/DolomiteOfficial
• Email: contact@dolomite.io