Handling High-Volume Transactions on Ethereum via Off-Chain Asynchronous Queuing
In our third technical blog, Adrian Lopez and Julien Déray from our back-end team take us through our current off-chain solution for pushing high volumes of tickets onto the blockchain, and how we might embrace a more decentralised approach in future.
Anyone who’s ever sat online waiting for tickets to be released knows that there can be very high, short-term peaks for ticket sales when systems are suddenly hit by a huge amount of traffic. The current load limits of Ethereum clearly present challenges for ticketing. While it is true to say that both Ethereum and Aventus have scaling challenges, it is important to note that the solutions don’t need to be the same.
At present, we’re exploring solving this challenge from two different perspectives:
- Smoothing out the flow of ticket purchases by buffering transactions using an off-chain queueing solution (the topic of this blog post).
- Developing a solution that preserves the principles of decentralisation by checking in with the MainNet (currently top secret, we’ll explain more another time).
Challenge
We’ve developed an off-chain solution which handles this challenge by holding all ticket transactions within an asynchronous queueing system and then pushing them onto the Protocol incrementally. There are a couple of requirements here:
- We want to ensure that all tickets will eventually be processed and delivered to the blockchain within a reasonable amount of time.
- We want to monitor gas prices on Ethereum throughout the delivery process, and automatically utilise this information to optimise the flow of transactions based on cost. This gives us the choice of keeping either the gas price or the pace of transactions constant.
Solution
The solution we have developed is a general, multi-purpose queueing system for Ethereum: any data transaction (with the exception of the creation of smart contracts) can be drip-fed onto the blockchain via this solution. For us, of course, these transactions are ticketing transactions; however we anticipate the adoption of this (or a similar solution) for many other types of data transaction.
We utilise a write-to API in order to push tickets to the blockchain, which can either be integrated directly into a ticketing agent’s technology platform, or carried out semi-manually via the Aventus team (i.e. the ticketing agent can send us all signed ticket-related transactions and we can push them incrementally onto the blockchain).
The solution is built on Amazon SQS, which is a highly scalable queuing system, in order to ensure that all tickets can be pushed to the blockchain in a efficient and reliable way.
In order to ensure that transaction cost is optimised, we utilise data from EthGasStation in order to retrieve the current standard and low-safe gas prices from the Ethereum MainNet in real-time.
Gas Decision Making
In order to optimise transaction flow, we needed to set some parameters around the gas prices we are willing to pay (we will go into our current calculations around this later in this article). For example, if we want transactions to be confirmed faster, we need to pay a higher gas price; if we are happy to wait and accept a lower transaction speed, our set maximum price can be lower.
If the standard gas price (ie. ~1.5 min / block) falls above our selected threshold, we use our maximum price. If EthGasStation’s lowest price (ie ~15 min / block) is above our maximum price, then we will stop pushing transactions until it goes down.
Technology Stack
- Aventus Write-to API
- Amazon SQS
- EthGasStation
- Akka: Akka is a toolkit used to write distributed, message-driven applications for Java and Scala. We made extensive use of reactive streams to model each of the operations, allowing for a more resilient and simple design for the resulting solution.
- Scala: Scala allows us to take a functional approach to our systems design, which greatly helps with modularity and reusability. It also allows us to create more elegant solutions that are less error prone.
- Aventus Protocol (built on top of the Ethereum blockchain)
The flow of transactions from creation to submission to Ethereum is as follows:
All services -> SQS (Simple Queue Service) -> KillSwitch -> PriceGate -> TicketProcessingFlow -> Delete from SQS
This is translated into Akka streams as a high level set of “pipe connections” as shown in the following code:
- As can be seen into the code, the different stages of processing are represented as flows. This high abstraction level over what is a multi-threaded process allows us to write clear and succinct code.
- ‘mapAsyncUnordered(10)(processActivity)’ : Currently we try to push tickets concurrently with 10 workers, leading to a maximum of 10 tickets per block (ie. ~10 tickets/minute).
- Both ‘KillSwitch’ and ‘PriceCheckerGate’ are designed to stop the stream’s ‘pull’ commands using custom graph stages, but will otherwise let push requests through. This way we allow messages in-flight to finish their processing and prevent only the retrieval of new messages from the queue.
Current Limitations
There are several fields of the current solution where more research and experimentation will allow us to implement a more efficient and suitable approach.
- While the gas price can be optimised, we still need to pay gas for every transaction — meaning that this queueing solution doesn’t fully offer economies of scale for the volume of transactions required.
- The current gas price calculation is a naive implementation based on EthGasStation. While this currently sufficient for the alpha version of the solution, we understand that the Aventus use-case is not the same as a regular user placing isolated transactions. We are exploring the possibility of creating a custom in-house implementation which would monitor both the Ethereum network and our system’s load in order to calculate the best price.
- The solution currently relies on a fixed maximum rate of transactions per block, which is a setting defined at the application start (currently ~10 transactions/minute). We would like to optimise this value using dynamic metrics such as gas price, our system’s relative load on the Ethereum network, or queue size.
Future Roadmap
While our current solution handles the process of pushing large amounts of ticketing transactions to the blockchain centrally, we are actively exploring decentralised solutions, both by developing internal proof of concepts and evaluating new externally-developed options.
About the Authors
Adrian Lopez is Lead Back-End Engineer at Aventus with extensive experience in Scala and functional programming. His ten years spent working in development includes a background in ticketing technology, working with ClubTickets and TicketLogic.
Julien Déray is a Back-end Engineer at Aventus, with a background in computer science spent initially as a full-stack software engineer at PDX Technology. After co-founding of early-stage FinTech start-up Mespo, Julien joined Aventus in 2017.
