In our latest Code Talks, we talk all about Digital Currency Exchanges including the business mechanisms behind them and the biggest challenges they face.
PPIO Code Talks is an open platform for high-quality presentations and discussions on blockchain technology with the aim of engaging the community and spreading ideas. Previous Code Talks covered The A-to-Z on zkSnarks and Zero-Knowledge Proof, Libra & PPIO: The Genius of Libra and What We’ve Learned and Tendermint: The Inventiveness of their Consensus Mechanism. The following is an adaption of a slideshow presentation given on July 27 by Wang Boyang, on the structure of digital currency exchanges. Talks were also given by other prominent members of the Shanghai blockchain and internet industry.
I believe that no one can deny that cryptocurrency exchanges are an extremely important part of the entire digital currency and blockchain ecosystem. It has even been said that exchanges reside at the top of the entire industry in that they have the most resources and the highest returns. To illustrate this point, the annual revenue of the top five digital asset exchanges in the world is worth more than 1.4 billion USD.
In the face of huge profits, it is inevitable that exchanges will compete with each other. With so much competition, in addition to marketing innovation, exchanges must constantly improve their technology and enhance the overall strength of their platform to succeed. Without advancements, the platforms will suffer whether its congestion caused by excessive volume of transactions, hackers attack, or malicious transactions by trading teams.
Today, we will provide a close analysis of the architecture of the digital currency exchange and how their business model is implemented from a technical point of view.
Exchange Overview
Taking cues from stock and commodity exchanges, and the rising volume of cryptocurrencies, digital currency exchanges emerged. Whether it is buying and selling stocks, futures, commodities, or cryptocurrencies, the main technical logic and business logic behind them all are almost the same. The only subtle difference is that the digital currency exchange’s unit accuracy is higher, and the way to recharge and extract tokens is slightly different from traditional exchanges.
At the same time, a crypto exchange is a business that emphasizes back-end operation and maintenance. Most exchanges tend to have more non-technical teams than technical teams.
Above is the backend OA system in an exchange. It includes some basic functions such as identity authentication, user management, wallet management, recharging token, extracting token, etc. It also features essentials for a functioning ecosystem: checking, clearing accounts, adjusting rates, currency equivalent trading functions. Finally, it includes auxiliary functions such as document configuration, rotation map configuration, and other marketing aids.
The Business of Exchanges
The real problem that the exchange essentially solves is to realize the purchase and sale of orders by users, how to maintain the volume of transactions, how to match the transaction of orders, and finally, to complete the liquidation and transfer.
More specifically, what currency pairs are included in the data structure, at what price, and what currencies are purchased for each order placed by the user. After that, the transaction order will enter the system, and the user’s purchase order will have a corresponding sell order for the matching transaction. If it cannot be combined, the purchase order will become a pending order. If a transaction can be made with another user, then liquidation and transfers will be carried out.
For example, A trades ETH with B in exchange for BTC. In this situation, A gives ETH to B, B gives BTC to A. There are four transactions involved. A reduces ETH, B increases ETH, A increases BTC and B reduces BTC. At the same time, A and B pay corresponding fees; and through system optimization, two fee transfers can be merged into one. So a complete transaction takes at least five transfers. From the point of view of transaction link, it is relatively simple. But when A’s unit volume is large, it will trade with many people at the same time, which presents a more complicated situation. In addition, individual areas are also very complex, such as state management and maintenance, or how to deal with performance problems when there is a large amount of data. Because of the high requirement for data correctness, it is necessary to ensure that even when the server and other hardware are down or need to be restarted, the user’s assets remain correct.
Of course, this is also a trade-off issue. In order to improve performance, some exchanges allow a small number of errors in the process of trading. The exchange then compensates the users to achieve the required volume and to ensure efficiency when trading. For example, in the past, a kind of transaction that occurred in individual exchanges was mining activity. Users could get a platform currency by trading. For users, the larger the number of funds they held, the more platform incentives they got. That is to encourage users to recharge their funds to the exchanges and increase the volume of transactions. At the highest time, the transaction volume reached 10 billion US dollars a day, and the turnover rate even reached 1000% or higher, therefore, in order to handle such a large volume of transactions, the system should be fault-tolerant. For example, some head-end exchanges can also trade between $2 billion and $3 billion a day, about 10,000 units per second, so the overall throughput is relatively large.
The three main steps in its business model include:
1. Order sales
2, Crosslinking
3. Clearing and transfer
Among them, the order types include:
- Limit order
- Market order
- Stop-loss order
- Termination limit order
- Buy or terminate a transaction in full
- Order or cancel immediately at the specified price
- Attempt to place an order
Among these many order types, except for the types of orders that are common to our traders, there are some that are more suitable for quantifying traders to run quantitative strategies.
Joint Transaction
For a variety of different orders, the system first determines the order of the orders based on price and time priority. For example, if A and B have placed an order, and the price offered by A is better than B, regardless of the time, A takes priority. If A and B place an order and the price of A is equal to B, the person who placed the order first will take priority. Therefore, the priority of the transaction is based on the price priority. When the price is the same, the transaction is completed in chronological order. The system will sort the orders according to the order of arrival, and then perform the matching transaction according to the order.
The depth of the transaction between the system and the exchange is also closely related. After a deep match, a series of clearing accounts are generated, and the transaction is completed.
A matching service is a state machine with strict definitions of input and output. The matching service uses the design of the state machine, so in the input module, only one order is processed at a time. The state itself is strictly based on the current pending order volume, and the price of the buy and sell orders once sorted. The output is the result of clearing, therefore the combination is the state machine defined by the input and output.
What are the advantages of this design?
1. It operates purely on memory so there is no need to read the cache, let alone the library. The design allows you to save all data yourself, so the transaction speed can be done very quickly.
2. If there are any service crashes server problems, message queue problems, etc. during the operation, you can still produce the same output if you later enter the same data.
This not only ensures the efficiency of matching, but also ensures the correct result when matching. Later, we’ll look at the implementation of matching at the code level.
The Technical Architecture of Exchanges
Transaction Module
The technical realization of the trading module is Order, Matching, Trading, and Liquidation. During the transaction process, different forms are generated. Within these different forms, the information of each link is recorded.
The Orders table records all pending order information. Among them, ‘charge Quote’ means whether only the denominated currency is accepted. Major mainstream exchanges may have the choice to not only collect valued currencies but also to collect valued currencies for smaller exchanges, which makes the calculation of handling fees more convenient. Something like ‘sequences index’ is the label for the specified order service. For example, if we need to establish 32 ordering services, one currency can only be ordered in one service.
The Order Sequences table records the results of pending ordering. The table ensures each ‘id’ and ‘previous id’ are unique from others.
Match Details matches each pending order information, including the user’s account, price, handling fee, etc.
The Account Flows table records the clearing and transfer between the accounts of the order. All information, including fees, and the address of the handling fee will be abstracted into account numbers in for transfer clearing.
Blockchain Module
The blockchain module is mainly reflected in the charging and withdrawing of all cryptocurrencies on the exchange.
When the coin is requested, the exchange will scan the block data, obtain the information of the coin, decide whether it needs to be audited according to the exchange’s own strategy, and then charge to the user’s address. In addition, the user raises the coin. Under normal circumstances, the exchange will conduct the currency review. After the confirmation is correct, the exchange will broadcast the transaction, then the exchange’s hot wallet pays out the money.
The exchange’s management of private keys is also an important link that will directly affect any issues related to the exchange’s financial security. The private key management of a hot wallet is a “vault” which is mainly for multi-signatures because it supports multiple chains; therefore does not use the original multi-signature of BTC and ETH. Cold wallets can be imported into the hardware wallet, or can be managed by some industry hosting solutions, such as Cobo wallet.
Challenges
Exchanges encounter several technical issues; here are the biggest challenges they face:
- Large amounts of data. As mentioned, the transaction volume of the exchange will be very large, plus there is additional data for common database additions, deletions, and changes to the operation optimization, such as sub-library, sub-table. It is common for sub-tables to be processed by date. Because of the large amounts of data, exchanges have historical records that are not unlimited. Records can only be found within a few months and cannot be queried later.
- The correctness of data. In all cases, the correctness of account reconciliation and asset data should be guaranteed.
- Safety. If you want to be absolutely safe, it may mean that the system is relatively difficult to use. As there are many attacks on the exchange, exchanges must constantly optimize, or even employ security audit agencies to help increase security.
- Front-end performance and state management. As day trading is common, there will be users logging in for long periods of time in order to trade. As a result, memory management must ensure that there is no memory overflow in all links, and a large amount of data must be structured with an upper limit. The exchange must also be able to configure activities such as marketing.
In regards to decentralized trading, before there were many projects did decentralized exchanges on the ETH public chain, such as IDEX, which is relatively large in China. However, ETH is characterized by relatively slow speeds. There are also individual decentralized exchanges that optimize the process by ordering off-line and clearing on-line. Then there are public chains that run relatively fast, such as EOS. Decentralized exchanges on EOS will be relatively easier and begin to mature. All matching, all orders directly on the chain. At the same time, because all the data are directly linked, the exchange needs to take into account the cost of RAM, CPU, net, the use of storage resources and so on. But the advantage is that it directly replaces the ordering function of centralized exchanges. At the same time, all matching is open, which is more fair and open for traders. The problem is that only EOS-based tokens can be traded. If other tokens are traded, cross-chain support is required. However, if the current cross-chain technology is to be used in a large number of transactions, there will be unsatisfactory performance or convenience.
In this article, we look at the exchange industry, their business and technology implementation, and some challenges the exchange industry faces. If you have any questions, leave a comment below and we’ll answer them. Get trading!
Want more? Join the PPIO community on Discord or follow us on Twitter.
