How to be sure that your transfer is complete
Different approaches to solving the atomicity problem
Imagine you would like to conduct a transaction, but you are not 100% sure whether it will be fully executed and reach its final destination or not. Also, if there would be a probability that an operation could be recorded partially or any of the parties could lose their money. We believe that no one would be willing to use such a network.
Power failures or event failures could lead to serious problems in databases. To guarantee validity each database transaction should meet four main criteria. This criteria form so called ACID model which means that transaction must be A-Atomic, C-consistent, I-Isolate, and D-Durable. Today we want to go into one of this properties — Atomicity.
So what is an atomic transaction in the first place? It is a database (DB) transaction that can be either executed in whole or not at all — never partway. But why use these transactions?
Let us first unravel the problems we approach, taking as a case in point a distributed database (DB). Assume for the moment that we want to change some data. Should the DB not support the atomic changes, it may result in one part of the DB having data inconsistent with the other part. Unaware of the changes, a casual user will see alternative data, and be unable to tell which are correct. However, if the DB does support the atomic transactions, any changes may be executed in two ways only: either everywhere or nowhere. We’ll begin by digging into the problem with up-to-date distributed databases and then turn to off-chain implementations.
Conventional Solutions
Multi-phase commits are in considerable current use with the aim of atomicity in mind. The two-phase commit protocol is the most common, but one-phase and three-phase commit protocols are also widely accepted.
One-phase commit is the most straightforward way to accomplish atomicity; however, it is crude and inefficient. The transaction manager sends and the participants execute the instruction to make changes. Cleary, this model has too many inherent threats and pitfalls, e.g. a participant may fall out and miss the commit when they come back.
The two-phase commit is more comprehensive as each transaction is split into two phases. First, the transaction manager queries each participant to determine whether a transaction should be committed. They create the necessary provisional entries (make allocations, in the case of multi-hop payment systems) and vote to commit. If the manager receives the response “Yes, I am ready to conduct payment” from all participants, he sends them the commit request, but should even one respond “No” or not respond at all, the manager revokes the payment. This version is much more robust and secure than the first one but hardly airtight. Indeed, if a participant rejects the transaction, the entry (allocation) remains there and cannot be deleted until a roll-back instruction from the manager has been received, resulting in a network lock-up.
To close this gap, the three-phase commit version has been developed, splitting the first stage in two. The manager also starts with querying the participants for their votes to commit, but the prepare instruction is postponed until the affirmative responses from all participants have been received. Then the participants create entries (make allocations) and confirm their preparedness for the next stage. Like in the two-phase commit, the last phase is performed only upon receipt of all confirmations.
This is about it in terms of conventional solutions. One cannot say for sure which version is better. While three-phase commit has a higher security level, it underperforms because of more phases and messages to exchange.
Atomicity in a single DB node is also worth mentioning. It is implemented with the help of a feed-forward ledger. Basically, whenever the user requests a transaction to be reflected in the DB, the entry is first made durable (in case of any low-level problems, e.g. blackout, the change is still there when the operation is resumed), then written into the disk ledger. It gives us two possible outcomes in the event of a system fault halfway through the process. If the ledger entry on the disk is missing, the transaction can be rolled back. If it is there, the transaction can be restored from the disk upon restart.
Atomic Transactions in Off-Chain Networks
The number of Bitcoin users has been growing exponentially since the cryptocurrency’s appearance in 2009. Respectively, commissions and transaction latency have increased too. The scalability problem impelled the community to aggressively search for a solution. Their efforts brought into existence the Lightning Network that is a layer 2 solution and implements payment channels and multi-hop transactions. Atomicity was also required to prevent the loss of funds through somebody’s fault. A variety of specific solutions which emerged is described below.
HTLC
To support atomicity, up-to-date solutions are mostly using HTLC (hashed timelock contracts) that allow for spending funds upon presentation of the original secret before a preset timelock. To explore more on this, let’s look at the Lightning-based transaction process flow. First, the receiver node generates the secret and calculates its hash. The hash then is sent to the sender node as the basis for HTLC generation. The sender generates the contract and sends it to node1, the next node on the route that creates a new contract (using the same hash) with the decremented timelock. This newly-generated contract is further sent by node1 over the route to node2 that repeats the actions and decrements the time-lock again. It goes all the way to the receiver who signs off the funds spending (unlocks payment) using its own secret generated at the beginning and receives the money from the node that sent the contract. This action, in turn, reveals the secret to the node closest to the sender, empowering it to unlock the payment and get the funds, thus revealing the secret to the preceding node. The process is repeated over the route to the node closest to the sender and, finally, everybody is paid, and the payment is completed. Hence, the only way for a node to lose funds is its inability to sign off funds before the timelock, e.g. by going offline after having received the secret.
The drawbacks to this solution are in regards to occasional disagreements about the contract timelock expiration and the loss of funds by a customer due to the concurrence of unfavourable circumstances.
HTLA
Interledger is an open protocol suite for transfers through various ledgers. It is this project’s white paper where the use of notaries is proposed. To support atomicity, it was suggested originally to use Universal and Atomic modes.
Under Universal mode, Interledger operation of atomicity is provided by HTLA, which is essentially an HTLC modification. The difference is that HTLA can be utilized even if the blockchain does not support HTLC as various connection types are supported including Conditional Payment Channels (with HTLCs), On-Ledger Holds/Escrow (using HTLCs), Simple Payment Channels, Trustlines, etc.
To put it differently, if a cross-chain payment passes through a blockchain not supporting HTLC, the connectors (special Interledger nodes that are responsible for routing) can replicate it using alternative methods, so that all contract provisions (e.g. payment time, amount, payment unlock conditions) are met.
PM
Sprite channels is a project aimed at the development of a new payment channel version to resolve certain Lightning problems related to atomicity, partial deposit and funds write-off from a channel.
The HTLC has been significantly upgraded by adding the preimage manager (PM). The developers’ idea was to make the PM a sort of arbiter for HTLC and delegate the decision-making on contract expiry from any individual node to the respective software. This prevents situations where a participant goes offline and loses their funds. The arbiter should be a regular Ethereum (or any other blockchain) smart-contract registering statement like ‘X preimage of H hash has been published on blockchain before expiry’.
Sprite channels should also have a unified contract expiry time. If a preimage has been timely published, all disputes are accepted. Logic excludes situations where somebody has received the funds whereas somebody else has noted the expiry time is the same for everybody. If, however, the preimage is published at the wrong time, no payment can be disputed.
HTLR
Celer Network is a solution for scaling public blockchains and maximising their performance through off-chain technologies. Here the PM has become a hashed timelock registry (HTLR) with mainly the same features. HTLR has two dependency endpoints, IsFinalized and QueryResult. The former returns whether or not the preimage has been registered earlier than the block number, and the latter returns whether or not it has been registered at all. The two features can eventually be merged. It should be noted that HTLR is always on-chain.
Notaries
Interledger should have used notaries in atomic mode. A payment conducted through the notary would look almost like a payment in Lightning with HTLC. The only difference is that before revealing the secret, the receiver node would transfer the contract for verification to notaries, the special entities selected randomly from their general register. The idea is to allow the sender to set the number of entities to verify the payment and the acceptable number of malicious notaries (within 30%). The notaries would have to vote on payment approval on the basis of BFT consensus. If everything was OK, they would ‘stamp’ the transaction, thus enabling the receiver node to unblock the funds. The concept looks good in the white paper but appears hard to implement cross-chain. Moreover, it requires that users should trust notaries.
Observers
This role is present in the GEO Protocol concept which offers a unique approach to resolving the atomicity problem. The project team is creating a decentralized peer-to-peer off-chain network that allows for exchanging assets.
The observers are involved whenever a transaction participant faces problems during its execution. The observers may neither influence the transaction direction nor change anything in it. They are not used for verifying every transaction, and rather interfere on the user’s request only. The GEO is using a framework similar to two-phase commit for regular transactions. Firstly, all the participants sign something like preparedness for payment. Secondly, the payment is executed, provided everybody has the list of signatures (the ‘document’). The observers act between the phases if a participant states the absence of the document. In this case, the observer gets the list of signatures from any node and sends it to all the participants or does nothing if it is impossible. In the latter case, the transaction expires in due time.
Atomic Cross-Chain Payment
The atomic swap is currently the most common way to perform cross-chain payments. The atomic swap enables the cross-chain exchange without escrow service or any other third-party involvement. It uses timelock contracts that have to be supported by the blockchains performing the transactions. In a real-life environment the user locks an amount they want to exchange (e.g. 1 BTC) on their blockchain for a preset period. They then generate a preimage, calculate hash, and state that another user can get the money only upon presenting this preimage. The lock and hash can be seen on the blockchain (but not the preimage). Another user wants to exchange his 3 Litecoins for this 1 Bitcoin. To do this, he locks 3 Litecoins on the contract with the same hash as the first user’s contract. This is possible as he sees this hash on another blockchain. To get his money from the second user’s contract, the first user has to reveal his preimage. The second user can use the revealed preimage to take funds from the first user’s smart contract. To close the deal, the first user has to reveal the preimage. If he does not, the deal cannot be closed. To avoid blocking the funds of both the first and the second user in this situation, the contract validity is limited in time.
Multi-path
Lightning designers came up with the multi-path atomic payment technology but have not implemented it yet. The technology is meant to mitigate the network liquidity problem by splitting a large payment into many smaller ones. The atomicity is supported by the modified HTLC. The receiver creates a base preimage (BP) to be further used for creating partial preimages for each small payment. As soon as the receiver gets the partial payments, they can unlock the funds using BP regardless of the partial payments arrival order.
In the GEO, the multi-path atomicity is provided in exactly the same way as the single-path atomicity (allocations, signature collection, list of signatures dissemination, addressing observers in case of problems).
Conclusion
Atomicity developments in decentralized networks have been driven by new concepts.
The first to appear was the hashed timelock contracts method that had its advantages and drawbacks. Its strength is no doubt the mitigation of losses when a node falls offline and the security for both the sender and the receiver. The issue is that the funds have to be frozen in the channels, and the participants have to be constantly online to avoid losses.
Then HTLC were modified to get HTLA and HTLR. The first solution rendered it possible to use HTLC in various registries, even those not supporting the respective contracts. The second one resolved the node falling offline problem.
After a time, brand new solutions arose employing observers and notaries. We should be extremely careful when implementing these solutions though, as centralization of observers/notaries may harm the network, but a suitably designed system may keep them decentralized.
We continue with our research in the field and welcome everyone’s contributions to distributed systems development and problem-solving.
