Till it’s lightning-fast — Uncover the Development of the Lightning Network

A step by step guide to the development of the lightning network. It is assumed that you have a good understanding of the concepts of UTXO, Bitcoin Script, and digital signature. Otherwise, please check part one, Till it’s lightning-fast — Uncover Bitcoin Transactions.

The idea of the lightning network is to create a system aside from the Bitcoin network. As the transactions in the Bitcoin network are heavily yet necessarily restricted within a very limited time and space, that roughly only 2MB data can be transferred every 10 minutes, the lightning network breaks the restrictions by moving bitcoins into its network and circulating them under a different rule.

If two participants decide to join the lightning network, the first step for them is to create a multisig address to build a mutual account, i.e., open a channel between them. Once established, the channel will be maintained for transferring money within the network. If for any reason, the money needs to be pulled out into the Bitcoin network, the participants can close the channel cooperatively. Money flows from the Bitcoin network to the lightning network if people decide to open channels, and it should never flow reversely unless people want to close their channels. Generally speaking, the two anticipated scenarios when cross-layer transaction shall happen are opening and closing a channel, with the exception that, when a participant becomes unresponsive or malicious, other related participants can unilaterally close the channel.

Build the Mutual Account

To use the lightning network, the first step is to move some bitcoins into it. The participants will create multisig addresses together, to which the funds will be sent. For instance, Alice and Bob decide to open a channel with a total balance of 1.5 BTC, to which Alice will contribute 0.5 BTC and Bob will contribute 1 BTC.

Each of them will create a transaction to fund this channel and the money locked in this address cannot be spent unless both of them agree.

Transaction ID and Output Number in Inputs are omitted for the sake of demonstration.

Transaction ID and Output Number in Inputs are omitted for the sake of demonstration.

Now consider the following scenarios,

• What if one of the participants refuses to cooperate?
• What if Alice, or Bob, or both decide to close the channel and reclaim their money?
• What if the channel balance needs to be updated? Say, Alice pays Bob 0.1 BTC for something she buys from Bob?

Before tackling these issues, it should be emphasized that the lightning network shall be designed in a way that provides security while maintaining its decentralization. It should remain a trustless network like the Bitcoin network itself — you don’t rely on the integrity of anyone or any organization to successfully transfer money. As such, a more sophisticated design using Bitcoin Script is needed. Unlike the Bitcoin network, where security relies on computation power, the lightning network will secure itself by introducing punishments in its system.

Let’s solve the funding issue first. Keep in mind that, a layer-one transaction is done in three steps,

1. Create the transaction;
2. Sign the transaction, or, in a more general term, provide the unlocking script;
3. Broadcast the signed transaction;

Maybe there’s a genius design we can devise in-between the steps?

Funding without Signing

Alice and Bob can create the same transactions aforementioned without signing them. The purpose of staying unsigned is to leave room for exit in case anyone refuses to cooperate. Because the transactions are unsigned, no one has committed to anything thus no harm can be done at the moment. By referencing the unsigned transactions, Alice and Bob can create a new transaction to decide where the funds locked in the multisig address should go. In the current situation, it should give 0.5 BTC to Alice and 1 BTC to Bob.

Once created, two questions relating to this transaction should be asked,

1. Should they sign the new transaction? Yes. At this moment, no one is risking losing money. If Alice signed the transaction but Bob refused, she could immediately abort the cooperation at no cost and vice versa.

2. Should they broadcast the new transaction? No. The purpose of the new transaction is to prevent loss caused by unresponsive participants. Unless the other side becomes unresponsive, or the participants decide the close the channel, this new transaction shall never be broadcasted.

What should be signed and broadcasted instead is the original funding transaction, once Alice and Bob each have a copy of the fully signed new transaction. This new transaction is functioning similarly to the security deposit when renting an apartment — you place it to show your commitment to the contract and get refunded when the contract is voided. Thus, this new transaction is often referred to as the commitment transaction.

To summarize, the procedure for Alice and Bob to securely open a channel is as follows,

1. Alice and Bob each create a funding transaction without signing it.
2. Alice and Bob create and sign a commitment transaction based on the funding transaction. For instance, Alice creates the commitment transaction, signs it, sends it to Bob, and Bob will send it back with his signature. Or they could exchange their signatures for this particular transaction. Either way, the result is that they both have the same spendable commitment transaction.
3. Alice and Bob will broadcast their funding transactions.

Once their funding transactions are confirmed, the channel would be funded and they need to find a way to send bitcoins. However, before crafting the Bitcoin Script to make the balance update feasible, a crucial question here is, how can we create the funding transaction?

The Alternative — Unilateral Funding

Instead of the dual funding process shown above, the actual implementation of the lightning network uses unilateral funding by applying a clever twist in how the funding transaction is signed and broadcasted. To begin with, either participant can be the funder. Maybe Alice decides to purchase from Bob, she can treat the payment as the funds to open a channel with Bob. Later on, when Bob makes a purchase from others, he can then use the money in the channel without interacting with the Bitcoin network. As the number of channels increases, every participant will end up having some balance in its account, making it equivalent to have dual funding in the first place.

If Alice was to be the initial funder, it’s critical to ensure that when Bob becomes unresponsive, her funding transaction is recoverable.

When constructing transactions in Bitcoin, it’s the referenced transaction ID and the output number are needed in the input sections. Hence, it’s perfectly safe for Alice to create her funding transaction without broadcasting it first, and share her funding transaction ID and output number with Bob. When Bob gets the information, because he doesn’t have the signature from Alice, there’s nothing he can do with Alice’s funding transaction but to create a commitment transaction, which will refund Alice the money. The steps are summarized as follows,

1. Alice creates a funding transaction, signs it, and shares the transaction ID and output number with Bob.
2. Alice and Bob create and sign a commitment transaction based on the funding transaction’s ID and output number. Notice that the funding transaction puts Alice’s money into a multisig address, which requires both Alice and Bob’s signatures if they want to send it back to Alice using the commitment transaction.
3. Alice broadcasts her funding transaction. At this point, if Bob becomes unresponsive, Alice can broadcast the commitment transaction to recover her funds.

Alice’s funding transaction remains unchanged. However, she won’t broadcast it, nor share the signed transaction with Bob, but only give Bob the transaction ID(00) and the output number(0).

This unilateral funding scheme removes the risk of fund loss for the funder, thanks to how Bitcoin spends a UTXO in its transactions. Although the scheme can be further expanded into dual funding, the actual lightning protocol chose the unilateral funding. While not explicitly explained, the reason might be it requires less communication between nodes if funded unilaterally. As for the dual funding, there is an ongoing discussion on how to implement it.

Update Channel Balances

Now we have an account that is terminable, how about making updates? Alice has put down 0.5 BTC to open the channel, she will send 0.1 BTC to Bob in exchange for his laptop, how should they do that within the lightning network?

A trivial solution would be that Alice and Bob can create new transactions to reflect their latest balance.

For Alice, she will create a transaction as follows, sign it, and send it to Bob. Because it’s already signed by Alice, Bob can sign the transaction and broadcast it at his will.

Notice that we are referencing Alice’s funding transaction, not the commitment transaction.

For Bob, it’s the same except that he will sign and send the transaction to Alice. Currently, both of the participants have two valid transactions in parallel, together we’ve seen five transactions so far, listed as follows,

• The funding transaction from Alice, which has been broadcasted already;
• The commitment transaction for Alice, which gives her money back, we label it TX A1.
• Bob also has a copy of the commitment transaction, we label it TX B1.
• The newly created transaction for Alice, which updates the balance and gives Alice 0.4 BTC and Bob 0.1 BTC, we label it TX A2.
• Bob also has a copy of the newly created transaction, labeled as TX B2.

The transactions are simplified as follows,

All four transactions are valid before entering the Bitcoin network and have equal chances of being accepted by the miners if broadcasted simultaneously. If you find it confusing, think of Bitcoin as your uncle Tom, who is a bookkeeper that updates the ledger based on the firstly heard transaction and discards the later ones.

In the current case, speaking as economically rational beings, it would be Alice’s best interest to broadcast TX A1 to steal the 0.1 BTC she promised to pay Bob. For Bob, he’d rather broadcast TX B2 since he will get 0.1 BTC instead of nothing. If the two transactions are broadcasted at the same time, it’s a fifty-fifty chance that this is a fair trade, which is considered unacceptable.

How can we solve the problem? Specifically, how can we ensure that all the participants always broadcast the latest transaction but not the older ones?

Before moving on, let’s recall the types of transactions here as it’s easy to get perplexed when looking into all the transactions that happened in the lightning network.

• If the transaction has nothing to do with the lightning network, in other words, it only happens in the Bitcoin network, then it’s called the layer-one transaction.
• If the transaction will bring money from the Bitcoin network into the lightning network, i.e., open a channel, or bring money from the lightning network to the Bitcoin network, i.e., close a channel, then it’s called the cross-layer transaction as it’s interacting with both networks.
• If the transaction is meant to stay in the lightning network, then it’s called the layer-two transaction. Although, if a layer-two transaction is maliciously or accidentally broadcasted to the Bitcoin network, it becomes a cross-layer transaction. The lightning network is all about making sure layer-two transactions stay uncrossed.

Essentially, transactions made in the lightning network can legitly be broadcasted into the Bitcoin network, which means the layer-two transaction can be viewed as a specialized layer-one transaction. Because the Bitcoin network is slow and the lightning network is fast, the whole design is to make sure layer-one transactions happen as infrequently as possible.

Upgrading the Locking Script

Previously, all the locking scripts involved simply applied a multisig function. Now we want to see if we could find a secure way to update the balances by manipulating the output sections in those transactions. The goal is straightforward, we will implement punishment on the cheaters, which are defined as anyone who broadcasts old transactions. Without diving into various possible ways of punishment, what the lightning network uses is to make the adversary lose its money once the other side notices the malicious activity within a time window.

Back to our example, if Alice tries to broadcast TX A1, which credits her 0.5 BTC, we want to incorporate a design so that Bob will have enough time to notice and act upon by taking all Alice’s money as a punishment.

And the timelock comes into play, particularly, OP_CHECKSEQUENCEVERIFY. What it does is to begin the countdown when the transaction is broadcasted(or more precisely, accepted by the miners) and lock the funds until the specified time range has passed. If a transaction with 40 days sequence timelock is broadcasted today, then it cannot be spent until 40 days later. If it was broadcasted a year later, then it cannot be spent until a year plus 40 days later. This relative timelock can be incorporated into our new locking script,

• Alice can broadcast TX A1 to claim her money in case Bob becomes unresponsive.
• She has to wait for 2 weeks before she is able to spend the UTXO presented in TX A1 if broadcasted.
• If Bob is not unresponsive and finds Alice has broadcasted TX A1, he can take all the money as punishment to her.

The upgraded TX A1 looks like(we only focus on the output sections),

If Alice tries to make a cross-layer transaction on TX A1 she cannot get the money until two weeks have passed, which is enforced by a timelock. Meanwhile, if Bob monitors the Bitcoin network and finds out TX A1 has been broadcasted, he can immediately spend the 0.5 BTC.

The current design will make sure if an older transaction was broadcasted, Alice’s money will be taken as a penalty. On the other hand, it grants Bob too much power since his mere signature can spend all the money, which encourages him to be the bad guy (when your fans become your haters). The intention of having Alice being able to get her 0.5 BTC back is to protect her from suffering from an unresponsive participant. Nonetheless, Bob will be motivated to trick Alice into broadcasting it by deliberately being unresponsive and take all the money thereafter.

We have to escalate our design.

Much keys, many confusion — the Historical RSMC

When we use phrases like Alice’s signature, what we are referring to is a signature generated by one of the keys controlled by Alice. There is the concept of Alice’s signature, while in essence, there is always a private key creating this signature. On the other hand, Alice can create as many private keys as needed. This is important because it’s an integral property that can diminish the advantage for Bob created by the previous design.

Instead of asking for Bob’s signature in the locking script, a multisig function is applied to stop Bob from stealing the money. When the participants agree to update their balances by creating new transactions, the private keys specified by the multisig function from the old transactions are exchanged.

For the sake of demonstration, the private keys are named to be kept on track. As a starting point, Alice will generate four private keys, A1, Rick, Batman and Tom. For Bob, respectively, B1, Morty, Robin and Jerry.

For Alice, the new TX A1 looks like,

For Bob, the locking script in TX B1 is simple as he has no motivation to broadcast it.

By applying a multisig function in TX A1, Bob’s advantage has been taken away. If no updates are made, i.e., TX A2 and TX B2 do not exist at the moment, Alice’s fund stays protected even if Bob becomes unresponsive because he doesn’t have the key Rick. If they want to further update their balances, they can create new transactions with similar locking scripts. During the creation, they will exchange their private keys specified in the last transactions. Particularly, Alice will send Bob the private key Rick when creating TX A2. This move is a game-changer because it destroys Alice’s motivation to broadcast TX A1, as doing so will cause her to lose all the money to Bob since Bob will use Rick and Morty to unlock the script.

For Alice, the new TX A2 looks like,

And for Bob, TX B2 is similar,

If Alice decides to make another update that pays Bob 0.2 BTC, the steps are as follows,

1. Create new transactions. Both Alice and Bob will generate new private keys to create similar transactions shown above. The new transactions, TX A3 and TX B3 will credit Alice 0.2 BTC and Bob 0.3 BTC.
2. Exchange old private keys. Alice will give Bob the private key Batman, and Bob will give Alice the private key Jerry. This way, no one will be interested in broadcasting old transactions as the other side will take all the money.

Again, Bob has no interest in broadcasting the older transaction TX B2. If Alice decides to cheat by broadcasting TX A2, she will have to wait for two weeks before she could spend the money. However, because Bob has the private key Batman, which was given by Alice when updating their balances, and Robin, which was controlled by him, he can spend the money immediately. It’s Alice’s best interest to not broadcast an older transaction, and Bob must monitor the blockchain to check whether Alice has cheated or not. The problem is solved, as long as everyone is acting out of their best interests, no one will cheat. This sophisticated, economical incentive-driven design is what’s called RSMC, Revocable Sequence Maturing Contract, which serves as the inspiration for the development of the lightning network.

As its name indicates, it’s revocable because a layer-two transaction can be replaced by a newer one. It uses the timelock function OP_CHECKSEQUENCEVERIFY, so there is a sequence, which represents the time in the blockchain(as in block height or timestamp). Because the money is locked in time, it’s maturing till it becomes spendable when enough time has passed. Together this design was given the name Revocable Sequence Maturing Contract.

RSMC has its limitations as it can only serve two participants at a time. What we want is to expand the network so that people can make payments on the lightning network seamlessly. With a little tweak on RSMC, we will reach the foundation of the lightning network, the HTLC, Hashed Time Lock Contract.

Hashed Time Lock Contract

Imagine we have three participants and two channels. One channel is open between Alice and Bob, the other is open between Bob and Charlie. If Alice wants to send Charlie some coins, there are two options,

1. Alice and Charlie can open a channel and make the transaction;
2. Alice and Charlie can take advantage of the current channels.

The first approach seems to work but it doesn’t solve the problem when the number of participants grows. Let’s do the simple math.

• For 3 participants, we need 3 channels;
• For 4 participants, we need 6;
• For 10 participants, we need 45;
• For 1000 participants, we need 499,500 channels!

That is just too many channels(the formula for calculation is Cn2). Compared to the alternative in which Alice and Charlie ask Bob for help, this approach is much less favorable and feasible. Since they each have already built a channel with Bob, why not further develop it into a route to transfer the money back and forth?

About the Cryptographic Hash Function

To implement a route among channels, we need to borrow a tool from cryptography — a cryptographic hash function and understand its instincts and properties.

A hash function simply maps any inputs into a fixed size output. For example, a stupid hash function can be, take a word and group them by the first letters.

The inputs are the words(apple, agent), and the outputs are the letters(a). Another naive hash function can be a simple modulo operation. For instance, we can define a hash function that accepts any arbitrary number as inputs, mod them by 7 to produce the outputs. All the outputs will have a fixed range between 0 and 6.

These hash functions are used here to demonstrate the two properties of a cryptographic hash function,

• One-way function, meaning given the output, it’s hard to find the input. If the output is 1, it’s difficult to find whether the input value is 2, 9, or any number that produces the remainder 1. Or, given the letter a, it’s difficult to determine whether the input is apple or agent.
• Collision resistance, meaning it’s hard to find two inputs that produce the same output. Our dumb hash functions don’t have this property as it’s easy to find that both 2 and 9 produce the same result 2 and both apple and agent are hashed into the letter a.

Cryptographic hash functions achieved these properties by applying complicated mathematical functions. Without going down the rabbit hole, the instincts behind their design can be easily understood. To build a one-way function, it requires a mathematical function that is easy to verify when answers are supplied, but hard to solve. Or, more precisely, it is harder to find a valid answer than to verify it. In cryptography, being harder implies more computing power and time are needed. Factorization is a good example. Consider the following,

• Challenge one, find two numbers that can be multiplied into 2449.
• Challenge two, confirm that 31 x 79 equals 2449.

It’s not hard to conclude that solving challenge two is easier than challenge one. Though cryptography relies on complicated mathematical models(e.g., discrete logarithm), the very principle stays unchanged. On the other hand, to achieve collision resistance, the size of possible unique outputs needs to be substantially increased so that different inputs are less likely to collide to generate the same output. One of the commonly used algorithms, SHA256, produces 2²⁵⁶ possible outputs, which is an astronomical number that’s beyond imagination.

A simple use case of cryptographic hash functions is to hide a secret and later reveal it(zero-knowledge proof). A bizarre example would be, Alice claims she is the first person ever knows a magic note that will make people billionaires but Charlie doesn’t buy it. If Alice tells Charlie the message directly, Bob will jump out to claim that he owns the authenticity. To avoid intellectual property theft, Alice will apply a cryptographic hash function on the magic note, share the results with everyone, and claim she knows the secret note without revealing it first.

She uses SHA256 to hash her secret note, which generates an output,

8816cee3feb85ccaad0557ffb2f6b38947a27bea4ccdea1835fded53ad71c31a

Alice then shares it with Charlie. Charlie can now ask Bob to show the original message corresponding to this output. Of course, Bob doesn’t know the answer, nor can he fake it. Now it’s safe for Alice to reveal the secret note,

money is a social construct

A Naive Design — Hide and Reveal a Secret

It would be naive to simply ask Bob to pass the money if Alice wants to send 1 BTC to Charlie. Bob would have all the reasons to take the money himself. To solve this issue, a cryptographic hash function is applied in the design. If Alice wants to send 1 BTC to Charlie, it’s up to Charlie to initiate a payment request. Here’s what he needs to do,

1. Charlie will generate a random number r and keep it to himself as a secret.
2. Charlie will apply the cryptographic function to hash the secret r, and send the hashed result R along with the request charge Alice for 1 BTC to Bob.
3. Charlie will wait for Bob to pay him 1 BTC.

Once Bob receives the payment request along with the hashed value R, he will forward them to Alice and ask her to send him 1 BTC by making a transaction with the following locking script,

• If 3 days passed, and the signature from Alice is presented, then Alice gets the money back.
• Else, if the signature from Bob and the value can be hashed into R(which is the secret r) are presented, then Bob gets the money.

If Bob can get the secret r from Charlie then he will be able to unlock the transaction and spend the UTXO. Otherwise, once 3 days passed, Alice will be able to get her 1 BTC back. The 3 days condition is an absolute timelock differing from the previous one. It allows Alice to take her money back if Bob cannot provide the secret in 3 days. Now that Bob is resting assured he will get the 1 BTC from Alice if he learns the value of r, he can then send Charlie 1 BTC by creating a similar transaction with the locking script as follows,

• If 2 days passed, and the signature from Bob is presented, then Bob gets the money back.
• Else, if signature from Charlie and the value can be hashed into R are presented, then Charlie gets the money.

To claim the money, Charlie has to reveal the secret value of r to Bob. Once Bob knows the value of r, he can spend the transaction created between him and Alice. The length of the locked time decreases as it moves closer to the final recipient, i.e., Charlie has 2 days to reveal the value r, while Bob has 3 days so that each participant will have enough time to act. The act of revealing the secret r to Alice from Bob proves that he has sent his 1 BTC to Charlie because that’s the only way Bob learns what the value is.

But can anyone cheat?

Alice’s money is solely constrained by the 3-day period. Once the timelock is unlocked, Alice is free to spend the money no matter Bob has the secret or not. As for Bob, his knowledge of the secret r exclusively determines the ability to spend the output. Even if 3 days have passed, the money remains spendable for Bob as long as he can provide the secret r. Regardless of updating balances, the current design seems fine if,

• Bob will spend his UTXO as soon as he learns the secret r , which means he will broadcast this transaction within 3 days if possible.
• Alice will spend her UTXO as soon as 3 days passed, which means she will broadcast this transaction 3 days later if applicable.

However, it becomes a cross-layer transaction and hits the Bitcoin network if a layer-two transaction is broadcasted, which violates the principle of the lightning network — to keep the transactions within it as much as possible. Therefore, a new design that has similar effects produced by an RSMC is needed — we don’t rely on people behaving honestly, rather, we will punish those who don’t.

With these backgrounds, it’s time to check the grand design — the actual implementation of the lightning network.

Notes and References

1. The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments, Joseph Poon, Thaddeus Dryja
2. Reaching the ground with lightning, Rusty Russell
3. Cryptographic Hash Function
4. A Fast and Scalable Payment Network with Bitcoin Duplex Micropayment Channels, Christian Decker, Roger Wattenhofer
5. The Lightning Network Specifications has all the implementation details. The actual locking scripts used are specified in BOLT #3: Bitcoin Transaction and Script Formats
6. I’ve set up a testing environment using bitcoind and lnd to help me understand the whole process. Here’s the gist of the trimmed log generated from lnd, which recorded how Alice and Bob opened a channel.
7. A curated list of awesome Lightning Network projects for developers and crypto enthusiasts.