How Bitcoin Works

An in-depth-but-not-too-in-depth technical look at the Bitcoin protocol

Introduction

This article provides a technical overview of how Bitcoin works, relevant concepts, it’s history, and notable issues with it. It assumes knowledge of foundational concepts such as hashing, digital signatures, asymmetric cryptography, and double spending, as well as basic principles of blockchain (that it’s immutable, decentralized, etc).

I highly recommend this video as a primer for what’s to come.

Table of Contents

How it works

• Accounts
• Transactions part 1: UTXOs
• Transactions part 2: Transaction Structure
• Blocks
• Mining part 1: Overview
• Mining part 2: Incentives
• Mining part 3: Mining
• Consensus
• Stale and Orphan Blocks
• The Network
• Summary

Other Key Concepts

• Fork
• Softfork
• Hardfork
• Sidechains

History

• August 24, 2017: SegWit
• August 1, 2017: Bitcoin Cash
• October 24, 2017: Bitcoin Gold
• February 28, 2018: Bitcoin Private

Issues

• Fungibility
• Scalability
• Centralization

How it works

From a technical standpoint, Bitcoin doesn’t really do anything better than other cryptocurrencies; however it’s widespread use, name recognition, and the fact that it came first have been enough for it to reign supreme in the crypto world. It is important to understand Bitcoin because the unique selling points of most new blockchains are that they improve upon some aspect of Bitcoin.

The following subsections build on one another and are meant to be read in order. I will take a bottom-up approach to explaining, starting with small component parts and then explaining the larger concepts.

Accounts

In order to create an an account for the Bitcoin blockchain, users must generate an ECDSA key pair. The hash of the public key serves as their Bitcoin address from which they send and receive Bitcoins, and the private key is used in various ways to prove ownership of that account.

Transactions part 1: UTXOs

Bitcoin balances are calculated in the form of unspent transaction outputs (UTXOs). UTXOs are exactly what they sound like: unspent output from a previous transaction. They consist of:

1. Some amount of Bitcoin
2. A “lock”, ensuring that the UTXO can only be unlocked (and used) by accounts that satisfy some criteria

Every transaction takes in UTXOs as input, and uses them to generate new UTXOs as output. Any change left over is sent back to the owner as a new UTXO. Here is a simple example where Dob owns a $50 UTXO, and sends $40 to his pal Gob:

As you can see in this low-budget diagram, the input UTXO, now that it has been referenced as an input in a transaction, is considered spent and has lost its value. Additionally, two new spendable UTXOs have been created: a $40 one for Gob, and a $10 one for Dob.

What’s important to note here is that Bitcoin aren’t really things in the traditional sense where you could pass them around or bury them under a tree. When you “own” a Bitcoin, it just means that there is a Bitcoin on the blockchain that you have access to. Similarly, Bitcoin wallets don’t actually store Bitcoins like a leather wallet would; instead they are keys that have access to UTXOs. You can think of UTXOs as bank accounts that must be withdrawn in their entirety before they can be used.

Transactions part 2: Transaction structure

Here is a simple example transaction with 1 input and 1 output, taken from the Bitcoin Wiki:

Input:
Previous tx: f5d8ee39a430901c91a5917b9f2dc19d6d1a0e9cea205b009ca73dd04470b9a6
Index: 0
scriptSig: 304502206e21798a42fae0e854281abd38bacd1aeed3ee3738d9e1446618c4571d10
90db022100e2ac980643b0b82c0e88ffdfec6b64e3e6ba35e7ba5fdd7d5d6cc8d25c6b241501

Output:
Value: 5000000000
scriptPubKey: OP_DUP OP_HASH160 404371705fa9bd789a2fcd52d2c580b65d35549d
OP_EQUALVERIFY OP_CHECKSIG

Lets break it down. An input contains 3 fields:

1. The hash of the Previous tx––in other words, the transaction that contains the UXTO being referenced
2. The Index of the UTXO in the previous transaction’s outputs. A transaction can have multiple outputs; setting the index to 0 simply references the first output in that transaction.
3. The scriptSig, which contains two things: a script and a sig. The script is the first part of a script that must be evaluated in order for the output to be claimed (more on that later), and the sig is an ECDSA signature that proves the transaction was created by owner of the previous tx .

Transaction inputs reference UTXOs

An output contains instructions for sending the Bitcoins. It has 2 fields:

1. Value : the value of this output, representing how much it will be worth when it’s claimed. The total value of the outputs of a transaction must equal the total value of the input. Note that values in transactions are measured in Satoshi, where 1 Satoshi is equal to 0.00000001 (1/100 million) BTC
2. The scriptPubKey field has to do with claiming the transaction, which I will now explain.

Bitcoin uses a scripting system to claim outputs. The script is just a set of conditions that must be met in order for the output to be claimed. It allows the sender to get clever with sending Bitcoins.

The evaluation of the script is very simple. If Jimbo wants to claim an output, the input’s script and the output’s scriptPubKey are concatenated and then evaluated as a simple stack-based programming language. If the evaluation of the script returns true, Jimbo can claim the output.

This particular example’s script contains a very common sequence: OP_DUP OP_HASH160 <hash> OP_EQUALVERIFY OP_CHECKSIG, which simply means that if an account––which, if you recall, is a hash of a public key––is equal equal to <hash>, that account can claim this output. It is the standard script for P2PKH (Pay to Public Key Hash) transactions, the most common type of Bitcoin transaction where (predictably) the output is paid to a public key hash. I own’t be going over other types of transactions, but here is a good resource if you’re interested in learning more.

There are 2 more things you should note before we continue:

1. Ever since SegWit, this is not how transactions are currently formatted; however it is necessary to understand the old transaction format to understand the current format. This will be explained more when we get to SegWit.
2. Transactions often contain a small donation that can be claimed by miners who include the transaction in their block. This donation is optional, but a higher value provides greater incentive for miners to include and validate the transaction, therefore causing the transaction to be uploaded to the blockchain faster. This will also become clearer later on, as it has to do with concepts we haven’t covered yet

Blocks

This is the structure of a bitcoin block:

Taken from the Wiki

As you can see, it contains 5 parts and is fairly simple. The magic number, just like in network packets and files, lets people know they’re looking at a Bitcoin block. The transaction counter is the number of transactions included in the block.

One thing to note is that there is a maximum limit for the “blocksize”, the purpose of which is to limit the throughput of transactions into the blockchain. This comes with pros and cons, and there has been a great deal of controversy over it in the past.

The “blockheader” field, in turn, consists of the following information:

Taken from the Wiki

Again, some of these fields––hashPrevBlock, hashMerkleRoot, Time––are pretty straighforward.

The Bits and nonce fields have to do with mining, which I will explain now.

Mining part 1: Overview

Mining is the process by which blocks are added to the blockchain. Bitcoin mining works as follows:

1. The miner takes a bunch of unconfirmed transactions
2. The miner verifies that those transactions are valid
3. The miner groups those transactions together into a block.
4. The miner works to sign the block by continually incrementing the nonce until the hash of the block header is less than the “target” value. This requires a lot of computational power.
5. Finally, the signed block is broadcasted to the other nodes, who will verify that it is valid and add it to their own local blockchains.

Mining part 2: Incentives

For this system to work, miners must be incentivized to mine––if they weren’t, nothing would ever be added to the blockchain. These incentives come in 2 forms:

1. Miners whose blocks are accepted by the network receive a reward in Bitcoin. This reward comes in the form of a transaction, known as the coinbase transaction, which has no inputs and is the first transaction in a block.
2. Nodes who submit transactions can optionally offer a donation to any miner that includes their transaction in their blocks.

Mining part 3: Mining

In order to make the block valid, the miner must adjust the “nonce” field of the block header until the hash of the block header is less than the “target” value. Once that happens, the block can be broadcasted to the other nodes and if they consider it valid they will add it to their blockchains.

This is called a proof-of-work system because miners must prove that they have done a significant amount of work in order to mine the block. It allows the Bitcoin protocol to control the rate blocks are added to the blockchain; as more miners are added to the system, the difficulty to mine blocks increases (i.e. the “target” number decreases) such that only 1 block can be mined every 10 minutes or so.

In addition, this system grants the authority to validate transactions––typically entrusted to a central authority––to whomever spends the computing power required to mine the block. Thus, participants are granted an amount of authority relative to the amount of computing power they provide the network, establishing a kind of equality among participants.

Consensus

An important issue to consider when mining is what happens when the same block is mined twice. For example, let’s say Egg and Yam are both working on mining block #100, and Egg finishes mining her block. But before Yam learns that Egg has finished her block, he finishes mining his own. Now all the other nodes receive both Yam and Egg’s blocks! Which do they decide to add to their blockchain?

Clearly there must be some way for nodes to accept one version of the blockchain as the “real” one in cases such as these where they have multiple versions of the blockchain. The method by which nodes do is is called the consensus protocol.

From Wikipedia. The main chain is in black, orphaned blocks are in purple, and the genesis block is green.

One of Bitcoin’s core innovations is Nakamoto consensus, which simply accepts the longest chain as the real one. Because each block on the blockchain represents a significant amount of computational power expended, the longest chain represents the chain that has had the most work done on it.

Stale and orphan blocks

There is a lot of confusion out there about the difference between stale blocks and orphans. I will use Bitcoin.org’s glossary here as a reference:

Stale blocks are blocks which were successfully mined but which aren’t included on the current best block chain, likely because some other block at the same height had its chain extended first.

For example, let’s go back to Yam and Egg. We’ll also introduce Christopher: a miner who decides to build on top of Yam’s block. If he finishes mining before everyone else, the longest chain now includes Yam’s block; therefore Egg’s block would be considered a stale. Nobody (except for nodes that are out of the loop) will build on top of Egg’s block, and Egg loses the block reward because it’s no longer part of the blockchain. Sorry, Egg.

Now, orphans:

Orphans are blocks whose parent block has not been processed by the local node, so they can’t be fully validated yet.

So, orphans can’t be added to the blockchain because everybody thinks they have no parents. Sorry, orphans.

It would also probably be good to note here that these definitions really aren’t that important. In true English-speaking fashion, I just disregard them and refer to everything as an “orphan” because that’s what everybody else seems to do.

It’s also worth considering the notion of orphaned transactions. Even if a block containing a transaction has been validated and passed around a bit, it still has the potential to become an orphan. Because of this, people will not consider a transaction to be final until it is 6 or 7 of blocks deep in the blockchain, at which point they can safely assume that it’s there to stay.

The Network

The Bitcoin P2P network is about as simple as it gets: ad hoc, unstructured, best effort, and nodes are free to leave or join as they please.

Summary

Transactions are broadcasted across the network. Miners can choose which received transactions they want to include in their blocks and once the block is mined it will be broadcasted to all other nodes. Nodes download and verify new blocks from their peers before adding them to their local version of the blockchain, using Nakamoto consensus to resolve conflicts.

And that’s how Bitcoin works.

Other Key Concepts

Fork

A fork occurs when a change is made to the blockchain in order to introduce new features or reverse failures caused by hacking or bugs. Because the blockchain is decentralized, when forks occur it is infeasible for every participating node to accept the change; therefore some nodes will continue to under the old rules, while the nodes that adopt the new protocol will fork off and operate in a separate network with a separate blockchain. A softfork is when this “forking off” is not permanent; a hardfork is when it is.

Softfork

A soft fork occurs when a change that restricts the ruleset is introduced. Blocks valid under the old rules are no longer valid, while new blocks are valid under both new rules and the old; Therefore soft forks are backwards-compatible.

For example, consider what happens when a soft fork reducing the maximum block size from 1MB to 500kB is introduced. Under the new rules, previously valid blocks that were larger than 500kB would no longer be valid; however all new blocks, which are bounded by the new 500kB limit, would still be valid under the old rules. So, once enough miners––enough to represent a majority of the network’s computing power, that is––have adopted the soft fork changes, the longest chain will consist only of blocks that conform to the new rules and everybody––even the old timers who haven’t adopted the changes––will be cool with that.

Hardfork

A hard fork is a radical change that causes a permanent divergence in the blockchain. Nodes that abide by the new rules will reject all old blocks and only add new blocks to their blockchain, while nodes that do not abide by the new rules will reject all new blocks and only add old blocks to their blockchain, resulting in two separate blockchains.

Sidechains

Simply put, sidechains allow data to be transferred between blockchains––that is, a main blockchain and separate blockchain sidechains. I found the abstract from the paper that proposed sidechains to be quite nice, so I’m going to dump it here:

We propose a new technology, pegged sidechains, which enables bitcoins and other ledger assets to be transferred between multiple blockchains. This gives users access to new and innovative cryptocurrency systems using the assets they already own. By reusing Bitcoin’s currency, these systems can more easily interoperate with each other and with Bitcoin, avoiding the liquidity shortages and market fluctuations associated with new currencies. Since sidechains are separate systems, technical and economic innovation is not hindered. Despite bidirectional transferability between Bitcoin and pegged sidechains, they are isolated: in the case of a cryptographic break (or malicious design) in a sidechain, the damage is entirely confined to the sidechain itself.

If you’d like to go deeper with sidechains you can check out that paper, but for our purposes all you need to understand about them is that they allow other blockchains to securely run in parallel with and exchange data between the main blockchain.

History

Bitcoin was conceived pseudonymously by Satoshi Nakamoto in 2008. It introduced blockchain to the world as a solution to the Byzantine general’s problem with which a currency (i.e. Bitcoin) can be safely exchanged anonymously and without the need for a central authority or trust between participants.

I won’t be covering all forks, because there are a lot of them, most of which I have never heard of. I also won’t cover forks that have died out (Bitcoin XT, Bitcoin Classic) or were cancelled (SegWit2x). Also keep in mind that hardforks are essentially entirely different currencies/blockchains (that merit their own articles) so I won’t go into too much detail.

August 24, 2017: SegWit

I know this one is out of order and the Bitcoin Cash hardfork came before it, but in order to understand Bitcoin Cash you need to know what SegWit is. So here goes.

SegWit was a softfork in the Bitcoin protocol. I’m going to focus on 2 of its main effects:

1. Resolves the signature malleability issue
2. Improves scalability

To explain this further, let’s look back at the structure of a Bitcoin transaction:

Input:
Previous tx: f5d8ee39a430901c91a5917b9f2dc19d6d1a0e9cea205b009ca73dd04470b9a6
Index: 0
scriptSig: 304502206e21798a42fae0e854281abd38bacd1aeed3ee3738d9e1446618c4571d10
90db022100e2ac980643b0b82c0e88ffdfec6b64e3e6ba35e7ba5fdd7d5d6cc8d25c6b241501

Output:
Value: 5000000000
scriptPubKey: OP_DUP OP_HASH160 404371705fa9bd789a2fcd52d2c580b65d35549d
OP_EQUALVERIFY OP_CHECKSIG

Also, recall that the transaction ID of this transaction was created by taking the hash of all of this data. In theory, this transaction ID would be impossible to spoof because it is derived from all the data in the transaction. In practice, however, this was not the case. The format of the digital signatures were not strictly enforced, meaning the hash of a transaction with a slightly tweaked digital signature would still be accepted as valid. This allowed the existence of multiple transaction IDs for a single transaction, and is the essence of the signature malleability issue. You can read more about it here.

Also, notice how much data the signatures take up––before SegWit, they took up almost 65% of the transaction data. In addition, because of the restrictions on block size and mining difficulty, only a limited number of transactions could be added to the blockchain over a given period of time. As Bitcoin grew in popularity and transactions began to occur more frequently, serious concerns over backlogged transactions began to arise.

Enter SegWit. What SegWit did was segregate the signature data––called the witness––from the rest of the transaction data. In doing so, it resolved the signature malleability issue because it eliminated the possibility of tweaking the transaction ID by tweaking the digital signature.

SegWit improved scalability by introducing the concept of weight. In SegWit, weight replaced bytes as the measure of a block’s size, and the size limit was changed from 1MB to 4 million weight. The weight of a transaction is calculated as follows:

• A non-witness byte is 4 weight units
• A witness byte is 1 weight units

Therefore, the signature data in a SegWit block takes only 1/4 of the space it previously did.

In order for SegWit to achieve backwards compatibility and be implemented as a softfork, nodes that have not upgraded to SegWit simply don’t receive all of the witness data. Because new blocks have at most 1MB of non-witness data, they will still satisfy the old constraints.

You can read more about SegWit’s effect’s here, and see the BIP (Bitcoin Improvement Proposal) here.

August 1, 2017: Bitcoin Cash

Not everyone liked SegWit though, and Bitcoin Cash was a hardfork made for people who didn’t want to adopt the changes (notice how this fork came shortly before SegWit was implemented). It is the most successful hard fork to date, currently sitting at the 4th largest market cap out of all cryptocurrencies.

Bitcoin Cash also introduced the following changes:

• Maximum block size increased to 8 MB
• Modify the transaction ID hashing algorithm
• Replaced the Bitcoin Difficulty Adjustment algorithm with their new Emergency Difficulty Adjustment (EDA) algorithm, which was intended to be more responsive.

October 24, 2017: Bitcoin Gold

The main purpose of Bitcoin Gold is to prevent ASIC mining and empower GPU mining, thereby making mining more accessible and less centralized. In order to do this, Bitcoin Gold replaced Bitcoin’s SHA256 proof-of-work algorithm with Equihash, a memory-based algorithm.

February 28, 2018: Bitcoin Private

Bitcoin private provides privacy by making the sender, receiver, and amount in transactions private (in contrast to Bitcoin, where while accounts are anonymous, transaction histories are completely transparent). Just like Bitcoin Gold, it also uses the Equihash ASIC-resistant mining algorithm.

Issues

This section only focuses on the technical issues with Bitcoin, not legal or social issues.

Fungibility

Due to Bitcoin’s transparent transaction history, Bitcoin are not always fungible. Coins that have belonged to accounts associated with criminal activity––“tainted coins”––are not worth as much as “clean” coins.

Project such as Monero and Bitcoin Private seek to resolve this issue by making transactions private.

Scalability

Bitcoin’s scalability problem is due to the fact that blocks are limited in size and frequency. As a result, a backlog of transactions is formed, which causes transactions to take longer to be validated, which causes speed of validation to be in more scarce, which causes transaction fees to increases in demand, which…

I could go on. Here’s the bottom line: this bottleneck is a bad thing.

Centralization

Centralization––the very thing that Bitcoin sought to eliminate––is a growing concern about Bitcoin.

Think back to the Nakamoto consensus. Because the longest chain––because it has had the most work done on it––is accepted as the main chain, it follows that if one miner were to control a majority of the network’s computing power, they could construct the longest chain however they please and essentially have complete control over the network. This is known as the 51% attack.

While obtaining that much computing power would be infeasible for any one person, with mining pools all things are possible. Take a look at this graph:

Obtained from here

Kind of intimidating, huh? Fortunately, it’s extremely unlikely that any mining pool every grows to 51% as people who realize this could simply switch to a different pool (unless mining pools collude, in which case it’s going to be a bad time); it’s also unlikely that anyone with a 51% stake does anything bad because that would devalue all of their hard-earned power and Bitcoin.

But that’s not the only source of centralization. In addition to the fact that mining pools absolutely dominate the world’s Bitcoin mining resources, the existence of ASIC miners––expensive and blazingly fast integrated circuits that are specially tailored for mining Bitcoin––makes Bitcoin mining altogether less accessible, as your average Moe can’t really hope to compete with them.

As it stands, while it looks like mining pools are here to stay, ASIC-resistant proof-of-work algorithms such as CryptoNight and Equihash have been developed to prevent the use of ASICs in other cryptocurrencies.