A Hackers Guide to Layer 2: Merkle Trees from Scratch
In this tutorial, we will implement merkle trees, merkle proofs, and delta merkle proofs in JavaScript from first principles.
This is the first weekly installment of A Hackers Guide to Layer 2: Building a zkVM from Scratch.
Follow @cmpeq on twitter to stay up to date with future installments.
What I cannot create, I do not understand.
Richard Feynman
Introduction
Merkle trees are one of the most important tools in any would be layer 2 developer’s toolbox. In the spirit of Feynman’s approach to understanding a topic, we will implement Merkle Trees from scratch and build deep Merkle Tree intuition that will be essential later in our layer 2 development journey.
Inventing Merkle Trees: Data Integrity
Let’s imagine Jack wants to send a 400 gigabyte file to his friend Sally over an unreliable internet connection. It will take hours for the file to send, and Sally wants to make sure that the file she gets isn’t corrupted or changed while it is being transferred.
One way we might try to solve this is by having Jack create a digital signature of the whole file, and then have Sally verify the signature when the file transfer is complete. Unfortunately, however, digital signature technology has a property that: the longer the data you want to sign, the slower it takes to generate a digital signature, so it is impractical for Jack digitally sign the whole file.
They can get around this problem by taking advantage of digital fingerprint (also known as cryptographic hashing) technology. Taking the hash (or digital fingerprint) of a large file generates a short 32 byte number to represent the data that was hashed. Most importantly, if you change even a single byte of the large file, its fingerprint will also change:
- Jack sends the hash, the signature and the file to Sally
- When Sally receives the file, she computes the hash on her machine, and then checks to see if the hash matches the one that Jack digitally signed.
This is all well and good, but since the transfer can take hours and it is very likely that something gets corrupted during the transfer, Sally keeps on finding that the hash she computes is different than the one signed by Jack, and Jack has to send the 400 GB file all over again (very slow).
One way to fix this would be to have Jack first split the file into two pieces (A and B) and hash each piece. That way the size of any data that has to be resent is 200 GB instead of 400 GB.
If Jack is really clever, he can make it so he still only has to compute one digital signature, by signing the Hash(Hash(A)+Hash(B)) (called the Root). This way Sally knows the hashes of A and B weren’t modified during the transfer and can be confident that the file she receives is the same as Jack’s original file. If any data in A or B is modified, Hash(Hash(A)+Hash(B)) will not match Jack’s original signature.
In fact, we can split the file into even smaller pieces to make the size of the any data that has to be resent much smaller:
This business of splitting up data and recursively hashing is called a building a Merkle Tree.
Like in our example, the basic principle of a Merkle Tree is:
if you modify the value of any of the nodes at the bottom which represent our dataset, the value of the node at the top will also change.
The nodes at the bottom which contain our dataset are known as leaves and the node at the top (aka. N(0,0)) is known as the Merkle Root.
For convenience, we can identify each node N with two numbers:
- The level of the node (the top node is on level 0, and the nodes below it has level = 1, etc.)
- The index of the node (for the index, we just count from left to right with the node furthest left having index = 0)
Using our node definition, we can also say that every node N(level, index), has two children N(level+1, index*2) and N(level+1, index*2+1). This is to reflect that each level has twice as many nodes as the level above it.
In addition, using our previous rule of computing Hash(Hash(A), Hash(B)), we can define the value of each non-leaf node as:
N(level, index) = Hash(N(level+1, index*2), N(level+1, index*2+1))
We can also define the height of the merkle tree as the distance from the leaves from the merkle root (ie. count the lines in the path between the leaf nodes and the root node at the top). The leaves of the Merkle Tree always have a level number equal to the height of the tree.
Coding a Merkle Tree
Now that we have invented the Merkle Tree, we can demonstrate our new found knowledge by writing a JavaScript implementation.
First we will need to select a function to calculate our digital fingerprints. We can use the common sha256 hashing function to do the job of computing Hash(childA, childB):
const shajs = require("sha.js"); // npm install --save sha.js
function hash(leftNode, rightNode) {
return shajs("sha256")
.update(leftNode + rightNode, "hex")
.digest("hex");
}With a hash function in hand, we can write a simple class which computes the merkle root given a dataset of leaves and the corresponding tree height:
const shajs = require("sha.js"); // npm install --save sha.js
function hash(leftNode, rightNode) {
return shajs("sha256")
.update(leftNode + rightNode, "hex")
.digest("hex");
}
class MerkleTree {
constructor(height, leaves){
this.height = height;
this.leaves = leaves;
}
N(level, index){
if(level === this.height){
// if level == height, the node is a leaf,
// so just return the value from our dataset
return this.leaves[index];
}else{
// if the node is not a leaf, use our definition:
// N(level, index) = Hash(N(level+1, index*2), N(level+1, index*2+1))
return hash(
this.N(level+1, 2*index),
this.N(level+1, 2*index+1),
);
}
}
getRoot(){
// the node N(0,0) is always the root node
return this.N(0,0);
}
}Let’s test out our merkle tree implementation with a tree that has
leaves = [1, 3, 3, 7, 4, 2, 0, 6] and a height of 3:
![a tree that with leaves = [1, 3, 3, 7, 4, 2, 0, 6] and a height of 3](https://miro.medium.com/v2/resize:fit:1400/format:webp/1*5JOf6Sf59krEVTYJA2ZVRA@2x.png)
In our JavaScript file, we can write each leaf as a hexadecimal string, create a new instance of MerkleTree, and call the getRoot function to compute the root:
function example1(){
const height = 3;
const leaves = [
"0000000000000000000000000000000000000000000000000000000000000001", // 1
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000007", // 7
"0000000000000000000000000000000000000000000000000000000000000004", // 4
"0000000000000000000000000000000000000000000000000000000000000002", // 2
"0000000000000000000000000000000000000000000000000000000000000000", // 0
"0000000000000000000000000000000000000000000000000000000000000006", // 6
];
const tree = new MerkleTree(height, leaves);
console.log("[EX_1] the merkle tree's root is: "+tree.getRoot());
}
example1();If we run this code, we will get the output:
[EX_1] the merkle tree's root is: 7e286a6721a66675ea033a4dcdec5abbdc7d3c81580e2d6ded7433ed113b7737If we change any of the leaves (for example, changing the 2 to a 9), we should also see that the root changes:
function example2(){
const height = 3;
const leaves = [
"0000000000000000000000000000000000000000000000000000000000000001", // 1
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000007", // 7
"0000000000000000000000000000000000000000000000000000000000000004", // 4
"0000000000000000000000000000000000000000000000000000000000000002", // 2
"0000000000000000000000000000000000000000000000000000000000000000", // 0
"0000000000000000000000000000000000000000000000000000000000000006", // 6
];
const tree = new MerkleTree(height, leaves);
console.log("[EX_2] before root is: "+tree.getRoot());
// change the 2 to a 9
tree.leaves[5] = "0000000000000000000000000000000000000000000000000000000000000009";
console.log("[EX_2] after root is: "+tree.getRoot());
}
example2();If we run example2(), we will see the output:
[EX_2] before root is: 7e286a6721a66675ea033a4dcdec5abbdc7d3c81580e2d6ded7433ed113b7737
[EX_2] after root is: e74ae221cf067dc8e88195f5b30bfda60dc224a8cd5554bc6345c87ad8c6f925Success! The root before the change matches our original root because it had the same dataset, and the after root has updated because we changed one of the leaves in the dataset.
Let’s look deeper into why changing one of the leaves changes the root:
- Since we changed N(3,5), its parent N(2,2) changes because N(2,2) is defined as N(2,2) = Hash(N(3,4), N(3,5))
- Since N(2,2) changed, its parent N(1,1) changes because N(1,1) is defined as N(1,1) = Hash(N(2,2), N(2,3))
- Since N(1,1) changed, its parent N(0,0) changes because N(0,0) is defined as N(0,0) = Hash(N(1,0), N(1,1))
The nodes that change when you update a leaf, are called the node’s merkle path. So in this case, we could say the merkle path of N(3,5) is [N(3,5), N(2,2), N(1,1), N(0,0)].
To compute the merkle path of a node, we just recursively list out the ancestors of the node. (ancestors = parent of node, parent of parent, parent of parent of parent…).
Since we know from before that every node N(level, index), has two children N(level+1, index*2) and N(level+1, index*2+1), we can say that the parent of a node N(level, index) is N(level-1, floor(index/2)).
Using this knowledge, we can write a JavaScript function to calculate the merkle path of a node and check our earlier observation with N(3,5):
function getMerklePathOfNode(level, index){
const merklePath = [];
for(let currentLevel=level;currentLevel>=0;currentLevel--){
merklePath.push({
level: currentLevel,
index: index,
});
index = Math.floor(index/2);
}
return merklePath;
}
console.log(getMerklePathOfNode(3,5))When we run it, the log prints [N(3,5), N(2,2), N(1,1), N(0,0)]:
[
{ level: 3, index: 5 },
{ level: 2, index: 2 },
{ level: 1, index: 1 },
{ level: 0, index: 0 }
]In practice, we often exclude the root node from our returned merkle path as every node’s merkle path contains it, so we can rewrite our function to return the all nodes in the merkle path except the root (level>0):
function getMerklePathOfNode(level, index){
const merklePath = [];
for(let currentLevel=level;currentLevel>0;currentLevel--){
merklePath.push({
level: currentLevel,
index: index,
});
index = Math.floor(index/2);
}
return merklePath;
}Siblings
Before we continue it is important that we talk about the concept of left handed and right handed nodes on a merkle tree.
- We say that a node is left handed if it appears in the tree on the left side of its parent.
- We say that a node is right handed if it appears in the tree on the right side of its parent.
Given our original definition of the merkle tree where we say that every non-leaf node N(level, index) has a left child and a right child:
- Left Child: N(level+1, 2*index)
- Right Child: N(level+1, 2*index + 1)
Given this definition, we can see that all left handed nodes have an even index (2*x), and right handed nodes have an odd index (2*x+1).
Therefore:
- If a node N(level, index) has an even index, we say that its opposite handed sibling is N(level, index+1).
- If a node N(level, index) has an odd index, we say that its opposite handed sibling is N(level, index-1)
Let’s write a JavaScript function to return the sibling of any node we specify on a tree:
function getSiblingNode(level, index){
// if node is the root, it has no sibling
if(level === 0){
throw new Error("the root does not have a sibling")
}else if(index % 2 === 0){
// if node is even, its sibling is at index+1
return {level: level, index: index+1};
}else{
// if node is odd, its sibling is at index-1
return {level: level, index: index-1};
}
}Merkle Proofs
We often want to prove the inclusion of a certain leaf in a merkle tree with a known root. Let’s imagine Bob wants to prove to his friend Todd that the value 9 exists at leaf N(3,5) on a tree with a known merkle root R.
The most naive way for Bob to do this is:
- Bob gives Todd a list of all the leaves
- Todd uses our MerkleTree class to calculate N(0,0)
- Todd compares N(0,0) with R to make sure they are the same
For large trees, however, this is not very practical. If Bob wants to prove the value of a leaf in a tree with 1,000,000 leaves, Bob must send all 1,000,000 leaves to Todd, and Todd must do 999,999 computations to calculate the root.
If we look back at our merkle path diagram, however, there is a faster way we can do this:
In our diagram we can see that Todd can calculate the root if Bob gives him: the value of the leaf, the index of the leaf and the siblings of the non-root nodes on the leaf’s merkle path (N(3,4), N(2,3) and N(1,0)):
N(0,0) = Hash(N(1,0), Hash(Hash(N(3,4), 9), N(2,3)))
Using this method, for a tree with N leaves, Todd only needs to compute log(N) computations, so if the tree has 1,000,00 leaves, he only needs to perform 20 computations instead of instead of 999,999 computations!
The value, index and siblings together are known as a merkle proof, as they are the key ingredients required to prove a leaf’s existence in a merkle tree with a known root.
Let’s first write a function the calculate the siblings of a given node’s merkle path:
Let’s update our MerkleTree class to include a getMerkleProof function:
class MerkleTree {
// ...
getMerkleProof(level, index){
// get the value of the leaf node
const leafValue = this.N(level, index);
// get the levels and indexes of the nodes on the leaf's merkle path
const merklePath = getMerklePathOfNode(level, index);
// get the levels and indexes of the siblings of the nodes on the merkle path
const merklePathSiblings = merklePath.map((node) => {
return getSiblingNode(node.level, node.index);
});
// get the values of the sibling nodes
const siblingValues = merklePathSiblings.map((node) => {
return this.N(node.level, node.index);
});
return {
root: this.getRoot(), // the root we claim to be our tree's root
siblings: siblingValues, // the siblings of our leaf's merkle path
index: index, // the index of our leaf
value: leafValue, // the value of our leaf
};
}
}We can also write a function which calculates the merkle root using the proof’s siblings, index and value:
function computeMerkleRootFromProof(siblings, index, value){
// start our merkle node path at the leaf node
let merklePathNodeValue = value;
let merklePathNodeIndex = index;
// we follow the leaf's merkle path up to the root,
// computing the merkle path's nodes using the siblings provided as we go alone
for(let i=0;i<siblings.length;i++){
const merklePathNodeSibling = siblings[i];
if(merklePathNodeIndex%2===0){
// if the current index of the node on our merkle path is even:
// * merklePathNodeValue is the left hand node,
// * merklePathNodeSibling is the right hand node
// * parent node's value is hash(merklePathNodeValue, merklePathNodeSibling)
merklePathNodeValue = hash(merklePathNodeValue, merklePathNodeSibling);
}else{
// if the current index of the node on our merkle path is odd:
// * merklePathNodeSibling is the left hand node
// * merklePathNodeValue is the right hand node,
// * parent node's value is hash(merklePathNodeSibling, merklePathNodeValue)
merklePathNodeValue = hash(merklePathNodeSibling, merklePathNodeValue);
}
// using our definition, the parent node of our path node is N(level-1, floor(index/2))
merklePathNodeIndex = Math.floor(merklePathNodeIndex/2);
}
return merklePathNodeValue;
}Finally, let’s write a function which verifies the merkle proof:
function verifyMerkleProof(proof){
return proof.root === computeMerkleRootFromProof(proof.siblings, proof.index, proof.value);
}Putting it all together, we now have a class which can generate a merkle proof of inclusion that can be verified in O(log(n)) time 🎉
const shajs = require("sha.js"); // npm install --save sha.js
function hash(leftNode, rightNode) {
return shajs("sha256")
.update(leftNode + rightNode, "hex")
.digest("hex");
}
function getSiblingNode(level, index){
// if node is the root, it has no sibling
if(level === 0){
throw new Error("the root does not have a sibling")
}else if(index % 2 === 0){
// if node is even, its sibling is at index+1
return {level: level, index: index+1};
}else{
// if node is odd, its sibling is at index-1
return {level: level, index: index-1};
}
}
function getMerklePathOfNode(level, index){
const merklePath = [];
for(let currentLevel=level;currentLevel>0;currentLevel--){
merklePath.push({
level: currentLevel,
index: index,
});
index = Math.floor(index/2);
}
return merklePath;
}
class MerkleTree {
constructor(height, leaves){
this.height = height;
this.leaves = leaves;
}
N(level, index){
if(level === this.height){
// if level == height, the node is a leaf,
// so just return the value from our dataset
return this.leaves[index];
}else{
// if the node is not a leaf, use our definition:
// N(level, index) = Hash(N(level+1, index*2), N(level+1, index*2+1))
return hash(
this.N(level+1, 2*index),
this.N(level+1, 2*index+1),
);
}
}
getRoot(){
// the node N(0,0) is always the root node
return this.N(0,0);
}
getMerkleProof(level, index){
// get the value of the leaf node
const leafValue = this.N(level, index);
// get the levels and indexes of the nodes on the leaf's merkle path
const merklePath = getMerklePathOfNode(level, index);
// get the levels and indexes of the siblings of the nodes on the merkle path
const merklePathSiblings = merklePath.map((node) => {
return getSiblingNode(node.level, node.index);
});
// get the values of the sibling nodes
const siblingValues = merklePathSiblings.map((node) => {
return this.N(node.level, node.index);
});
return {
root: this.getRoot(), // the root we claim to be our tree's root
siblings: siblingValues, // the siblings of our leaf's merkle path
index: index, // the index of our leaf
value: leafValue, // the value of our leaf
};
}
}
function computeMerkleRootFromProof(siblings, index, value){
// start our merkle node path at the leaf node
let merklePathNodeValue = value;
let merklePathNodeIndex = index;
// we follow the leaf's merkle path up to the root,
// computing the merkle path's nodes using the siblings provided as we go alone
for(let i=0;i<siblings.length;i++){
const merklePathNodeSibling = siblings[i];
if(merklePathNodeIndex%2===0){
// if the current index of the node on our merkle path is even:
// * merklePathNodeValue is the left hand node,
// * merklePathNodeSibling is the right hand node
// * parent node's value is hash(merklePathNodeValue, merklePathNodeSibling)
merklePathNodeValue = hash(merklePathNodeValue, merklePathNodeSibling);
}else{
// if the current index of the node on our merkle path is odd:
// * merklePathNodeSibling is the left hand node
// * merklePathNodeValue is the right hand node,
// * parent node's value is hash(merklePathNodeSibling, merklePathNodeValue)
merklePathNodeValue = hash(merklePathNodeSibling, merklePathNodeValue);
}
// using our definition, the parent node of our path node is N(level-1, floor(index/2))
merklePathNodeIndex = Math.floor(merklePathNodeIndex/2);
}
return merklePathNodeValue;
}
function verifyMerkleProof(proof){
return proof.root === computeMerkleRootFromProof(proof.siblings, proof.index, proof.value);
}
function example4(){
const height = 3;
const leaves = [
"0000000000000000000000000000000000000000000000000000000000000001", // 1
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000007", // 7
"0000000000000000000000000000000000000000000000000000000000000004", // 4
"0000000000000000000000000000000000000000000000000000000000000009", // 9
"0000000000000000000000000000000000000000000000000000000000000000", // 0
"0000000000000000000000000000000000000000000000000000000000000006", // 6
];
const tree = new MerkleTree(height, leaves);
console.log("[EX_4] the root is: "+tree.getRoot());
const merkleProofOfN3_5 = tree.getMerkleProof(3,5);
console.log("[EX_4] the merkle proof of N(3,5):\n" + JSON.stringify(merkleProofOfN3_5, null, 2));
console.log("[EX_4] computed root: "+computeMerkleRootFromProof(merkleProofOfN3_5.siblings, merkleProofOfN3_5.index, merkleProofOfN3_5.value));
console.log("[EX_4] verify merkle proof: "+verifyMerkleProof(merkleProofOfN3_5));
}
example4();This prints:
[EX_4] the root is: e74ae221cf067dc8e88195f5b30bfda60dc224a8cd5554bc6345c87ad8c6f925
[EX_4] the merkle proof of N(3,5):
{
"root": "e74ae221cf067dc8e88195f5b30bfda60dc224a8cd5554bc6345c87ad8c6f925",
"siblings": [
"0000000000000000000000000000000000000000000000000000000000000004",
"deb2a27a00dbc46e3a59096a8d07d2b97f950158886411ff6a9c9ab9623dada6",
"2f4d3e941b602c50347af3f5c809a28737c27c7ce460e77b10739875ef957aa7"
],
"index": 5,
"value": "0000000000000000000000000000000000000000000000000000000000000009"
}
[EX_4] computed root: e74ae221cf067dc8e88195f5b30bfda60dc224a8cd5554bc6345c87ad8c6f925
[EX_4] verify merkle proof: trueDelta Merkle Proofs
If we have merkle tree with root A, and we generate a merkle proof that has siblings
In addition to proving that a certain leaf exists in a merkle tree, we will often want to prove the result of modifying a specific leaf in a tree from one value to another. This is called a delta merkle proof (delta means change).
In particular, delta merkle proofs prove:
- A leaf exists at index I with value X in a tree with merkle root A
- If you change the value of the leaf with index I from X to Y without modifying any other leaves in the tree, the new merkle root of the tree is B
We already know how to prove “A leaf exists at index I with value X in a tree with merkle root A”; we can just call the getMerkleProof function on our contract.
If we then modify the leaf, let’s say from 9 to 8, let’s call getMerkleProof again and see what happens:
function example5(){
const height = 3;
const leaves = [
"0000000000000000000000000000000000000000000000000000000000000001", // 1
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000007", // 7
"0000000000000000000000000000000000000000000000000000000000000004", // 4
"0000000000000000000000000000000000000000000000000000000000000009", // 9
"0000000000000000000000000000000000000000000000000000000000000000", // 0
"0000000000000000000000000000000000000000000000000000000000000006", // 6
];
const tree = new MerkleTree(height, leaves);
const beforeMerkleProofOfN3_5 = tree.getMerkleProof(3,5);
console.log("[EX_5] the merkle proof of N(3,5) before change:\n" + JSON.stringify(beforeMerkleProofOfN3_5, null, 2));
// change 9 to 8
tree.leaves[5] = "0000000000000000000000000000000000000000000000000000000000000008";
const afterMerkleProofOfN3_5 = tree.getMerkleProof(3,5);
console.log("[EX_5] the merkle proof of N(3,5) after change:\n" + JSON.stringify(afterMerkleProofOfN3_5, null, 2));
}
example5();This prints:
[EX_5] the merkle proof of N(3,5) before change:
{
"root": "e74ae221cf067dc8e88195f5b30bfda60dc224a8cd5554bc6345c87ad8c6f925",
"siblings": [
"0000000000000000000000000000000000000000000000000000000000000004",
"deb2a27a00dbc46e3a59096a8d07d2b97f950158886411ff6a9c9ab9623dada6",
"2f4d3e941b602c50347af3f5c809a28737c27c7ce460e77b10739875ef957aa7"
],
"index": 5,
"value": "0000000000000000000000000000000000000000000000000000000000000009"
}
[EX_5] the merkle proof of N(3,5) after change:
{
"root": "99ba6856d11cd514be35ed291e86ae7957ec43d6b83270f32aaa50a8a25dc5cc",
"siblings": [
"0000000000000000000000000000000000000000000000000000000000000004",
"deb2a27a00dbc46e3a59096a8d07d2b97f950158886411ff6a9c9ab9623dada6",
"2f4d3e941b602c50347af3f5c809a28737c27c7ce460e77b10739875ef957aa7"
],
"index": 5,
"value": "0000000000000000000000000000000000000000000000000000000000000008"
}Aha!
Notice that the siblings are the same in the before and after proof’s!
Remember that when we update a leaf, it only changes the nodes on the leaf’s merkle path, and since by definition the leaf’s merkle path does not include any siblings, the before and after proofs have the same siblings!
If we look deeper, the fact that both proofs provide the same siblings and index, verifying both proofs prove that we have only modified the leaf with index I:
Therefore, our delta merkle proof needs to provide:
- The leaf’s index (the same for both before and after proofs)
- The leaf’s siblings (the same for both before and after proofs)
- The leaf’s old value (before the change)
- The tree’s old root (the root of the tree before the change)
- The leaf’s new value (after we change the value of the leaf)
- The tree’s new root (after we change the value of the leaf)
We can implement a getDeltaMerkleProof function to do this for us:
class MerkleTree {
// ...
getDeltaMerkleProof(level, index, newValue){
// compute the root of the leaf at index I
const oldLeafProof = this.getMerkleProof(level, index);
// compute the merkle root from the proof, replacing the leaf's value with the new value
const newRoot = computeMerkleRootFromProof(oldLeafProof.siblings, index, newValue);
// return the data from the old proof and the new proof
return {
index: index,
siblings: oldLeafProof.siblings,
oldRoot: oldLeafProof.root,
oldValue: oldLeafProof.value,
newRoot: newRoot,
newValue: newValue,
}
}
}To verify the delta merkle proof, we can split the data of the delta merkle proofs into a newProof and oldProof which share the same siblings/index, and verify the proofs using our existing verifyMerkleProof function:
function verifyDeltaMerkleProof(deltaMerkleProof){
// split the delta merkle proof into a before and after merkle proof, reusing the same siblings and index
const oldProof = {
// reuse the same siblings for both old and new
siblings: deltaMerkleProof.siblings,
// reuse the same index for both old and new
index: deltaMerkleProof.index,
root: deltaMerkleProof.oldRoot,
value: deltaMerkleProof.oldValue,
};
const newProof = {
// reuse the same siblings for both old and new
siblings: deltaMerkleProof.siblings,
// reuse the same index for both old and new
index: deltaMerkleProof.index,
root: deltaMerkleProof.newRoot,
value: deltaMerkleProof.newValue,
};
return verifyMerkleProof(oldProof) && verifyMerkleProof(newProof);
}Let’s combine this together and see if it works:
const shajs = require("sha.js"); // npm install --save sha.js
function hash(leftNode, rightNode) {
return shajs("sha256")
.update(leftNode + rightNode, "hex")
.digest("hex");
}
function getSiblingNode(level, index){
// if node is the root, it has no sibling
if(level === 0){
throw new Error("the root does not have a sibling")
}else if(index % 2 === 0){
// if node is even, its sibling is at index+1
return {level: level, index: index+1};
}else{
// if node is odd, its sibling is at index-1
return {level: level, index: index-1};
}
}
function getMerklePathOfNode(level, index){
const merklePath = [];
for(let currentLevel=level;currentLevel>0;currentLevel--){
merklePath.push({
level: currentLevel,
index: index,
});
index = Math.floor(index/2);
}
return merklePath;
}
class MerkleTree {
constructor(height, leaves){
this.height = height;
this.leaves = leaves;
}
N(level, index){
if(level === this.height){
// if level == height, the node is a leaf,
// so just return the value from our dataset
return this.leaves[index];
}else{
// if the node is not a leaf, use our definition:
// N(level, index) = Hash(N(level+1, index*2), N(level+1, index*2+1))
return hash(
this.N(level+1, 2*index),
this.N(level+1, 2*index+1),
);
}
}
getRoot(){
// the node N(0,0) is always the root node
return this.N(0,0);
}
getMerkleProof(level, index){
// get the value of the leaf node
const leafValue = this.N(level, index);
// get the levels and indexes of the nodes on the leaf's merkle path
const merklePath = getMerklePathOfNode(level, index);
// get the levels and indexes of the siblings of the nodes on the merkle path
const merklePathSiblings = merklePath.map((node) => {
return getSiblingNode(node.level, node.index);
});
// get the values of the sibling nodes
const siblingValues = merklePathSiblings.map((node) => {
return this.N(node.level, node.index);
});
return {
root: this.getRoot(), // the root we claim to be our tree's root
siblings: siblingValues, // the siblings of our leaf's merkle path
index: index, // the index of our leaf
value: leafValue, // the value of our leaf
};
}
getDeltaMerkleProof(level, index, newValue){
// compute the root of the leaf at index I
const oldLeafProof = this.getMerkleProof(level, index);
// compute the merkle root from the proof, replacing the leaf's value with the new value
const newRoot = computeMerkleRootFromProof(oldLeafProof.siblings, index, newValue);
// return the data from the old proof and the new proof
return {
index: index,
siblings: oldLeafProof.siblings,
oldRoot: oldLeafProof.root,
oldValue: oldLeafProof.value,
newRoot: newRoot,
newValue: newValue,
}
}
}
function computeMerkleRootFromProof(siblings, index, value){
// start our merkle node path at the leaf node
let merklePathNodeValue = value;
let merklePathNodeIndex = index;
// we follow the leaf's merkle path up to the root,
// computing the merkle path's nodes using the siblings provided as we go alone
for(let i=0;i<siblings.length;i++){
const merklePathNodeSibling = siblings[i];
if(merklePathNodeIndex%2===0){
// if the current index of the node on our merkle path is even:
// * merklePathNodeValue is the left hand node,
// * merklePathNodeSibling is the right hand node
// * parent node's value is hash(merklePathNodeValue, merklePathNodeSibling)
merklePathNodeValue = hash(merklePathNodeValue, merklePathNodeSibling);
}else{
// if the current index of the node on our merkle path is odd:
// * merklePathNodeSibling is the left hand node
// * merklePathNodeValue is the right hand node,
// * parent node's value is hash(merklePathNodeSibling, merklePathNodeValue)
merklePathNodeValue = hash(merklePathNodeSibling, merklePathNodeValue);
}
// using our definition, the parent node of our path node is N(level-1, floor(index/2))
merklePathNodeIndex = Math.floor(merklePathNodeIndex/2);
}
return merklePathNodeValue;
}
function verifyMerkleProof(proof){
return proof.root === computeMerkleRootFromProof(proof.siblings, proof.index, proof.value);
}
function verifyDeltaMerkleProof(deltaMerkleProof){
// split the delta merkle proof into a before and after merkle proof, reusing the same siblings and index
const oldProof = {
// reuse the same siblings for both old and new
siblings: deltaMerkleProof.siblings,
// reuse the same index for both old and new
index: deltaMerkleProof.index,
root: deltaMerkleProof.oldRoot,
value: deltaMerkleProof.oldValue,
};
const newProof = {
// reuse the same siblings for both old and new
siblings: deltaMerkleProof.siblings,
// reuse the same index for both old and new
index: deltaMerkleProof.index,
root: deltaMerkleProof.newRoot,
value: deltaMerkleProof.newValue,
};
return verifyMerkleProof(oldProof) && verifyMerkleProof(newProof);
}
function example6(){
const height = 3;
const leaves = [
"0000000000000000000000000000000000000000000000000000000000000001", // 1
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000003", // 3
"0000000000000000000000000000000000000000000000000000000000000007", // 7
"0000000000000000000000000000000000000000000000000000000000000004", // 4
"0000000000000000000000000000000000000000000000000000000000000009", // 9
"0000000000000000000000000000000000000000000000000000000000000000", // 0
"0000000000000000000000000000000000000000000000000000000000000006", // 6
];
const tree = new MerkleTree(height, leaves);
const deltaMerkleProofOfN3_5 = tree.getDeltaMerkleProof(3,5, "0000000000000000000000000000000000000000000000000000000000000008");
console.log("[EX_6] delta merkle proof of changing N(3,5) from 9 to 8:\n" + JSON.stringify(deltaMerkleProofOfN3_5, null, 2));
console.log("[EX_6] verify delta merkle proof: "+verifyDeltaMerkleProof(deltaMerkleProofOfN3_5));
}
example6();Running this code prints:
[EX_6] delta merkle proof of changing N(3,5) from 9 to 8:
{
"index": 5,
"siblings": [
"0000000000000000000000000000000000000000000000000000000000000004",
"deb2a27a00dbc46e3a59096a8d07d2b97f950158886411ff6a9c9ab9623dada6",
"2f4d3e941b602c50347af3f5c809a28737c27c7ce460e77b10739875ef957aa7"
],
"oldRoot": "e74ae221cf067dc8e88195f5b30bfda60dc224a8cd5554bc6345c87ad8c6f925",
"oldValue": "0000000000000000000000000000000000000000000000000000000000000009",
"newRoot": "99ba6856d11cd514be35ed291e86ae7957ec43d6b83270f32aaa50a8a25dc5cc",
"newValue": "0000000000000000000000000000000000000000000000000000000000000008"
}
[EX_6] verify delta merkle proof: trueSuccess! We have now invented the core concepts behind merkle trees, merkle proofs and delta merkle proofs.
More importantly, with this intuition we can use our full understanding of how merkle proofs work to implement variants of these proofs to create useful and efficient data stores for our layer 2 protocol.
In the next tutorial, we will build an efficient merkle tree database for large sparse merkle trees, write an append only merkle tree that only uses O(log(N)) storage, and investigate some merkle proof variants that allow us to merge trees and prove historical state.
about the author — follow @cmpeq on twitter or check out my about page.
Vist QED at https://qedprotocol.com
