Building Layer 1 blockchain from scratch (PART — III Database)
This part is in continuation of part 2
In the previous parts of our series, we laid down the foundation of a basic blockchain by implementing transaction handling, block creation, and mining. In part 3 we will focus on integrating a database to persist blockchain data and using Merkle Tree for efficient and secure storage of transactions.
Environment:
Python Version: 3.11.6
Operating System: Linux Ubuntu
Database: LevelDB (from the previous part)
Understanding Merkle Trees
Before diving into the code, let’s first understand Merkle Trees and their importance in blockchain technology. A Merkle Tree is a binary tree where each leaf node contains a cryptographic hash of a data block, and each non-leaf node contains the cryptographic hash of its child nodes. The top node, known as the root, provides a single hash representing the entire dataset.
Merkle Trees are crucial in blockchain for the following reasons:
- Efficiency: They allow for efficient and secure verification of large datasets.
- Integrity: Any change in the data (transactions) would alter the hashes in the tree, making tampering easily detectable.
- Proof of Inclusion: They allow for the verification of the inclusion of a specific transaction without revealing the entire dataset.
Database Integration with LevelDB
To persist blockchain data, we’ll use LevelDB, a fast key-value storage library. In our implementation, we’ll store account states and transactions using LevelDB and organize them with Merkle Trees for integrity and efficiency.
Refer to code here
Code
1. Importing Required Modules
import json
from eth_account import Account
from eth_account.messages import encode_defunct
from typing import Dict, Optional
from eth_utils import keccak
import plyvel
from trie import HexaryTrie- Here, we import essential libraries, including
eth_accountfor handling Ethereum accounts,plyvelfor interfacing with LevelDB, andHexaryTriefrom thetrielibrary to implement Merkle Trees (in the form of tries).
2. .Creating a wrapper for Plyvel
class PlyvelDictWrapper:
def __init__(self, db):
self.db = db
def __getitem__(self, key):
value = self.db.get(key)
if value is None:
raise KeyError(key)
return value
def __setitem__(self, key, value):
self.db.put(key, value)
def __delitem__(self, key):
if self.db.get(key) is None:
raise KeyError(key)
self.db.delete(key)
def __contains__(self, key):
return self.db.get(key) is not None- The
PlyvelDictWrapperclass provides a dictionary-like interface for interacting with the LevelDB database. This allows us to use the trie structure with LevelDB as if it were a simple dictionary, abstracting the complexity of direct database operations.
3. Initialising the data base
# Initialize the LevelDB database
db = plyvel.DB('b1', create_if_missing=True)
# Load the existing root hash from the database, or use the default empty root
root_key = b'state_latest_root'
existing_root = db.get(root_key) or HexaryTrie().root_hash
# Initialize the tries with the wrapper
state_trie = HexaryTrie(PlyvelDictWrapper(db.prefixed_db(b'state_')), root_hash=existing_root)
storage_trie = HexaryTrie(PlyvelDictWrapper(db.prefixed_db(b'storage_')))
transaction_trie = HexaryTrie(PlyvelDictWrapper(db.prefixed_db(b'transaction_')))
receipt_trie = HexaryTrie(PlyvelDictWrapper(db.prefixed_db(b'receipt_')))- Database Initialization: We initialize the LevelDB database and load the latest state trie root hash from the database. If no root hash is found, we start with an empty root.
- Trie Initialization: We create instances of
HexaryTriefor different types of data: state_trie: Stores the state of accounts (e.g., balances and nonces).storage_trie: (Placeholder) Could be used for smart contract storage.transaction_trie: Stores the transactions.receipt_trie: (Placeholder) Could be used for storing transaction receipts.
4. Managing Account States and Balances
#State Root Retrieval
def get_latest_state_trie_root() -> str:
return state_trie.root_hash.hex()- State Root Retrieval: This function retrieves the latest root hash of the state trie, which is crucial for verifying the integrity of the entire state of accounts on the blockchain.
#Account State Retrieval
def get_account_state(address: str) -> Dict[str, int]:
state = state_trie.get(address.encode())
if state:
return json.loads(state)
return {"balance": 0, "nonce": 0}- Account State Retrieval: This function retrieves the state (balance and nonce) of a specific account from the state trie. If the account does not exist, it returns a default state with a balance of 0 and a nonce of 0.
#Balance Retrieval
def get_balance(address: str):
state = get_account_state(address)
return state['balance']- Balance Retrieval: This function simply returns the balance of a given account by calling
get_account_stateand extracting the balance from the returned state.
#State update
def update_account_state(address: str, balance_change: int):
state = get_account_state(address)
state['balance'] += balance_change
state['nonce'] += 1
state_trie[address.encode()] = json.dumps(state).encode()
# Persist the latest state trie root in the database
db.put(root_key, state_trie.root_hash)- State Update: This function updates the state of an account after a transaction by adjusting the balance and incrementing the nonce (to prevent replay attacks). The updated state is then saved back into the state trie, and the root hash is updated in the database.
5. Adding and Retrieving Transactions
#Signature varification
def verify_signature(transaction: Dict[str, str]) -> bool:
message = f"{transaction['Sender']}:{transaction['Receiver']}:{transaction['Amount']}"
encoded_message = encode_defunct(text=message)
recovered_address = Account.recover_message(encoded_message, signature=transaction['signature'])
return recovered_address == transaction['Sender']- Signature Verification: This function verifies that a transaction was signed by the actual sender by reconstructing the signed message and checking it against the sender’s public key.
#Adding a Transaction
def add_transaction(transaction: Dict[str, str]) -> Optional[str]:
if verify_signature(transaction):
sender = transaction['Sender']
receiver = transaction['Receiver']
amount = int(transaction['Amount'])
sender_state = get_account_state(sender)
if sender_state['balance'] < amount:
print("Insufficient balance")
return None
update_account_state(sender, -amount)
update_account_state(receiver, amount)
transaction_data = json.dumps(transaction)
txn_key = keccak(transaction_data.encode()).hex()
transaction_trie[txn_key.encode()] = transaction_data.encode()
return txn_key
return NoneAdding a Transaction:
- Step 1: The transaction signature is verified.
- Step 2: The sender’s account is checked for sufficient balance.
- Step 3: If valid, the sender’s balance is reduced, and the receiver’s balance is increased.
- Step 4: The transaction is stored in the transaction trie, and its key is returned.
#Retreiving a transaction
def get_transaction(txn_key: str) -> Optional[Dict[str, str]]:
txn_data = transaction_trie.get(txn_key.encode())
if txn_data:
return json.loads(txn_data)
return None- Retrieving a Transaction: This function retrieves a transaction from the transaction trie using its unique transaction key.
🏁 Conclusion and What’s Next
In this part , we integrated a database and implemented Merkle Trees to manage and verify our blockchain’s data. We explored LevelDB for data persistence, managed account states and balances, and handled transaction verification and retrieval.
This part provided a hands-on experience with the core concepts of blockchain technology, including cryptographic security and decentralized storage.
In the final part, we’ll dive into peer-to-peer (P2P) networking, enabling our blockchain to operate in a decentralized environment. We’ll explore how nodes communicate, share blocks, and maintain consensus across the network.