SDBS #34: Final Testnet Push before Mainnet Junaeth
Summary
Today’s post will review all the changes in the latest code commits to github. This push represents the last major push before taking three years of development live to mainnet. Highlights of the recent changes are:
- An Easter egg: NFT art! Each StealthNode now has NFT art associated with it, created by the renowned illustrator Johanna Westerman.
- A new authority for StealthNodes, called “Manager authority”, that allows a StealthNode owner/operator to exercise full control over a StealthNode without exposing the Owner key.
- Forced delay for re-enabling disabled StealthNodes. Requiring disabled StealthNodes to wait before re-enabling adds to blockchain security by increasing the immediate penalty for misbehaving StealthNodes.
- Un-crippling of network nodes. Network nodes now operate with nearly 100x (one hundred times!) more capacity thanks to removing unnecessarily wasteful code.
- Thread-safe feework buffers. This testnet push adds this necessary thread safety to memory buffers used in creating and verifying feeless transactions.
— — — — — — —
NFTs!
The essence of a StealthNode has always been that it is a blockchain token that permits the owner to validate blocks and claim block rewards. No StealthNode is the same as any other, making them non-fungible. To underscore that StealthNodes are non-fungible tokens (NFTs), I have added character identities to StealthNodes.
The fact that StealthNodes are NFTs can easily get lost on most people who think of NFTs as “blockchain artwork”. To remedy this confusion, we have hired a renowned and well-published illustrator, Johanna Westerman, to produce 100 works of art to be associated with StealthNodes as character cards.
When a user purchases a StealthNode, they have the option to specify which character card they want to be permanently associated with their StealthNode. In other words, once the StealthNode is purchased, the only way to transfer ownership of the NFT artwork is to transfer ownership of the NFT itself (i.e the StealthNode).
To illustrate how claiming the character works, here is the command that purchases a StealthNode:
purchasestaker <txid> <vout> <alias> <owner>For simplicity, I have left off the optional arguments (these options are discussed below).
Here, a user can specify a character with the “alias” argument as a number (the unique character identifier) or as the nickname. For example, to specify Kaleo (pictured above), the purchaser would either put “Kaleo” or “16” as the alias when making the purchase. When specifying a character, the StealthNode is given that character nickname.
A user can also forego specifying a character, and thus provide any name they want for their StealthNode, except for any existing StealthNode name or any character name. In this case, the purchaser will be assigned, permanently, the next available character in numerical order of character identifier.
StealthNode NFTs are a quantum improvement over existing models for NFTs. Not only can StealthNode NFTs secure a blockchain, but the precise identity of the NFT artwork (the character card) is part of the Stealth protocol. At block 3,635,099 on mainnet, the hash (fingerprint) of the first 100 character cards will be catenated, and then hashed, and then prepended to the block data that is itself hashed to create the block hash. In Pythonic pseudocode it looks like
hashes = catenate([hash(card) for card in character_cards])block_hash = hash(catenate[hash(hashes), block_data])
This process creates an immutable blockchain commitment to all 100 of the first NFTs, meaning that even if the URL for the NFT ceases to be accessible, a StealthNode owner can prove ownership of the character card image simply by calculating the hash of the image and comparing it to the corresponding hash in the code. For this proof, the owner should retain a copy of the image by saving it from the original URL, a very simple task possible from any web browser.
With this instructive (and admittedly fun) addition to Stealth’s blockchain protocol, we believe that StealthNodes are the “ultimate NFTs”, surpassing any previous NFT in several ways. (1) StealthNode NFTs are functional in that they tokenize blockchain validation rights; (2) StealthNodes are based on an immutable commitment to the actual artwork; (3) the artwork commitment is protocol-level in that it is part of the reference codebase; (4) the NFT (and its art) is purchased from the blockchain and not from the artist; and (5) the artwork is created by a highly respected artist specifically for this project.
Stay tuned for the gallery of mainnet character cards, and be ready for the StealthNode purchase period if you have a favorite NFT character card that you want to acquire as part of your StealthNode.
— — — — — — —
New StealthNode Manager Authority
In short, the new Manager authority can do everything that the Owner authority can do, except for transfering ownership.
The cardinal functionality of a StealthNode is to validate blocks for the network. Each StealthNode has a cryptographic key pair, known as a “Delegate key pair”, that is used to sign blocks. This key pair is termed the “Delegate authority”. Notably, the Stealth blockchain protocol will ensure that Delegates get an owner-specified payout for each and every block produced. In other words, after the Owner sets the payout, the blockchain takes care of paying the Delegate. This automated system simplifies the fiducial relationship between Owner and operator (Delegate). Stealth is unique in that it is the only blockchain that fully mediates remittances between block validators (Delegates) and stakeholders (StealthNode Owners).
Along with the Delegate authority, StealthNodes have other authorities important for operation. For example, the “Owner” authority is uniquely necessary and sufficient to transfer ownership of a StealthNode. The Owner and Delegate key pairs can be different, allowing Owners to hire a second-party service to validate blocks. In these cases, the Owner can also hire a third-party service to observe the Delegate and disable the StealthNode if the Delegate service abuses its authority to sign blocks. This overseer utilizes the “Controller” authority. Importantly, the Controller authority is limited to just this oversight function.
With the latest changes, StealthNodes now have an additional authority known as the “Manager” authority. Managers can perform all the functions of an Owner except for transferring ownership. On mainnet (and soon on testnet), all non-Delegate block validation rewards will be paid to Managers, and not to Owners, as has been the case on testnet.
The most important consequence of introducing the new Manager authority is that one never needs to expose the Owner private key in a “hot wallet” environment. For example, Owners can generate a public-private key pair on a Ledger hardware wallet, get the compressed public key, and transfer ownership to that public key, making the StealthNode as protected against theft as any asset on a hardware wallet. I’ll provide detailed instructions for this process soon. The added security will be worth the effort.
Without going too deeply into how to interact with the Manager authority, I’ll explain its introduction changes purchases. Prior to introducing the Manager authority, a testnet StealthNode buyer could specify three different public keys when purchasing a StealthNode. Now the specification has expanded to four keys. The command takes the form:
purchasestaker <txid> <vout> <alias> <owner> [amount] [manager delegate controller payout]
Make note of the part where it says “manager delegate controller payout”. Previously, “manager” was absent from this list. It’s inclusion now means a purchaser has to supply all nine (9) arguments to specify a manager public key. If the purchaser doesn’t supply these latter four arguments, all authorities will be the same as Owner, which I should emphasize is an insecure way to assign authorities.
The following list summarizes StealthNode authorities:
Controller Authority:
- Enabling the StealthNode
- Disabling the StealthNode
Delegate Authority:
- Signing blocks
- Setting certain StealthNode extensible attributes
Manager Authority:
- Enabling the StealthNode
- Disabling the StealthNode
- Setting ALL StealthNode extensible attributes
- Setting the Controller authority
- Setting the Delegate authority
- Setting the Delegate payout
- Setting the Manager authority
Owner Authority:
- All functions of the Manager authority
- Transferring ownership
— — — — — — —
Delay to Re-enable Disabled StealthNodes
StealthNodes can be disabled automatically if they (1) miss too many consecutive blocks or (2) StealthNodes immediately preceding them in the queue miss too many consecutive blocks. Disabling problematic StealthNodes improves blockchain performance. Additionally, the threat of being disabled incentivizes good performance. To prevent StealthNodes from immediately re-enabling and continuing bad behavior, any StealthNode that has been disabled must wait 12 hr (8640 blocks) before it can be re-enabled.
One feature that StealthNodes have is the ability to be enabled and disabled. Enabled stealth nodes are included in the block signing queue and are obligated to validate blocks during allotted time intervals (called “slots”). A node that is in the queue but does not validate a block during its slot faces a couple of penalties. First, its reputation is docked, resulting in fewer earnings for each block it validates in the future. The second penalty is that a counter is incremented that tracks whether a StealthNode should be disabled. After 512 consecutive misses, a StealthNode will be disabled.
The primary reason for disabling a StealthNode for missing too many consecutive blocks is not so much to penalize it, but to give the operator time to fix the problem, and to allow blocks to proceed regularly, without gaps created by the problematic StealthNode.
But missing blocks is a form of misbehavior, and penalties are used to discourage misbehavior. Previously, a StealthNode operator could immediately enable a disabled StealthNode, making this penalty for misbehavior limited to the small fee required to re-enable the StealthNode. With the latest changes, the mainnet policy is that a StealthNode that has been disabled will need to wait at least 12 hours (8640 blocks) before it can be enabled. This wait time is enforced not only for StealthNodes that have been disabled automatically for missing blocks, but also for StealthNodes that are disabled manually by StealthNode operators.
— — — — — — —
Un-crippling of Testnet Nodes
Anyone who has been part of the testnet for the past 2+ years knows that the testnet never really worked quite right. Nodes often became completely disconnected from their peers, created blocks too late to be valid, and did not receive blocks created by their peers in a timely manner. All these problems put stress on the network to stay on the same chain.
I have a confession: I wanted the network nodes as crippled as possible. One goal of testnet was to root every last consensus bug from the novel blockchain consensus protocol, called Junaeth, that lies at the heart of the forthcoming mainnet release.
To find consensus bugs, a testnet that functioned perfectly because all the nodes functioned perfectly was of little use. Perfect function means perfect communication and perfect communication means consensus is never challenged. That scenario is useless for finding problems with the consensus protocol itself.
Instead, we needed a big wrench bouncing around knocking things loose so we could see what parts of the protocol needed tightening. To this end, I let a horrible performance bug remain that had its origins in early versions of the bitcoin codebase.
The bug went like this: Each block in a blockchain points to the block immediately preceding it. For example block 100 points to block 99, and block 99 points to block 98. The data structure for each block also stores the block’s height as a number. To find a block by its number, which is a common task in blockchain consensus protocols, the old code would start at the best (highest numbered) block, and follow this train of pointers back until it found the block with the matching height.
To see how this height-finding algorithm could cause inefficiency, imagine a blockchain protocol with four million blocks (for example) that needed to provide block information all the way back to block 1. Every time it needed to do this it would thumb through all four million blocks. Nodes actually do this very frequently when they inform peers of their blockchain histories, to help the peers decide which blocks they want to request when updating their blockchains.
It might be surprising, but nodes do this process so often that, prior to the latest code changes, testnet nodes were spending well over 99% of their time thumbing backwards through blocks! This ridiculous burden crippled nodes to where they were running at less than 1% capacity. It sounds bad, but this situation is perfect for discovering consensus bugs. A consensus protocol needs to be excellent to work under conditions where every node is nearly 100% crippled.
With the mainnet release coming up in a couple of weeks, I couldn’t let this efficiency bug persist. The fix was simple. In the new code, I created an in-memory lookup that indexes blocks by their height. With four million blocks, this lookup is over 1,000 times faster than thumbing through blocks.
Testnet performance is now nearly perfect. Before testnet, nodes missed about 13% of blocks on average. Since the new changes went into effect, testnet StealthNodes miss about 0.04%. This small fraction of misses is easily attributable to my own testing of nodes, starting and restarting them, which incidentally causes them to miss blocks.
The bottom line is that not only will Stealth’s Junaeth blockchain consensus protocol be unique, it is now ready for mainnet. I predict that Stealth will be one of the most robust and high performance blockchain protocols available.
— — — — — — —
Feework Buffer Thread Safety
We recently introduced feeless transactions to the Stealth testnet. To speed feework creation and validation, these buffers are initialized at startup and persist until shutdown, saving a lot of time allocating and deallocating memory. These buffers were originally not safe for multithreading. The new changes will allow a highly multithreaded approach to validation by StealthNodes. Additionally, the reference client can be used by third-party services that concurrently calculate feework for multiple low power devices that host user wallets, like mobile devices.
— — — — — — —
The Stealth Team
— — — — — — —
