Crust Show：Design and Implementation of Crust Core Protocol Stack
On June 18 Crust Show 3rd event, we invited Parity engineer Mr. kai chao together with Crust Network CTO Zikun to have an informational discussion related to the core technology of Crust.
In this sharing meeting, Zikun had detailed explanations about from the Crust protocol group design and technical implementation, to other four aspects including the design and thoughts about seamless upgrade, how Crust to use of the cipher library and on-chain timer (Scheduler) on Runtime Interface extension chain, and Crust on-chain identity binding design and implementation.
This sharing content has been divided into two parts. This part is mainly focusing on the detailed introduction about the design and technical implementation of Crust protocol group.
The below is the sharing content about Crust Show (first part) .
1.Would you please specify how Crust protocol group design and implement MPoW, GPoS, Market and Storage?
The Crust protocol group is designed to address two key issues in the DS (Decentralized Storage) incentive layer: proof and motivation.
The proof mainly refers to how to correctly reflect the off-chain storage state onto the chain; and the incentive is referring to the incentive scheme for the storage node, that is, the way the node gets the reward.
GPoS protocol group: Crust defines a protocol group, which includes the MPoW protocol for quantifying the off-chain storage state, the GPoS protocol for distributing on-chain block rewards to nodes, the Market module for decentralization (storage/retrieval) market, and the Storage module for adapting multiple storage protocols.
2.I notice that your GPoS protocol group contains TEE modules. Can you tell me more about how TEE apply inside of the protocol group?
TEE is the core module of the MPoW protocol, which provides the trusted and effective input off the chain. MPoW also provides initial input for GPoS protocols and Market modules, so MPoW is the basis for the entire protocol group. Specifically, MPoW have three core functions:
1.TEE through identity binding can authenticate the node of the trusted identity (explained later)
2.TEE credits and quantifies the meaningful files and SRD (Sealed Random Data, aka) of users and is submitted onto the chain through workload reports (data block proofs)
3.Provide the correct initial input (storage and file information {Merkle Root, File Size}) for the transfer of the functions of state on the chain;
Moreover, The MPoW proof process (both off and on the chain) is very efficient:
1. Since TEE submissions occur locally and can be self-validated ( non-tampering feature of SGX programs), we can constantly optimizing the entire validation process by eliminating the network spending of mutual validation of data blocks and the computational spending of native zero-knowledge proof. The current performance of the data is very satisfactory ,the timing of full verification of 1 TB of data (whether SRD or meaningful documents here) is 18s. Also, under the circumstance of the selective examination of verification and the examination rate of 2%, our verification process only needs 0.36.
2. In addition, the verification on the chain is benefited from the efficient Native implement of Rust and the interactive mechanism of Native & Wasm on the top of Substrate Runtime Interface. The data currently being tested is that the signature verification(behind which stands the verification of data blocks proof) of conducting p256 elliptic curve 100 times costs only 0.016s.
3.What does it mean specifically for GPoS and Market of the on-chain protocol?
GPOS:
GPoS (Guaranteed Proof Stake, proof of security interest), a protocol for distributing block incentives to nodes, is inspired by Pokaldot’s NPoS and combined with innovations in our system.
You can focus on the recent release of Crust Network Economic White Paper Draft. Let’s mainly talk about how GPoS achieved. First, as we talked about MPoW as the basis for the entire protocol group, we set the initial input of the on-chain state functions, which is the workload/storage of the nodes, containing how much storage capacity the node has, how many meaningful files are stored, and the size of each meaningful file. The data matrix is transformed into a key value, StakeLimit. in the GPoS by a formula.
In the GPoS, Stake Limit be used as a node-deposited/guaranteed amount, throughout the whole GPoS protocol flow.
Certainly, the realization mechanism of the whole GPoS is more complex, including active detection, passive detection, valid ticket, delay guarantee, multiple guarantee and so on.
This technology sharing probably won’t cover all the implementation details, and then we plan to arrange a technology forum to talk about all GPoS detail rules.
Market:
The current design and implementation of the Crust version is the storage market; the retrieval market will be online with the launch of the Maxwell 2.0 version. In this technology forum we can first talk about the design and implementation of the storage market. Which is also based on given initial input of MPoW, Market module only focuses on the meaningful file submitted by the node Merkle Root.
According to the corresponding Market storage order information of Merkle Root, thus it reflects whether the node stores the file effectively. Market will also periodically check whether the files of all valid orders under Provider are reasonably stored, and once the detection fails, the order inefficiency of provider will also be recorded on the chain and it will be punished according to different degrees of file loss.
The implementation challenge of Market module is the huge order quantity, and about this part we will adopt Substrate Offchain Worker to make order record and upgrade by Forkless Upgrade in Maxwell subsequent version.
4.We know that Crust is a decentralized storage incentive layer. Can you introduce the storage protocols in the protocol group?
About the storage part, we mainly want to talk about Crust’s supported storage layer protocol, Kars. For Karst, we define it as a Storage Adapter.
What is a storage adapter, that is, Karst protocol will transform different storage protocols (including IPFS, FastDFS ,HDFS, etc.) into storage structures that can be identified by TEE and persist locally.
Within the Karst, a range of common interfaces are defined, such as storage transformer, which can be qualified by TEE through realizing this interface and return the speculated structures by Karst, Sealer (calling TEE Seal interfaces and Unseal interfaces to encrypt and decrypt storage, preventing Sybil Attack and Generating Attacks), and Checker (interfaces that can be selectively examined by TEE).
The interesting thing about this design is that, no matter how the underlying storage and the requirements change, if it fit the FS in the Karst protocol, it can be quantified into the Crust network. And this also enables Crust networks to coexist with multiple protocols, and decoupling storage protocols and quantization layers is what we think is the right direction.
▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼
During the AMA session, as here come too many questions from the community, we have to select some of the more concerned questions and sorted out:
1.I want to ask a question: when the miner pledge, there should be both the storage pledge and the pledge from the verifier, so in the punishment, the two parts of the pledge is logically separated or related?
Zikun: It’s separated. Regarding GPoS and Market, they are our two games designed on the basis on CRU token. The rules of the game are independent, whose independence includes reward, mortgage and fine, in which GPoS gets block reward, and Market gets order reward.
The contribution degree of the two modules is also different for the Crust network, where GPoS ensures the basic operation (determine generating blocks) of the Crust network, and Market supports the application characteristics of the decentralized storage of the Crust network.
2. All decentralized storage have to face a problem, that is, how strongly fluctuating price of currency coordinate with locked storage costs and storage pledge. Assuming that the currency price plummeted, resulting in the miner’s deposit is no longer worth any money, or the currency price skyrocketing, resulting in the miners think that the previous storage order was lost. How do you use a fluctuating Token payment system to keep storage stable?
Zikun: Well, we’ll introduce a stable currency mechanism, which is now designed out, but there are still some details on the implementation that we need to think over and will share it out when it is ready. We all have consider on within the ecology such as Acala (through Parachain), and USDT outside of the ecology.
A stable currency is at the heart of the storage market, and we’re grinding it over and over again, but it’s not very mature yet. But good question, answer is to keep an eye on us.
3. According to the Crust plan, the bottom layer only realizes the interfaces such as simple access, and for reliability and availability assurance will be placed on the top, this upper layer is also developed by Crust? And is its guarantee guaranteed by the chain, or a certain centralization factor? For example, a user saved four copies of the data, he hopes that after losing two copies, someone can help him re-order to others to save.
Zikun: Well, two questions.
1. Reliability & Availability. Concretely speaking, they are node acceleration and redundancy deletion, and these must be two layers to improve the mechanism, Crust will develop a template for community use.
2. Chain will record the quality of service of Provider through the order status of the Market module, including order failure rate, continuous failure time, cumulative failure time. These QoS will slash on the first layer, but on the second layer there is more to be done (such as the QoS of history).
The mechanism you’ve just used as metaphor is the second layer. The Crust network layer one will only define some meta-words: the order status is the QoS meta-language, the second layer can stack this part for complex combination, so as to evaluate the richer QoS mechanism.
4. Do you decide how to deal with Storage node Slash? For example, give the penalty money to a new node, and let him retrieve to continue to save. Either burn a part, or return it to the user or something, I see Filecoin’s current design does not take reliability into consideration.
Zikun: Maxwell 1.0 will have a simple Slashing. File missing is 5%, which will Slash 50% of the deposit of this order ,10% will be directly fined.
This parameter, however, will always be polished, including the mechanism you mentioned, that is, whether the Slashed money is burned or which is quite similar to the Graceful Quit of Storj that distribute to other nodes, and it will be changed by on-chain update.
