Crust Show：How does Crust Network play the “black magic” of Substrate?

On June 18 Crust Show 3rd event, we invited Parity engineer Mr. kai chao together with Crust Network CTO Zikun to have an informational discussion related to the core technology of Crust.

In this sharing meeting, Zikun had detailed explanations about from the Crust protocol group design and technical implementation, to other four aspects including the design and thoughts about seamless upgrade, how Crust to use of the cipher library and on-chain timer (Scheduler) on Runtime Interface extension chain, and Crust on-chain identity binding design and implementation.

This sharing content has been divided into two parts. The last part is talking about the design and technical implementation of Crust protocol group. This part is mainly focusing on how Crust applies Substrate.

The below is the sharing content about Crust Show (second part) .

1.As Substrate is an open source, modular, extensible framework for blockchain development, can you briefly explain why you chose Substrate for development?

First of all, Substrate is a particularly good development tool. Thanks to good design patterns of the Generic of Rust language and Substrate, Substrate forms a unique development model of building blocks, which allows developers to easily get started and use. And that’s the first feeling when I’ ve firstly used Substrate.

With an increasingly deeper understanding of Substrate, many of its “black magic” are getting interesting, including well-known Forkless Upgrade, Off-chain Worker, on-chain governance, or Crust applied Runtime interface and chain timers, all making protocol development of different types and functions pure and simple.

Crust is a relatively complex decentralized storage public chain with protocol stack, and its off-chain complex interactive mechanism, SGX special verification mechanism, and exquisite economic model design and so on make it more difficult to implement Crust protocol.

Before this, Crust tried over a month of a combination of Ethereum +Tendermint development. Since using Substrate began at the end of last year, however, Substrate “black magic” has greatly accelerated the implementation of the Crust protocol. Crust has completed 70% of the protocol development, and today will also share with you how these “black magic” are used.

2.Some of the components provided by the Substrate include the underlying database, point-to-point networks, consensus algorithms, trading pools, and modular Runtime, the state transition function. What components Crust reuse on a Substrate basis and what new features?

There are 5 main parts in the current Crust code base, Staking, Payment, TEE, Market and Balance. The Staking and Balance modules are multiplexed Substrate native components. The NPoS provided by native Staking and the Phragmen Election algorithm are modified to fit the protocol of GPoS by Crust. We also modify the native Balance modules to match the function of delaying release.

New features include TEE modules to implement MPoW protocols, Market module to realize decentralized storage/retrieval market and Payment module responsible for order payment. The Provider Slashing mechanism of the Market module also refers to the Slashing, of the Staking module and Slashing Span mechanisms have also been partially transplanted.

Last but not lest, Crust also refers to many Substrate design patterns, such as introducing Trait and Frame Support, Primitives of the common type definition and Mock/Tests to address circular references, all of which greatly improve the quality of Crust code.

3.You just mentioned the no-fork upgrade, I know it is one of Substrate core strengths, but how it is used by Crust, and what are the problems with the process?

Because the Crust protocol group consists of two parts, on-chain and off-chain, the no-fork upgrade is actually divided into two same parts: on-chain and off-chain.

The first is the on-chain upgrade: because the Crust is developed on top of the public chain, so the on-chain part of the upgrade Crust directly used the Substrate Forkless Upgrade.

The Runtime code will be stored on the chain in the form of Wasm in the Substrate. This part is the main logic of the on-chain, and Runtime modification can be done by no fork upgrade way. However, non- Runtime code modification need to upgrade Native program.

For this point, Crust have two aspects of consideration, that is how to upgrade and restrictions on the upgrade.

The first is how to upgrade. The current Crust approach is to initiate an on-chain timer on the Sudo or parliamentary module, agreeing to an on-chain upgrade on the specified block, which is also officially recommended.

The principle is to apportion set Code operation to each block of the timer Duration by Scheduler, so don’t worry about that code file too large to exceed the packaging capacity of one block or exceeding the value of the MaxWeight to cause the Extrinsic not to be accepted. The official rule for restrictions on on-chain upgrades is that they can be made without Native changes and with logical and storage matching.

But for some details there do not give a special description. Crust actually has conducted detailed experiments about the involved business and the situation may rise. These experiments include:

1. Upgrade of Off-chain Worker
2. upgrade of on-chain cipher library
3. change of on-chain storage
4. upgrade of DB
5. upgrade of Substrate Client module
6. adjustment of parameters on-chain

All the above have been tested, and offered detailed upgrade instructions, and then we will give Upgrade Red Line report, and you can focus on Crust Github Wiki.

Overall, however, Crust could complete more than 99 percent of the upgrades through the on-chain upgrade mechanism provided by Substrate. And also, Crust will be in the Maxwell test network to fully determine the case of 1%, so that the probability of Hard Fork made to the minimum. What followed by the off-chain upgrade, that is, the upgrade of the SGX program.

The significance of this upgrade is to fix or improve the efficiency of some quantification and encryption and decryption programs. Crust design an AB Upgrade mechanism to solve problems that Enclave cannot trust each other.

The figure above means that when Block Number=500, what is considered legal on the chain is Enclave A. And the Id A that submitted by Enclave A which is bound by Account A and its workload reports, are considered legal inputs by the chain.

When the Sudo /Democracy changes the on-chain storage status through a Set TEE Code operation and certifies the Enclave B as legitimate code, all Enclave A will be specified to fail after the block height reaching 1000.
But in the middle of block height of 500 and 1500, a simultaneous efficient intermediate state of A & B occurs, which is intended to allow B (new) to synchronize storage states of A (old) .

Until the Block Number becomes 1500, Enclave A’s submissions will be considered invalid by the chain. In short, Enclave A and Enclave B will exist simultaneously and quantify the storage resources during the upgrade process, and the chain will allow for the workload reported by both Enclave Code A & B.

In another word, Enclave A and Enclave B will both exist and quantify the storage resources during the upgrade process, and the chain will also allow for A & B workload reported by both Enclave Code.

But within the agreed period, the node needs to complete the upgrade from A to B. Beyond the agreed period, all information reported by the A will no longer be accepted on the chain.

This mechanism is a bit similar to the ETH difficulty bomb upgrade, using A & B Upgrade can be efficient to avoid and solve the problem of mutual trust in A & B, and can achieve smooth upgrade without letting users sense it.

4.Next question, you mentioned Runtime Interface at the beginning, and this is actually a more complex technical point in the Substrate, please tell us more about it.

To be honest, the introduction about Runtime Interface is little. Because Substrate doesn’t expect you to use Crate directly, but Off-chain worker which built on the base of this.

In other words, Runtime Interface is the basic component for Off-chain Worker. The principle is to bridge some fixed interfaces in Native and Wasm environments, and accept input parameters from the Wasm by the defined external functions by Wasm executor, and execute any programs in the Native and return the results to the Wasm environment.

It sounds quite similar to the Off-chain Worker, but it’s lighter or less than Offchain Worker. Because you don’ t have to make a life cycle like Off-chain Worker, just can use it as a function call. But like Off-chain Worker, the use of Runtime Interface also needs to notice that do not to have side effects on the chain, because full nudity does not have a timeout mechanism, and Runtime Interface makes it easier for you to have side effects on the chain.

So according to the Crust experience, you’d better not to introduce complex computing or IO cost, such as network waiting. Crust’s application of Runtime Interface is simply to extend the cipher library, which is also part of the MPoW protocol that cannot be missing, including CA X509 signature algorithm, ECDSA P256 elliptic curve algorithm and EVP Sha256 algorithm.

The introduction of Runtime Interface does greatly reduce the workload of Crust’s implementation on-chain validation.

5.When Runtime upgrade, you can usually use a chain timer (Scheduler) to specify the upgrade block, so does Crust use it?

The on-chain timer (Scheduler) is the more characteristic module in the Substrate that allows you to arrange a task or function to execute at a certain block number and to support periodically repeated operations.

The Scheduler module provided by the Substrate perfectly matches one of the Crust business scenarios, that is, the linear release of the storage order amount, and that is, the amount paid by the user to the storage node, which is linearly released to the storage node during the validity period of the file.

For example, user Alice store a photo onto the node A with a storage period of is 3 months, and she shall pay A total of 300CRUs. Market modules will split the CRUs into 3*30*24*60*0.0023 CRUs, that is, every 10 blocks or one minute, it will pay A 0.0023 CR and complete payment in 3 months.

So Crust introduced the Scheduler into Market module, but we can’t assign a Scheduler to a single order because the network of Substrate has a limit on the number of Scheduler on the entire network (about 30 to 40).

Based on this consideration, Crust designed an off-chain settlement system, that is, using Substrate specific Offchain Worker, to let Scheduler perform a periodic off-chain settlement system, and submit a Quote from off-chain to on-chain, and then perform a Batch clearing transfer work.

So we can allocate an average of 10 Scheduler over a period of era (300 blocks) to achieve about 300,000 orders releases. At present, this scheme has not been fully implemented, so there is no Benchmark or accurate test data, just a relatively rough estimate.

On the whole, however, Scheduler provides a simple way to delay Crust release. Without a chain timer, we may have to do a lot of things based on on_initialize or on_finalize, and we need to calculate our own cost.

6.My last question is, how does identity on Crust chain be designed and implemented?

Crust identity binding has three layers of logic :

1.obtain Intel authentication and TEE Enclave Code (environment reliability: to ensure a trusted operating environment), environmental reliability is to ensure that no malicious modification on Enclave Code and to ensure that the environment in which the Enclave runs is credible.

2.bind TEE Enclave Code and generated public keys (reliable output: to ensure matching of running environment and TEE identity).

Since all behaviors outside the Enclave can be controlled and modified by the users, the public and private keys of the user must be generated within the Enclave (the private key is only available within the Enclave) and the output of all Enclave is encrypted with the private key.

This makes it impossible for the user to modify all the content output from the Enclave and ensures the reliability of the output.

3. public key and chain identity generated by the binding (Reliable chain identity: to ensure TEE identity and chain identity matching), avoid Sybil attacks with multiple chain identity binding one TEE.

This can lead to the theft of TEE identity, so this step will generate on-chanin identity binding information within the TEE, and conduct the unique on-chain confirmation.

Through these three-layer binding, it can assure the whole chain from Enclave to Chain be trusted, which provides a credible basis for MPoW.