Atticbee’s AMA in the China community, English transcript | 2021/03/03

Guest: Atticbee, a member of the China community of RChain Cooperative, graduated from Zhejiang University and Carnegie Mellon University with a major in computer science, and later settled in the United States and worked in the IT industry for a long time. As a blockchain technology enthusiast, I began to pay attention to RChain’s technology in 2017, had a lot of communications with members of the RChain cooperative, and have been promoting the awareness of RChain’s unique technology in the Chinese community.

Zora: People familiar with the RChain project know that Atticbee has made a lot of contributions to the RChain China community. He follows closely the development of the RChain project, and organizes and shares knowledge so that everyone can easily understand this technology.

Zora: Q1 — Please briefly introduce the RChain project itself, let us deepen our understanding of the project.

Atticbee: We recently made a one-page introduction of RChain. I just copy and paste here:

Based on the revolutionary Rho calculus, RChain solves a series of problems preventing blockchain platforms from mainstream adoption.

Its fast and scalable conflict detection algorithm accompanied with Casper CBC consensus allows:

– all nodes to produce and verify blocks concurrently without global epochs so it becomes the first smart contract platform to achieve single-shard scalability.

– large data to be stored directly on chain, removing the dependency on other data storage solutions such as IPFS.

Built on Rho calculus, RChain is the first chain to allow any complicated cross-shard transactions to be verified and finalized atomically and concurrently by all involved shards. This means the cross shard transactions can be done seamlessly and safely just as within a single shard.

It will also be equipped with a Rho calculus based behavioural type system. This allows smart contracts to be verified quickly in a concurrent and sharded setting, which makes possible the orchestration of large quantities of smart contracts.

Its unique reactive smart contract system is more suitable than other chains’ active systems for time sensitive applications such as Defi.

RChain’s unique technology makes it the best candidate to build a functional world computer.

However I want to talk more about this today.

Unlike many other projects, RChain is not a flash in the pan project based around the new hype of blockchain. Instead, Greg and his followers have been researching the revolutionary Rho calculus for more than ten years.

They have always wanted to use Rho calculus to build a usable global computer, because it is almost the only logically and mathematically available option, as can be seen from the following table:

However, they could not solve several problems: redundancy, trustable computation and no tampering with results. With the rise of blockchain, all these challenges were solved.

After the rise of Ethereum, Greg felt that the Turing machine based Ethereum (EVM is a Turing machine or state machine computing model) was unable to become a world computer due to its fundamental defects, including its ability to scale. He went to lobby Vitalik to adopt Rho calculus. Vitalik did not accept this idea due to various reasons, so RChain was started as an individual project.

Zora: In recent years, great attention is paid to the development of public chain projects, as they have also faced many difficulties in technology. RChain just launched its mainnet. As a rookie public chain project, it brings some technological innovations to the blockchain industry.

So Q2: RChain was originally an ETH smart contracts based token. What are the reasons for RChain to research and solve the concurrency problem?

Atticbee: It’s very simple, concurrency is the foundation of all scalability. To do concurrency, logically, many nodes should be able to produce blocks at the same time.

These blocks are like cars on a highway. If you put them in a single lane, it becomes single-threaded. You need to let them run together in many lanes. But these blocks may be related to each other, so a DAG is formed to allow for concurrent and conflict free transactions.

There are already many DAG based projects that do token transfers, but there are almost no smart contract platforms that have a truly implemented DAG to concurrently execute transactions.

For example, Conflux, Casper Labs, FTM, etc., if you dig deeper, you will find that they are all projects that pretend to be concurrent executing but still have hidden single-threaded bottlenecks. That is to say, yes you have many lanes, but all your traffic may converge into one lane at a certain point and certain moment, which is not scalable.

All in all, the real problem is: for all Turing machine based smart contract platforms (almost all existing projects use EVM or WASM), it is extremely difficult to hook up a new block to an existing DAG.

This is a simple diagram. The picture shows “block merge”, which is a milestone recently completed by RChain.

Suppose one node has seen blocks 1, 2 and 3 in the current DAG, and now it produces block 4. It wants to hook up block 4 to blocks 1, 2 and 3. That means block 4 will end with same state after running blocks 1, 2 and 3 and block 1, 2 and 3 can run concurrently.

This process is called “block merge”. For maximum concurrency, when you produce block 4, you want to hook it to as many ending blocks of the existing DAG as possible, as long as there is no conflict among them. So back and forth, you’ll grow a DAG like this:

For RChain, contract deployments are like Lego blocks, which will not change as environment changes, are easy to analyze, and are easy to hook up to the DAG.

For Turing machine contract deployments, they are like shapeshifters. If environment changes, those shapeshifters need to be run again, otherwise you don’t know what they may turn into. For RChain’s LEGO blocks, you can analyze their behavior by just looking at them and you easily know whether deploys are conflict free.

When one programmer merges a Pull Request on Github, he can easily find and resolve conflicts in the code, because the texts are also like LEGO blocks, rather than shapeshifters changing constantly.

This is why RChain can handle concurrency easily, while it is extremely difficult for other projects.

Casper Labs is a good example. They were forked from RChain partly because they feel that Rho calculations are not popular and it is better to switch to WASM so that the new platform can attract more developers.

But this is what their DAG looks like:

Speaking of this, it is time to show this correspondence table.

Correspondence between blockchain and traditional computer

Consensus ProtocolI/O layer with redundancy and error correctionPOS Staking groupAccess controlConcurrencyMultithreading, multicore executionShardingMulti-process, clusterFormal verificationFirewall

Shards are similar to processes, and concurrent execution is similar to multi-threads. You do sharding because you want to divide different consensus groups and related resources. For example, you don’t need to let the whole network know the contents of a chat dApp, as long as the nodes of the chat dApp shards reach a consensus, it is totally fine.

Many projects rely on sharding as a TPS scaling solution. This is wrong from the beginning. It is equivalent to using multiple processes to speed up execution.

Sharding with resource and access control isolation is the process’s job. To improve TPS within a single shard, you need multi-threading instead, which means that all nodes can produce blocks at the same time (multithreading) to achieve the TPS scalability within the shard.

A process is very heavy with a lot of overhead; a thread is much smaller.

So the correct approach is this: first, use sharding to expand the “memory space” of the blockchain computer, which is the state space of the virtual machine; secondly, use concurrency to improve TPS. Each does its own work.

In the whole crypto world, only RChain has planned for both scaling solutions, and it is the first one to truly do concurrent execution of smart contracts.

Zora: Q3. Most public chain projects now emphasize sharding and scalability. Could you please explain what kind of relationship exists between concurrency and sharding?

Atticbee: As already explained above: Concurrency = multithreading, sharding = multiprocess.

Sharding is used to expand the memory space of the world’s computers, not to improve TPS. The industry’s use of sharding is actually a desperate move because they cannot achieve concurrency within a shard.

BTW my tweet with the metaphor of “concurrency = multithreading, sharding = multi-process” was retweeted by the RChain coop, so it seems they also agree with my view.

Zora: Q4. Since Bitcoin, there have been many well-known public chain projects. Even so, public chain projects still face many difficulties. How did RChain solve and achieve technical break throughs?

Atticbee: This question is relatively broad. For my understanding, a public chain is a world computer. The ultimate goal is to do general computing on chain and implement a real web 3.0.

To make a functional world computer, several prerequisites are needed: scalability, seamless interoperability across various shards and chains, and fast and low-cost formal verification.

RChain uses both multi-threading (concurrency) and multi-process (sharding) to solve scalability. The latter (multi-process) is used to isolate resources. Now we come to an interesting problem: the question of seamless interoperability across shards.

One of the most important aspects of sharding is the interoperability. If you divide network into many shards, and those shards just do their own things and can not interoperate very well, it defeats the original purpose of sharding.

Generally speaking, there are two types of cross-shard transaction consensus mechanisms: synchronous consensus and asynchronous consensus.

Synchronous consensus: RChain.

Asynchronous consensus: Ether 2.0, Near, Polkadot.

Synchronous consensus means that cross-shard transactions need to be finalized by the validators of two shards at the same time, so it guarantees the atomicity of the cross-chain transaction. Asynchronous consensus is to divide the cross-shard transaction into stages, and each shard only processes its own part. In this way, the transaction is no longer atomic, and there are many problems with this approach.

For example, I exchange Token A in Shard A for Token B in Shard B. Then under the synchronous consensus protocol, nodes from shard A and B work together to approve this transaction at once, so there is no problem.

If you use the asynchronous consensus solution, then first you need to transfer Token A in Shard A, but the balance of this transfer is not immediately available, and the system temporarily locks it and generates a receipt. Then Shard B sees this receipt, and then completes the transfer of Token B. Finally, Shard A lifts the lock of the transferred Token A, making the balance available. If Token B does not complete the transfer within a certain period of time, Shard A will cancel and roll back the transaction of Token A.

So you can see that this asynchronous consensus protocol is very cumbersome. And because of the atomicity of transactions is compromised, many types of transactions are difficult to get done, such as the famous example of “air tickets or hotels, either all booked or all cancelled.”

This question is listed in the “Ethereum Sharding FAQ”, which is still an open question to be solved.

When Greg worked on the distributed transaction system biztalk at Microsoft decades ago (it is not a blockchain, but there are many similarities, a bit like a private chain without considering malicious nodes), it was very troublesome for this kind of multi-stage transaction processing. He found that once the transaction becomes complicated (for example, involving more shards, more general computations instead of just token transfers), this multi-stage transaction processing explodes exponentially, with temporary locks, rollbacks, etc all intertwined with each other, and there is no way to analyze them.

This diagram shows asynchronous and synchronous cross-shard transactions:

Synchronous consensus is so simple, it is no different from calling a contract within a single shard, but why not use it? Because it is difficult to implement under the state machine system, it is necessary to do “block merge” as well.

It is similar to the intra-shard concurrency: you need to hook up a block to several parent blocks, one from each shard.

If each shard is a single chain like many existing projects, and you occasionally hook up a block on both single chains for the cross shard transactions, problems will arise. When there are more and more cross-chain transactions, all the chains are glued together like when we cook homemade noodles without dispersing them first — you end up with a giant dough.

It is similar to traffic in highways: you have many lanes, if each car changes lane, you need to join two lanes together, then finally all traffic will be squeezed into one single lane. This is not scalable.

Therefore, the correct way to do the cross shard cooperation is to do synchronous intra-shard consensus, and at the same time do concurrency and block merge inside each shard, just like what RChain is doing.

The basic technical difficulties of concurrency and cross-chain interoperability are actually very similar. RChain has already completed concurrency, so it is much easier for RChain to complete a seamless synchronized cross-chain consensus — you only need to compete the corresponding security model for sharding.

The third important point is fast and low-cost formal verification.

Everyone can do formal verification, and so can Ethereum. The key is the cost and its operating environment.

To use the metaphor of my favorite transformation shapeshifter and LEGO blocks, the Turing machine contract is a shapeshifter. To get formal verification, you need to consider all its possible transformations. The contract of Rho calculus is just a LEGO block: you can see what it can do just by looking at them.

At present, the contracts of Ethereum and Polkadot are all executed in a single thread mode, and there is not much collaboration between contracts. So the behavior of the shapeshifters is relatively more predictable.

Suppose that one day they really get concurrency working, or they want to encourage lots of collaboration between contracts — one contract calls another contract, and then calls a third contract, then disasters will happen.

Formal verification in a single thread is generally done by Hoare logic. For each line of code, pre-conditions, post-conditions, and commands are abstracted, and then you check whether a piece of code together can derive the required assertion.

Formal verification under concurrency generally uses tools such as TLA+, you first abstract the states and state transitions, and then exhaustively explore the state space to see if you can prove that no matter what the state transition path is, the desired states set can be obtained. Both of these are not scalable and they are costly.

There is another way — don’t explore the state space, but instead derive in mathematics. This is the process calculus way.

Rho calculus is like Pi calculus; there is no interaction between the contract and the virtual machine state space like Turing machine smart contracts. Instead, all affected information are contained in the processes themselves. And, the evolution of processes are done by a very simple reduction rule. Therefore it is very convenient to do math derivation.

In the RChain roadmap, the formal verification part is OSLF (Operational Semantics in Logical Form), which is one of the math derivation methods. It is used to quickly generate behavioral types. If the code satisfies a certain behavioral type, it satisfies the corresponding properties (for example when you call another contract, the access privilege you give to it won’t be given to others) as well, without exploring the whole state spaces.

Formal verification algorithms in Pi calculus are all very mature, and there are many papers about it. If you want to know the level of maturity (of formal verification) in Rho calculus, you’d better ask Greg, Steve, or Christian. They are all masters of process calculus. This kind of formal verification based on the behavioral types is very fast and cheap.

The ultimate goal of RChain is to achieve “the orchestration of thousands of smart contracts on chain”.

And now we have the big picture of RChain:

Bock merge is the foundation, shards are the framing, OSLF (Operational Semantics in Logical Form) is decoration.

RChain’s skyscraper will be built in this way, step by step. Once all the steps are compete, a functional world computer will be born.

Zora: Q5- Security is the most important point of a public chain project. It is understood that RChain has its own unique design in terms of security. Can you talk about it?

Atticbee: It is essentially the formal verification. And I’ve already covered most of this topic in the previous question. I’d like to add a bit more here.

Formal verification is not the same as software testing.

Formal verification is to prove that your code “will satisfy certain properties at any time”, for example, your account balance will never become negative, the total token supply will not increase, etc.

Software testing is to feed a variety of different inputs to your code to see if the outputs are expected.

For traditional software engineering, testing is sufficient, and formal verification is not required.

But blockchain is very different.

Because blockchain is mission critical, which means “zero fault tolerance”.

Suppose you have a bug. In the traditional computer industry, the problem is not that big. You can kill the process and restart it and debug. At least, you can roll back the wrong transactions. You can do any thing because you own the server.

But you don’t have this luxury in blockchain.

Especially under concurrency, anyone who has done concurrent programming knows that it is very likely that your problem may have one error out of 1 million runs. Software testing is very likely not able to catch that.

However, if you use a formal verification tool, you are likely to find this bug. If you don’t find it, the hackers will find it for you.

Zora: Q6- The RChain mainnet has been online for a while, what about the next steps in the development plan?

Atticbee: The Coop’s official website has the latest roadmap. Block merge will be complete soon. Next external nodes will be allowed. These are expected to happen in the first half of the year.

The main task in the second half of the year is to complete sharding. As mentioned earlier, RChain does difficult parts of the roadmap work first and the easier parts later. After concurrency with fast block merging is done, a lot of the sharding work has already been completed.

Another task is to complete Rholang v1.1. After this is done, the readability of the contract code will have meaningful improvements, and the TPS performance will also be greatly improved. At the same time, there are still a lot of optimizations to be done, as we need to constantly tap the potential of this platform.

After sharding, it is the final decoration stage: formal verification based on the behavioraltypes system.

In the middle, some other interesting tasks may be added: for example, directly accelerate Layer 1 with zero-knowledge proof. We don’t have time to go over this today.

Zora: Q7 — It is understood that Dfinity also said to study the world’s computers. Compared with this, what are the advantages of RChain?

Atticbee: Not only DFN, Algorand and Solana etc are all “single-threaded”.

It is like you get trained to be a knight capable of wielding swords with unprecedented speed. However you are still just one person. RChain summons a legion of solders to fight against you. If one legion is not enough, RChain summons another one.

So it is not a fair play. It shows the power of scalability. In addition, RChain’s sword may be actually faster than theirs. The above projects all have epochs such as 1 block for every 100ms. In the future, RChain may have no epoch at all — all nodes can fire at will without synchronization.

If you follow the wrong path of VRF, concurrency will be completely impossible. For example, if you start two threads, how can the two randomly picked nodes by VRF coordinate together to guarantee there is no conflict between the blocks they are producing?

Similarly, the synchronous execution of cross-shard transactions will also be impossible. Therefore from technology perspective, I have a very pessimistic view of these “take turns to produce block” projects.

Of course, people in the crypto world always say “just blindly all in and you’ll be a millionaire; too much technical analysis will make you lose all money”. My position in the crypto circle is a nerdy blockchain technology enthusiast. I only comment from a technical perspective. If your purpose is to speculate the coins’ prices, don’t take my views too seriously.