The Shyft Ring Node

An Ethereum Fork With So Much More

Before we deep dive into the Ring Node, a little bit about myself and my role at the Shyft Network. My name is Gregory (github), and I’m the Head of Development at ChainSafe Systems (github). I’m leading the team building the Shyft Ring Node.

At the heart of every blockchain protocol is the network that powers it; Shyft utilizes Ethereum.

Shyft’s Ethereum Virtual Machine (EVM) is a fork of the Go-Ethereum (Geth) implementation of Ethereum, which contains a multitude of custom modifications. Some of these modifications include a built in block explorer API, a lightweight block explorer UI, custom inflation and dynamic tracing of internal transactions.

The Shyft Network uses EVM nodes to power its Ring Node infrastructure (we’ll discuss that later), an essential component to the ecosystem’s security and scalability.

On a high level, the Shyft Ring Node is the collection of EVM nodes that are running the Shyft Network specification.

Each node runs a modified version of the EVM that contains custom functionality for a Postgres database that stores block, transaction and account data specifically designed to populate a block explorer interface. An API is also supplied to facilitate database calls to retrieve the information for a block explorer. A React app also exists within the project that contains a lightweight block explorer that can easily be styled and customized as needed.

The rest of this article will dive deep into the for why certain design choices were made, as well as their wider implications.

Geth

For the Ring Node, we ultimately decided to go with the Geth implementation for quicker development cycles. Initially, we played around with Parity, but we ran into some issues. The first major issue was the compile time: every change required us to recompile the code base, which would take anywhere from four to twenty minutes depending on the files we changed. The most up-to date version removed support for CPU mining, meaning the majority of laptops would become unusable when mining blocks. Also, Parity transaction tracers were difficult to use (at the time). Geth also has its limitations, but it provided us with features like CPU mining, fast compile time, and transaction tracing out of the box.

Block Explorer

Intro

The block explorer is arguably the most vital piece of infrastructure baked into the Ring Nodes. It allows anyone running a node to support the network and prevent centralized points of failure.

Wait… Ethereum isn’t centralized?

Yes, I did just say that, and we encounter our first major problem within the Ethereum ecosystem. It unfortunately has a massive centralized flaw: Etherscan, MyEtherWallet and Infura are the main entry points to the blockchain. For those that are not aware, the three websites above all run full nodes in order for transactions to go through, and everyone uses them. If there was a massive DDOS attack on all three of those websites, the Ethereum Network would be borderline unusable, and the vast majority of users would not be able to send ether across the Network. Most dApps rely on Metamask, which subsequently relies on Infura.

I think by now everyone should understand the issue: Ethereum has a bit of centralization.

At Shyft, we took this weakness into account, and that’s why we created the block explorer API as a core component for every node on the network. When a node is booted up, the block explorer API is run alongside in order to support the Ring node network.

Under The Hood

Database

In order to populate a block explorer, we need to query the blockchain for all the information. This would normally be a very time-expensive process since we would need to use RPC to perform a look up on the Merkle tree and Patricia trie databases. The Merkle tree stores all block and transaction information while the Patricia store account balances. This process is slow because both databases are written using levelDB which is a key value store, therefore everything is hashed as byte arrays and must be serialized for the API. For these reasons we decided to use a relational database, specifically Postgres. Since Postgres is a relational database we can link transactions to blocks, accounts, and many other data points to make querying faster. It also makes reverting blocks a breeze in case of diverging chains.

Blockexplorer API

As addressed earlier, to solve the problem of having a centralized block explorer, we created a very simple GO API that runs alongside the geth node to get access to the blockchain data in the database. It also facilitates the ability to send pre-signed transactions into the mem pool.

Pre-signed Transactions

In order to prevent censorship on the node level, a heartbeat feature is built into the API for sending pre-signed transactions. A wallet client would utilize the Shyft Wallet npm package to interact with nodes on the network, making on-boarding the Shyft chain into clients seamless and headache-free. The npm package attempts to make a TCP/IP connection to two nodes on the network, the first being the primary connection and the second being a backup. Once a connection is made successfully, the wallet can then send pre-signed transactions onto the Shyft chain. In turn, the TCP/IP connection is made to help adjust the reputation score of the selected node.

Dynamic Tracing

A personal favourite (get ready for a PR into Geth soon!), we have built out functionality to utilize the tracing methods that currently exist through the Geth console (accessed by calling geth attach). This allows us to trace transactions as they are mined, giving the ability to peer into internal transactions. Internal transactions are composed of token transfers, contract calls, and just about everything else you can think of when interacting with a contract. For example, executing a function on a contract that sends Ether through multiple contracts is inherently difficult to figure out without tracing. Tracing also gives us access to the opcodes and contract storage so we can see what exactly is going on within our Shyft-specific contracts.

Modifications To Core Code

Thankfully, so far, we haven’t needed to make any changes to the core Geth code. All the files with changes extend their existing functionality to give us access to functions, data or data types that are needed to store block information and run the tracer dynamically. We did modify the inflation (block reward) to 5 fuel (or native eth) which is split evenly between the miner of the block and the Shyft Conduit Contract (covered in another article) which is used for reward mechanisms.

Extended Functionality

In regards to extending Geth’s functionality, a few changes were made. We added a method From() to the transaction type to allow us to quickly get the sender’s address, which natively was not needed because that information is not stored in the merkel or patricia trees. A few types are being exported globally such as the Ethereum object and certain configs for the tracer. Outside of calling functions to write to our Postgres database, we left everything the same.

Road Blocks

We hit two major roadblocks while developing the block explorer, namely; databases and import cycles.

Initially, we started working with a LevelDB instance to stay consistent with the existing Geth architecture which proved to be a challenge when retrieving the data. The whole purpose of the API is to have fast retrieval of the data. As explained above, LevelDB is a key value store which means we had to convert structs into byte arrays to store them, this meant retrieving the data required us to convert byte array structs, then convert to JSON. Naturally this was an overly complicated workflow, and had some foreseen scaling issues.

Go import cycles are not fun. In order to get the dynamic tracing going we had to pull objects and methods from all sorts of packages that depend on each other, which caused us to enter an import cycle hell. Thankfully we found a solution thanks to an amazing article that let us create an interface to make a link between packages. Without the interface solution we probably would have had to rewrite the tracers from the ground up, or at least make a carbon copy of them in a separate package, which would have added considerable bloat to the codebase.

Current State of the Chain

The big question, where are we today?

After roughly four months of learning the inner workings of Geth and attempting different strategies at solving the problems at hand, we’re pretty excited to say the testnet is ready to go live at the end of July (or early August). Currently, we have the API finished, Postgres is storing the correct values, the basic block explorer displays the correct data, and the TCP/IP handshake and npm package are wrapping up. As of mid-July we finished the dynamic tracer, writing tests for it and getting it ready for a pull request into Geth! The final work in progress is extending our testing suite to cover more test cases. As we approach testnet launch, we will have the entire setup hooked up with Docker and a custom CLI to make running a node a breeze.

***

Shyft is building the world’s first modern, secure, multi-stakeholder Blockchain-based network that enables KYC/AML attested data transfers. Join our Telegram (https://t.me/joinchat/HhrB_hKGQDQKU7mhpzor_g), follow us on Twitter (https://twitter.com/shyftnetwork), GitHub (https://github.com/ShyftNetwork) and other channels found on https://www.shyft.network/