Ethereum Classic Labs Joins Forces with Storj Labs

Increasing ETC Node Performance Using Core-geth

Published in

Ethereum Classic Labs

5 min readSep 10, 2020

Running an Ethereum Classic node requires downloading and syncing an entire copy of the blockchain. One of the most demanding requirements to support a copy of the blockchain is storage, and scaling that storage since the blockchain is a continuously growing database. Syncing a new mainnet node can take a few hours, days, or weeks (depending on your setup of course). In the event of a client bug or network incident, recycling a node from scratch could be expensive downtime.

In collaboration with Storj Labs, we demonstrate how we can massively increase the syncing performance of an Ethereum Classic node in a trustless manner with Core-geth’s Freezer feature and the Storj decentralized cloud.

“Blockchains are breaking into more mainstream use cases, and enabling the simple and scalable growth of networks is critical in driving this adoption. Decentralized cloud storage solutions, like Tardigrade, offer the ability to store state history in a way that is secure, decentralized, and highly resilient to common attacks. We’re excited to be working with ETC Labs and its community of miners to store state history on the decentralized cloud — making it easier and faster to spin up new nodes, greatly reducing the expense of downtime,” said John Gleeson, Storj Labs VP of Operations.
James Wo, Founder and Chairman of ETC Labs added, “incorporating Tardigrade decentralized cloud storage into the process for GETH has allowed us to accelerate sync times, making it much more efficient to spin up new mining nodes around the world. This helps miners avoid expensive downtime, create a more diverse set of endpoints, and contribute to a larger, more secure blockchain.”
“This partnership with Storj is a great way forward for clients to take ownership of their data that serves their usage needs. Using Storj for long-term data storage helps developers and infrastructure providers keep productivity high and downtime minimal,” said Zane, Member of the ETC Core development team.

Freezer

Core-geth separates the database into two parts (done by Péter Szilágyi, Martin Holst Swende, and Gary Rong).

Recent blocks, all state, and acceleration structures are kept in a fast key-value store (LevelDB) as until now. This is meant to be run on top of an SSD as disk IO performance is crucial.

Blocks and receipts that are older than a cutoff threshold (three epochs) are moved out of LevelDB into a custom freezer database, backed by a handful of append-only flat files. Since the node rarely needs to read this data, and only ever appends to them, an HDD should be more than suitable to cover it.

By default, Core-geth places the freezer inside the chaindata folder, into the ancient subfolder. Moving the freezer location can be easily done with the — datadir.ancient CLI flag. If you have an existing freezer, then you can manually move it by copying the existing ancient, but ensure — datadir.ancient is set to the correct path.

Getting Started with Core-geth & Storj

Requirements

Core-geth: The protocol providing Ethereum Classic client.
Storj environment variables STORJ_API_KEY, STORJ_SECRET, STORJ_SATELLITE
Ancient-Server: The Ancient-store-storj (done by Zane Starr and Isaac Ardis of ETC Core) program is a persistence manager for core-geth “ancient” block data using Storj, and can be used as a replacement for Core-geth (and Go-Ethereum) default of persisting data in local flat, compressed files.
Go-lang: To build the ancient-server from source sudo snap install go — classic.

Installing Core-geth

The easiest way to install Core-geth is via binary. You can find the latest releases of Core-geth on Github.

wget https://github.com/etclabscore/core-geth/releases/download/v1.11.12/core-geth-linux-v1.11.12.zip # Download release for linux
sudo unzip core-geth-linux-v1.11.5.zip -d /bin/ # Unzip and move contents to /bin/ directory
geth version # Check install

Install Ancient-Server for Storj

git clone https://github.com/etclabscore/ancient-store-storj.git
cd ancient-store-storj
go build .
./ancient-store-storj — help

If you get exec: “gcc”: executable file not found in $PATH errors when attempting to build, then ensure you have GCC compiler installed sudo apt update, sudo apt install build-essential, and check it’s installed gcc — version, then run the build again.

If you are in the project directory you should see the built executable ancient-store-storj like so:

~/ancient-store-storj$ ls
LICENSE ancient-store-storj freezer_remote_storj.go go.mod main.go
README.md cmd.go freezer_remote_storj_test.go go.sum

Running the Ancient-Server

Using SystemD to run the Ancient-Server. Of course, use your Storj environment variables.

ExecStart=./ancient-store-storj — bucket %I — ipcpath /data/classic/.core-geth/storj_%I_stable.ipc — loglevel 5
Enviorment=”STORJ_API_KEY=”
Enviorment=”STORJ_SECRET=”
Enviorment=”STORJ_SATELLITE=”

Running this service allows the ability to store ancient-store data on Storj.

Running Core-geth

Now that that the Ancient-Server is running, all we have to do is tell Core-geth to use the Ancient-server’s IPC path like so:

geth — %I — datadir/classic/core-geth/%I/stable — ancient.rpc /data/classic/core-geth/storj_%I_stable.ipc

Now, Core-geth will download and sync the Ethereum Classic blockchain and will update the Storj cloud bucket with the ancient data.

Use-cases

Data-recovery: If you lose your chain data, then you can sync from the latest block height of the remote ancient store.
Sync speed: The ancient-store data basically acts as a backup of the blockchain. If the ancient-store contains x amount of history, then that’s x amount of history you don’t need to download and sync from scratch. Theoretically, days of syncing can be done in hours.

Resources

About Storj Labs

Storj Labs provides end-to-end encrypted, affordable, distributed cloud storage that gives data ownership back to the user, instead of major corporations. The company’s open-source cloud storage platform utilizes spare disk drive space shared by its community members to create a secure network for developers, operations teams, companies, and others in need of secure cloud storage. The distributed architecture of the Storj network protects against attacks, improves reliability, increases download/upload speeds, and enhances performance when compared to many traditional cloud storage approaches.

Storj Labs also created the Tardigrade decentralized cloud service, the world’s first enterprise-grade, decentralized cloud object storage service. Tardigrade offers decentralized cloud storage that’s automatically encrypted, easily implemented, highly performant, exceptionally economical, and ridiculously resilient.

Now in production, the Tardigrade service delivers durability, performance, and security that is better than major cloud providers, at a fraction of the cost. Through the Tardigrade Open Source Partner Program, any open source projects that enable users to store data on Tardigrade via connectors will receive a portion of the revenue generated by those users.

About Ethereum Classic Labs (ETC Labs)

The mission of ETC Labs is to build relevant, accessible, and high-quality technology, to create communities of value in a mature and regulated ecosystem. Our goal is to fulfill the promise of blockchain to improve people’s lives using the Ethereum Classic technology, one of the world’s major public blockchains. The ETC Labs team of experts fosters partnerships with organizations and institutions in order to address fundamental challenges in developing and deploying this innovative technology. We fulfill the mission in three ways: The ETC Labs Accelerator, which invests in up to 25 blockchain projects annually that contribute to sustaining a robust ecosystem; strategic investments in innovative projects focused on economic and social development; and the ETC Core, a team of experts and developers who maintain the Ethereum Classic blockchain and build key applications, solutions, and tools. For more information visit, https://etclabs.org/.