Waves node in Go

In this post, we’ll discuss how Waves’ node in the Go programming language came into existence and how you can launch it.

Thanks to the simplicity of Go, the node in that language is attracting new developers to the Waves community.

Code error

In early January 2018, when Russia was on New Year’s vacation, the Waves blockchain experienced a down time that lasted for about 40 minutes. Subsequent analysis revealed that the cause was a simple error in the node’s code. Errors of that kind are extremely hard to detect, and that specific one had been in the code for more than six months before it “went off.”

However, the error was not in Waves’ protocol, but in its implementation. The existence of just one implementation made the entire Waves network vulnerable in situations like that. Therefore, a decision was made to do another implementation of the Waves node.

The language for a new implementation was selected based on the following criteria:

• existence of cryptographic libraries
• popularity in other crypto projects and in the crypto community
• simplicity of the language for developers.

There were two main candidates, Go and Rust. But Rust could be hardly called a simple language because of its memory management model and ensuing multiplication of entities. Unlike Rust, Go has a minimal number of concepts and, consequently, its code is readable.

Other advantages of developing a node in Go included the opportunity to test new approaches to the implementation of already existing algorithms and the addition to Waves protocol specifications of implementation details hidden in the Scala code.

The path to implementation

A project for developing a Waves node in Go was launched in the fall of 2018. The team featured three developers, although its lineup partially changed several times.

The development plan was simple. The starting point was basic primitives of the Waves protocol, including cryptographic functions, transactions and network messages. Then, to benefit the community, a client library for the node’s REST API in Scala was implemented. It was used in several internal Waves services. Subsequently, to demonstrate the capacities of the client library, the utility tool chaincpm and the services Fork Detector, Retransmitter and Waves Market Data were built.

In the spring 2019, node development began. The developers were tasked with ensuring high speed of the node operation and accelerating block imports. Algorithms for consensus, block and transaction checking were implemented as part of developing the utility tool importer.

Simultaneously, a concept for block and node state storage was created. In the Go implementation, block storage in leveldb was abandoned: new blocks are added in full to a simple binary file, and their meta information is stored in leveldb.

Another improvement in the Go realization was an adaptive buffer of applied blocks. The Scala node applies blocks in equal batches of 100, and the Go node constantly tracks block size and adjusts the number of blocks in the buffer, based on their size. That is, if blocks are empty, the Go node can apply them in batches of several thousands, and, if blocks contain a large number of transactions, the buffer can be dynamically reduced to several dozen blocks. This mechanism facilitates the significant speeding up of initial import of the Waves blockchain.

Some improvements in the Go node were also added to the Scala node. For instance, the addition of bloom filters when calling the state storage, helped reduce the number of costly transactions for non-existing keys, which is especially vital for blockchain import as all blocks and transactions are new at the time and, as a rule, are absent from the storage. Disableable bloom filters were implemented in the Scala node, which helped speed up blockchain import.

The Go node was developed under the grey-box model. The implementation of a functionality would only start based on relevant documented data. For testing, existing blockchain data was used, and only at the final implementation stage, comparisons with the Scala implementation were made.

As a result, in the winter of 2019, a fully functioning implementation of a validating node in the Go language was completed. The only limitation was the node’s inability to generate new blocks. This was the developers’ next task, and, in the spring of 2020, generating nodes in the Go language were launched in the testnet and, a few weeks later, in the mainnet.

Simultaneously, Scala developers actively expanded the node’s functionality and prepared version 1.2 Malibu, released in August 2020. In October, support for the Waves 1.2 Malibu protocol was also added in the Go version.

In future, all major Waves protocol releases will be launched simultaneously in Scala and Go.

In the development process, special attention was paid to the Ride script interpreter. Several interpreter versions were developed, with varying optimization degrees. Currently, a virtual machine for Ride script execution is in an active development stage. The creation of a virtual machine is expected to further increase the speed of node operation and create a capacity margin for further blockchain growth.

During the joint work on new 1.2 Malibu protocol functions, several errors in the Scala implementation were detected, which were promptly corrected before they could make it to users’ computers.

Room for improvement

The Waves node implementation in Go is a relatively young product that can’t offer as many user interfaces as the Scala node. In the Go node, a new gRPC interface (gRPC API) is fully implemented to access node and blockchain data. But support for the node’s original REST API has been implemented only partially. This imposes some restrictions on the use of the Go node.

However, the Go node can be properly used for the development and operation of new services that obtain data over gPRC API, as well as for block generation in the testnet and stagenet. Still, the Go node’s record in block generation is not long enough, so we recommend limited use of it for this purpose on the mainnet.

The Go node development team has no intention of slowing down and will work on introducing new functionalities, as well as on supporting interfaces already familiar to users. The most immediate plans include adding the Go node to full-cycle integration testing, debugging and improving security.

We invite developers to study the Waves node, detect and correct errors, propose improvement or new functionality. Thanks to the Go implementation, all of this is much easier to do.

Launching a Go node

The process of launching a Go node is very simple. Go to the Github release page, download an executable file and launch it, following the short guidelines.

If you have problems, questions or suggestions, create a Github issue or chat with the developers on Telegram or Discord.

Read Waves News channel
Follow Waves Twitter
Watch Waves Youtube
Subscribe to Waves Subreddit