Feb 4, 2021
Cosmos SDK and the Launch of the 0.40 Stargate Release
The Stargate release is live, and we are close to seeing this code live on the Cosmos Hub. The vision of the internet of blockchains is now a reality. Using Stargate, different sovereign blockchains are already transacting with one another in a multi-chain testnet.
As you may be aware, Regen Network took responsibility for maintaining the Cosmos SDK and stewarded the non-IBC parts of the Stargate release. Regen Network led the SDK architecture for the migration to Protobuf and worked closely with the larger community. Now that Stargate is released, this is a good time to share more about the technical side of the new Cosmos SDK, including the design principles underpinning the new release, the paradigm of evolutionary schema-based development, and the performance increases that the refactoring to Protobuf brings to the Cosmos SDK.
Let’s begin with the big picture.
The Stargate 0.40 release of the Cosmos SDK resulted from many teams working together over almost 10 months. This release took some time due to the intersection of three large pieces of interconnected software: Tendermint Core 0.34, Inter Blockchain Communication (IBC), and the Cosmos SDK. Each codebase is complex on its own. Bringing the three together in the 0.40 Stargate release brings IBC to the world, running on the most secure state machines possible, with a flexible application development framework.
This was the work of a whole community. As a community, we conducted thorough code reviews and many tenets, which slowed our release time, but ensured the safety and integrity of the release.
Combining semantic versioning with evolutionary schema engineering for a long-running, interoperable distributed system design:
A major design principle around the architecture of the refactored Cosmos SDK is in service to long-running, interoperable distributed systems. Defining stable interfaces for long-running distributed systems must be effectively executed before interesting features can be built and adopted and see large scale usage.
- The first elements of executing this vision in the Stargate release are the SDK upgrade module and the upgrade daemon Cosmovisor. Long-running chains need upgrade support.
- The second major element in the new Stargate SDK was the refactoring of the SDK to Protobuf. Protobuf brings a transformational upgrade in performance and improved UX for clients. However, what drove the architectural decision was an understanding of the importance of evolutionary schema design for a long running distributed system. A core goal of this engineering paradigm is to try not to break existing functionality when you add new functionality.
There are two main ways we accomplish the goal of resisting breaking changes for client developers.
One is simply good hygiene and refraining from the arbitrary renaming of fields. The other is the use of tools that check for breaking changes in order to ensure architecture and engineering is done in a way that avoids these breaking changes whenever possible.
Here is a quick example of what an evolutionary schema approach to software engineering looks like. You will quickly see why it is so important for our goal of stability for client interfaces in service to a long-running, interoperable distributed systems.
- Proto messages have field numbers. This makes it possible to add new fields to a schema and delete unused fields without causing breaking changes for downstream clients. When depreciation occurs, we simply deprecate the whole field and it’s number, and add a new field with a new number for new functionality. This ensures that clients do not arbitrarily break.
- For instance, if you were to build a form with the following fields: 1- Name, 2- Date of Birth, 3 — Country, using amino it would be very easy to simply replace 2 — Date of birth with 2- Birthday. This breaks all downstream clients because computers just do what you tell them to. Protobuf makes it easy to check for these sorts of breaking changes and easy to deprecate field 2 if you no longer want to collect Date of Birth information. Instead of clobbering field 2 via renaming, instead, you create a new field 4 — “Account Balance,’’ to add new functionality in a way that downstream clients don’t break, but the update is reflected. This is an essential attribute in creating a stable interface for long-running distributed systems, and Protobuf has the tooling to make developing in this paradigm easy for developers.
The tooling for checking and returning an error if changes are breaking is called Buf. Buf allows developers to easily ensure that changes are backward compatible. The term for the process outlined above is called schema evolution.
It is important to note that in the evolution of the Cosmos SDK, or any other software, breaking changes will still have to be made. However, we want to reserve breaking changes for very important upgrades, while working hard at the client level to avoid breaking changes whenever possible.
We now have the concept of evolutionary schema design under our belts, and understand why it is important for long-running distributed systems like the Cosmos Hub, and other blockchains running on the Cosmos SDK. Let’s explore the concept of Semantic Versioning and its importance and relationship to evolutionary schema-based engineering.
- Semantic Versioning is a paradigm that dictates how to version software releases. The basic idea is to have consistency in indicating breaking changes to an API in a common way. Implied within it is an engineering paradigm, in which breaking changes are postponed or minimized. For instance, you don’t want to push a software version 2.0 (denoting a large breaking change to your API) just because you can. Instead, you want to focus on a 1.1 release, and then a 1.1.1 after that, in which changes, as much as possible, are not breaking the core API. In the case of the Cosmos SDK, in order to embrace this Semantic Versioning concept, it is very important to apply the evolutionary schema paradigm described above.
- If you are not careful, Semantic versioning can lead developers to stop worrying about breaking changes as long as they are appropriately named, which is a nightmare for client developers and what we need to avoid in the ecosystem. Instead of being aggressive with breaking changes, we want to postpone large breaking changes as long as possible. The fusion of evolutionary schema with semantic versioning ensures we have discipline about naming breaking changes, while also striving to avoid those breaking changes whenever possible.
This combination of Semantic versioning for the state machine, and evolutionary schema design for the client interface, is what will make the Cosmos SDK the right mix of stable, customizable, modular, and extensible. These are the attributes needed by developers to ensure that sovereignty AND interoperability, the two core concepts of Cosmos’s blockchain paradigm, are both achieved.
0.40 proto files were released with a beta tag, which means that we may still make some breaking changes, as needed. This is in service to ensure a stable long-running distributed system as the outcome as we march as a community towards SDK 1.0. Regen Network will do everything we can to avoid breaking change, but we are also aware that the battle testing happening in testnets and main nets around the ecosystem could uncover issues that need to be addressed with a schema breaking change.
Protobuf brings large performance upgrades due to its use of code generation. This allows more transactions per second using the Tendermint Consensus engine, increasing the performance of a Cosmos SDK.
In short, Protobuf was adopted by the Cosmos Community because it makes it easier to do a better job of something that is already very hard and important: striving to provide stable clients interfaces for as long as possible
—
Thank you to Gregory Landua and Aaron Craelius for their contributions to this post.
Follow us on Discord to join the conversation!
Regen Network is a platform serving to align economics with ecology to drive regenerative land management. Regen’s Registry allows land stewards to sell their ecosystem services to buyers around the world, functioning to reverse climate change through incentivized carbon removal.
