We are happy to announce that we have finalized our Exonum 1.0 release candidate (in preparation for the full release of Exonum 1.0). In this blog, we’ll talk through the features of the Exonum 1.0 release candidate as well as the changes from Exonum version 0.12.
Introducing the Service Lifecycle
The core feature introduced in this release is a logically complete service lifecycle. This means it is now possible to add new services to the blockchain during blockchain operation, stop or upgrade existing service instances, and even perform data migration in background in order to change your data layout to the one required by the new service version.
Some examples of the benefits of using the service lifecycle include allowing you to define new APIs to interact with the service (in form of either transactions or HTTP endpoints); add new parameters to existing APIs; or remove your existing APIs altogether. (We also now offer semantic versioning to help determine the compatibility of this change). This upgradeability brings Exonum services closer to traditional applications, enabling you to build a more durable and agile blockchain infrastructure.
All service lifecycle events are managed by the Exonum supervisor service, which brings the following benefits:
- Service state is agreed among all nodes in the network by the virtue of the consensus algorithm.
- Service administration is fully auditable and properly authorized.
- Service administration can be fine-tuned according to the use case. For example, in a small blockchain, lifecycle events could be authorized by a single administrator, while in a consortium network their authorization could require an agreement of a (super)majority of administrators.
- Supervisor logic itself is fully upgradeable.
The reference supervisor implementation can be found in the exonum-supervisor crate.
We will be sharing more details on our blog about the service lifecycle soon.
Data Migrations
As a part of this service upgrade workflow, Exonum now supports background (aka asynchronous) data migrations. This allows you to migrate data from one schema to the other without needing to stop a node or your blockchain network; instead, the whole process is performed in the background while the node continues processing transactions and other requests.
Data migrations are performed with the help of migration scripts, which can be written in the same languages that Exonum supports (currently Rust — Java migration support is in progress). The migration workflow defined in the Exonum core ensures that the outcome of a data migration is the same on all blockchain nodes, and that migration finishes by a certain blockchain height. A node will bring the migration script into the foreground if it hasn’t managed to finish it in time, and block consensus logic until it is completed.
Relevant migration support was also added to the storage backend (MerkleDB) and can be used independently of Exonum.
Splitting up Crates
The crate structure was significantly refactored in this version of Exonum. The node logic, Rust runtime, HTTP API wrapper, explorer endpoints and system endpoints (all of which were previously parts of the main crate) have now been moved to separate crates. This provides greater flexibility for core and service developers and allows you to evolve downstream crates independently of the core.
An additional benefit of this change is that both the development and testing of Rust services can now be performed without relying on node-specific dependencies, such as tokio providing the event loop or snow implementing the Noise protocol for encrypting communication channels among nodes. As a result, compilation times are sped up, and the build artifacts become slimmer.
MerkleDB Updates
MerkleDB now automatically aggregates the storage state, which was previously performed manually by the core. State aggregation produces a single Merkelized map (the state aggregator), the keys of which are the names of the Merkelized indexes in the database, and values are the corresponding index hashes. The hash of the aggregator thus commits to the Merkelized part of the database state and can be used to prove the existence or absence of elements in any Merkelized index.
With automated aggregation, all aggregation logic previously present in the core and runtimes was removed. The new approach requires zero effort from the service developers — it is enough to declare the data schema, and all Merkelized indexes will be added to aggregation automatically!
Speaking of data schemas, the Rust services can now utilize a more intuitive declarative format to declare them. For example:
Indexes or index groups can then be simply accessed via the corresponding properties of the Schema.
Compared to 0.12 release, MerkleDB provides new tools to enforce database access restrictions. First, the access can be restricted to a namespace (this is used by Exonum to provide read/write restrictions for services during their execution — a service should not be able to write to the schema of another service). Besides the general namespaces, MerkleDB provides two types of namespaces for migrations — Migration for storing new data and Scratchpad for temporary data which is discarded after a migration is flushed to the general storage or rolled back.
Besides namespaces, MerkleDB now provides separation between read-only and read-write access, which is enforced on the level of the Rust type system. This helps you to write safer code and eliminates potential unexpected behavior of services.
To assist in developing bindings in other programming languages, MerkleDB now supports generic database accesses and shares iterator types among all indexes. Any kind of access can be converted to a generic access, allowing to decouple and/or reuse much code in the “native” part of a binding.
Service Interfaces in Rust
To make service development easier (and more aesthetically pleasing), service interfaces were reworked in the Rust runtime. Now they are represented by the ordinary Rust traits, like this one from the cryptocurrency tutorial service:
Transactions are now plain data types, and business logic of a service is defined by implementing the service interface trait, which is idiomatic from the Rust perspective and easier for general comprehension.
Besides the service implementation, any interface trait is automatically implemented for an assortment of stub types. For example, the KeyPair stub allows to generate signed transactions to the service — it’s enough to import the interface trait to the scope:
Moreover, new stub types can be defined in the user code! For example, the middleware service implements stubs in order to create checked calls, that is transactions, which are only routed if the target service has a specified name and satisfies a version requirement.
We will publish an article about how stubs work soon. In the meantime, you can read a literary code explainer in the stubs module of the Rust runtime crate.
Other Changes
Besides functional changes, this new release improves user experience and prepares Exonum APIs for future compatibility in the following ways:
- Protobuf declarations in the core and upstream crates have been reviewed, documented and structured according to their packages.
- Data types in the core and upstream crates have been made non-exhaustive.
- Experimental or unstable APIs have been marked as such in the documentation or hidden from the documentation.
You can find a more detailed list of changes in CHANGELOG.md.
Coming Soon: Exonum 1.0
We are working to release Exonum 1.0 by the end of February. No substantial functional changes are expected before then. At the same time, there may be breaking changes to the APIs documented as unstable (for example, data migration). Another kind of potential breaking changes is related to updating core dependencies. Right now, Exonum core targets compatibility with Rust 1.36 due to the problems of compiling the Java runtime with the more recent versions. If this restriction is lifted, some core dependencies will be updated, which could result in breaking changes (at least technically).
Along with release preparations, we have updated our technical roadmap. It can be found on the Exonum website.
Our team is working hard to integrate these Exonum features into Exonum Enterprise, our Blockchain-as-a-Service platform that allows any business to launch a blockchain in just 5 minutes. You can request a trial of Exonum Enterprise today on our website: https://exonum.com/enterprise
