Insights on integrating the Rosetta API developed by Coinbase

Published in

Harmony

6 min readNov 16, 2020

Harmony is a sharded Proof of Stake blockchain that is EVM compatible. Our production mainnet supports 4 shards of 1000 nodes, producing blocks in 5 seconds with finality.

Harmony has finished it’s initial integration with the Rosetta API specification — an open-source framework to simplify blockchain interactions. We hope to share a little bit about why we decided to integrate with Rosetta as well as share some of our learnings. You can find our Rosetta implementation here.

Check out the technical details of our implementation on our Rosetta community forum post: https://community.rosetta-api.org/t/harmonys-rosetta-data-construction-api/293

Why Rosetta?

Earlier this year, we launched open staking on our network followed by our grant program, and with that came a bunch of interaction with our Node API. We quickly realized that the nuances of our sharded & staking mechanisms, along with a slew of new API methods, made it difficult for exchanges and developers to onboard quickly & build. Therefore, we set out to massively overhaul our Node API. Fortunately for us, Coinbase just announced their Rosetta specification right as we were getting started and we felt that it would be a perfect addition to our overhaul.

So why did we choose Rosetta? Well, the biggest thing for us is that it allows for faster onboarding since it is a standard, chain agnostic, framework with sharding as a first-class citizen. Among other things, this greatly reduces interoperability friction between products spanning multiple chains. Moreover, Rosetta’s growing community and tooling were major factors as it means less effort from the Harmony team is needed to implement and, more importantly, maintain our Node API.

Painless Implementation

The Rosetta spec is defined using the OpenAPI 3.0 format (spec files here). This makes it easy to implement the spec in any language as you can easily generate boilerplate code from the spec files using existing OpenAPI tools. That said, it is even easier to implement the Rosetta spec in golang (the language we use in our core) since Coinbase provides a full go SDK that can be used by implementers and clients. A member of Rosetta’s community has also ported the go SDK to Javascript (seen here) and we expect more languages to come in the future. Coinbase has also provided detailed documentation on the spec and example (golang) implementation like rosetta-bitcoin and rosetta-ethereum. Lastly, the Rosetta community is a great place to ask any questions and has served as our primary source on any questions regarding the spec.

A tip we would give to anyone looking to implement the Rosetta spec is that, if possible, you should add Rosetta as a service on some existing ‘block explorer’ type program. This should greatly simplify any required block syncs and there probably exist some utility and/or tracing functionality in the said program that will speed up coding your implementation. We chose to natively support Rosetta as a service on our core node program for these reasons.

Testing our Rosetta implementation was also very easy using the rosetta-cli provided by Coinbase. To any project looking to implement the Rosetta spec, we urge you include testing your implementation with the latest rosetta-cli as part of your continuous integration pipeline ASAP (even before implementing the Construction API).

Expressiveness

At first, it may seem that the Rosetta spec is only for some specific set of tokens transfers. However, the spec provides plenty of ways to express most (if not all) of your on-chain functionality through well placed metadata fields (which can contain any JSON object) and the notion of a sub_account for the spec’s AccountIdentifier model.

For example, Harmony has staking transactions to facilitate validator edits/creation, token delegations, and reward collection. All of these staking transactions involve locking/unlocking tokens tied to some validator on the network. We chose to express many of the staking information (validator name, fee, etc…) in the metadata of these transactions so a rosetta client could easily obtain such information when a staking transaction is found in a block. Moreover, we are currently implementing a way to track one’s locked/delegated token by using the sub_account field to indicate delegated balance for a specific validator (who’s address will be given in the sub_account field).

This presents a bit of a tradeoff, however. A client would need to know a little bit about Harmony and our specific implementation to be able to use our Rosetta implementation to its fullest capabilities. That said, a generic rosetta client would still function just fine for most operations of interest such as token accounting. The ability to be interoperable yet specific when needed is why we love the Rosetta spec so much.

Internal Transactions

One of the toughest things for us (and other projects from what we’ve seen) is the need to report all native token balance changing operations, including any native balance changes executed by a smart contract — commonly referred to as internal transactions. On top of that, we have to consider our current staking logic as well as our “pre-staking” rewards logic since our mainnet had a phased release (from a permissioned committee to an open committee), which had many internal balance changes to handle our permissioned-phase block rewards.

For us, we had to implement a lot of extra machinery to account for all of our balance changes. Fortunately, our Dapp developers can immediately benefit from some of this hard work as the ability to trace smart contract calls is now exposed as an RPC (for now).

The machinery needed to report all balance changes took us far longer than expected to implement. Therefore, other projects that want to implement the Rosetta spec should carefully assess their existing ability to track all balance changes while planning.

Standardized Transaction Construction

The Rosetta spec includes a standardized way to construct transactions for a network. This was one of the biggest selling points for us. We saw a ton of effort from both wallet integrators and Harmony engineers to correctly construct a transaction on Harmony, as we are a sharded network with cross-shard transactions. With the Rosetta Construction API implemented, we believe it is the easiest and clearest way for anyone to create a transaction on Harmony without a language-specific SDK.

We hope to see wallet integrators adopt the Rosetta construction API so that wallets can more easily support new projects/chains.

Common Interface For All

Implementing the Rosetta spec was a great way for us to look back at what information is easily accessible from our nodes with absolute accuracy and improve it. In a sense, the Rosetta spec can act as a checklist of what integrators — such as exchanges, wallets, and developers — expect when working with a blockchain.

We hope to see more adoption of the Rosetta spec in the industry as standardization will help bridge all sorts of different chains, and hopefully, ultimately lead to greater widespread adoption.

Developers can build on Harmony today, a fast and open blockchain for decentralized applications, suited to handle DeFi and cross-border finance. Harmony Mainnet supports state sharding with instant finality. Our staking mechanism reduces centralization while supporting delegation and slashing.

We ❤️ Developers.

Want to build on Harmony? We have grants available. 👉 Apply HERE 👈