A Deep Audit of Events on the Road to IBC: Game of Zones đ„
Having participated throughout this yearâs Game of Zones, at Stake Capital weâve conducted a thorough examination of the competition and all of the events that transpired. We see it as our duty to transparently document everything we witnessed â including the good, the bad, and even the highly controversial â in this comprehensive ânovelâ of events.
Last yearâs Cosmosâ Game of Stakes not only marked the launch of the first ever implementation of a BFT decentralized network, but ultimately went on to become the archetypal launch method for bootstrapping new, decentralized networks and protocols.
This year, it was time for the next step in the Cosmos vision: the launch of IBC (Inter-Blockchain Communication), marking the beginning of functional data interoperability between completely separate Cosmos-based chains.
Although initially planned for a much earlier launch, Game of Zones suffered from a number of significant delays, primarily resulting from internal issues within the Cosmos & Tendermint teams. Although a number of key members of these teams left their roles, IBC was nonetheless still effectively implemented and completed, enabling Game of Zones to finally go-ahead as a final thorough testing ground for IBC.
This yearâs Game of Zones was run by three, key core contributors to Cosmos: Zaki Manian, Jack Zampolin, and Jessy Irwin, who handled relayer support, planning, communication, coordination and all other responsibilities associated with operating the competition. Game of Zones was planned to run in three separate week-long stages with different Capture-the-Flag style objectives, each one testing out and encouraging experimentation with distinct aspects of the Cosmos Relayer, enabling cross-chain interoperability.
To launch each phase, the team set as a require that the Game of Zones Hub (the version of the Cosmos Hub created specifically for Game of Zones, and state-reset prior to the beginning of each new phase) must always achieve at least 24 hours of stability prior to the launch of a new phase. However, prior to the initial launch of Phase 1 of the Game of Zones, an issue was discovered by Yelong from the IRISnet team which detailed the phase from launching.
The issue consisted of a failure to create a channel after a node restart due to an apparent capability initialization failure. The launch date target was thus had to be set back to Wednesday, May 6th.
Phase 1
After the required 24 hours of liveness and a slight clarification of the Phase 1 challenge scope, Phase 1 successfully launched on May 6th. The main competition objective of the first week of the competition was liveness, thus each teamâs competition rewards were dependent on their ability to keep their connection alive. More specifically, the challenge objective metric for teams was set as sending the fewest possible packets while keeping the client alive.
In the initial hours of the beginning of Phase 1 the hub suffered a number of capacity-related issues, particularly around congestion, resulting in significant difficulty for teams when trying to connect to the GoZ Hub. However the GoZ team did respond quickly by increasing the number of sentry nodes on the Hub and setting transaction fees higher to better mitigate network transaction spamâalthough many congestion-related issues sadly did persist throughout the following phases.
Phase 1 officially ended on May 10th, and although Phase 2 had initially been set to launch on Monday, May 11th, several technical issues made the necessary network stability difficult to achieve, Phase 2 was thus postponed so that these stability issues could be properly addressed.
Due to this mounting software instability and a plethora of phase 1 scoring confusion a number of key community members decided to publicly exit the Game of Zones:
Phase 1b
A few days later, the GoZ team decided to relaunch GoZ Phase 1 once the hub stability issues were addressed â naming this new phase, Phase 1b â with the aim of ensuring that no teamâs contributions got overlooked or forgotten due to the Hub-related issues that plagued Phase 1a prior. For the new phase, the amount of tokens allocated to teams was tightly controlled to better help ensure network stability and alleviate network load by reducing the risk of saturation due to high transaction volume. Additionally, software-level Relayer trust-period restrictions were removed and disqualification was announced for any teams that pool their genesis allocated doubloons (GoZ Hub tokens) to obtain additional gas. Later in the week, a revised phasesâ timeline was laid out.
For Phase 1b, the winning team was set as the team with the smallest trust period specified on their client while maintaining the longest period of liveness:
If Team A were to achieve a client trust period of 11 minutes, and Team B were to achieve a trust period of 15 minutes and both teams keep their clients alive for 72 hours, Team A would score higher. If no team is able to maintain a connection for the full 4320 minutes of Phase 1b, the winner will be decided by scoring the length of the longest lived connection over the trust period.The GoZ team also announced that, in terms of judging, the data collected during both Phases 1a and 1b would be combined to determine a challenge winner.
With the GoZ relaunch, Phase 1b proceeded more successfully, with a number of Spam Reflection Attacks helping to effectively raise the stakes for phase competitorsâweeding out those who had insufficiently robust infrastructure or relayer configurations.
At Stake Capital we experienced a particularly bumpy start to Phase 1b due to, what we went on to discover to be, a bug in the relayer implementation.
When reestablishing a connection with the GoZ Hub, both client-ids had to be forcibly changed. This, of course, was incorrect Relayer behaviour, since any non-expired client-id associated with a chain should be able to be effectively reused to reestablish a connection without issue. Thankfully, the GOZ team responded quickly to push a patch for our bug. You can read more about the specifics of the bug in our issue filed here.
The official winners of Phases 1a & 1b were later announced, with the prize for 1a split between four teams â due to scoring ambiguity associated with the fact that multiple teams performed exceptionally well â and prize 1b going solely to the Kira Core team. The GoZ Liveness Award winners were also later announced: splitting 50,000 ATOMs, Stake Capital was named as a joint-winner.
Phase 2
Phase 2 began on Monday, May 25th, with the main objective metric of throughput â meaning that teams would compete to relay as many packets as possible with their respective Relayer keys. Scoring varied based on the direction and origin / destination of the packets. Packets relayed to the hub by a teamâs relayer key were the most important metric, worth 1 point; packets relayed from the hub were the second most important metric, worth 0.5 points; finally, packets relayed among other (non-Hub) chains were less important to performance evaluation, worth only 0.1 points.
For Phase 2, a number of unique approaches were deployed by the highest scoring teams, aiming to send an abundance of packets to and from the hub, as well as between different zones. Most notably, the Sentinel team very successfully implemented and deployed a modified relayer which allowed for the transfer of multiple packets within a single transaction at a very competitive throughput â you can read more about their compelling work here. The Stake Capital team also hacked the relayer to achieve a competitive throughput.
Considering the unique success of Sentinelâs custom relayer modification, they were selected as the sole Phase 2 winner. (See below for our thoughts on this questionable decision to solely reward a single successful performer from each phase. đł)
Phase 3
The main objective for Week 3 was to stress-test the IBC security model, with the winner being the team that executes the most successful âconfusion or deception attacksâ against other zones.
Although a number of unique strategies of deception were deployed during GoZ Phase 3, P2P.org was selected as the Phase 3 winner for their unique double spending attack achieved via double signing â you can read a great write-up from their team about the attack here.
Our Take on the GoZ Reward Distribution
Although Phases 1a and 1b enabled multiple teams to earn an ATOM reward for their contribution via the Liveness Award, both Phase 2 and 3 did not have this incentive. As such, only the single very best team was eligible to directly earn a reward from participation in these phases. Although a number of other prizes were offered to teams (e.g. content creation / documentation, Most Packets Relayed, and Most Creative Zone â see the full list here), this lack of direct participation incentivization led to a significant drop in the number of participants by Phase 3.
Rather than giving many thousands of ATOMs to single phase winners, active phase participation could have been better incentivization via a more egalitarian reward distribution across successful participants â e.g. spreading the phase rewards across the top 10 most successful participants for each phaseâs respective success metrics. At Stake Capital, we did lobby for this more distributed reward distribution approach, however rewards were established from the outset of GoZ and thus remained the same â albeit with some tweaking for the aforementioned multiple winners of Phase 1a.
Scoreboards
For Phases 1a and 1b the official scoreboard, constructed with Sagan and DataDog, provided real-time visibility into participant performance. Two separate scoreboards were provided, one showing Client Update By Relayer, the other showing ICS 20 packets by Relayer â see the complete descriptions of the Phase I scoreboards here and here.
For Phase 2, the team provided one comprehensive DataDog dashboard showing all key evaluation metrics for the phase.
In addition to the official scoreboards provided by the GoZ teams, a number of other fantastic visibility tools and explorers being developed by the community specifically for GoZ, including:
- goz.p2p.org and dash-goz by the P2P.org team â used widely by participants throughout the competition.
- Map of Zones by Bitquasar.com & Ztake.org â a great visualizer showing a data-rich workflow of the different zones and their IBC-based connections.
- IRISnet BlockChain Explorer by the IRIS Network team â another data-rich GoZ visualizer.
Looking Forward
Although riddled with challenges and some instability, overall Game of Zones did function as a testing ground for IBC and, more than anything, exposed large swathes of the Cosmos community â validators and devs alike â to the setup and use of relayers and the practical operation of IBC in the wild, at scale. This collective experience, in addition to many of the useful tools and scoreboards created by teams during GoZ, will undoubtedly prove invaluable when IBC hits mainnet soon.
Leopold Joy & Stake Capitalâs TEAM
