Experiment #4: Turning Harmony Questions into Challenges
Goal: Turn question ideas into actionable Challenges, tied to the right difficulty levels, incentives and timelines, for analysts to successfully solve.
MetricsDAO provides a 6 step process for Organized, On-Demand Analytics Delivery, or as we like to call it, OODAD.
After curating the Harmony data and sourcing questions from the community, the third step is to turn the most upvoted questions into Challenges for analysts.
Turning questions into Challenges involves completing 3 key steps, in this order:
- Determine the level of difficulty of solving each question
- Determine an appropriate payout amount for doing that type of work
- Determine a fair timeline that will both allow the analysts enough time to complete it, as well as provide the requested analysis to the Harmony community in a timely manner.
Workshop Set Up
Harmony gave us the 10 questions to start with, which made things so much easier. Knowing directly from their core team what needed to be prioritized helped us focus on producing the right outcomes.
The workshop we organized was to turn those questions into challenges: re-working the language to make the questions even better and determining the level of difficulty for each based on the availability of the data.
Outcomes
Amazing international turnout. About a dozen people joined from all over the world, including analytically minded community members and expert bounty hunters; people who had completed Olympus bounties and members of the Harmony data curation team; as well as members of the MetricsDAO Counsel — the OG leadership group.
Valuable complement to the question vault. This was our first time running a workshop, we had only been using the question vault until now. Turns out workshops are a valuable tool in our toolkit and a nice complement to the question vault. It also made us realize that one way the question vault could improve is to let people vote on question difficulty.
The need for the data curation team. The workshop was especially valuable because we had the data curation team join, and they can quickly tell us how hard a question will be and if the data is readily available for it.
Agreeing on difficulty levels. It took the group only half an hour to agree on the difficulty level for 10 questions. This is the key to determining payout amounts. We then work backwards from our budget:
- Very Easy and Easy Questions receive $100
- Hard Questions receive $150
- Grand Prize winners receive $450; see note below
- The top 25 submissions, as judged by reviewers, will receive payment
Each of these prizes is subject to our reviewers’ discretion. For example, if an Easy submission does not meet the basic criteria outlined in the rubric, the reviewer may recommend half payment ($50).
Some feedback we have encountered is that analysts may focus on Very Easy and Easy questions and ignore the Hard questions. To counteract this, at least 3 of the 5 grand prizes will be reserved for answers to Hard questions.
Challenges are launching next week! Here’s a preview:
- Display the number of wallets with >0 balance (as of current date; show change over a time period of your choice, minimum last 7 days)
- Daily number of active wallets (wallets with any transactions in/out that day)
- What is the sum of value (in $ONE) for all existing wallets?
- Display the daily number of transactions
- Locked assets and number of transactions
- Locked assets and number of transactions (monthly) per project (DFK, Tranquil)
- Locked assets $ on ETH bridge
- Number of on-chain delegators
- Total $ staked
- Number of validator pools
Join us on Discord to get involved!
