Introducing: AI Trainer Bounty 2.0
04/04/2024 Update: We have completed implementation of MIP-12, which creates a sustainable source of rewards for AI Trainers by allocating 10% of the weekly rewards pool. The bounty is now complete. With this change, there is now a rewards pool available every week based on Global Map Progress. It is shared among AI Trainers based on how many tasks they completed, the weight of those tasks and the reputation of the AI Trainer. The weights of tasks will be adjusted over time with the most up-to-date weights being published here.
TL;DR — Today, the Hivemapper Foundation is announcing an update to its AI Trainer bounty program. Under the new bounty, which applies to tasks completed after Monday, Oct. 9, the amount of HONEY rewards generated will vary based on the complexity and time required for each type of AI Trainer task. Each reviewer’s rewards will be scaled up or down based on their reputation score and other quality metrics.
–
Since the launch of AI Trainers, more than 16,000 contributors have done more than 70 million reviews to train Hivemapper’s cutting-edge Map AI.
There are now more than 20 active AI Trainer games, which use the human intelligence of the Hivemapper community to validate how the Map AI classifies, positions and decodes objects in the real world. With this feedback, our Map AI gets more and more precise every day.
We launched a bounty program in May to reward contributors better without taking a larger share of the HONEY rewards that are minted on a weekly basis. The bounty offered a flat amount of 0.1 HONEY per review, which was redistributed based on reviewers’ reputation.
This bounty did its job, incentivizing thousands of contributors to participate in AI Trainers. However, it did not account for the fact that some games are more complex and time consuming than others.
Some of the most skilled contributors responded to these incentives, grinding simpler games to maximize rewards. We can do better. To put it simply: reviewers should be fairly rewarded for “leveling up” and spending their time on challenging tasks that are more valuable to the network.
More rewards for more complex tasks
Under the new bounty, which takes effect on Monday, Oct. 9, AI Trainer tasks will be divided into five tiers. The amount of HONEY rewards generated will vary based on the tier.
These amounts will be multiplied by the reputation of the reviewer to determine the final reward. A reviewer with a reputation of 1 will get the full amount. A reviewer with a reputation of 0.5 will get half as much. And a reviewer with a reputation of 0 will get 0 HONEY.
As an example, the No Turns 2 game from Tier 1 requires the reviewer to sort signs by type: no left turn, no u-turn, and so on. It’s a simple job and it moves quickly. At an average of 3 seconds per task, a motivated reviewer could do 1200 reviews in an hour. Under the new bounty, this would result in an hourly reward of 1200 * 0.01 = 12 HONEY for a reviewer with the maximum reputation score of 1.
For comparison, the Positional Accuracy game from Tier 5 asks the reviewer to confirm the location that Map AI assigned to an object. This task can be rather complex if done well, requiring the reviewer to compare the assigned location against satellite imagery to validate that the location makes sense. At 45 seconds per task, a motivated reviewer could do 80 reviews in an hour. This would result in an hourly reward of 80 * 0.3 = 24 HONEY for a reviewer with the maximum reputation score of 1.
As you can see, this creates a strong incentive for reviewers to level up and focus their time on tasks that require higher skill.
Behind the scenes, we have also invested a significant amount of work into overhauling our reputation system, so that anyone who attempts to spam the system is swiftly identified and receives no rewards.
Even skilled reviewers who make it to the highest reputation levels will fall back down if they try to move too quickly and start making errors, resulting in a lower bounty per task completed.
There is no scheduled end date for this bounty; we will continue to iterate over time and will provide advanced notice for any planned change. We are committed to providing fair rewards for AI Trainers on an ongoing basis, making it an attractive place to spend time relative to other tasking options.
If you have ideas or suggestions, we would recommend that you provide feedback in the #ai-trainers channel on Discord.
