Implementation of MIP-3

4/4/2024 Update: With the implementation of MIP-12, the AI Trainer bounty is officially complete. All rewards for AI Trainer tasks will be paid through the weekly rewards pool moving forward.

TL;DR — Today, the Hivemapper Foundation is announcing a temporary bounty to increase rewards for AI Trainers. The bounty will take effect for the current rewards cycle running from May 8 to May 14, 2023, with the first rewards issued on May 17. This approach differs from the original MIP-3 proposal, which indicated that AI Training rewards would be increased through a change to the Map Progress formula. Although we are postponing that change, we plan to develop a long-term methodology for AI Training rewards as part of a more comprehensive MIP related to Map Progress this summer.

–

In the three weeks since the launch of AI Trainers on April 18, more than 10,000 contributors have completed more than 7 million reviews to help train Hivemapper’s cutting-edge Map AI. Many of them have shared useful ideas for making AI Trainer as fun and rewarding as possible.

One common piece of feedback is that the current HONEY rewards from AI Trainers are too small to be satisfying. To help with this, the Hivemapper Foundation released a Map Improvement Proposal called MIP-3 two weeks ago aimed at growing the weekly rewards pool and increasing the share of HONEY rewards for AI Trainers without impacting rewards for map contributors.

After taking feedback from the community and carefully considering how the proposal could be implemented, we’ve decided to take a different approach.

Rather than changing Map Progress formulas to grow and reallocate the rewards pool, the Hivemapper Foundation will instead pay a temporary bounty from its own holdings of HONEY, and propose a more comprehensive revision to the Map Progress formulas at a later date.

The bounty will take effect for the current rewards cycle, which runs from May 8 to May 14, 2023, and the first enhanced rewards will be issued next week.

How the bounty will work

Under the terms of the bounty, AI Trainers will generate 0.1 HONEY for the weekly rewards pool with every review they complete. This means that every 10 reviews will generate 1 additional HONEY for contributors. We expect this change to result in approximately a 5x increase in weekly rewards issued to AI Trainers at current rates of activity, and we expect that number to increase in the coming weeks as more AI Trainers come online.

In the longer term, Hivemapper may have 100 or more AI Trainer tasks. These will range from today’s relatively simple tasks such as answering “Is this a stop sign?” to more complex tasks such as typing the text of an exit sign to indicate the name of the road. Each task could take as little as 1 or 2 seconds, or as much as 30 seconds, depending on the task at hand. For the time being, all AI Trainer tasks will generate the same amount of rewards per review, regardless of how time-intensive they are. This will likely change in the future as more complex tasks come online.

As an example of how the new bounty would work in practice, let’s say that we had 20,000 contributors, each averaging 500 reviews for the week. In this scenario, 10 million reviews would have been completed for the week and 1 million incremental HONEY would have been generated for the weekly bounty pool.

This pool of bounty rewards will be allocated in the same way that all other rewards are distributed today, by assigning a “user score” to each AI Trainer. This score will be based on the number of reviews they completed during the week and their AI Trainer reputation score.

To be specific:

Weekly bounty pool = 0.1 * [total number of reviews by all contributors]

Individual score for each contributor = [number of reviews] * [reputation]

Individual reward for each contributor = [individual score] / [sum of all individual scores]

Reputation score is one of several guardrails we use to deter bots and spammers. The visible ones are rate limiting and captchas, but there are other invisible ones that we have implemented and won’t disclose (for obvious reasons). Crossing these guardrails results in a reputation score of 0, meaning no rewards and no attention paid to these submissions when updating the map.

Although reputation score is not currently surfaced anywhere on the Explorer or in the Hivemapper app, it may be in the future.

Why a bounty?

You might be wondering why we decided to make this change as a bounty.

On the one hand, the Hivemapper Foundation believes that AI Training should be fairly compensated in a permissionless way through the pool of rewards allocated to network contributors. AI Training is a fundamental part of ensuring the quality of data collected by the Hivemapper Network, and should be rewarded accordingly. On the other hand, the current rewards pool creates some deep-rooted issues for AI Training rewards that cannot be quickly resolved. There are two that are particularly important to consider:

• Rewards for AI Trainers are tied to Map Progress. If mapping activity remains constant but the number of AI Training reviews goes up by 10x, the total amount of HONEY going to AI Trainers will be unchanged, resulting in less satisfying rewards for individuals. Increasing the share of the rewards pool going to AI Trainers may help in the short term, but it will not address this in a systematic way.
• AI Trainer rewards are issued based on region. AI Training rewards for imagery from a given region are calculated as a percentage of the map rewards issued in that region. If the AI Training activity is concentrated in certain regions (due to regional differences in objects such as road signs and traffic lights) it can result in a smaller overall reward pool for AI Trainers regardless of the number of reviews completed or the underlying value that AI Trainers generated for the map.

Although this bounty program will not address these underlying issues, we plan to develop a long-term methodology for issuing AI Training rewards this summer. It will likely be included in a package of changes to Map Progress designed to ensure contributors are fairly rewarded throughout the “life cycle” of a region as it transitions from no coverage to full coverage.

We would like to thank everyone who shared feedback and questions through the #mip-3 channel on Discord to help us arrive at this decision.