Economics of Tokenized Incentives 2: Selecting Effective Performance Metrics
In our previous post, we explained that pay for performance is useful when key desired outcomes cannot be specified in advance. Under those conditions, pay-for-performance systems allow us to measure performance after the fact and reward contributors accordingly.
When creating a pay-for-performance scheme, the most important design element is the choice of performance metric. A well-chosen performance metric can align the incentives of contributors with the goals of the platform, while a poorly chosen metric can be ineffective at best and encourage destructive behavior at worst.
Poorly chosen performance metrics abound. Even top companies with a plethora of resources are not immune to them. Wells Fargo famously instituted a system in which branch employees were rewarded for the number of new accounts they opened. This system was so successful in motivating the creation of accounts that employees felt compelled to open fraudulent accounts without customers’ consent. As a result, this system violated customers’ trust, created an unpleasant work environment for employees, and cost the bank hundreds of millions of dollars in fines.
Finding appropriate performance metrics can help blockchain platforms to avoid costly mistakes. Sometimes, though, the best course of action is to use other incentive structures to reward desired behavior rather than using pay-for-performance. This post outlines a process for determining whether a good performance metric exists, and if so how to identify it.
As the name suggests, a performance metric is intended to measure the contributions of participants in a pay-for-performance system. By their nature, performance metrics must be variables that can be observed, measured, and verified, such as the revenue generated by a salesperson. In the context of blockchain, a performance metric must also be readable by a smart contract. But it is not enough to simply find a variable that satisfies these criteria. As the old saying goes: not everything that can be measured matters, and not everything that matters can be measured.
The two essential criteria for an effective performance metric are:
- It should be well aligned with activities or behaviors that are under the control of individual payee. This ensures participants are motivated to act.
- It should be well aligned with the goals or overall long-run value of the payer. This ensures that the actions participants take are beneficial.
Going back to the Wells Fargo example, the bank wanted to generate more revenues by getting customers to utilize more of the bank’s services. In theory, bank branch employees might be able to help achieve this goal by alerting customers to services that would benefit them and up- or cross-selling to those customers. But the performance metric they chose — the number of new accounts opened — did not satisfy the criteria we’ve discussed. While the number of accounts opened is clearly under the control of the branch employees — they were able to open thousands of accounts at will — the number of accounts a customer has open is not closely related to the revenue that the customer generates for the bank. In fact, 95% of the accounts that were opened under this incentive program generated no revenue at all. A poor choice of performance metric led to organizational dysfunction that Wells Fargo will be cleaning up for years to come.
These same essential criteria can be applied in the context of blockchain platforms, where pay-for-performance might be used to align the interests of payers and payees.
For example, a number of projects are attempting to use blockchain to disintermediate the advertising industry. The idea is that advertisers, rather than engaging a bunch of middlemen, can go directly to the source and compensate consumers for their time and attention.
The goal of the payers on these platforms, the advertisers, is to direct consumers toward the brands that they are advertising and possibly to direct them away from competing brands. When a participants pays attention to an ad and is persuaded to buy a product they would not have otherwise bought, that is a win for the advertiser who might pay them.
The performance metrics used in these systems usually revolve around the number of seconds that an ad appears on a participant’s screen. The problem with such a metric is very similar to the problems with Wells Fargo’s account opening performance metric; it aligns well with what the user can control, but not with the goals of the company paying them. Participants can certainly control how long an advertisement stays on their screen; however screen time is not a good indication of attention, let alone of intent to purchase. For example, a user could easily keep an advertisement on their screen for an extended period and not even read it. This misalignment is likely to be problematic for companies considering advertising on these platforms.
It is important to note that sometimes it may not be possible to find a performance metric that satisfies both essential criteria. For this reason, there are some contexts in which pay-for-performance is rarely used.
For example, finding good metrics can be difficult in a field like Public Relations, where there are many steps between the activities that a PR professional can engage in — such as outreach to journalists and publications — and the goals of the client — for example, to increase revenue. In such a context, any performance metric that a payee might choose — ranging from number of publications contacted, to number of features landed, to increase in revenues — is problematic because it is poorly aligned with payer goals or because it is poorly aligned with payee behaviors.
For this reason, pay-for-performance in PR is usually avoided in favor of other compensation arrangements. Other incentive mechanisms, such as long-term relationships and reputation systems, are better at aligning the goals of payers and payees in PR and similar contexts, even though they are less objective or precise.
Blockchain platforms that are attempting to use smart contracts or other mechanisms in order to streamline and disintermediate transactions must think carefully about how those contracts are designed. A poorly aligned pay-for-performance scheme may result in participants not behaving as expected or worse, abandonment of the platform. In some cases, an effective performance metric — one that satisfies both essential criteria — may not exist. For this reason, teams should consider a broader range of incentive structures than simply rewarding participants with tokens.
