TCR Design Flaws: Why Blockchain Needs Reputation

It seems like Token Curated-Registries (TCRs) are all the rage right now and virtually every crypto startup is incorporating them into their protocols.

To summarize, TCRs are supposed to offer a way to create decentralized, crowdsourced lists. The criteria for inclusion in these lists ranges from objective questions like “who are the soccer players that scored at least one goal in the world cup,” to subjective ones like “what are the best restaurants in NYC.” The answers are based on crypto-economics alone and anyone can participate by proposing list candidates and casting token-weighted votes.

However some are starting to suspect that TCR designs are insufficient to answer subjective questions in a reliable manner (more on this below). I tend to agree; crypto-economics alone simply cannot provide enough nuance in these situations. In order to build TCRs that can reliably answer subjective questions we need a reputation system. At Relevant, we are building a reputation protocol that enables information curation based on subjective criteria and can be used with TCRs and other curation market mechanisms.

Why Subjective TCRs Don’t Work

Aleksandr Bulkin has contributed some great analysis of subjective vs objective TCRs:

Curate This: Token Curated Registries That Don’t Work.

Bulkin outlines 3 criteria for a successful TCR: (1) an objective answer exists (i.e subjective TCRs don’t work), (2) it is publicly observable, and (3) it is very cheap to observe it.

The reason for criteria (1) is that in an open network, subjective questions will lack a coordination signal and hence cannot be accurately answered by a TCR. One may counter this with the notion that the coordination signals exist around the intended purpose of the TCR and the incentive to increase the value of the token by attracting more list applicants. However we are blissfully forgetting about the tragedy of the commons when we make this assumption. Token holders will always be more interested in their own well-being, and given a chance, will choose to make a short-term profit at the expense of slight devaluation of the token at some arbitrary point in the future. This becomes particularly true as the market size of a TCR grows (and it needs to be fairly big to begin with in order to resist manipulation). The main motivation for curators will always be “will I win a reward this round?” because even if they consider long-term downside risk, they will be able to mitigate it via a timely exit.

The Power of Context

To create a stronger coordination signal around a subjective question we need a well defined context. For example, given a question with a seemingly arbitrary answer, like: “What is better, Bitcoin or Ethereum?” in the context of a Bitcoin chat room, the answer and the coordination signal become very clear.

But defining context is not easy. The question boils down to who are the people curating information are and why are they doing it. In a permissionless network these questions are hard to answer. The stated purpose of a given TCR can provide some signaling, for example — “curate the best restaurants.” But things get complicated once we begin to consider the various actors involved in curation. For example, our restaurant TCR token holders may include:

1. Food connoisseurs eager to tell the world about the best restaurants
2. Restaurant owners that want to promote their business
3. People that don’t care much for food but want to earn some tokens

If Alice assumes most of the token holders are in category 1, she will try to curate the restaurants that have the most unique menu and high-quality food. In the second case, Alice will vote for the biggest, fanciest restaurants independent of quality because those are the restaurants that will hold enough tokens to sway the vote. In the third case, Alice will vote for the most popular restaurants independent of quality because that’s what the public seems to prefer.

There is no clear coordination signal because the context is not well defined and we end up with the exact problem Bulkin describes. Furthermore, with token-based voting, the context will always be contaminated by the context of wealth. Alice will always be incentivized to vote for the choices she believes the whales will vote for.

TCRs + Social Reputation

We can design a system with a well-defined context by adding a social reputation system. Traditional oracle designs sometimes include a reputation system that represents honesty in reporting objective observations. In our case we are not interested in just honesty, but a general alignment of subjective social values, so we want reputation to stem from social interactions of the curator community. This will enable communities to effectively encode and maintain alignment of community values.

Let’s set aside for a moment the problem of constructing an on-chain reputation (I’ll describe how Relevant is approaching this below), and assume that one exists.

Ex: a community of food critics with a reputation system where a high reputation score represents both commitment to the stated mission of the community and gastronomic expertise. These food critics curate quality restaurants using a TCR.

Most of the TCR dynamics remain the same, but when curators vote for inclusion of a candidate in a list, their votes a weighted by their reputation, not their token balance. Participants can still stake tokens when casting votes, but the tokens are only used to allocate rewards to the winning side (the more tokens you stake, the bigger your reward). This design gives all users a clear coordination signal to vote in alignment with the reputation system. In the case of multiple conflicting coordination signals — i.e. a restaurant with a celebrity chef but low quality food — we can rely on the reputable actors to be similar enough to attach relative importance to the signals in a consistent manner. A community of celebrity chefs will likely accept the submission but a community of food critics will likely reject it.

An interesting side effect is that Bulkin’s third criteria, that “[the answer] is very cheap to observe,” now only applies to reputable actors and not the general public*. For a lay user, judging whether a given code commit to the Ethereum core codebase should be accepted or rejected may be prohibitively expensive, but not so for a high-ranking Ethereum dev. Users with no reputation can still participate if they want, they just won’t impact the final outcome. For them the action of voting is akin to betting in a prediction market (the outcome of token-weighed votes can even be a useful secondary soft signal)**.

*For other potential solutions for curating answers that are hard to observe, check out this article by Moshe Praver:

Subjective vs. Objective TCRs

** I’m assuming incentive dynamics of TCR 1.1, where a portion of the minority voters get slashed in order to award voters voting for the winning outcome (similar incentive structure can also be accomplished via an inflationary token).

Token Curated Registries 1.1, 2.0 TCRs, new theory, and dev updates

A well governed reputation system can also solve the problem of vote-mememing (aka voting ring) attacks as described in the TCR white paper. If the community has a way to punish bad actors by reducing their reputation & blacklisting accounts, voting rings and possibly some cases of vote buying attacks can be mitigated.

To summarize, a TCR combined with a reputation system has several advantages over vanilla TCRs:

• Context that helps create a coordination signal around based on subjective community values
• A well defined community of curators that hold certain values in common
• Deep engagement in the community is incentivized because reputable actors are able to earn greater economic rewards
• Reputation scores that are more likely to be evenly distributed than wealth
• TCRs are easier to bootstrap when given a reputation system (regular TCRs require a large market to create enough resistance against wealthy attackers)
• Reputation systems can potentially help with voting rings and vote-buying

Building an On-Chain Reputation

Below is a high-level overview describing how a curation community can bootstrap a reputation system using the Relevant protocol and community tools. I’m leaving the in-depth description of the protocol design for a later blog post.

Bootstrapping a Top-Restaurant TCR:

1. A few food critics decide to start a restaurant TCR, verify their identities on-chain (to signal trust), and designate themselves as ‘admins’ of the food critic community.
2. Founders invite pseudonymous members to casually post comments and discuss restaurant recommendations in the community chat and begin to upvote quality contributions.
3. Reputation scores get computed for the members of the community using a sybil-resistant pagerank-type algorithm and comment votes. For example, when an admin upvotes Alice, her reputation goes up. Since Alice now has a positive reputation score, when she upvotes Bob, Bob’s reputation increases as well.
4. Users nominate and elect new members for admin positions and remove inactive or inept admins via reputation-weighted elections***.
5. Community launches a Quality Restaurant TCR that is curated via reputation-weighed votes****.

***In order to make elections resistant to vote-buying and manipulation attacks we propose a Wikipedia-style election process where votes are only used for signalling and final outcome requires an approval by one of the admins.

**** Even if the reputation system is is sybil-proof, to enable sybil-resistant voting one would need to use either reputation*stake, or compute votes using a pagerank algorithm with modified parameters.

Relevant’s reputation design has the ability to reinforce its intended context. We can observe a circular dynamic in the flow of reputation — it stems from initial admins, and is then transferred to community members who can use it to elect new admins. And of course we have the final benefit of being able to curate quality restaurants via TCR. Our reputation system can also be applied to continuous curation markets (similar to Steemit) in which content is ranked by reputation-weighted votes and users can stake (or bet) on the ranking of a given piece of content. In fact, we are focusing on this use case in our incentive protocol. You can check out a very early beta version here: https://relevant.community/home

Challenges:

• Reputation computation is resource-intensive and hard to implement on Ethereum right now. We are designing a custom federated blockchain network built on top of Tendermint & Cosmos that will be able to facilitate this.
• Transparent reputation systems are easier to game then opaque centralized ones. To mitigate attacks, community members will need to monitor activity using off-chain tools and remove malicious actors via governance methods — for example by maintaining a list of blacklisted accounts.

Reputation systems will not solve all of blockchain’s challenges, but they are an essential feature of digital platforms and cannot be replaced with token economics. Just because Bob is holding a bag of FOOD tokens, doesn’t mean anyone should trust Bob’s taste in food. At the end of the day any token is money — a reputation system in its own right, but a limited one. There is a reason meatspace is rife with reputation systems — social, academic, cultural. We rely on myriad reputational systems, both explicit and implicit, in our day-to-day life. The idea that we can just swap all of this out for tokens and expect anything good to come of it is at best misguided and at worst pathological.

If you want to get in touch you can find us on Slack or follow Relevant on Twitter.

Thanks to billy rennekamp, Sam Hart and Taylore Scarabelli for edits and feedback.