Motivations & Reputations: Mapping Reputation (Eco)systems in Web3

18 min readJul 21, 2023

Reputation systems play a vital role in decentralized technology communities. They serve as proxies for established social relationships in the absence of close connections between individuals online. In Web3, reputation systems seek to enable trust, incentivize participation, and foster community. However, DAO & protocol leaders are often unclear on how to implement reputation systems. In the face of complexity about how to approach reputation, they default to the question, “Which reputation tool should we use?” This approach prioritizes the most popular tools rather than designing and building systems that address specific community objectives. Consequently, reputation systems are often utilized in a misaligned and ad-hoc manner, failing to adequately meet the needs of DAOs.

To address this challenge, our research provides a framework to help DAO & protocol leaders (1) clarify their core aims and (2) select reputation mechanisms and tools to support their objectives. Our conclusions in this article come from:

Clearly defining reputation systems and how they are different but interplay with the concepts of identity, membership, and governance.
Mapping the nine objectives that reputation tools have been developed to address and grouping them into core aims.
Correlating the reputation mechanisms and tools that exist today to the objectives they are trying to achieve.
Outlining common barriers to building reputation systems.

I. Introduction

What is a reputation system?

Reputation systems are a method of evaluation (programs, mechanisms, and metrics) typically developed on a set of criteria, such as behavior, history, or performance. Throughout centuries, reputation has remained a ubiquitous and spontaneous measure of social interaction. People rely on the trusted opinions and experiences of others, and themselves, to inform whom to trust, discourage harmful behavior, measure influence, encourage quality participation, and so on.

Today, reputation systems occur formally (e.g., credit scores¹, rideshare ratings², e-commerce³, government social systems⁴, etc.) and informally (e.g., call-out culture, social media follower count, and the rise of product influencers, etc.).

More specifically, reputation systems have been integral to the success of technology organizations. Because online interactions often lack familiarity or established connections between individuals, digital reputation systems have emerged as proxies for in-person social relationships. In the absence of direct or network-based social connections, reputation systems serve as proxies to assess and evaluate individuals’ trustworthiness and credibility. Peer-to-peer feedback (e.g., user ratings) is an example of a reputation mechanism that has fueled the globalization of e-commerce and the rise of the gig economy.⁵ Other reputation mechanisms provide structure for decision-making (e.g., Wikipedia’s Barnstar system) and verifiability of information in online communities (e.g., Reddit’s Karma system).

Reputation and Web3

Why are reputation systems relevant to decentralized technology?

Similarly, within the context of Web3, reputation systems offer a way to foster trust among digitally dispersed communities. They are essential because:

Identity is not a given. Improving trust is increasingly complex when one’s identity can remain unverifiable.
Decentralized ecosystems do not often rely on traditional employment criteria. Similar to the gig economy, reputation in decentralized ecosystems serves as a mechanism to gauge contributors’ accountability and ensure quality assurance since a central employer is not evaluating one’s work.
Multiple stakeholders add complexity. Decentralized ecosystems often resemble marketplaces, where multiple stakeholder groups contribute to the success of the system. This can make it challenging to determine, measure, evaluate, and attribute what specific behaviors and contributions are necessary from each individual to ensure the system to operates effectively.

What is the challenge with reputation systems in Web3?

DAO & protocol leaders see the inherent need for reputation systems, but are often unsure of where to begin and which systems to implement. In many cases, DAO & protocol leaders approach building a reputation system with the question: “How can we manage our community? Which reputation tool should we use?”

The demand side of the market (DAOs and protocols) is inspired by new — and often shiny — tooling technologies rather than choosing systems based on the objectives needed in a community. Meanwhile, the supply side of the market (the DAO tooling) pushes forward with tools that garner immediate attention rather than designing and building with specific objectives in mind.

This approach results in reputation systems that have yet to prove they address the underlying challenges DAOs face.

For instance, MirrorDAO attempted to incentivize participation through healthy competition with their $WRITE RACE. However, their gamified approach to membership resulted in tension between the intended audience (writers) and others with differing interests (VCs and DAOs). Mirror has since pivoted from gated access to open membership access. In another reputation system, Token Engineering Commons implemented a leaderboard determined that distributed tokens using a “praise system.” However, the system favored highly visible behaviors, resulting in an internal debate known as “praisemageddon.” Many teams have found evaluating behaviors is a recurring challenge in the reputation space (e.g., Gitcoin’s nrGIV reputation).

We believe a root cause of the challenges many teams face reflects a lack of clarity about the impact a reputation system is intended to have. Reputation systems are often associated with “building trust”; however, the term “build trust” is often a catch-all statement for more detailed objectives (e.g., increasing participation, growing community, and more effectively distributing power). Often, reputation systems are defined by the how (e.g., reputation systems are a way of tracking contributions) or the what (e.g., “a reputation system is a mechanism that evaluates the trustworthiness of a user based on their actions and interactions within a network.”) In other instances, reputation systems are vaguely used to describe decentralized membership and identity verification projects.

To navigate this confusion and build systems that work, we believe teams should start with the question of why not how. Why we are building them and what we seek to achieve with them?

Apiary’s Approach to Reputation Systems

We put forth a definition of reputation systems and a framework that maps reputation systems’ core objectives within the Web3 ecosystem.

Our Definition of Reputation Systems:

Reputation systems are sociotechnical systems within DAOs and protocol organizations that encourage desirable behaviors essential for the functioning of an ecosystem. They foster trust, cooperation, and accountability by leveraging mechanisms beyond (or in addition to) financial compensation to evaluate individuals or entities based on behavior, history, or performance. These systems aim to promote positive participation, distribute ownership, and ensure the overall health and success of the system.

Our Approach:

We take the first step at mapping the “big picture” of the reputation systems landscape in Web3. In our framework, we begin with a macroscopic view of the ecosystem and then narrow into core aims, mechanisms, and specific tools. Our goal is to help DAO & protocol leaders understand which mechanisms support their specific goals and identify the necessary aims and mechanisms for their ecosystem.

Specifically, we seek to help DAO & protocol leaders answer the following question:

(1) What are our core objectives? Do we really know why we need a reputation system? What challenge(s) are we trying to solve? Are we clear on whether we’re tackling reputation, identity, and/or governance issues and how those interplay?

2) How do we craft, design, and select reputation-based tools that align with desired objectives? Do you know what behaviors we want to incentivize with a reputation system? Is it possible to measure them within our organization or across an ecosystem? Did we select tooling to support our reputation system and its mechanisms aligned with our core aims? What would happen in our organization or ecosystem if our reputation system were successful? How would we know it was working?

To accomplish this, we:

Explain how reputation systems differ, but overlap with identity, membership, and governance.
List the common objectives of reputation systems. Map the mechanisms and tools in relation to the objectives they are trying to achieve.
Outline the core barriers to building reputation systems.
Offer an actionable framework for building a reputation system.

[Figure 1. Nested outline of our approach]

II. Reputation System Landscape

Reputation is often conflated with decentralized identity, membership systems, and governance.

Reputation systems often converge and diverge with the objectives of decentralized identity, membership systems, and governance. Often these terms are conflated with one another as they all seek to create decentralized, trust-based ecosystems. For clarity, we summarize their specific objectives:

Decentralized identity systems promote self-sovereign identity, interoperability, privacy, and portability, empowering individuals with ownership over their digital identities, enabling seamless cross-platform identity management, minimizing centralized data storage, and facilitating the transfer of identities without dependence on specific providers or ecosystems.
Membership systems seek to grant or restrict access to communities (and benefits), empower members in decision-making processes, and ensure portability across platforms to enhance participation and engagement within decentralized communities.
Governance systems refer to the structures and processes that together determine how decisions are made in a given context. In short, it includes the rules and processes on what decisions can be made, how a system and its rules can be changed, and, finally, bodies, institutions, and procedures to monitor compliance with the rules, hold actors accountable, and resolve conflict.⁶ In this light, governance encapsulates reputation, identity, and membership systems.

[Figure 2 Reputation Systems, Membership Systems, Decentralized Identity, and Governance]

How Reputation, Identity, Membership, and Governance interplay:⁷

This preliminary mapping highlights the interdependencies of these systems.

Reputation <> Identity

Reputation systems provide identity systems information about contributors’ activity and contributions, often across ecosystems.
Identity systems provide reputation systems verified IDs to attribute compensation (e.g., token airdrops) and information about contributors’ history and network.

Reputation <> Membership

Reputation systems provide membership systems information to access individuals on the bases of criteria (e.g., evaluate the quality of contributions, participation levels, voting history, etc.) and manage overall access to resources.
Membership systems provide reputation systems criteria to identify quality (and/or mission-aligned) contributors.

Reputation, Identity, Membership <> Governance

Reputation systems provide governance systems with the necessary information to identify and reward quality contributors, inform dispute resolution, incentivize active participation, allocate rewards and compensation, distribute power, and ensure alignment with the overall mission of the ecosystem. Conversely, governance systems provide reputation systems with contextual data on participants’ actions and contributions, and governance systems decide on the criteria by which the community will be evaluated, tracked, and assessed.
Identity systems provide governance systems with verifiable proof of individual identity, which can inform decision-making around the distribution of influence, prove individual ownership of assets, track their contributions and reputation, and engage in decision-making processes while protecting against Sybil attacks and other forms of identity fraud. Conversely, governance systems provide identity systems verified contributions that strengthen the credibility of digital identities.
Membership systems provide governance systems criteria for establishing the affiliation and eligibility of individuals, assessing mission alignment, and distributing a more meritocratic approach to decision-making. Conversely, governance systems provide membership systems rights by allowing participants to gain formal or recognized rights (e.g., proposals, contributing to decision-making processes, accessing certain resources) and privileges (e.g., receiving rewards or benefits based on their level of participation and contribution).

III. Reputation Systems’ Core Aims

Having outlined where reputation, identity, membership, and governance overlap, we narrow our focus to reputation systems specifically. We group reputation objectives into three main categories: trust, community, and incentives.

[Figure 3 High-Level Objectives of Reputation Systems]⁸

Let’s delve deeper into each of these sub-objectives:

1. Trust.

Reputation systems aim to build trust by implementing mechanisms and tools to make decisions easier, promote accountability and transparency, and reduce risk. Many seek to build trust by implementing a second (reputation) token, but this approach often fails to address the fundamental design challenge. Trust is a multi-layered concept that cannot be adequately captured by a single token alone.

[Figure 4 Building Trust is an Aim of Reputation Systems]

The sub-objectives of trust include:

Improve and establish context for decision making. Information about past behavior lends itself to an evaluation of trustworthiness. This can inform decisions about whom to collaborate with or place in leadership roles. Putting power in knowledgeable and deserving hands is intended to improve security.
Reduce risk, bad actors, and harm. Access to information about past behavior and contributions can help identify reliable contributors and weed out bad actors.
Promote accountability and transparency. Tracking and evaluating behavior encourages responsible conduct and discourages bad actors.

2. Community.

Reputation systems aim to foster and grow community by establishing social connections, encouraging value-aligned contributions, and expanding the network of participants. Reputation systems become a key way of encouraging interactions, collaboration, and the formation of relationships, strengthening the cohesion of digitally dispersed communities.

[Figure 5 Building Community is an Aim of Reputation Systems]

The sub-objectives of community include:

Fostering community. Facilitate a shared sense of belonging, identity, and ownership amongst contributors and also between individual participants and the larger ecosystem. Recognizing and rewarding contributors’ efforts fosters a sense of value and commitment, encouraging their sustained involvement and closer collaboration within the community.
Growing community. Attract new talent to decentralized networks by providing a means to assess the credibility and trustworthiness of existing participants, reducing the barriers to entry, and encouraging individuals to onboard. Rewarding past contributors and encouraging them to invite others to join the ecosystem (e.g., referral programs, retroactive airdrops, loyalty programs, etc.) seeks to stimulate a network effect.
Preserving and protecting the mission. By rewarding positive contributions and penalizing negative behavior, reputation systems create a self-regulating environment where participants are encouraged to uphold the organization’s principles. This, in turn, fosters a sense of community and common purpose.

3. Incentives and Motivations.

Reputation systems seek to incentivize positive contributions that drive adoption and active participation. By linking a criteria of evaluation (e.g., contribution, history, etc.) to rewards and/or benefits, reputation systems motivate users to act in ways that benefit the collective.

[Figure 6 Incentivizing Positive Contributions is an Aim of Reputation Systems]

The sub-objectives of incentives and motivations include:

Increase positive participation. Rewarding positive contributions helps drive adoption, attract new participants, and strengthen the overall network and its value proposition. “Positive,” here, is a placeholder for the uniquely specific behaviors an ecosystem deems valuable, such as verifying code, participating in voting, etc.
Distribute power. Recognize and reward participants (e.g., retroactive airdrops, burn token compensation, etc.) throughout the ecosystem who have made valuable contributions and may not have been originally compensated or have voting power in the ecosystem.
Inspire creation and collaboration. Inspire creation and collaboration by providing recognition and incentives for individuals to contribute their skills, knowledge, and resources to the community or platform. Recognizing quality contributors aims to highlight leaders and invite others to follow in-suite.

Reputation Systems’ Mechanisms

Reputation Mechanisms (highlighted in orange in Figure 7)⁹ are the processes designed to support reputation systems’ core aims. They functionally make up a reputation system by offering processes to support the core aim. Reputation mechanisms often exist beyond (or in addition to) financial compensation. This includes but is not limited to peer-to-peer evaluations, proof of work systems, dispute resolution systems, loyalty/mentor/referral programs, reputation-based delegation voting, proof of stake systems, reputation mining, talent recruitment, and retroactive incentives.

[Figure 7. Reputation Systems Mechanisms]

Optimism’s Layered Mechanisms

In practice, many of these mechanisms and tools are layered to achieve specific goals. Consider, for instance, Optimism’s retroactive delegate rewards program. Optimism is an L2 that aims to enhance the scalability and usability of the Ethereum blockchain. They employ a reputation-based delegation system, but have struggled with participation.

In Spring 2023, they implemented a retroactive reward that compensated the top delegates who voted in Season 3 (on the Agora test) and embraced participation and contribution across the Token House and Citizens’ House. In this instance, Optimism sought to address three core aims: (1) incentivize participation, (2) distribute power across the ecosystem, and (3) foster community. Here, we see how various mechanisms converge: reputation-based delegation (only top delegates) indicated through contribution tracking and retroactive token airdrop for governance participation. This example illustrates how various mechanisms can be combined to support an organization’s objectives. Thus, understanding one’s core aim(s) becomes essential to navigate the array of options. The need for clarity compounds when considering the chaotic reputation tooling landscape.

[Figure 9 Optimism’s Season 3 Reputation Systems]

IV. Reputation Tooling

Reputation Tools (highlighted in pink in Figure 8) help developers integrate reputation mechanisms into decentralized communities and platforms. These tools often offer features like reputation scoring algorithms, data visualization, reputation analytics, etc. Early reputation tooling was quite rudimentary– often focusing on aggregating quantitative metrics and single integrations, such as NFT badges with limited usability. However, there is an increase in tools aggregating off-chain and on-chain information; offering integrations that can be built on top of or in addition to other tools; supporting cross-chain and cross-ecosystem individual identification; and allowing communities to decide for themselves how the tools should be used and valued. Below we outline ~70 current digital tools that aim to facilitate and support specific mechanisms and core aims.

[Figure 8. Reputation Systems Tooling]¹⁰

How does reputation tooling support core aims and mechanisms?

A closer look at reputation systems’ tooling landscape reveals an array of approaches to designing a reputation system. Tools emerged as a market response to the challenges of building reputation mechanisms, namely: unclear or inaccessible information about decentralized communities, lack of verifiable identity, technical barriers to creating, distributing, and managing badges and token distribution, etc. [see Figure 8 for full list].

That said, there is a bilateral relationship between tools and mechanisms. Reputation tools can both support reputation mechanisms, as well as offer the mechanism itself. Consider the tool Source Cred, a compensation algorithm for decentralized communities. More specifically, Source Cred is a tool that offers contribution tracking as a way of enabling compensation. Here, we can see how it is both the mechanism (contribution tracking) as well as a key input of a different mechanism (retroactive token incentives, for instance). Thus, choosing the right tool is increasingly complex and important.

Together, reputation tools make up the connective tissue between the core aims and mechanisms of reputation systems. But for these tools to be utilized most effectively, DAOs and protocols must first understand their core aims. Doing so will assist in (1) navigating modular tooling and (2) measuring if the intended mechanism is working. The value of a core aim is exemplified in the case of Otterspace:

Otterspace’s Modular Tooling

Otterspace is a community management tool that offers non-transferable NFT “badges.” Badges can enable DAOs to engage in non-token governance, automate membership, manage reputation credentials, and align incentive systems. Otterspace offers a range of combinations with different possible options for governance tooling. The core offerings of Otterspace tooling can be understood as

Foster community. Badges can be awarded to different teams to create a sense of shared identity or rewarded to contributors for their achievements.
Reduce risk and bad actors. Badges can be used to indicate different levels of membership and prevent bad actors by gating access to the community.
Preserve mission. Badges can provide visible and tangible recognition of those who embody the values of the community.
Increasing participation and making decisions easier. Off-chain Otterpacks can be used for internal team coordination, such as Core teams gauging support for various proposals to the treasury.
Distributing power. Otterhats are “role-bound” NFTs to delegate different access and power to different contributors.

[Figure 10 Otterspace Tooling]

All of these mechanisms can function alongside or plug into existing token governance processes. For instance, Otterspace can be integrated with Snapshot Labs to delineate voting powers, on Gnosis Safe to manage collections, Guild, and Collab.Land, and Wonderverse to gate membership through badges.

Together, the reputation mechanisms and tool mapping underscores the need for clarity around a DAO & community’s core aim. Because a mechanism (and the tooling options) can be leveraged in many ways, identifying the core aim becomes a north star to navigate the array of tooling options and integrations.

V. Key Challenges Facing Reputation Systems

While motivations and mechanisms are central to understanding the current reputation-system landscape, they do not exist without challenges. Before diving into building reputation systems, we have identified the (high-level) core challenges/barriers to the objectives mentioned above.

VI. Conclusion. Building Reputation Systems that Support Core Aims

Digital reputation systems are actively evolving. In order to iterate and build reputation systems, we need to clearly define our experiments. Emergent strategies only work when experiments are clearly defined.

Below is an actionable set of questions for those engaging in reputation systems:

If you are a VC investing in tools, utilize these questions to hold the development team accountable to clear objectives and to measure success.
If you are a DAO leader or operator implementing these tools, come to the table with a sense of which objectives you are trying to achieve. Measure progress towards those objectives.
If you are a tooling team, you can use these questions to articulate understand your client’s needs, position your offering in the space, and measure if the objectives you’ve set out to change for DAOs evolve.

Questions to consider:

Do you really know why you need a reputation system? What challenge(s) are you trying to solve? Are you clear on whether you’re tackling reputation, identity, and/or governance issues and how those interplay?
Do you know what behaviors you want to incentivize with a reputation system? Is it possible to measure them within your organization or across an ecosystem?
Did you select tooling to support your reputation system and its mechanisms aligned with your core aims?
What would happen in your organization or ecosystem if your reputation system were successful? How would you know it was working?

If you are exploring questions of reputation systems, a great place to start is a better understanding of your community. You can reach us at hello@apiary.xyz.

This content is provided for informational purposes only and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters.

Footnotes

Credit Scores systems are used to assess the creditworthiness and reputation of individuals or businesses seeking loans or credit. These systems aim to evaluate “quantitative” factors such as repayment history, income stability, and credit utilization to generate a credit score, but not without bias.
Ridesharing apps such as Uber and Lyft utilize reputation systems to assess and evaluate both drivers and passengers based on two-way ratings, user reviews, account suspensions, and profile history. These systems help users make informed decisions by providing average ratings and feedback, while also enabling the platform to take appropriate actions against users who violate guidelines or engage in misconduct.
E-commerce sites like Amazon, Etsy, Craigslist, etc use reputation systems to promote trust and reliability among buyers and sellers. These systems allow customers to rate and review products and sellers based on their experiences, helping other buyers make informed purchasing decisions.
China’s “social credit system,” a system that aggregates information to reward actions that build trust amongst and hold individuals, companies, legal institutions, and government agencies accountable.
Of note, there is increasing pushback for these mechanisms. https://www.forbes.com/sites/adigaskell/2022/05/16/how-gig-workers-strive-to-exert-some-control-over-their-work/?sh=67a8244a1e26 / https://www.wired.com/story/how-to-fix-ratings-in-the-gig-economy/ / https://insight.kellogg.northwestern.edu/article/gig-workers-customer-reviews
These definitions are specific to decentralized technology communities.
The use cases listed are non-exhaustive. Moreover, these systems are not mutually exclusive and often coexist in a non-tokenized governance mechanism.
Many of these aims are mutually reinforcing. For instance, the aim of “reduce risk” and “promoting accountability and transparency” are two sides of the same coin.
Again, we note that this is a theoretical mapping based on high-level motivation and aspiration. We are not saying that “reputation mining” will always “incentivize participation;” rather, that it is simply one aspect of the mechanism’s intention.
The yellow circles present the core aims, and the orange circles represent the various machines that can be used to achieve the desired aim. Finally, the pink circles represent the existing tools and platforms that sell services that support the mechanism.

Works Cited

Reputation

Sybil Resistance

Information Aggregation

Decentralized Identity

Membership