In this post we talk about reputation in reputation systems. At Verify, we’re building a reputation protocol on the Ethereum blockchain and are sharing these posts in an effort to share our knowledge with the wider crypto community.
In our previous series of posts we discussed the core concepts behind the Blockchain, Bitcoin and Ethereum. Let us talk about the next component of verify and is reputation.
Reputation is important to implant trust between two unfamiliar entities. Online services become more efficient with reputation systems as they allow prediction of who is trustworthy and who is not, based on previous interactions.
Reputation systems are implemented in various web services and applications, and they have various forms. You have seen them in the form of numbers, badges, words or even drawings. Their importance is not only limited to consumers making decisions but it is also important to tune the service and deliver better quality and performance.
Take Stackoverflow for instance. Users have badges and points to represent their reputation. Users rate each other and have the ability to vote up and down based on their reputation. The higher the reputation the higher the benefits they get in the website. Similarly Amazon. Amazon allows users to rate items and write reviews on them; other users can review reviews and rate them using a discrete approach (helpful, not helpful) the higher helpful votes a review gets, the greater benefits to the review author and the higher his ranking. Many other applications implement a reputation system. Good reputation of users lead to higher trust; in the context of sales the higher the reputation, the higher the returns.
In cases where a system or group needs a reputation, it can be obtained from the reputation of its members.
Reputation systems vary in the type of data they use to calculate reputation. Some systems use a single metric to measure reputation for example, a buyer buys an item and he likes it, he rates the seller with 5 stars, that is accumulated or averaged with other ratings from other buyers and used as reputation score.
Other systems aggregate different information; for example when you want to take a loan from the bank, they ask for several documents (your salary, how long you have been employed, your address…etc) all that information is aggregated to decide on the level of trust to give that borrower and accordingly decide if a loan is to be handed out or not. One thing should be made clear here: trust is not reputation.
Trust is the result of reputation. Someone with reputation is someone you can trust. So trust decisions are easily made if you have a reputation. We see this evident in commercials. Many commercials utilize some trusted icons to market their brand and inherently gain the trust of the viewers. The better the reputation the greater the resulting trust.
It should be noted that trust is a subjective matter. For example, you may be close to someone with a bad reputation and may still engage with him due to that relationship despite the bad reputation. So decision making concerning a person’s reputation is subjective. “I trust you, despite your reputation”
Systems that aggregate information to calculate the reputation score can be further categorized into two sub-categories:
1) Systems that aggregate different types of information from the same single source: this is information that was obtained from direct interaction. The type of information collected to measure reputation of a person can be of the same context or different context.
An example of multiple attributes of the same context is of a transaction; the system can utilize the rating of buyers along with the timestamp, transaction value, etc.
An example of different context is when a system uses more than just the transaction’s context. One common example is in online food ordering; aside from the price of the plate, other contexts that should be considered include how fresh the food is as well as delivery time.
2) Systems that aggregate different types of information from different sources can also be divided to information used under the same context or different context. Utilizing information from different sources require that the source be trusted.
Let us start with information obtained from different sources and under a single same context: it means that the system obtains information like ratings from different systems. Or say in a network, a node A wants to communicate with an unknown node B; before establishing a connection with that node B, A checks its records and does not find any previous connection with B. It communicates with other trusted nodes or nodes it connected with before. It ask the nodes whether it is safe to connect with the unknown node B; A gets a boolean answer (yes or no); A aggregates all that and decides if it is safe to connect with B. The aggregation can even be weighted, the trusted nodes’ response would have a higher weight and accordingly A decides on whether to connect with B or not. In this case the context was “safe connection”.
Information obtained sometimes must be derived; from the previous example the information obtained on B may not be a direct Yes/No response; it can be a bunch of historical connections and A needs to derive conclusions on its own and accordingly decide whether to connect with the machine or not
An example of information obtained from different sources and under different contexts is when it obtains transaction history of a person and also his social activity and aggregates all of that to generate a reputation score.
It means that systems use various information from various sources. Another example is an employer looking to hire someone. The employer collects data from different sources, universities where the candidate attended and previous employers and accordingly the employer aggregates all that information and creates a reputation score.
For reputation systems to be effective they should provide incentives for users to report their experiences. Something to note here is a difference between reputation systems and recommender systems. For example when a user likes an item and similarly likes another item the second item is recommended to other users given the first user likes it and he is a trusted user.
Reputation systems should be cautious about the type of information they rate because it can be subjective. For example rating an ice cream flavor; some users might like other might hate it. However if users are rating a store on availability of an item, say apples, then the result is objective and one can expect consistent ratings.
Systems that deal with various opinions on a single matter are called collaborative filtering systems. The distinction between them is subtle. Collaborative filtering systems assume genuinity and trust in its user’s; reputation systems on the other hand assume that there could be some users who can’t be trusted or faulty opinions/votes.
Combining Collaborative filtering with Reputation systems provides better results.
A point to consider is that reputation systems sometimes fail to reflect a fair rating. Consider a buyer and a seller. The buyer buys an item and the seller ships it. The delivery service damages the item and it arrives damaged. The buyer would in turn give a bad rating to the seller, even though it was not directly his fault.
Reputation systems, if centralized, should meet certain criteria they should have a communication protocol that allows users to give ratings as well as obtain information on ratings. Another criteria is a standard algorithm/engine that would do the computations and outputs a score. If a reputation system is distributed then the information is requested from different parties that hold the information (mostly information that concerns their transactions). The entity requesting the information would accordingly do the necessary calculation and output the decision.
There are various ways of computing reputation scores. A common and straightforward one is the summation of positive and negative ratings, sometimes subtracted, averaged or make use of the mode to get a single score. In some systems, negative and positive ratings are kept separate. This approach is easy to understand but gives a general picture of reputation and not specific or accurate. There are advanced approaches in which weights are used to calculate the score. The weights are distributed according to the interaction that took place. An entity A that had a recent experience with entity B would have a higher weight than entity C that had an older experience.
Another form of reputation computing is to use Bayesian Systems; Bayesian is about predicting using known information or past information. Here, past experiences and present experiences are used to predict the next experience the user would carry out.
The exact system to use would depend on the purpose of the score (what it is used for).
