Reputation Assessment vs. Reputation Management
Given the distortion caused by reputation management systems, how can reputation assessment be handled at all so that “organic” reliability and trustworthiness can be truthfully measured?
Humans are social animals able to efficiently handle social relationships and thereby obtain survival and competitive advantages from group behaviours. The British anthropologist Robin Dunbar proposed a limit, now called Dunbar’s number, on “the number of people with whom one can maintain stable social relationships — relationships in which an individual knows who each person is and how each person relates to every other person.” The value of Dunbar’s number has been evaluated as between 100 to 250. In practice for highly efficient social performance, however, the number is limited to 5–15, which corresponds to average team size or army command operating under the direct management of a single leader or officer.
In our modern online world, the connectivity in social networks is often several orders of magnitude larger than that from “face to face” encounters, exposing people to thousands of “friends” and marketplace “offerings.” Such exposure makes the problem of assessment of reliability and trustability of “friends” and “offerings” unmanageable. It quickly becomes impossible for users to reliably evaluate the quality of their social connections, opening the door for ruthless actors to manipulate connections through cheating of ratings, reputation gaming, and astroturfing recommendations and reviews. Indeed, an entire business domain and market, called “reputation management”, has emerged to artificially manage or shape one’s reputation.
Given the distortion caused by such reputation management systems, how can reputation assessment be handled at all so that “organic” reliability and trustworthiness can be truthfully measured? The SingularityNET reputation system, based on the “Liquid Rank” algorithm can be applied to a wide domain of use cases, improving the safety, reliability, and trustworthiness of marketplaces and social networks.
We unveiled the SingularityNET reputation system in August 2019 through two presentations as part of the “Artificial Intelligence for Social Good” workshop held during the International Joint Conference on Artificial Intelligence in Macau, China.
The first presentation, “A Liquid Democracy System for Human-Computer Societies” focused on tuning the Reputation Assessment System to ensure financial security in marketplaces to minimise losses due to scam caused to honest suppliers by reputation gamers. We demonstrated how different market conditions and reputation gaming attack vectors can be prevented or have their impact minimised via extended versions of our Weighted Liquid Rank reputation algorithm presented in our earlier publication.
In our second presentation, “A Reputation System for Market Security and Equity” we focused on finding a balance between the security of honest participants and overall consumer satisfaction. We also focused on the equity of the suppliers, so that supplier income corresponds to the true quality of their goods. We further explored how different reputation system use patterns can achieve this balance. We used a Loss To Scam (LTS) metric to identify the fraction of market value paid by consumers to scammers; a Utility metric to measure the overall customer satisfaction on the market; and an Inequity metric for suppliers as an extended Gini coefficient, normalised to account for organic service or product quality. Lower values of the Inequity measure indicate that suppliers providing better services are generally compensated better. The study shows that using a “Winner Takes All” (WTA) policy, in which the supplier with the top reputation score is always selected leads to increased LTS and Inequity and decreased Utility. In contrast, when the shortlist of highly scored suppliers is used for semi-random selection implementing “Roulette” policy, it is possible for the LTS to decrease below the level in which no reputation system is used. This case also allows Utility to increase making Iniquity lower than in the WTA case, even if remains higher than in the case when a reputation system is not used.
We demonstrated another application of the Reputation Assessment System at the IMCIC 2019 conference in Orlando, Florida, USA as “Reputation Systems for Human-Computer Environments” (with slides). We showed that the Reputation Assessment System can track reputation dynamics in social networks such as the blockchain-based Steemit social-financial network and render social hierarchies in the network based on the whole volume of financial (transactions), textual (comments) and emotional (likes/votes) interactions between network participants of the network, as shown in the following illustration.
For more detailed information on design, implementation and applications of the Reputation Assessment System see the following video.
For further information, please refer to the reference implementation of the Reputation Assessment System in Python within SingularityNET project or its Java implementation as part of Aigents project and stay tuned to our blog further.
