Rating Quality system on Privatix P2P Network

THE IDEA

Thinking, as always, about our users’ convenience, we came up with the idea to create a rating of service providers which would reflect their authenticity and quality of service. After all, picking the right provider should be an educated guess — otherwise the user is likely to lose their money.

THE PROBLEM

Immediately we encountered the difficulties that stem from the very nature of the decentralized blockchain/cryptocurrency world. Everyone loves cryptocurrency exactly because information about the participants of a given network is limited to each account’s transaction history, without any links to the physical world. But that leaves precious little data to base our rating on.

The Privatix network keeps records of each provider’s transaction history, reflecting their interactions with past customers. These records could give us the stats on the amount of services rendered by the provider and the amount of money they earned. Also, we could indirectly evaluate the quality of service by looking at the outcomes of the provider’s contracts: if a contract was terminated early by the customer, chances are the provider’s quality of service failed to meet the customer’s expectations. And vice versa, if the customer has been satisfied with the quality of service, they will use the service until the contract expires — and quite possibly even extend the contract’s duration by making additional payments. Thus, both a contract that ended on the initially agreed amount and a contract where the amount has been updated through extra deposits are positive flags indicating good quality of service.

In reality, however, we need to take into account possible unfair competition practices as well as technical issues that may hinder even the best provider’s ability to render quality services. If we just place at the top of our rating the providers whose customers spent a lot of money, pumped a large amount of traffic and did not initiate termination of their contracts, then what is barring a dishonest provider to create fake accounts, simulate their activity and thus get to the top without having any real customer?

So we needed a way to counter this possibility. The situation was actually similar to the problem of mutual trust that arose at the birth of cryptocurrencies. Theoretically, it could be reduced to the Byzantine generals’ problem, which was solved by introducing blockchain. But creating a blockchain within another blockchain with the purpose of rating participants looked cumbersome. Instead, we decided to use a game theory strategy and make cheating in our rating system more expensive than any potential benefit it could bring. No matter how enterprising cheats may be, they are unlikely to “invest” more money into the rating than they hope to make off of it.

The next step was to create a rating system that would work like that.

THE SOLUTION

We could see in the transaction history how much money a given agent earned. However, we could not fully rely on this data: the agent could have been creating fake accounts and imitating activity all along, paying money to themselves. The only real amount spent in this case would have been the Gas paid for executing transactions. And this was one piece of data we could trust.

Unfortunately, we could not say the same about the agent’s transaction history because the identity of their customers remained unknown. And here we came to a rather unexpected conclusion: in order to evaluate the provider, we must evaluate their users.

When a customer purchases services from a single provider, the relative authenticity of these transactions is low. Purchasing services from many providers gives the customer a higher degree of authenticity. And if each of these providers has a pool of customers, each of which, in turn, has a variety of providers, it is higher still.

Having arrived at this logic, we needed to express it in the form of an algorithm.

Clients and agents are connected with one another via contracts. The number of these connections together with a certain array of signals related to them determines the degree of authenticity. In mathematical terms, it resembles a directed graph, where each vertex represents a network participant and each edge stands for one or more contracts. Each vertex has a parameter — a rating that “spreads” from vertex to vertex with a certain speed, depending on the nature of the connection between the vertices (the number of contracts, the amount and quality of the services rendered).

This schematic, in turn, is surprisingly similar to the task of ranking web pages that search engines faced at the dawn of the World Wide Web: they had to find a mechanism for arranging a number of pages connected with hyperlinks in some sort of a pecking order. The solution was the PageRank algorithm and its various modifications. Over the years, this family of algorithms has changed greatly, but the initial PageRank model had a lot in common with the model of transactions between Privatix network users.

THE IMPLEMENTATION

The calculation formula we came up with looked like this:

To put it in words, the agent’s rating is equal to the total of the ratios of each of his customers’ ratings R(c) to the number of agents with which the customer L(c) is associated:

1) the rating of each customer is calculated;

and 2) divided by the number of agents with which the customer is associated;

3) then, for each agent, the resulting ratings of the customers who have used this agent’s services are taken and added up.

This calculation is iterative (the operations are repeated several times) and convergent (each time the results change less and the ratings approach certain values, or converge).

The initial rating values in step (1) are calculated based on two parameters: authenticity and service quality. To accomplish this, we collect the data and do calculations as follows:

Thus, for each agent / customer pair, the weight of the channel is calculated as the total of the funds spent in all transactions (both PRIX tokens and Gas spent are taken in ETH at the exchange rate at the time of calculation), times the “age” coefficient taken as the ratio of the number of the block where the channel was closed to the number of the current block. Thus, the more time has passed since the interaction, the less weight it has.

Authenticity is simply the amount earned / spent in all agent / customer transactions.

Quality is calculated as the ratio of the amount spent on successfully completed contracts to the total amount spent in all transactions. It is calculated separately for each agent / customer pair.

THE PRACTICE

Okay, no more math. Let’s take a quick look at how this algorithm helps create our rating system and solves the fraud problem.

The first and foremost condition is that all participants of the network get evaluated. That is, the rating is calculated for both agents and customers. Here is why this is important. If a customer purchases large amounts of services from good providers and then, after entering at some point into a contract with another provider, suddenly terminates it — in all likelihood, that provider’s services are of lower quality than the services the customer had been buying before. Vice versa, if the agent has the majority of customers with a good history who have been purchasing services both from this and other providers, and then a new customer with a short history comes along and, after buying services from this agent, terminates the contract early, the authenticity of this signal is rather low. In both cases, the degree of authenticity is determined by the evaluation of the customers and by the ways in which the particular customer’s actions affect the evaluation of the agent.

Here is how our algorithm works in various cheating scenarios.

Scenario A. Inflating one’s own rating through fake customer accounts.

This one is simple: if a fraudulent agent creates fake accounts and imitates their activity, it will have a weak effect on the agent’s own rating, since fake customers have an extremely low rating to begin with. Moreover, if all customer’s contracts are concluded with only one agent, that customer is awarded zero rating and so does not affect the rating of the agents at all. In order to inflate their own provider rating, fraudulent agents will need to do more than just imitate activity. First, they will need to pump up the ratings of their fake customer accounts by actually purchasing services from other agents through these fake accounts. But then the money does not merely flow between the perpetrator’s many wallets but ends up going to other participants of the network. Remember the objective we set when talking about the Byzantine generals? Make cheating more expensive than it can earn. If we take a close look at the algorithm, it becomes clear that the cheat will have to counter the entire money flow between the top-rated participants. And this renders the cheat’s strategy completely useless.

Scenario B. Wrecking competitors’ ratings through fake customer accounts.

If a prematurely terminated contract is a negative signal resulting in lowered rating, a fraudulent player may try entering into contracts with other providers through fake customer accounts and then breaking these contracts ahead of time, in hopes to lower the rating of the competitors. However, like in the previous scenario, in order for this to work, the fraud will have to pump up the fake accounts’ ratings, so that the signals from them could carry some weight and actually affect the calculation. As we have seen, this requires quite a lot of money and renders the attack unreasonably expensive.

Scenario C. Subsequent changes in service quality.

Another way things can make a wrong impression on users is the changes in service quality over time. Let’s say there is an agent with a good history who has been providing high-quality services for a long time. Then, for one reason or another, the quality of his services decreases. Note that this can occur for technical or real-life reasons or as a result of a fraudulent scheme — selling a high-reputation account to someone else. How fast will the rating algorithm respond to this turn of events? If signals of different age are counted in the same way, the response will be rather slow, and the longer the history of the agent, the slower its rating will change. But this problem is easily solved if we include the age of the signal in the equation (see the formula for the initial distribution of weights): older signals weigh less than newer ones.

In addition to solving the problem of subsequent quality changes, the signal age feature has another useful effect. With lack of activity the agent’s rating declines, since all signals increase their age with time, resulting in a decrease in their weight. As a result, other things being equal, an active agent gets a higher rating than a less active one. This, in turn, basically takes care of “old-money” monopolies, where several agents with high ratings may occupy the top of the rating with new members unable to break through. Since the age of the signal affects its weight, providers who wish to stay at the top need to remain active and maintain the quality of their services at a certain level. Incidentally, this is exactly what our customers need — a wide selection of high-quality services.

Take a look at the real-time visualization of the rating of Privatix network agents.

The code of the algorithm, as well as all our other code is on github.com