PoCo Series #5 — Open decentralized brokering on the iExec platform

Published in

iExec

12 min readOct 23, 2018

Introduction

iExec V2, released May 2018, introduced the Marketplace. The iExec Marketplace is composed of both on-chain and off-chain entities that allow anyone within the iExec ecosystem to trade cloud computing resources power, buying and selling computing power like a commodity. While ‘PoCo’ is the consensus protocol that governs the execution of jobs, it is the marketplace that enables trading, locking required ‘stake’ and initiates jobs. The marketplace and PoCo are two complementary elements that will evolve independently.

While the PoCo future evolutions will answer the need for new “technical” features, the development of the marketplace is motivated by the need for actors to easily and efficiently trade computing power, applications and datasets.

Brokering objectives

The ‘brokering’ aspect of the iExec solution addresses a simple issue: to provide a mechanism for different actors to reach an agreement. In an enterprise context, this agreement could be based on authorization and certificates. In the context of the public cloud, this agreement is governed by economic consideration.

Similar to other markets, a transaction can only happen when the buyer and the seller agree on the price of a given commodity. In the iExec case, an agreement corresponds to a price as well as a selected category and trust level.

In addition to the price, the agreement construction could consider authorization mechanisms to enforce constraints:

OTC: Limiting a worker pool ask orders to specific requesters.
Licencing: Limiting the execution of an application to a specific worker pool / Limiting the execution of a dataset to a particular application / etc…

In addition to the specificities of the agreement, the brokering also has to achieve some functionalities. Easily finding a counterpart and closing a deal is essential to the user experience. Entities such as smart-contracts clients should be considered, and mechanisms should be provided for them to go through this brokering process and submit jobs on the iExec platform.

To evaluate the performances of brokering solutions we will consider:

The ability or a user/provider to find a counterpart, particularly in finding the best price for a given category
The ability of a user/provider to “sign” custom agreements with restrictions (licensing / OTC)
The cost of reaching the agreement (gas fees)
The difficulty to implement that solution

Deconstructing brokering

To better understand the process of brokering, we should go down to its fundamental elements.

The first step is formalization and publication of orders. Actors that are to take part in an agreement have to formulate their requirements and publish them for counterparts to answer them. This publication process can be performed on-chain or off-chain.

The next step is order-matching. Actors need to have access to an organized order book to emit counter orders that are compatibles with the market. For example, if I want to buy RLC tokens on an exchange, I want access to an order book where I can see how much other people are selling them. With this information, I will formulate a buy order that matches market value.

Performing this matching, whether it is between two existing orders, or between an existing order and an intention (buy/sell at market price), requires data and algorithms that can be run on-chain or off-chain.

Existing implementation

The solution implemented by iExec V2 relies on an on-chain order book and off-chain pairing. This solution only supports ask.

The protocol works as follows:

ⓐ The workerpool submits a sell (ask) order to the marketplace smart-contract. This requires writing to the blockchain which is costly.
The marketplace website provides an interface to the on-chain data. Filtering and sorting are performed off-chain (gas free) so users can select which order to answer to. This is off-chain pairing.
ⓑ The user submits a buy order (with a pointer to the corresponding sell order) to the blockchain.
① The blockchain verifies the orders validity and creates (writes) the agreement on-chain.
② Execution can start, the PoCo takes over.

This solution is functional, but the use of on-chain orders has a cost. Off-chain pairing is performed on the marketplace website, with means many tools can be imagined. However, for a smart-contract to initiate a job, it must know which sell order it should answer. Last but not least, this solution does not implement the bid, which significantly limits the versatility of the marketplace.

Evolution of this mechanism

After V2 we studied many possible evolutions of the brokering process. The first requirement was to include bid orders in addition to the already available ask orders. It soon became clear that the evolution had to go beyond a simple interaction. We considered on-chain and off-chain approach and finally went for a solution where both the pairing and the order book are off-chain. It might sound counterintuitive, but it has many advantages.

If the orders and the pairing are handled off-chain, how can we build an on-chain agreement knowing that all parties have agreed? Is there a threat to the platform security?

This option relies on the use of cryptographic signatures for order authentication. We represent orders using structures containing all the required details. The hash of this structure uniquely identifies the order. The structure by itself is worthless as anyone could write and publish it. However, if we add a valid cryptographic signature (of the identifying hash), then the origin of the order can be certified with the same level of security as if it was published on-chain. This role is fulfilled by a smart-contract called the iExecClerk.

The open decentralised brokering protocol unfolds as follows:

User and Workerpool (and any other actors like dapp and dataset owners) write their orders in dedicated structures. This can be initial orders or counter orders (answers).
The actors sign their orders using their ethereum private key and append the signature to the structure.
ⓐ/ⓑ The orders are broadcasted to potential clients. This can mean pushing the orders to an off-chain website (such as, but not limited to, the marketplace), using an on-chain relay (see later) or communicating them OTC.
When orders match, anyone can send them to the iExecClerk to produce an on-chain agreement. This can be the user, the workerpool, or an external service.
① The iExecClerk (on-chain) verifies the orders’ validity (signatures) and compatibility, writes the agreement and locks the required stake.
② Execution can start, the PoCo takes over.

This is a little bit more complex than the previous brokering approaches so let’s explain some points:

Creating orders and broadcasting them off-chain is free of charge! While the previous approach required a write operation on the blockchain (expensive) this approach does not consume any gas for order creation.
Canceling an order has a (small) cost as it requires you to inform the iExecClerk (on-chain) in order to cancel an otherwise valid (signed) order.
The signature requires the private key of the account publishing the order. If such signature is difficult to produce (for any reason) the actor could submit a short transaction to the iExecClerk to pre-sign the order. This fallback is not strictly speaking necessary but is an equally secure option that could find a use case (at the cost of a minor gas fee)
For a smart-contract to buy/sell computing power, the smart-contract would have to advertise its orders. We believe the best solution for that is to use a relay. A relay is a smart-contract with would broadcast the order as an event. Workerpool would be incentivized to listen to the relay to answer these orders and profit out of them.
Compatible ask and bid orders that were emitted independently can be paired.
Orders can be broadcasted to the world or transmitted privately to partners. This enables OTC transactions such as reduced cost to use a resource (workerpool, dapp, data) for partners, promotional events (1000 free / discounted run of an app) …

What does an order look like and how do I sign it?

An order for a dapp looks like this:

The signature is computed using the pattern proposed in EIP712 and already implemented by MetaMask since version 4.10.0.

MetaMask confirmation popup for EIP712 signature

For a message to be valid, the signature must be performed by:

The requester for a UserOrder
The pool owner (scheduler) for a PoolOrder
The dapp owner for a DappOrder
The data owner for a DataOrder

Ownership is recorded in the corresponding on-chain registries.

The salt is used to distinguish two orders with the same parameters (for example two ask orders from the same worker pool with the same price, trust, and category).

Thanks to the domain separator, the signature are attached to a version of the protocol, a chain ID, and a clerk smart-contract’s address. This is a security measure that restrict orders to a context controlled by the signer.

Overview of an iExec platform with open decentralised brokering

iExec Hub: Smart-contract in charge of PoCo. The entry point for the middleware.
iExec Clerk: Agreement validation / storage, token management. The entry point for users.
Marketplace: Off-chain order store and interface.
Relay: Smart-contract in charge of broadcasting orders from on-chain participants to the marketplace.

Consequences for users

For simple human users, the process would not change at all. Answering to ask orders through the marketplace would feel very similar and would cost about the same as today. Publishing bid orders would be just as easy and would be free of charge.

Consequences for workerpool

The only modification to the current workerpool experience would be that submitting ask order to the marketplace would be free.

The open decentralised brokering would, however, enable workerpool that are willing to do so to implement services that would benefit the users. For example, a workerpool could listen to the iExec relay, or provide its own relay, so smart-contracts can easily target it.

Consequences for Dapp and Dataset providers

The consequence would be more important for Dapp and Dataset provider. They could match the previous experience by providing a single order for anyone to use. However, they could also decide to cancel this order and replace it with a new one to change the price of their application / dataset. They could also decide to have a secondary order, with a lower price and a limited number of executions to do some sort of sales. They could also apply specific prices for partners.

All this is in the domain of what is possible. It is likely that these features will not be used straight away. They will, however, prepare the future of the platform, particularly in term of OTC trading and partnerships between actors of the platform.

Consequences for smart-contracts

Today the smart-contracts experience is far from perfect. Smart-contracts that could be users of the iExec platform have to know which ask order to answer to. This could somehow work if a workerpool provides an order with enough volume, but there are today no mechanisms to prevent this ask order to be used by other users. With the new brokering approach this solution becomes viable if a workerpool and a smart-contract agree on an OTC order.

Another solution would be to rely on relays. In a fully on-chain approach, a smart-contract could “buy at market price” in exchange for a very high gas fee. In the new brokering approach, the same smart-contract could subscribe to a relay service. A broker would be in charge of watching a relay, performing the brokering off-chain and push the paired orders to the iExecClerk. A commercial broker could make a profit by asking for fees for this service.

To put it in a nutshell, these are the 5 solutions a Dapp smart-contract has to access iExec resources:

Buy from a generic order: This is similar to what V2 offers. It starts with a workerpool advertising an ask order that accepts jobs coming from anyone. The Dapp would have to report this ask order when submitting a compatible job. The issue here is that the order could have been consumed by someone else.
Buy from a specific order: Another solution is to ask a workerpool for a specific ask order, which would be limited to a specific app or a specific requester. This involves the same workflow as previously, but provide guarantees on the remaining uses of the ask order as they cannot be consumed by anyone.
Use a relay in targeted mode: Moving away from the ask logic, a Dapp smart-contract can push its order to a relay that a scheduler will be listening to. By putting the address of the targeted workerpool in its message he helps the scheduler filter it and ensures the scheduler that no one else races to answer this order before it. Schedulers have a strong incentive to listen to BID orders as it extends their market and potential profits.
Use a relay in public mode: Similarly to the targeted relay mode, a Dapp smart-contract can push a order without a workerpool specification. Any pool could answer such a order, and they will have to race to answer this order.
Use a relay in public mode with an external broker: If the Dapp is afraid schedulers might not answer its BID orders, it could rely on an external broker. Similarly to what we find in the other commodity markets, this service will be in charge of listening to a client’s orders, find compatible orders and combine them in an agreement. This service would have a cost (previously supported by the scheduler) and would, therefore, charge some fees to its clients.

Responsibility and certification on the iExec platform

One of the fundamental aspect of PoCo is that the protocol is not an arbitration mechanism but rather a framework that ensure auditability and that gives incentives for the agents to behave correctly. If a worker misbehave it gets punished (loss of stake) and can be evicted by the workerpool (responsibility of the scheduler). If a scheduler misbehave he also losses some stake, but he also risks being boycotted by the users. This however requires a way for the community to achieve consensus on which scheduler is worthy and which is not.

The strength of the iExec platform is that it is decentralised. The on-chain part manages the stake and score of the different actors. No one has the privilege to blacklist someone, which removes the incentive of anyone to fork the platform but rather to join it.

Rather than having the community voting for any on-chain restriction, we believe that the iExecClerk smart-contract should only manage the RLC stake and verify the validity of orders. Any community decision to avoid an actor should be performed off-chain. This would be cheaper in terms of gas and would be easily upgradable. It would also mean that any divergence of the community could be solved off-chain with the iExecClerk and the iExecHub continuing to play their role.

An example of attack that would be solved by the off-chain pairing is the “free execution attack”.

When a scheduler proposes to sell computing resources, it’s incentive to behave correctly is divided in two part. If the scheduler wants to make money, the jobs he received have to be validated by the PoCo, and if the consensus is not reached the scheduler will not only fail to gain any money but it will also lose some stake (proportional to the price of the execution). However, if a scheduler only wants to attack the quality of service and accepts to lose some gas in the process, it can simply propose large computing power for free and never deliver. The only cost involved is the gas required to flood the order book with fake orders.

In a fully on-chain approach, each time a user wants to perform a computation at market prices, the free order will be selected, the job will never be performed and the platform will be stuck. No user would get any computation done and no workerpool would make any benefit. For a free execution, the (proportional) scheduler stake is zero, meaning the attacker will not lose any RLC. The solution to this attack is to blacklist the scheduler. The sooner the better.

If the blacklisting is to be performed on-chain we need some sort of voting which is time consuming and expensive. We also need on-chain mechanisms to purge the order book, which is also gas-intensive.

In an off-chain approach, modifying the brokering algorithm can be done much faster. Some brokers might refuse orders with suspicious prices while others might want to wait and witness the issue before taking any decision. Other might want to keep the brokering as pure as possible and refuse to take actions. The thing is, with an off-chain brokering and orderbook we have a decentralised market where different approach could coexist and social consensus among the users would be the only judge. Like in the PoCo’s philosophy, the users actions are similar to votes and their choice of service will shape the evolution of the platform.