iExec Dev Letter #18: V2, The Dragon in the Clouds — April 23, 2018
This week, we celebrated the first anniversary of the RLC token’s release on mainnet. Now, one year later, we are edging closer to iExec Version 2. This is a major release for the team and this Dev Letter will introduce its various innovations that have never been seen before in cloud computing.
V2: “The Dragon in the Clouds” — 公孫勝 Gōngsūn Shèng
iExec introduces a new paradigm in cloud computing: it allows the trading of computing resources as commodities; in the same way we may observe with resources such as oil, gold or rice.
To understand the benefit of a global market for computing power, let us draw comparisons with the oil market. When you are stopped at a gas station, filling your car with oil, you have little to no idea where that oil comes from or how it arrived at that gas pump. There is an entire industry behind the scenes, that has standardized the whole process from petroleum extraction, to processing, to transport and delivery, and eventual utility being consumed by vehicles.
Now take the example of an application developer. A developer needs resources too, in the form of computing power from cloud vendors to ‘fuel’ his applications. However, in contrast, they do not have the luxury of benefiting from an organized and global market with abundant choice of vendors and competitive prices.
Let’s imagine you are a driver in the same situation, with no choice but to fuel a car in the same way developers fuel their apps. Being a driver in this case, you would have to call a specific Iranian or Venezuelan extractor to organize oil transported directly to your car. What’s worse is that, because each oil company produces its specific oil without standardization, the driver would probably have to process or ‘transform’ the oil so that it is compatible with his car. Today, this is the situation developers find themselves in, in the current age of cloud computing.
Therefore, the entire infrastructure must be changed. Our vision with iExec V2 is to create the first global market for computing resources. So, allow me to give you a primer on what we’ll deliver on May 29, 2018, at Blockshow Berlin, Germany.
Worker pools
The main evolution of V2 is the ability for computing resources to join the iExec network, making it the first decentralized cloud that is able to execute any type of applications using (almost) any type of resource. Up to now, off-chain computations with iExec were executed on our own machines, and dapp developers had to trust iExec for their executions.
Soon, any machine will be able to become a “worker”, i.e. getting paid in RLC for executing application tasks.
How will it work?
Workers will be organized in worker pools. Each worker pool is managed by a scheduler, whose responsibility is to distribute tasks to workers. Thus, a worker pool is somewhat similar to what we know as ‘mining’ pools. An individual miner often joins a mining pool to maximize their chances of getting reward computations. Similarly, as a worker, you would join a public worker pool that will make sure that it has sufficient workload to distribute.
An interesting feature is that several public worker pools will compete, therefore allowing for the best possible quality of service. As a worker, if you switch to a different worker pool, you will still be able maintain your reputation, bringing it ‘with you’ to the new pool, as this is all recorded on the blockchain!
In the marketplace, you will also find what we call “private” workers pools, where all machines are provided by a single cloud provider running his own scheduler. We already have sealed agreement deals with several startups in the area of ‘Green IT’ that offer more sustainable approaches to data centers. We are currently in discussion with more traditional cloud providers — but you will have to wait for May to get the final list ;-)
What does it change for developers ? A piece of ‘work’ is now defined as triplet: an application, a dataset, and a worker pool. It means that every dapp can now have access to unlimited off-chain computing resources just by specifying this triplet. Developers simply have to deploy their legacy applications (as Docker containers) and datasets, and to connect to an existing worker pool. Et voilà !
The Marketplace
In Version 2, we’re introducing a new element that will become central to the iExec cloud: the marketplace. Here, you will be able to view all the different worker pools, the prices for using resources, the available resources and so on.
Based on these different offers, users and developers will select the corresponding pool to execute their task. The marketplace is implemented as a smart contract and is already part of our ‘PoCo’ protocol. Version 2 will also provide an easy-to-use interface so that users and developers can see how the market and ecosystem is dynamically evolving.
Pay-per-Task
To make this market possible, we have to change the way pricing is done in traditional cloud computing. With AWS for instance, you use a particular instance, which have known characteristics for some time. Hence, the common way of pricing the usage is the renting of an instance per hour, depending on the zone. Such method cannot be used to build a global marketplace, because the resources will come from many providers (including Internet users) and thus will be very heterogeneous.
To address this issue we are introducing a new method for pricing that we call Pay-per-Task, and we have defined several task categories that describe the execution boundaries. We’ll start with a very simple definition for task categories, namely wall clock time on a reference machine and amount of data transferred. We’ll setup a test infrastructure so that application developers can evaluate the category of their submissions. Conversely, worker pools will be able to benchmark their infrastructures against the reference machine. Later, we’ll refine the categories, and provide more advanced tools for helping developers to maximize the usage of the infrastructure.
Proof-of-Contribution
PoCo is our consensus protocol for ‘off-chain’ computing. This protocol verifies that computations that happened off-chain have been executed according to the customers requirements. Moreover, PoCo stores all the results of the consensus back on the blockchain so that every step in the off-chain execution is open and auditable by everyone.
I will not elaborate too much on it as Hadrien, our scientific consultant, has already published detailed articles on this topic. In a nutshell, the protocol has the following features: staking to implement economic incentives, result verification through majority voting and reputation based on previous consensus results.
What You Will Not Find in V2.0 (But Later in V2.x)
As with all major releases, we’ll first deploy V2 on testnet and let developers as well as users test and evaluate it. This will work in the form of a ‘feature freeze’. Deployment on mainnet will happen soon after — probably in a matter of weeks. In May, we plan to have only selected private worker pools. Public worker pools, i.e. worker pools that connect machines from the Internet, will come later. The reason for this is that it requires specific developments, tools to administer and support public worker pools. Those will be developed in the coming months after the initial V2.0 release. Although, we don’t have yet a definitive roadmap, a credible goal for this would be September/October.
The first version of PoCo deployed on Ethereum will not have yet reached its full potential. Improvements are still to be made on performance (round-trip) as well as costs view. At the moment, it may be slower than a traditional cloud and costly due to the high transaction fees on Ethereum. We’re evaluating several sidechain technologies (such as 0x or Parity bridges) to decrease the cost of running on mainnet. The eventual goal is to run PoCo on a dedicated sidechain, using the mainchain only for payments and storage of PoCo consensus results.
Conclusion
We hope you are as excited as we are for this significant milestone in the iExec journey. This is certainly our biggest release yet, one step closer to realising our vision of the future internet infrastructure, and the very first decentralized marketplace for free trading of computing resources. Until then, we will continue to work non-stop until the very end to bring the best possible product on May 29th.
Just in case you missed it, hereby our two latest highlights:
iExec on social networks
- Website: http://iex.ec
- Blog: https://medium.com/iex-ec
- Slack: http://slack.iex.ec
- Telegram: https://goo.gl/fH3EHT
- Reddit: https://www.reddit.com/r/iexec/
- Twitter: https://twitter.com/iEx_ec
- Facebook: https://www.facebook.com/iexecteam/
- LinkedIn: https://www.linkedin.com/company/iex.ec/
- Youtube: https://www.youtube.com/c/iExec
