iEx.ec Architecture
In preparation for the EDCON conference, we present a series of articles to introduce the technology behind the iEx.ec Blockchain-based Distributed Cloud. The previous article gave an overview of iEx.ec. This article introduces the technology we use: XtremWeb-HEP, a middleware that allows us to use many machines on the Internet to execute distributed applications.
Introduction
XtremWeb-HEP (XWHEP), developed by CNRS-IN2P3, is based on XtremWeb by INRIA. It is a middleware permitting to deploy a distributed data processing infrastructure (computing grid). XWHEP belongs to the so called “Cycle Stealing” family that uses idle resources. Like some other grid middleware stacks, XWHEP uses remote resources (PCs, workstations, PDA, servers) connected to Internet, or a pool of resources inside a LAN. Thus this middleware is the cornerstone technology of the iEx.ec distributed Cloud because it allows the participant to share their computing resources, such as processor, application and/or data.
XWHEP is written in Java language permitting to run on different architectures. It’s a Free Software under Gnu Public License that easily enables applications of Global Computing and Peer to Peer distributed systems. XWHEP allows to setup and run Distributed Systems. Such project infrastructure may be based on a community of participants. For example, XWHEP allows high schools, universities and companies to setup and run a Global Computing or Peer to Peer distributed system for either a specific application or a range of applications.
XWHEP has been developed and used in the context of international IT projects funded by the European Union: EDGeS, EDGI, DEGISCO, and is part of the International Desktop Grid Federation.
Features
XWHEP features include:
- P2P communications
- Fault tolerance
- security: authentication (X509, OpenId, OAuth, login/password); authorization; confinement; blacklisting
- user management
- access rights
- data management
- data driven scheduling
- application management
- bag of tasks
- replication
- volunteer sharing
- deployment policy
- bridges
- virtual machine management
Architecture
Server — as shown in figure “iEx.ec : a secured 3 tiers architecture” :
- one ormore Services are maintained by system administrators to host services such as the ‘Scheduler’, the ‘Result Collector’ and the ‘Bridge’ between the iEx.ec infrastructure and the Ethereum Blockchain.
- Clients are installed by iEx.ec users on their PCs to interact with the infrastructure. The client permits users to manage the Services and use the distributed resources (register applications, submit tasks, retrieve results…).
- Workers are deployed on computing resources. Thanks to the worker, a machine deployed on the Internet will be able to join the iEx.ec network and execute client’s tasks.
Workflow
The workflow described below does not intentionally take security into consideration to ease the understanding. Security will be the subject of an upcoming article.
In short, iEx.ec works as follow:
- The XWHEP services permit to manage registered applications, data, users etc.
- Client registers applications, data and any item that are needed to successfully compute jobs. These item may be stored in the iEx.ec infrastructure itself or in any repository as soon as they can be described by an URI and are accessible through the network. There is no limit on item size, but exceeding a few hundred of mega-bytes may be very long to transfer.
- A Client prepares jobs (units of work) containing the reference of a registered application, optional parameters, optional references to additional files as well as any item.
- Finally, the Client submits the prepared jobs to the scheduler.
- Independently, workers contact the scheduler to get jobs suitable for their architecture.
- In response, the scheduler sends a suitable job description to a worker.
- For each item referenced by the job which is not present in the local cache of the worker, the worker fetches it from iEx.ec Data Repository or an external data repository.
- As soon as a job has finished on worker side, the worker contacts the Result Collector to send the results.
- The result can either be retrieved by the Client, or fetch back to the smart contract.
- The client may want to develop a front-end for his Dapp application using a framework such as Embark or Truffle. Still, his application will be able to communicate with the iEx.ec network through the iEx.ec smart contracts.
Dashboard
The iEx.ec comes with a public dashboard aiming to present a view of the platform. Please note that this public dashboard does not propose any interaction.
It is available at http://xw.iex.ec/dashboard.
Next figures show some examples:
Conclusion
This article introduced the iEx.ec architecture. The upcoming article will detail security. It will present the communication secured layer; it will also introduce authentication, authorization, access rights, confinement and more.
Media
You are kindly invited to:
- join our Slack: https://iexec-team.slack.com
- follow us on Twitter: https://twitter.com/iEx_ec
- interact with us on Reddit : https://www.reddit.com/r/iexec/
Resources
- iEx.ec Github
- XtremWeb-HEP Github
- XWHEP documentation
- Video introducing XWHEP
I presented at CrowComputing 2014, in Almere, NL.
https://vimeo.com/113122296
Next Article
Next article describes the security: available here.
