Scaling Across the Edge Network — Challenges for wide adoption
To recap our previous post, Edge/Fog computing can bring various benefits to IoT systems including lower transmission latency, better compliance, higher data privacy and security, conserved network bandwidth and reduced operational cost (communication, processing, and storage). However, scaling issues can be seen as a bottleneck for the large scale adoption of Edge/Fog computing solutions.
Vertical scaling approaches are cloud-dependent architectures that try to address challenges such as limited capability (i.e. computational and storage) of a single Edge device. Horizontal scaling approaches, on the other hand, rely on scaling across the edge network and better utilization of the available Edge/Fog resources. To elaborate further, in horizontal scaling, resources across devices and servers can be better utilized by pooling them when available, to create computing systems of different capabilities. Opportunistic Edge Computing (OEC) is an example of such a framework in which the devices are owned distributedly (technically by anyone) and leased to the network through short-term contracts.
How about governance?
The more open and public your edge network is, the harder its governance would be. The governance of a horizontal scaling Edge network can be centralized or decentralized. In the centralized architecture, similar to traditional cloud infrastructure, a central authority is responsible for collecting information about edge devices (e.g. computing/storage capacity, location, etc.), for building an opportunistic pool and orchestrating the computation task. In a decentralized system, on the other hand, edge devices do these tasks by cooperating among themselves to orchestrate the system without the involvement of a centralized authority. Central governance can ensure the QoS (Quality of Service) but it can also create a monopoly, add extra cost, remove transparency and cause security and privacy issues.
Maybe too good to be true?
Horizontal scaling for Edge Networks sounds quite tempting. Who doesn't want a cloud-independent, self-healing, ad-hoc fleet of edge devices with autonomous and decentralized governance?
However, as good as it sounds, there are still ongoing challenges that need to be addressed before horizontal scaling solutions are ready for large scale adoption. Here, I have tried to provide a short summary of these challenges for OEC, which can be seen as a complicated scenario due to the openness and public nature of the network:
a) Service orchestration architectures designing efficient schemes for:
- Selection or election of the device(s) that should host the management and orchestration of the computation job in hand.
- Resource allocation scheme: discovery and pooling schemes based on the requirements of the job at hand (e.g. latency, the computing power required, reliability, available connectivity, bandwidth performance, etc.) and giving the computation demander complete flexibility over the execution strategy (e.g. when, where, how).
b) Platform programmability in application level (as opposed to hardware- and network-level): this feature is required to be able to perform activities such as task distribution (e.g. MapReduce technique) and collecting/consolidating the results and delivery to the end-user with optional use of security methods (e.g. encryption) and to employ existing solutions (e.g. Spark).
c) Fault Tolerance and Reliability (checking the reliability of the results):
- A mechanism to adapt to the highly dynamic physical topology of OEC (e.g. resource migration strategies in case of service denial or unavailability of a computation resource).
d) Pricing and Incentives:
- Optimum pricing models with high flexibility for the computation power demander to choose the right service based on the requirements of the job at hand.
- A proper incentivization scheme for participants to encourage them to provide their resources to the network.
e) Privacy and Security: adequate measures for such an interactive system involving many participants and stakeholders to ensure their privacy and security (tasks and slave edge devices have different trust levels, based on this a different security measure might be needed).
f) Trust Management: a proper mechanism for a decentrally governed OEC to perform in a given trustless environment — centralized approach is out of the question due to the lack of existing trust in the operator (i.e. a third party) to run and govern the network.
Addressing all the above-mentioned challenges can take such a framework a step closer to user acceptance and mass adoption (end-user to contribute their resources as well as use the network/other’s resources) and can make this solution technically and economically feasible.
DISCLAIMER: This post just reflects the author’s personal opinion, not any other organization. This is not official advice. The author is not responsible for any decisions that readers choose to make.
