Federated Learning
The future of technology will be the intersections of different fields. In the next world — and perhaps the current world — there is an intrinsic need for the collaboration of different fields and subfields together.
This collaboration sometimes happens between different fields; for instance, in Genetics and Computational Biology, we can see significant collaborations between “biologists”, “geneticists”, and “computer engineers” which have resulted in outstanding outputs in the human daily-life.
There is another type of collaboration which happens when 2 people of a field — but different subfield — make a decision to work together, which is the case study of this medium article which we are going to talk about.
Federate Learning, also called Distributed Learning, is a new — somehow new! — field that is in the intersection of “Learning Concepts” and “Embedded Systems” issues.
Definition
When we want to aggregate the learned information of different models, into a single model.
Why?
Because of privacy, security, and time we cannot have all our information in a single place, but we have information in a distributed setting. We have some nodes (lime different hospitals), and we are going to train different models for each node (hospital) with its own data.
Pros
We can train the models in parallel, so we are saving the time.
The privacy is obtained, and data leakage is at its lowest probability.
Cons
Data is non-IID which is a fundamental issue in centralized machine learning (normal machine learning!), and we have some problems with generalization.
The other issue is models' parameters’ synchronization. The hospitals may send their parameters and hyperparameters on different times, so inferring a single model or feedback at the same time needs some more efforts.
Data imbalance is another issue that is probable. Assume we have 3 hospitals that have x, 2x, and 10x data of patients respectively. How are we going to infer a single model out of the 3 tuples which are received? It seems somehow tricky.
Another thing is the explanation of the models — which is related to the previous reasons, too. We are gathering data from different hospitals which may located far-away from each other. Each location or country may have its own features or specific issues which is not related at any way to other locations’ issues. This problem is handled by different approaches. One of them is splitting the models into 2 parts: General part (Same part between all the models) — Specific part (individualized part of each model regarding to the region and other features related to its location and patients).
Future Work?
There is always a possible work for incremental issues. We can always do some steps to make something enhance from performance of x into performance of 1.01x, and there is sometimes a way to make outstanding changes to this fields.
The concerns are as follows:
- Time
- Energy (Consumed by nodes and the central node)
- Models Explainability and Generalization
- Non-IID handling
Thanks for reading this paper. If you have any feedback regarding to this paper, don’t hesitate to contact me at m.j.maheronnaghsh@gmail.com.