Federated Learning : The future of Artificial Intelligence
One of the most recent developments in the fields of Deep Learning and Artificial Intelligence is the concept of “Federated Learning”.
On Thursday, April 6 2017, Google AI announced this concept of Federated Learning in their Blog, which literally blew away Machine Learning enthusiasts, Data Scientists, and researches and made them believe that such a decentralized AI model is indeed possible.
As mentioned above, the concept of Federated Learning was brought about by Google right in this blog.
What is Federated Learning ?
Many of us are still wondering what is Federated Learning. Well, Federated Learning is a concept which involves the intersection of on-device AI, block chain, and edge computing/IoT. Unlike other traditional Machine Learning Algorithms Federated Learning believes in training samples of various servers.
This means its a technique which trains the algorithm over multiple decentralized devices or servers which have their own data samples (local data samples) without exchanging their data. After the training procedure from the local data is complete, the modified models are uploaded from these local servers to one centralized global server, where changes are applied to the main model according to the information received from the local servers.
So how is it different from Traditional Machine Learning? Unlike Machine Learning techniques where training takes place in a single server, Federated Learning lies in clear contrast to this concept as it distributes the data samples identically over various local servers and collects data from them once the training is complete.
The above diagram explains the concept of Federated Learning. Here we have two components : The Global Server ( referred as Server in the above diagram) , and the Local Servers ( referred as Nodes ).
The first step involves training the algorithm over various data samples to generate a model in the Global Server. This model is then distributed among various Local Servers.
The Local Servers train this model , based on local data samples that is present on the device and update the model accordingly. Then, the Global Server calculates the average of these updated local server models and trains the Global model according to the information received.
This process is repeated again and again until the best model with minimum errors and high accuracy is generated and serves the purpose for which it was built efficiently.
However in a few cases it is also seen that this entire process is orchestrator -less. Which means there is no Central Server or Global Server. Each local node sends its outputs to a several randomly-selected others, which aggregate their results locally. This reduces the number of transactions and thus greatly reducing time and computing cost.
Types of Training Algorithms in Federated Learning
The training procedures that takes place over the local global servers follows mainly two algorithms as of now:
Federated Stochastic Gradient Descent (FedSGD) and Federative averaging(FedAvg).
Federated Stochastic Gradient Descent(FedSGD)
The concept of FEDSGD is based on Stochastic Gradient Descent where the global optimum is approximated using only one data saving a lot of time and computation power.
FEDSGD is a direct transposition to this algorithm using a random fraction C and all the data present in a single node. The parameter C is optimized to select that node and then the gradients are averaged by the server proportionally to the number of training samples on each node, and used to make a gradient descent step.
Federative averaging(FedAvg)
Unlike FEDSGD here more than one batch update is allowed and thus the weights are updated rather than the gradients. The averaged tuned weights are used to update the model in the Global Server.
Applications of Federated Learning
Federated Learning is indeed a brand new concept and the works have only scratched the surface of what is possible.
However Google Keyboard is the most renowned application of Federated Learning.
Earlier it was impossible to upload every user’s text message to train the predictive algorithm for word guessing. Besides, such a process would hijack too much of the user’s data. So Google Keyboard came up with this new concept of Federated Learning which made the process much more efficient .
As of now, Federated Learning has limited applications, but in the near future it is said that as more and more probabilities and possibilities of this concept is discovered there is no stopping it. Now the only thing we can do is analyse the concept , wait and discover. As a wise man once said : “Learning Never Stops”
