New Alternative for Machine learning?
In this story I observe Federated learning techinique, all of it’s pros & cons and will performe practical analysis.
Standard approaches to machine learning require the centralization of training data on a single machine or in a data center. This particular concept I analyzed in one of the previous stories. And many global concerns, like Amazon and Google, have developed many cloud infrastructures for processing this data in order to improve existing services. Now there are more and more questions about reliability and security for models trained on the basis of user interaction with mobile devices. It is to solve these problems that a new approach is coming on the scene: Federated learning.
What is Federated learning?
Federated learning allows mobile phones to learn a common prediction model together while keeping all the training data on the device, separating the possibility of machine learning from the need to store data in the cloud. That is, your data does not go to a third-party server (they preserve security and privacy), but at the same time they contribute to the training of the general model (they adjust the parameters). This goes beyond using local models that make predictions on mobile devices, for example, Google’s Mobile Vision API, since model training is also transferred to the device.
The workflow is so: your device downloads the current model, improves it by studying the data on your phone, and then returns the updated model parameters to the server. Only this data becomes available, where it is immediately averaged with other custom metaparameters to improve the overall model. All training data remains on your device, and no individual updates are stored in the cloud, at the same time, the model you use trains on the widest massive.
Why is Federated Learning a better solution?
Strong sites
Federated learning provides smarter models, less latency, and less power consumption while ensuring privacy. And this approach has another immediate advantage: in addition to updating the overall model, the improved model on your phone can also be used immediately, which greatly improves the user experience, as it reduces time. A good example of federated learning is the Google Board keyboard or the use of an approach in the medical sector.
Silly sites
However, Federal Education also extols new problems. In a typical machine learning system, an optimization algorithm such as stochastic gradient descent (SGD) is performed on a large dataset evenly distributed across servers in the cloud. Such highly iterative algorithms require connection to training data with low latency and high throughput. But in conditions of federated learning, data is distributed among millions of devices extremely unevenly. In addition, these devices have significantly higher latency, lower connection throughput, and are only available for training periodically.
Various solutions to this problem have already been presented on the market, for example, using a special approach of federated averaging, the key idea of which is to use powerful processors in modern mobile devices to calculate updates of higher quality than simple gradient steps. Other solutions are based on reducing the iterations of data transmission over the network, since this is the slowest and most vulnerable process. In general, solutions exist and it is only a matter of the diligence of the minds of mankind and the time spent on the thought process.
Summary
Federated learning also cannot solve all the problems of machine learning, for example, learning to recognize finely tuned classes of images based on their common data. And also for many problems, the necessary training data is already stored in the cloud (for example, spam training filters for Gmail).
The use of federated learning requires machine learning specialists to introduce new tools and a new way of thinking: model development, training and evaluation without direct access to raw data or labeling, and the cost of communication is a limiting factor.
Be human, do science 🕊
