Architecture of three Federated learning
Federated Machine Learning: Concept and Applications
As the concern of privacy raising, the application of AI on the business will face the obstacle. Federated learning is AI technology that focuses on privacy.
Privacy techniques for federated learning
- Secure Multi-party computation:
provide security proof in a well-defined simulation framework to guarantee zero-knowledge, each party knows nothing except its input and output.
- Differential privacy:
Maximize the accuracy and minimize the opportunity of recognizing the individual by adding noise or generalization methods.
To know more about differential privacy
- Homomorphic encryption :
a form of encryption that allows computation on ciphertexts, generating an encrypted result which, when decrypted, matches the result of the operations as if they had been performed on the plaintext.(from wiki)
You get the same answer by calculating the number directly or calculate by calculating encrypted and then decrypt it.
Categorization of Federated learning
- Horizontal Federated learning:
The most famous example is Gboard which is google's keyboard on Android. Each participant will get a new model and train on their own device. Sending gradient to server. Participants get a new model after the server aggregate the gradient.
The example from the paper is the bank in the same region and get familiar features, but little intersection set of clients. However, I am wording that is no difference with federated transfer learning. I believe Gboard is a better example.
- Vertical Federated learning:
First of all, use encryption-based user ID alignment to confirm the common user from company A, B. System does not expose users that do not overlap with each other.
Secondly, we will need a collaborator C to handle the homomorphic encryption. A and B will send encrypt and exchange the gradient and loss. A, B compute the gradient and add the additional mask(prevent the leakage from C), sending it to C. C will return decrypted gradients and loss to A, B.
I omit a few training steps. Here is the detail of the training.
Vertical federated learning is an exciting AI technology since banks and retail stores can cooperate. We can infer the user purchasing power by the bank saving and preference by trading records from retail. We can get better-personalized service without compromising privacy.
- Federated Transfer learning:
Similar business but have a little set of overlapping samples. It does not change the architecture of vertical federated learning, but different in detail which tries to find the common representation among the parties.
Combining the incentive mechanism may improve this architecture. Encourage more organizations to provide their data.
Want to know more about Federated learning?
(Summary)Federated Learning: Strategies for Improving Communication Efficiency
Federated Learning Aggregate Method (1)
(Summary)Biscotti: A Ledger for Private and Secure Peer-to-Peer Machine Learning
