Member-only story
Privacy Preserving Machine Learning (PPML).
The Balance of Interests !!
PPML represents a line of research focusing on preserving privacy throughout the machine learning process by addressing various threats to privacy.
One area in PPML is DP (Differential Privacy), which provides formal guarantees for the privacy of training data.
Yue demonstrated that models can be trained with DP while maintaining competitive performance in synthetic text generation tasks.
As the synthetic data generated can be further used for training models without significant compromise on performance for certain tasks.
These downstream models are mostly small models for specific tasks.
The synthetic data used for training LLMs only makes up a small proportion of the all data fed into LLMs and is used for very specific purposes such as instruction tuning, which do not safeguard privacy effectively.
To be able to solve the root of these issues by fixing the original model we have to use the idea of machine unlearning.