Many privacy attacks rely on the fact that many models overfit to the training data, and leverage this property to extract personal information. It is therefore important to ensure the models do not overfit. Besides many of the techniques for reducing features, you can refer to the roadmap for improving generalization.
Another technique for privacy protection is to perturb the values of the data points.
Noise injection. One simple way of achieving this is by injecting noise into the data. Notice that this needs to be done in a way that preserves some of the statistical properties of the data. This should ensure that, while the predictive accuracy on the individual data is diminished, the performance on the dataset as a whole should be maintained. If X is the original data point, ϵ is the noise and Z is the transformed data point, we can add noise using these modalities:
Additive noise: Z=X+ϵ
Multiplicative noise: Z=Xϵ
Logarithmic multiplicative noise: Z=ln(X)+ϵ
Differential Privacy. A technique that perturbs data in such a way that it makes it impossible to tell whether any individual's data was part of the original dataset by just looking at the output. These techniques are founded on rigorous mathematical definitions of privacy (Dwork 2006).
Anonymization or pseudo-anonymization. Some of the techniques listed, like dimensionality reduction or noise perturbation, may produce anonymization or pseudo-anonymization results as a side effect. Please note that these are privacy-preserving techniques in itself (Iyengar 2002 , Neubauer and Heurix 2011) which can be very useful to mitigate risks concerning data privacy
You can find more information about data perturbation in Mivule 2013, and about data obfuscation in Zhang et al. 2018.
Another technique for privacy protection consists of changing where the data is held and accessed.
Federated learning. This type of machine learning does not rely on a centralized approach, but rather allows training to happen across multiple decentralized devices relying only on their own local data. For example, the autocorrect feature of a smartphone can be trained on an individual device using solely the text messages sent by that specific user. This would not only allow for a more personalized outcome, but would also preserve the privacy of the user as those messages would not need to be shared with a central system.
Making inferences locally. Another way to mitigate privacy risks, would be to intervene at inference time. If we could host the machine learning model on an individual device, then inference could be triggered locally. This would avoid any superfluous data sharing.
