How to Preserve Privacy of Text Representations in NLP

Problem overview

Recently, we have been experiencing numerous breakthroughs in Natural Language Processing (NLP) owing to the evolution of Deep Learning (DL). The successes emerged from word2vec, or distributed word representation, which is capable of projecting discrete words into vector space. Such mappings have revolutionized the understanding and manipulation of syntactic and semantic relations among words. One famous example is we can play the following equations of word embeddings in word2vec:

Powered by this technique, a myriad of NLP tasks have achieved human parity and are widely deployed on commercial systems [2,3].

The core of the accomplishments is representation learning, which is able to extract the necessary information, such as semantics, sentiment, intent, etc, required by task. However, because of the over-parameterization, DL models also memorize certain unnecessary but sensitive attributes, such as gender, age, location, etc.

The private information can be explored by malicious parties in different settings. Firstly, cloud AI services have been widespread. Users can easily annotate their unlabelled datasets via cloud AI platforms such as Microsoft Cognitive Services, Google Cloud API, etc. However, if eavesdroppers intercept the immediate representation of users’ inputs from cloud AI services, they can perform some reverse engineering to obtain the original text. Considering privacy concerns, users are unwilling to upload their data to servers. Instead, they can transmit their extracted representations to servers. Nevertheless, the input representation after the embedding layer or the intermediate hidden representation may still carry sensitive information that can be exploited for adversarial usages. It has been justified that an attacker can recover private variables with higher than chance accuracy, using only the hidden representation [4,5]. Such an attack would occur in scenarios where end-users send their learned representations to the cloud for grammar correction, translation, or text analysis tasks, as shown in Fig.1.

Fig.1 Inference attack on representation.

How to preserve representation privacy in NLP

More recently, Li et al. [4] and Coavoux et al. [5] proposed to train deep models with adversarial learning. However, both works provide only empirical privacy, without any formal privacy guarantees. To address this issue and protect privacy against an untrusted server and the eavesdropper, we are inspired to take a different approach by utilizing Local Differential Privacy (LDP) defined as follows.

Compared to the centralized DP (CDP) adopted by Google [1], LDP offers a stronger level of protection. As illustrated in Fig.2, in DL with CDP, the trusted server owns the data of all users [1], and the server implements CDP algorithm before answering queries from end-users. This approach can pose a privacy threat to data owners when the server is untrusted. By contrast, in DL with LDP, data owners are willing to contribute their data for social good but do not fully trust the server, so it necessitates data perturbation before releasing it to the server for further learning.

Fig. 2: Deep Learning with CDP and LDP.

Basically, in order to achieve privacy protection, LDP employs a protocol named Unary Encoding (UE), which is comprised of two steps:

1. Encoding: one can encode an input into a d-bit vector, where only one digit is 1, and the rest elements are all 0’s.
2. Perturbing: the one-valued digit can be flipped with a probability of (1-p), while zeros are preserved with a probability of (1-q).

Depending on the choice of p and q, UE based LDP protocols can be classified into [6] :

1. Symmetric UE (SUE): p and q must satisfy equality, p+q=1
2. Optimized UE (OUE): Setting p and q can be viewed as splitting ε into ε1+ε2 such that

However, both SUE and OUE are dependent on the domain size d, which may not scale well when d is large. To remove the dependence on d, we propose a new LDP protocol called Optimized Multiple Encoding (OME) [7]. The key idea is to map each real value vi of the embedding vector into a binary vector with a fixed size l. Then the privacy protection can be accomplished via the following perturbation:

where λ and ε are tunable hyperparameters.

Our proposed framework

As shown in Fig. 3, the general setting for our proposed deep learning with LDP consists of three main modules: (1) embedding module outputs a 1-D real representation with length r; (2) randomization module produces local differentially private representation; and (3) classifier module trains on the randomized binary representations to generate a differentially private classifier.

Fig. 3: General setting for deep learning with LDP.

Performance evaluation

To examine the performance of our proposed local differentially private NN (LDPNN), we first compare it with the non-private NN (NPNN), where the randomization module is removed. We evaluate these two models on three NLP tasks: 1) sentiment analysis (IMDb, Amazon, and Yelp dataset), 2) intent detection (Intent dataset) and 3) paraphrase identification (MRPC dataset). Table 1 shows that our LDPNN delivers comparable or even better results than the NPNN across various privacy budgets ϵ when the randomization factor λ ≥ 50. We hypothesize that LDP acts as a regularization technique to avoid overfitting.

Secondly, we also compare with the other two LDP protocols, i.e. SUE and OUE, on sentiment analysis tasks. Table 2 suggests that our OME significantly outperforms both SUE and OUE.

Conclusion

To conclude, we formulated a new deep learning framework, which allows data owners to send differentially private representations for further learning on the untrusted servers. A novel LDP protocol was proposed to adjust the randomization probabilities of the binary representation while maintaining both high privacy and accuracy under a wide range of privacy budgets. Experimental results on various NLP tasks confirm the effectiveness and superiority of our framework.

Where to find the paper and code?

Paper: http://arxiv.org/abs/2006.14170

Code: https://github.com/lingjuanlv/Differentially-Private-Text-Representations

Reference

[1] Martín Abadi, Andy Chu, Ian Goodfellow, HBrendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of CCS. ACM, 308–318.

[2] Peters, Matthew et al. “Deep Contextualized Word Representations.” Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (2018): n. pag. Crossref. Web.

[3] Devlin, Jacob et al. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (2019): n. pag. Crossref. Web.

[4] Yitong Li, Timothy Baldwin, and Trevor Cohn.2018. Towards robust and privacy-preserving text representations. In Proceedings of ACL. 25–30.

[5] Maximin Coavoux, Shashi Narayan, and Shay B Cohen.2018. Privacy-preserving neural representations of text. In Proceedings of EMNLP. 1–10.

[6] Tianhao Wang, Jeremiah Blocki, Ninghui Li, and Somesh Jha. 2017. Locally differentially private protocols for frequency estimation. In USENIX Security. 729–745.

[7] L. Lyu, Y. Li, X. He, and T. Xiao, “Towards differentially private text representations,” in SIGIR, 2020.