NAACL’19: Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence
Natural Language Processing paper challenge (2/30)
What problem do they solve?
Given a sentence, we have to classify the sentiment.
TABSA: Given a sentence, we have multiple target and for each target we want to know the aspects and its polarity (sentiment).
For example,
s: LOCATION1 is often considered the coolest area of London.
It has 1 target: LOCATION1. And for this target, we have 4 aspects: (general, price, safety, transit-location) and its polarities (positive, None, None, None)
ABSA: The target-aspect pairs {t, a} become only aspects a.
For example,
s: LOCATION1 is often considered the coolest area of London.
This sentence has 4 aspects: (general, price, safety, transit-location) and its polarities (positive, None, None, None).
What model do they propose?
Construction of the auxiliary sentence:
Sentences for QA-M:
input: target-aspect pair (LOCATION1, safety)
output: “what do you think of the safety of location — 1?”
Sentences for NLI-M:
input: target-aspect pair (LOCATION1, safety)
output:“location — 1 — safety”
Sentences for QA-B:
input: target-aspect pair (LOCATION1, safety, positive)
output: “the polarity of the aspect safety of location — 1 is positive”
Sentences for NLI-B:
For QA-B, we add the label information and temporarily convert TABSA into a binary classification problem.
input: target-aspect pair (LOCATION1, safety, positive)
output: “location — 1 — safety — positive”
They convert the original ABSA problem to question answering (QA) problem and natural language inference (NLI) problem.
BERT-single-TABSA: We consider TABSA as a combination of target-aspect-related sentiment classification problems.
BERT-pair-QA-M: They applying the sentence-pair classification approach to solve TABSA with QA-M.
BERT-pair-NLI-M: They applying the sentence-pair classification approach to solve TABSA with NLI-M.
BERT-pair-QA-B: They applying the sentence-pair classification approach to solve TABSA with QA-B.
BERT-pair-NLI-B: They applying the sentence-pair classification approach to solve TABSA with NLI-B.
Data
SentiHood dataset:
it consists of 5,215 sentences:
3,862 of which contain a single target
1,353 of which contain multiple targets
For each sentence, it contains a list of target-aspect-sentiment (t-a-y) pairs <target, aspect, sentiment)
SemEval-2014:
The only difference from the SentiHood is that the target-aspect pairs {t, a} become only aspects a.
Hyperparameters
The number of Transformer blocks: 12
The hidden layer size:768
The number of self-attention heads: 12
The total number of parameters for the pretrained model is 110M
Baseline model
LR: a logistic regression classifier with n-gram and pos-tag features.
LSTM-Final: a biLSTM model with the final state as a representation
LSTM-Loc: a biLSTM model with the state associated with the target position as a representation
LSTM+TA+SA: a biLSTM model which introduces complex target-level and sentence-level attention mechanisms
SenticLSTM: an upgraded version of the LSTM+TA+SA model which introduces external information from SenticNet
Dmu-Entnet: a bidirectional EntNet with external “memory chains” with a delayed memory update mechanism to track entities
Result
Other related blogs:
Beyond Clicks: Dwell Time for Personalization
RecSys’15: Context-Aware Event Recommendation in Event-based Social Networks
RecSys’11: Utilizing related products for post-purchase recommendation in e-commerce
RecSys11: OrdRec: an ordinal model for predicting personalized item rating distributions
RecSys16: Adaptive, Personalized Diversity for Visual Discovery
RecSys ’16: Local Item-Item Models for Top-N Recommendation
COLING’14: Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts
Best paper in RecSys:
https://recsys.acm.org/best-papers/
My Website: