Wide-Ranging Review Manipulation...
Transcript of Wide-Ranging Review Manipulation...
![Page 1: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/1.jpg)
Department of Computer Science and EngineeringTexas A&M University
CIKM 2019
Parisa Kaghazgaran, Majid Alfifi, James Caverlee
Wide-Ranging Review Manipulation Attacks:
Model, Empirical Study and Countermeasure
![Page 2: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/2.jpg)
User Reviews are Everywhere
Online Retailers
Business Review Forums
… and so are targets of manipulation.
Media Platforms
!2
![Page 3: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/3.jpg)
Amazon Headphones, 2019
!3
![Page 4: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/4.jpg)
Amazon Headphones, 2019
!4
![Page 5: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/5.jpg)
Crowd-based Manipulation Campaigns
Crowdsourcing websites
✓Read the product description before writing down a review.
✓Go to https://goo.gl/7QfW0h✓Leave a relevant 5-star review with at
least 40 words.✓Provide the proof that you left the
review your self.
Target Review Platforms
!5
![Page 6: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/6.jpg)
Realistic Reviews: Written by Humans
• Reviews arrive synchronized in time.e.g., Mukherjee et al. 2013, Akoglu et al. 2015, Kaghazgaran et al. 2019
• Dense community over co-review graph.e.g., Kijung et al. Shin 2017, Kaghazgaran et al. 2018
Typical detection approaches:
Manipulators can launch scalable and difficult to detect attacks by removing manipulation traces.
!6
![Page 7: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/7.jpg)
Crowd vs. Machine
!7
Review PlatformsCrowdsourcing
websites
Crowdworkers
Attack Strengths• Scalability: do not rely on paying workers.• Increased deception: obfuscate signals left by crowd campaigns.
![Page 8: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/8.jpg)
!8
How machine produces human readable reviews?
Seminal work Yao, Yuanshun, et al. "Automated crowdturfing attacks and defenses in online review systems." CCS, ACM, 2017.
• Domain dependent reviews.e.i., training process needs to be replicated for any domain, needs large training data for each domain
• Character-level language model.e.i., capture longer dependency, learn spelling in addition to semantics, grammatically more error-prone
Leveraging neural language models for the specific domain of restaurant reviews at Yelp
![Page 9: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/9.jpg)
Our Proposed DIOR Framework
1.WeproposeDIORforDomainIndependentOnlineReviewGenera9on.
2.Empiricalstudytoevaluatethequalityofmachine-generatedreviews.
3.Embedding-basedclassifiertodetectsuchfakereviews.
!9
Goal:validatethewide-ranginga2acksonreviewpla7ormsandproposetodetectthem.
![Page 10: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/10.jpg)
!10
Neural Language Models (A quick refresher)
• Recurrent Neural Networks have shown success in generating meaningful text.
• They learn from a sequence of words to predict the next word
Ht
ot
xt
Each word in the review as input
Learn the information from sequence until time-step t
Predict the next word in the review
x( < = t)(P(xt+1 |x1, …, xt)) H0
o0
x0
H1
o1
x1
H2
o2
x2
H3
o3
x3
I
ate
ate
at this restaurant
At thisIn generation step, the predicted word at time-step t is feed back to the model as input along with hidden state to predict the next word
![Page 11: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/11.jpg)
!11
RQ1: Can we generate reviews across different domains?
I ’ve eaten here about 8 times . I ’ve been introduced to this place . Its always busy and their food is consistently great . I LOVE their food , hence the name . It is so clean , the staff is so friendly , and the food is great . I especially like the chicken pad thai , volcano roll , and the yellow curry .
the case works great ! it has a soft rubber insert that goes over the hard shell . The hard plastic shell has a soft inner shell and the hard case is hard plastic . It is very sticky and has not fallen out or dropped or fallen apart .
this app is a great tool for discovering new things : being able to search for films and putting reviews on particular items as well as having a way to download stories from the app .
![Page 12: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/12.jpg)
!12
Yelp
LSTM3
wt
Encoder
word to id
Input word
id
LSTM2
Decoder
id to word
wt+1
Next word
LSTM1id
Embedding of input word (400)
ht1(1150)
ht2(1150)
ht3(400)Embedding of output word
Transfer Learning to the Rescue!
Amazon
Universal Model
Transferred Model
App Store
Universal model parameters —>
Transferred model parameters —>
θYelp
θAmazon
θAppStore
![Page 13: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/13.jpg)
!13
Example of Synthetic Reviews
I ’ve eaten here about 8 times . I ’ve been introduced to this place . Its always busy and their food is consistently great . I LOVE their food , hence the name . It is so clean , the staff is so friendly , and the food is great . I especially like the chicken pad thai , volcano roll , and the yellow curry .
this is a nice case . It ’s a little difficult to remove , but that ’s to be expected . The case is slightly thicker than a regular screen protector , but that is to be expected . It ’s a great phone case and I highly recommend it .
this app is great for learning the basics of math ! I love that it has a different function that can help you learn the words that you understand . I wish all apps were this simple .
![Page 14: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/14.jpg)
Temperature
Labe
led
“Rea
l” (%
)
020406080
0.2 0.4 0.6 0.8 1.0
Labe
led
“Rea
l” (%
)
022.5
4567.5
90
0.2 0.4 0.6 0.8 1.0
Labe
led
“Rea
l” (%
)
020406080
0.2 0.4 0.6 0.8 1.0
Yelp
Amazon
App Store
!14
RQ2: Can Model-generated Reviews Pass Human Test?
Takeaway 1: Reviews generated at temperature 0.8 can fool human readers and go undetected.Takeaway 2: Human readers are more sensitive to repetition errors than they are to small grammar mistakes.
AMT Guidelines
• 95% approval rate
• Dwell time >= 7 minutes
• Located in US
• Ask for a trivial question
![Page 15: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/15.jpg)
!15
RQ3: Can Spam Detector Catch Model-generated Reviews?
Takeaway: The textual-based spam detector does not distinguish synthetic reviews from real reviews.
Yelp Amazon App Store
Accuracy (%) 64 61 62
Precision (%) 65 64 62
Recall (%) 65 61 62
F1 score (%) 65 60 62
Textual Features
• Similarity
• Structural
• Syntactic
• Semantic
![Page 16: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/16.jpg)
!16
RQ4: How DIOR Works versus Crowd Manipulators?
Takeaway: Users find reviews generated by DIOR as reliable as fake reviews written by manipulation campaigns.
DIOR31%
Neither37%
Crowd32%
![Page 17: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/17.jpg)
!17
RQ5: How DIOR works versus Individual Models?
Takeaway: Using transfer learning not only facilitate the domain shift but also improves the performance significantly.
Pref
eren
ce (%
)
0
25
50
75
100
0
25
50
75
100Amazon App Store
TransferredModel Transferred
Model
IndividualModel
IndividualModelBoth Both
![Page 18: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/18.jpg)
!18
RQ6: How Much Training Data for Transferred Model?
Takeaway: The transferred models need reasonably low number of samples compared to universal model to reach stable performance.
Valid
atio
n Lo
ss
3
3.2
3.4
3.6
25k 50k 75k 100k 125k 150k 200k
App Store Amazon
Training Size
![Page 19: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/19.jpg)
!19
RQ7: How We Can Detect Model-generated Reviews?
Takeaway: Model-generated reviews are detectable in the embedding space with high accuracy.
• Embedding based Classifier
![Page 20: Wide-Ranging Review Manipulation Attackspeople.tamu.edu/~kaghazgaran/papers/CIKM_Slides_Parisa.pdf · 1. We propose DIOR for Domain Independent Online Review Generaon. 2. Empirical](https://reader034.fdocuments.in/reader034/viewer/2022042413/5f2d3f3a2fb46428e71ed98b/html5/thumbnails/20.jpg)
Conclusion and Next Steps
!20
Explored how transferred learning technology could lead to a wide-ranging review manipulation attacks.
Proposed DIOR framework demonstrates:
(1) Model-generated reviews can be perceived as real by human examiners, pass the traditional textual-based spam detectors, and beat the crowd-based review manipulators.
(2) Fake reviews tend to cluster together in the embedding space that provide the intuition for our proposed discriminator.
Next steps: study the performance of other neural network architectures to develop more powerful discriminator.