Retweet Prediction with Attention-based Deep Neural Network
22
Retweet Prediction with Attention- based Deep Neural Network #CIKM2016 Authors: Qi Zhang, Yeyun Gong, Jindou Wu, Haoran Huang, Xuanjing Huang Reading group: 25/10/2017 Presenter: Guangyuan Piao (Unit for Social Semantics) Mentor: Subhasis Thakur | Supervisor: John G. Breslin
-
Upload
guangyuan-piao -
Category
Data & Analytics
-
view
421 -
download
2
Transcript of Retweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural Network #CIKM2016
Authors: Qi Zhang, Yeyun Gong, Jindou Wu, Haoran Huang, Xuanjing Huang Reading group: 25/10/2017 Presenter: Guangyuan Piao (Unit for Social Semantics) Mentor: Subhasis Thakur | Supervisor: John G. Breslin
Agenda • Background & Related Work
• Proposed Approach
• Experimental Setup & Results
• Conclusions
• Summary
2
Background
3
• easy real-time information sharing
• 1 billion unique visits / month for Twitter
Background – Retweeting Behavior
4
Background – Retweeting Behavior
5
• key mechanism for spreading information
• can help information spreading prediction, popularity prediction etc.
(Some) Related Work
6
• Retweeting behavior
• study of a number of features for retweetability of tweets [Suh et al., SocialCom’10]
• feature-aware factorization model [Feng et al, WSDM’13] • considering information about user, tweet, and author
• who will retweet me? [Luo et al., SIGIR’13] • using learning-to-rank framework
• non-parametric statistical models [Zhang et al. AAAI’15] • combining structural, textual & temporal info.
(Some) Related Work
7
• Retweeting behavior
• study of a number of features for retweetability of tweets [Suh et al., SocialCom’10]
• feature-aware factorization model [Feng et al, WSDM’13] • considering information about user, tweet, and author
• who will retweet me? [Luo et al., SIGIR’13] • using learning-to-rank framework
• non-parametric statistical models [Zhang et al. AAAI’15] • combining structural, textual & temporal info.
feature engineering is required
(Some) Related Work
8
• Convolutional Neural Network (CNN)
• image recognition • video processing • natural language processing
• Attention-based Neural Network
• machine translation • speech recognition • visual object classification
Proposed Approach – Variants of CNN approach
9
words of a tweet
• Vu: user embedding vector • Vp: tweet embedding vector
Proposed Approach – Variants of CNN approach
10
• Vu: user embedding vector • Vp: tweet embedding vector • Va: author embedding vector
Proposed Approach
11
• Modeling User Interests based on Tweet History [t1, t2 … tm]
• clustering m tweets of each user into n groups using K-means
• using the central tweet of each group as an interest of user
• user interest profile [t1, t2 … tn]
• Modeling User Interests based on Tweet History [t1, t2 … tm]
• clustering m tweets of each user into n groups
• using the central tweet of each group as an interest of user
• user interest profile [t1, t2 … tn]
• apply CNN for each tweet to obtain tweet embeddings
Proposed Approach
12
Proposed Approach
13
• Attention
• Folding
the value in the i-th position of the embedding of the j-th attention interests
Proposed Approach
14
Proposed Approach
15
• Vu: user embedding vector
• Vi: user interest embedding vector
• S: similarity(user interest vector, tweet vector)
• Vp: tweet embedding vector
• Va: author embedding vector
Experiment Setup
16
• Twitter Dataset
• 75% (training, 10% for validation), 25% (test)
• Evaluation Metrics • precision • recall • F1-score
Experiment Setup
17
• Model Parameters
• dropout rate: 0.5 • window size: (1, 2) • feature maps num.: 100 • L2 constraint: 3 • mini-batch size: 40
• cluster number: 5 • word vector: word2vec trained based on Google News • user & author vector dimensions: 300 (the same as word embedding)
Experiment Setup
18
• Compared Methods
• Random: random decision
• Ave-SVM, Sum-SVM: average, sum of word vectors for tweet vectors
• ASC-HDP: non-parametric statistical models [Zhang et al. AAAI’15]
• CNN, U-CNN, UA-CNN
• SUA-ACNN: with attention
Experimental Results
19
Experimental Results
20
Conclusions
21
• Proposed a novel attention-based deep neural network
• that can perform better than state-of-the-art methods for retweet prediction
• user, author embeddings, the similarity score and the user’s attention interests can each significantly improve the performance
• the integration of these components provides the best performance
22
Guangyuan Piao e-mail: [email protected] twitter: https://twitter.com/parklize slideshare: http://www.slideshare.net/parklize
