Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf ·...
Transcript of Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf ·...
![Page 1: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/1.jpg)
Sentiment of Emojis Petra Kralj Novak
4th Conference on CMC and Social Media Corpora for the Humanities
![Page 2: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/2.jpg)
- EMOJI FACTS - OPINION MINING - EMOJI SENTIMENT
Outline
2
![Page 3: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/3.jpg)
Emoticon • Facial expression composed of
keyboard characters • Emotion + icon (English)
O:-) Angel X-( Angry ~:0 Baby :-D Big Grin :-# Braces </3 Broken Heart =^.^= Cat *<:o) Clown O.o Confused B-) Cool :'( Crying …
Emoji • Pictographs (pictorial symbols): things,
emotions, activities, …
• Presented in a colorful form
• E ‘picture’ + moji ‘letter, character’ (Japanese)
3
![Page 4: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/4.jpg)
Unicode® Emoji
• Single Unicode characters emoji counts
• Many more emojis are composed of sequences of two or more characters (e.g. regional indicators in pairs represent ≈ flags)
4
Version Emoji count Year 1.0.0 76 (+76) 1991 3 78 (+2) 1999 3.2 86 (+8) 2002 4 94 (+8) 2003 4.1 108 (+14) 2005 5.1 112 (+4) 2008 5.2 139 (+27) 2009 6 855 (+716) 2010 6.1 868 (+13) 2012 7 972 (+104) 2014 8 1,013 (+41) 2015 9 1,085 (+72) 2016
Emojis were first defined in Unicode 6.0, and pre-6.0 characters were only defined as emoji in 6.0 or later.
![Page 5: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/5.jpg)
167 new Emoji Unicode 9.0.0 (July 2016)
5
![Page 6: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/6.jpg)
Sequence of Emojis
6
![Page 7: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/7.jpg)
Unicode® Emoji
7
Version Emoji count Year 1.0.0 76 (+76) 1991 3 78 (+2) 1999 3.2 86 (+8) 2002 4 94 (+8) 2003 4.1 108 (+14) 2005 5.1 112 (+4) 2008 5.2 139 (+27) 2009 6 855 (+716) 2010 6.1 868 (+13) 2012 7 972 (+104) 2014 8 1,013 (+41) 2015 9 1,085 (+72) 2016
Emojis were first defined in Unicode 6.0, and pre-6.0 characters were only defined as emoji in 6.0 or later.
![Page 8: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/8.jpg)
Emoji Popularity:
8 http://instagram-engineering.tumblr.com/post/117889701472/emojineering-part-1-machine-learning-for-emoji
![Page 9: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/9.jpg)
Why do we Emoji?
Oxford Dictionaries named (Face With Tears of Joy) its 2015 Word of the year.
9
![Page 10: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/10.jpg)
10 Mehrabian, Albert (1971). Silent Messages (1st ed.). Belmont, CA: Wadsworth.ISBN 0-534-00910-7.
![Page 11: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/11.jpg)
Emoticons in Mind
When we look at a smiley face, the same parts of the brain are activated as when we look at a real human face.
11 Churches O, Nicholls M, Thiessen M, Kohler M, Keage H (2014) Emoticons in mind: An event-related potential study. Social Neuroscience
9(12)doi:10.1080/17470919.2013.873737
![Page 12: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/12.jpg)
- EMOJI FACTS - OPINION MINING - EMOJI SENTIMENT
Outline
12
![Page 13: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/13.jpg)
Opinion Mining
• Opinion mining (or sentiment analysis) is the computational study of people’s opinions, sentiments, emotions, and attitudes.
13
Approaches
Machine learning
Large corpus annotated with
sentiment
Lexical
Sentiment lexicon
Hybrid
Annotated corpus
Sentiment lexicon
Semantic parsing
![Page 14: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/14.jpg)
Sentiment Lexicons
• Lists of words associated to scores of sentiment polarity
• Well developed for English
– Bing Liu's Opinion Lexicon
– MPQA Subjectivity Lexicon
– Harvard General Inquirer
– SentiWordNet assigns to each synset of WordNet three sentiment scores: positivity, negativity, objectivity.
14 Baccianella S, Esuli A, Sebastiani F. "SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining.“
LREC. Vol. 10. 2010.
![Page 15: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/15.jpg)
- EMOJI FACTS - OPINION MINING - EMOJI SENTIMENT
Outline
15 Kralj Novak P, Smailović J, Sluban B, Mozetič I (2015) Sentiment of Emojis. PLoS ONE 10(12): e0144296.
doi:10.1371/journal.pone.0144296
![Page 16: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/16.jpg)
Why Sentiment of Emojis?
• As features for sentiment models
• As labels to generate samples of negative and positive texts
• Sentiment conveyed by emoticons is domain and topic independent
Emoticons in opinion mining
• B-)
Emojis are replacing emoticons in social media
16
![Page 17: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/17.jpg)
Dataset
• Over 1.6 million tweets in 13 European languages (Apr. 2013 – Feb. 2015)
• Labeled by 83 native speakers: negative (-1) < neutral (0) < positive (+1)
• 4% of our tweets contain emojis (69,546)
17 Mozetič I, Grčar M, Smailović J (2016) Multilingual Twitter Sentiment Classification: The Role of Human Annotators. PLoS ONE 11(5):
e0155036. doi:10.1371/journal.pone.0155036
![Page 19: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/19.jpg)
Sentiment Score
19
Discrete distribution
Discrete probability distribution
Sentiment Score
negative: 3 neutral: 17 positive: 75
negativity p- = 0.04 neutrality p0 = 0.18 positivity p+= 0.78
Sentiment score = 0.74
BOUQUET
95 occurrences
![Page 20: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/20.jpg)
Sentiment Score
Laplace estimate to estimate the probability
20
Discrete distribution
Discrete probability distribution
Sentiment Score
Sentiment score is the mean of the discrete probability distribution
![Page 21: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/21.jpg)
Sentiment Bar
21
BOUQUET • Negative: 0.04
• Neutral: 0.18
• Positive: 0.78
• Sentiment Score: 0.74
![Page 22: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/22.jpg)
Emoji Sentiment Lexicon http://kt.ijs.si/data/Emoji_sentiment_ranking/
22
![Page 23: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/23.jpg)
Language Independent?
23
Compare the ordered list with Emoji Sentiment Lexicon
Order emojis by their sentiment score
Create emoji sentiment lexicon for each language separately
![Page 24: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/24.jpg)
Emoji Sentiment in Different Languages
• Pearson’s correlation and Spearman’s rank correlation are relatively high, and significant for all the languages, except Albanian.
• Emoji Sentiment Ranking can be considered a universal, language-independent resource, at least for European languages.
24
![Page 25: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/25.jpg)
Emoji Sentiment Map
25
![Page 26: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/26.jpg)
Emoji Sentiment Map
26
![Page 27: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/27.jpg)
Emoji Sentiment Map: Zoom Positive Sentiment Score
27
![Page 28: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/28.jpg)
Emoji Sentiment Map: Zoom Negative Sentiment Score
28
![Page 29: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/29.jpg)
Emoji Sentiment Map: Zoom Neutral Sentiment Score
29
![Page 30: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/30.jpg)
Emoji Occurrence and Sentiment
30
![Page 31: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/31.jpg)
Tweets With and Without Emojis
31
• Does the sentiment distribution differ between tweets with and without emojis?
Sentiment
• Do annotators agree on the sentiment of the tweet any better if tweet contains emojis?
Annotator agreement
![Page 32: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/32.jpg)
Tweets With and Without Emojis: Sentiment
32
• Tweets with emojis are more positive.
![Page 33: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/33.jpg)
Tweets With and Without Emojis
33
• Does the sentiment distribution differ between tweets with and without emojis?
Sentiment
• Do annotators agree on the sentiment of the tweet any better if it contains emojis?
Annotator agreement
![Page 34: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/34.jpg)
Tweets With and Without Emojis: Annotator Agreement
34
• The presence of emojis has a positive impact on the emotional perception of the tweets by humans.
![Page 35: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/35.jpg)
Where in Tweet?
35
Beginning of tweet End of tweet
More emotionally loaded emojis, either negative or positive, tend to occur towards the end of the tweets.
![Page 36: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/36.jpg)
Applications
• Opinion mining and sentiment analysis
– Automated sentiment classification
– Automated dataset generation
• Fiverr.com is developing a system to automatically identify dissatisfaction, which is using our Emoji sentiment lexicon
36
![Page 37: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/37.jpg)
Caution
37 Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., and Hecht, B. (2016) “Blissfully happy” or “ready to fight”: Varying Interpretations of
Emoji. Proceedings of ICWSM 2016. Menlo Park, CA: AAAI Press.
![Page 38: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/38.jpg)
Conclusion
• Emojis are ubiquitous
• Most of the most frequently used emojis are positive
• Tweets with emojis are more positive
• Annotators perceive sentiment better if emojis are present
• Emotionally loaded emojis tend to appear at the end of the tweet
• Emoji sentiment map
• Emoji sentiment lexicon
– Language independent
– Clarin.si
38
![Page 39: Sentiment of Emojis - IJSnl.ijs.si/janes/wp-content/uploads/2016/09/CMC2016_kralj_novak.pdf · Sentiment of Emojis Petra Kralj Novak 4th Conference on ... • Opinion mining (or sentiment](https://reader033.fdocuments.in/reader033/viewer/2022060223/5f07e8487e708231d41f5b22/html5/thumbnails/39.jpg)
THANK YOU.
Thanks to coauthors
Jasmina Smailović, Borut Sluban and Igor Mozetič
39