Sentiment Analysis - Department of Computer Science...
Transcript of Sentiment Analysis - Department of Computer Science...
![Page 1: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/1.jpg)
SentimentAnalysis
![Page 2: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/2.jpg)
Announcements• Homework3dueat2:30pmnextTuesday.
![Page 3: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/3.jpg)
FromCoreNLPtoApplications
CORENLPParsingPOStaggingSemanBcs
APPLICATIONS SenBmentSummarizaBonInformaBonExtracBon
MachineTranslaBon
![Page 4: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/4.jpg)
Today• SenBmentanalysistasks:definiBon• SenBmentresources• TradiBonalsupervisedapproach• Neuralnetapproach
![Page 5: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/5.jpg)
Doembeddingshandlenegation?• 1:not1.0000000000000004• 2:n't0.8595728019811346• 3:but0.839545755064721• 4:did0.8378272618764329• 5:would0.8187187243474063• 6:should0.8147740055059252• 7:if0.8116091330796058• 8:because0.7987450091713499• 9:they0.7944962528430977• 10:be0.791361002418091• 11:could0.7894321710724349• 12:never0.7860447682817786• 13:any0.7842654035407371• 14:even0.776876305477035• 15:do0.7708686700685263• 16:only0.7691260950825682• 17:might0.7673607671887417• 18:so0.7670919214405183• 19:that0.7630105150824054• 20:though0.7625415104568314• 21:does0.7556821279593668• 22:cannot0.7536603867497527• 23:neither0.7520466068389566• 24:yet0.7470431538715656• 25:although0.7465924525156928
![Page 6: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/6.jpg)
![Page 7: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/7.jpg)
AntonymsandSynonyms:embeddingfor“hot”• 1:hot0.9999999999999996
• 2:cool0.6137860693915586• 3:hoVest0.5816319703075693• 4:heat0.5266680665228453• 5:warm0.51671900202736• 6:cold0.5093751774671291• 7:chili0.4624909077143189• 8:dry0.4613872561048547• 9:heated0.45721498258314297• 10:bubbling0.4534137094122158• 11:hoVer0.4529186992101415• 12:spots0.4416197356093728• 13:boiling0.44035866405318447• 14:billboard0.4340849896360003• 15:soW0.4268572343642097• 16:temperature0.42600188687018437• 17:wet0.4198371006642362• 18:chocolate0.41844508951954273• 19:water0.4174513613786725• 20:temperatures0.4160490504977998• 21:drink0.41476813237122767• 22:stove0.41431353491608697• 23:humid0.41044559731588987• 24:sizzling0.4083319186481177• 25:cooking0.408002615081707
![Page 8: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/8.jpg)
![Page 9: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/9.jpg)
Whatissentiment?• ExpressionofposiBveornegaBveopinions• ..Towardsatopic,person,event,enBty• ..Towardsanaspect
![Page 10: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/10.jpg)
Whysentimentanalysis?• SenBmentiscommoninonlineplaYorms• Peoplewriteabouttheirpersonalviewpoints
• UsefultounderstandwhatpeoplethinkaboutpoliBcalissues,poliBcalcandidates,importanteventsoftheday• UsefulforgeneraBngsummariesofreviews:restaurants,products,movies
![Page 11: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/11.jpg)
Thesentimentanalysistask(s)• SubjecBvevsobjecBve• PosiBve,negaBveorneutral• DowehavesenBmenttowardsatarget?OraspectbasedsenBment?• What/whoisthesenBmentsource?
![Page 12: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/12.jpg)
SubjectivevsObjective• �Atseveraldifferentlayers,it’safascina3ngtale.[“Who’sSpyingonOurComputers”,GeorgeMelloanWallStJournal.(Bookreview)• BellIndustriesIncincreaseditsquarterlyto10centsfrom7centsashare.
ExamplesfromWeibeetal2004
![Page 13: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/13.jpg)
Positive/Negative/Neutral• FromUseNet:
• NegaBve:Ihadinmindyourfacts,Buddy,nothers.• PosiBve:Nicetouch.“Alleges”whateverfactspostedarenotinyourpersonaofwhatis“real”• Neutral:Marchappearstobeanes3matewhileearlieradmissioncannotbeen3relyruledout,"accordingtoChen,alsoTaiwan'schiefWTOnego3ator
ExamplesfromWeibeetal2004andRosenthal2014
![Page 14: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/14.jpg)
SubjectivePhrases• TheforeignministrysaidThursdaythatitwas“surprised,toputitmildly”bytheU.S.StateDepartment’scri0cismofRussia’shumanrightsrecordandobjectedinparBculartothe“odious”secBononChechnya.[MoscowTimes,03/08/2002]• Subjec3vityanalysisiden3fiestextthatrevealsanauthor’sthoughts,beliefsorotherprivatestates.
ExamplesfromWeibeetal2004
![Page 15: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/15.jpg)
SubjectivePhrasesandSources• TheforeignministrysaidThursdaythatitwas“surprised,toputitmildly”bytheU.S.StateDepartment’scri0cismofRussia’shumanrightsrecordandobjectedinparBculartothe“odious”secBononChechnya.[MoscowTimes,03/08/2002]• Whowassurprised?• WhowascriBcal?
ExamplesfromWeibeetal2004
![Page 16: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/16.jpg)
AdditionalExamples• AuthoriBesareonlytooawarethatKashgaris4,000kilometres(2,500miles)fromBeijingbutonlyatenthofthedistancefromthePakistaniborder.
• Taiwan-madeproductsstoodagoodchanceofbecomingevenmorecompeBBvethankstowideraccesstooverseasmarketsandlowercostsformaterialimports,hesaid.
• "MarchappearstobeamorereasonableesBmatewhileearlieradmissioncannotbeenBrelyruledout,"accordingtoChen,alsoTaiwan'schiefWTOnegoBator.
• fridayeveningplansweregreat,butsaturday'splansdidntgoasexpected--iwentdancing&itwasanokclub,butterriblycrowded:-(
• WHYTHEHELLDOYOUGUYSALLHAVEMRS.KENNEDY!SHESAFUCKINGDOUCHE
• AT&Twasokaybutwhenevertheydosomethingniceinthenameofcustomerserviceitseemslikeafavor,whileT-Mobilemakesthatanormaleverydaythin
16
ExamplesfromRosenthal2014
![Page 17: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/17.jpg)
SentimenttowardsTarget• IpreUymuchenjoyedthewholemovie.Target=wholemovie,senBment=posiBve.• Bulgariaiscri3cizedbytheEUbecauseofslowreformsinthejudiciarybranch,thenewspapernotes.Target=Bulgaria,senBment=negaBve• Stanishevwaselectedprimeministerin2005.Sincethen,hehasbeenaprominentsupporterofhiscountry’saccessiontotheEU.Target=country’saccesstotheEU,senBment=posiBve
ExamplesfromBreck&Cardieforthcoming
![Page 18: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/18.jpg)
Datasets(Sem-evaldatasetsalsoused)
2000sentencesineachcorpus
18
Corpus AverageWordCount
AverageCharacterCount
Subjec6vePhrases
Objec6vePhrases
VocabularySize
CharacterLengthRestric6ons
LiveJournal 14.67 66.47 3035(39%) 4747(61%) 4747 30-120
MPQA 31.64 176.68 3325(41%) 4754(59%) 7614 none
TwiVer 25.22 118.55 2091(36%) 3640(64%) 8385 0-140
Wikipedia 15.57 77.20 2643(37%) 4496(63%) 4342 30-120
MPQA:extensivelyannotateddatasetbyStoyanav,CardieandWeibe2004.15opinionorientedqusBons,15factorientedquesBons.Alongwithtextspansfrom252arBcles.
RosenthalandMcKeown2013)
![Page 19: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/19.jpg)
ExampleSentences
LiveJournal iwillhavetosBcktomycanonfilmslrunBlinafewyearsicanaffordtoupgradeagain:)
MPQA ThesaleinfuriatedBeijingwhichregardsTaiwananintegralpartofitsterritoryawaiBngreunificaBon,byforceifnecessary.
TwiVer RT@tashjade:That’sreallysad,CharlieRT“UnBltonightIneverrealisedhowfuckedupIwas”-CharlieSheen#sheenroast
Wikipedia PerhapsifreportedcriBcallybyawesternsourcebutcertainlynotbyanIsraelisource.
19
SubjecBve ObjecBve
![Page 20: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/20.jpg)
SentimentLexicons• GeneralInquirer• SenBWordNet• DicBonaryofAffect(DAL)
![Page 21: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/21.jpg)
DictionaryofAffectinLanguage• DicBonaryof8742wordsbuilttomeasuretheemoBonalmeaningoftexts• Eachwordisgiventhreescores(scaleof1to3)• pleasantness-alsocalledevaluaBon(ee)• acBveness(aa)• andimagery(ii)
C.M.Whissel.1989.Thedic6onaryofaffectinlanguage.InR.PlutchikandH.Kellerman,editors,EmoBon:theoryresearchandexperience,volume4,London.Acad.Press.
21
![Page 22: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/22.jpg)
Wordnet• Propernouns(e.g.BritneySpears)areautomaBcallymarkedasobjecBve• WordsthatdonotexistintheDALarelookedupinWordnet• ComputetheaverageoftheDALscoresofallthesynonymsofthefirstsense• Iftherearenosynonyms,lookatthehypernym
22
![Page 23: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/23.jpg)
Wiktionary• WikBonaryisafreecontentdicBonary• hVp://www.wikBonary.org
• IfaworddoesnotappearintheDALorWordnetlookitupinWikBonary• ComputetheaverageoftheDALscoresforeachwordinthedefiniBonthathasitsownWikBonarypage
23
![Page 24: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/24.jpg)
Emoticons• 1000emoBconsweregatheredfromseverallistsavailableontheinternet• Wekeptthe192emoBconsthatappearedatleastonceandmappedeachemoBcontoasingleworddefiniBon
24
![Page 25: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/25.jpg)
Methods• Pre-processingsteps• EmoBconkeysandcontracBonexpansion• Chunkerandtagger*• LexicalFeatures*• SyntacBcFeatures*• SocialMediaFeatures
*ApoorvAgarwal,FadiBiadsy,andKathleenR.McKeown.2009.Contextualphrase-levelpolarityanalysisusinglexicalaffectscoringandsyntac6cn-grams.InProceedingsofEACL’09
25
![Page 26: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/26.jpg)
PreprocessingLiveJournal [i]/NPsub[willhavetos6ck]/VPobj[to]/PPobj[mycanonfilmslr]/NPobj[unBl]/
PPobj[in]/PPobj[afewyears]/NPsub[i]/NPsub[canaffordtoupgrade]/VPobj[again:)]/NPsub
MPQA [Thesale]/NPsub[infuriated]/VPobj[Beijing]/NPobj[which]/NPsub[regards]/VPsub[Taiwan]/NPobj[anintegralpart]/NPsub[of]/PPobj[itsterritoryawaiBngreunificaBon,]/NPobj[by]/PPobj[force]/NPsub[if]/obj[necessary.]/sub
TwiVer [RT@tashjade:]/NPobj [That]/Npobj[is]/VPsub[really]/sub[sad,]/sub[CharlieRT]/NPobj[”]/NPobj[UnBl]/PPobj[tonight]/NPsub[I]/NPsub[never]/sub[realised]/VPsub[how]/sub[fucked]/VPsub[up]/PPobj[I]/NPsub[was]/VPsub[”]/obj[-CharlieSheen#sheenroast]/NPobj
Wikipedia [Perhaps]/sub[if]/obj[reported]/VPsub[cri6cally]/sub[by]/PPobj[awesternsourcebut]/NPsub[certainlynot]/sub[by]/PPobj[anIsraelisource.]/NPsub
26Xuan-HieuPhan,CRFChunker:CRFEnglishPhraseChunkerhVp://crfchunker.sourceforge.net/,2006
![Page 27: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/27.jpg)
LexicalFeatures• POSTags*• N-grams*• Performedchi-squarefeatureselecBononthen-grams
*ApoorvAgarwal,FadiBiadsy,andKathleenR.McKeown.2009.Contextualphrase-levelpolarityanalysisusinglexicalaffectscoringandsyntac6cn-grams.InProceedingsofEACL’09
27
![Page 28: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/28.jpg)
SyntacticFeatures• Usethemarkedupchunkstoextractthefollowing:*• n-grams:1-3words• POS:NP,VP,PP,JJ,other• PosiBon:target,right,leW• SubjecBvity:subjecBve,objecBve• Minandmaxpleasantness
*ApoorvAgarwal,FadiBiadsy,andKathleenR.McKeown.2009.Contextualphrase-levelpolarityanalysisusinglexicalaffectscoringandsyntac6cn-grams.InProceedingsofEACL’09
28
![Page 29: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/29.jpg)
SocialMediaFeatures
29
![Page 30: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/30.jpg)
30
-
0.04
0.08
0.12
0.16
0.20CapitalW
ords s o
OutofV
ocabulary s o
EmoB
cons s o
Acronyms s o
Exclam
aBon
s s o
Repe
ated
ExclamaB
ons s o
Que
sBon
s s o
Repe
ated
Que
sBon
s s o
Ellipses s o
SocialMediaFeatures
twiVer
livejournal
wikipedia
mpqa
SMfeaturestendtobeveryrare.Thefrequencyforeachfeatureislessthan1%perdataset
![Page 31: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/31.jpg)
SingleCorpusClassiOication
BalancedUnbalanced
• LogisBcRegressioninWeka
• 10runsof10-foldcross-validaBon
• StaBsBcalsignificanceusingthet-testwithp=.001
31
Experiment LiveJournal MPQA TwiVer Wikipedia
n-gramsize 100 2000 none none
majority 58% 59% 64% 63%
JustDAL 76.5% 75.7% 83.6% 80.4%
DicBonaries+SM 77.1% 76.1% 84% 81.4%
Wordnet 76.7% 75.6% 84% 80.7%
Wordnet+SM 77.1% 76.1% 84.2% 81.4%
DicBonaries 76.6% 75.7% 83.9% 80.7%
SM 77% 76.1% 83.7% 81.2%
Experiment LiveJournal MPQA TwiVer Wikipedia
n-gramsize 100 200 none none
majority 50% 50% 50% 50%
JustDAL 74.7% 75.7% 81.9% 79.3%
DicBonaries+SM 76.7% 76.2% 82.6% 80.2%
Wordnet 75.1% 75.8% 82.4% 79.1%
Wordnet+SM 76.6% 75.3% 82.6% 80.3%
DicBonaries 75.3% 75.8% 82.4% 79.1%
SM 76.2% 76.3% 82.2% 80.4%
![Page 32: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/32.jpg)
SocialMediaErrorAnalysis• Wikipedia• PunctuaBonwasusefulasafeaturefordeterminingthataphraseisobjecBveifitisasmallphrase.However,severalsubjecBvephraseswereincorrectlyclassifiedbecauseofthis 32
0%
20%
40%
60%
80%
100%
Wikipedia
subjecBveobjecBve
![Page 33: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/33.jpg)
SocialMediaErrorAnalysis• TwiVer• ellipsesdohelpindicatethatasentenceisobjecBve.Theaccuracyimprovedfrom82%to92%forsentenceswiththisfeature• AllothersocialmediafeatureswereincorrectlyclassifiedasobjecBve/subjecBvedependingonthesocialmediapreference. 33
0%
20%
40%
60%
80%
100%
TwiQer
subjecBveobjecBve
![Page 34: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/34.jpg)
SocialMediaErrorAnalysis• LiveJournal• OutofVocabularywordsandpunctuaBonwerethemostusefulsocialmediafeatures.• InalldatasetsthepunctuaBonfeaturecausedcloseto50/50exchangebutthefeaturewasbestinLiveJournal. 34
0%
20%
40%
60%
80%
100%
Livejournal
subjecBveobjecBve
![Page 35: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/35.jpg)
Cross-GenreClassiOication
35
TwiQer LiveJournal MPQA Wikipedia
TwiQer 71.6% 62.1% 76.9%
LiveJournal 82.5% 65.4% 80.9%
MPQA 75.6% 69.3% 71.2%
Wikipedia 82.4% 76.7% 62.4%
Training
TesBng
Thischartdisplaysthebestresultsforeachexperiment
![Page 36: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/36.jpg)
Cross-GenreClassiOication
36
TwiQer LiveJournal MPQA Wikipedia
TwiQer 71.6% 62.1% 76.9%
LiveJournal 82.5% 65.4% 80.9%
MPQA 75.6% 69.3% 71.2%
Wikipedia 82.4% 76.7% 62.4%
Training
TesBng
• TheonlinegenresdonotdowellinpredicBngMPQAsentences
![Page 37: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/37.jpg)
Cross-GenreClassiOication
37
TwiQer LiveJournal MPQA Wikipedia
TwiQer 71.6% 62.1% 76.9%
LiveJournal 82.5% 65.4% 80.9%
MPQA 75.6% 69.3% 71.2%
Wikipedia 82.4% 76.7% 62.4%
Training
TesBng
• LiveJournaltrainingdatadoesagoodjobofpredicBngtheotheronlinegenres• WikipediatrainingdatadoesagoodjobofpredicBngTwiVer
![Page 38: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/38.jpg)
Cross-GenreClassiOication
38
TwiQer LiveJournal MPQA Wikipedia
TwiQer 71.6% 62.1% 76.9%
LiveJournal 82.5% 65.4% 80.9%
MPQA 75.6% 69.3% 71.2%
Wikipedia 82.4% 76.7% 62.4%
Training
TesBng
• TwiVertrainingdatadoesadecentjobofpredicBngWikipedia• WikipediatrainingdatadoesadecentjobofpredicBngLiveJournal
![Page 39: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/39.jpg)
Cross-GenreClassiOication
39
TwiQer LiveJournal MPQA Wikipedia
TwiQer 71.6% 62.1% 76.9%
LiveJournal 82.5% 65.4% 80.9%
MPQA 75.6% 69.3% 71.2%
Wikipedia 82.4% 76.7% 62.4%
Training
TesBng
• Ingeneral,usingtheMPQAastrainingdoesnotperformwell• UsingTwiVerastrainingdoesnotperformwellinpredicBngLiveJournal
sentences
![Page 40: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/40.jpg)
NeuralNetworkApproachestoSentiment• Goldberg:• TakeastandardRNNsuchasshowninclasslastBme• Takealabeleddataset(e.g.,IMDBsenBmentdataset)• IniBalizewithpre-trainedwordembeddings(wordtovecorglove)• UsesigmoidtopredictbinarysenBmentlabels:posiBvevsnegaBve.
![Page 41: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/41.jpg)
Example• Foreachsentenceinthetrainingcorpus,classify,comparetogoldstandardandcomputeloss,backpropagate.• Recallthatwemayusemini-batchessothatwe’renotback-propagaBngforeachexample
• Ihadinmindyourfacts,Buddy,nothers.
![Page 42: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/42.jpg)
RNN–Ihadinmindyourfacts,buddy,nothers.
wx
wh
x1
h0
h1
σwx
wh
x2
h2
σwx
wh
x3
h3
σ
y3sigmoid
I had in
wx
wh
x3
h3
σ
mind …
Inthisoverview,wreferstotheweightsButtherearedifferentkindsofweightsLet’sbemorespecific
![Page 43: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/43.jpg)
RNN–Ihadinmindyourfacts,buddy,nothers.
wx
U
x1
h0
h1
wx
U
x2
h2
σwx
U
x3
h3
σ
y3sigmoid
I had in
wx
U
x3
h3
σ
mind …
σ
![Page 44: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/44.jpg)
![Page 45: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/45.jpg)
![Page 46: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/46.jpg)
![Page 47: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/47.jpg)
RNN–Ihadinmindyourfacts,buddy,nothers.
wx
U
x1
h0
h1
wx
U
x2
h2
σwx
U
x3
h3
σ
y3sigmoid
I had in
wx
U
x3
h3
σ
mind …
Waretheweights:thewordembeddingmaatrixmulBplicaBonwithxiyieldstheembeddingforxUisanotherweightmatrixH0isoWennotspecified.Histhehiddenlayer.
σ
![Page 48: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/48.jpg)
RNN–Ihadinmindyourfacts,buddy,nothers.
wx
U
x1
h0
h1
wx
U
x2
h2
σwx
U
x3
h3
σ
y3sigmoid
I had in
wx
U
x3
h3
σ
mind …
ht=σ(Uwxt)ht-1
σ
![Page 49: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/49.jpg)
RNN–Ihadinmindyourfacts,buddy,nothers.
wx
wh
x1
h0
h1
σwx
wh
x2
h2
σwx
wh
x3
h3
σ
Sigmoid
I had in
wx
wh
x3
h3
σ
mind …
Y=posiBve?Y=negaBve?Finalembeddingrunthroughthesigmoid
funcBon->[0,1]1=posiBve0=negaBveOWenfinalhisusedaswordembeddingforthesentence
![Page 50: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/50.jpg)
UpdatingParametersofanRNN
wx
wh
x1
h0
h1
σwx
wh
x2
h2
σwx
wh
x3
h3
σ
y3sigmoid
Cost
wy
BackpropagaBonthroughBmeGoldlabel=0(negaBve)AdjustweightsusinggradientRepeatmanyBmeswithallexamples
SlidefromRadev
I had in
![Page 51: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/51.jpg)
RecursiveDeepModelsforSemanticCompositionalityoveraSentimentTreebank• Socheretal,Stanford2013hVps://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf• Problemwithpreviouswork:difficultyexpressingthemeaningoflongerphrases• Goal• TopredictsenBmentatthesentenceorphraselevel• CaptureeffectofnegaBonandconjuncBons• SenBmentTreebank• RecursiveNeuralTensorNetwork
![Page 52: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/52.jpg)
SentimentTreebank• MoviereviewexcerptsfromroVentomatoes.com(Pang&Lee2005)• 10,662sentences• ParsedbyStanfordParser(Klein&Manning2003)• 215,154phrases• EachphraselabeledforsenBmentusingAmazonMechanicalTurk(AMT)• 5classesemerge:negaBve,somewhatnegaBve,neutral,somewhatposiBve,posiBve
![Page 53: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/53.jpg)
Example--verynegaBve++veryposiBve- NegaBve+posiBve0neutral
![Page 54: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/54.jpg)
RecursiveNeuralModels
![Page 55: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/55.jpg)
RNN:RecursiveNeuralNetwork
WaretheweightstolearnWεf=tanh
![Page 56: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/56.jpg)
MV-RNNMatrixvectorRNN• Introduceweightmatrixassociatedwitheachnon-terminal(P2foradjP)andterminal(Afora)• a=not,b=very,c=good
![Page 57: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/57.jpg)
RNTN:RecursiveNeuralTensorNetwork• TheMV-RNNhastoomanyparameterstolearn(sizeofvocabulary)• CanwegetcomposiBonalitywithreducedparameters?• P1=f([ab]u1u2a)u3u4b=f([ab]u1a+u2b)u3a+u4b
=f(u1aa+u2ab+u3ab+u4bb)
![Page 58: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/58.jpg)
Results
![Page 59: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/59.jpg)
Positive–“mostcompelling”
![Page 60: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/60.jpg)
Negative–“leastcompelling”
![Page 61: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/61.jpg)
HandlingConjunctions
![Page 62: Sentiment Analysis - Department of Computer Science ...kathy/NLP/2017/ClassSlides/Class18Sentiment/... · Today • SenBment analysis tasks: definiBon • SenBment resources •](https://reader033.fdocuments.in/reader033/viewer/2022052517/5c6796ed09d3f28e058bf5db/html5/thumbnails/62.jpg)
NextTime• SummarizaBon