Entity and Aspect Extraction for Organizing News Comments · analysis and lexical database search....

EntityandAspectExtractionforOrganizingNewsComments

RadityoEkoPrasojo,MounaKacimi&WernerNuttEMCLWorkshop,11–12February2016

CommentsinNewsWebsiteTypically, commentsarelistedbasedondate-timeandreplyrelation.

Problem:difficultytocatchtheflowofthediscussionsandtounderstandtheirmainpointsofagreementanddisagreement.

Example:whyisindependencegood/bad fortheScots?Willtheireconomy beaffected?

EMCLWorkshop2016 Prasojo,Kacimi,&Nutt- EntityandAspectExtractionforOrganizingNewsComments

2

Thereisaneedfororganizingcomments

to help users to:(1) have a better understandingof the viewpoints related to each topic(2) facilitate the participation in discussions and thus increase the

chance of acquiring new viewpoints

by clustering comments containing similar discussions:

• they talk about the same entities

• they argue about the same aspects of those entities.EMCLWorkshop2016 Prasojo,Kacimi,&Nutt- EntityandAspectExtractionfor

OrganizingNewsComments3

Contributions

• Improvementsonstate-of-the-artunsupervisedentityextractiontools(Zemanta,NERD,AIDAYago)• Addressedissues:noisesandlowcoverage(duetocoreferences)

• Introduced aspectextractioninnewsdomain• Previously:aspectsonlyonproductreviewdomain(Zhang&Liu,2014)


4

EntityextractionBaseline:UnsupervisedTools

• Improvedbyapplying:• Entityfiltering• Namenormalization• EntitysearchonKB• Coreference resolution


5

Tools:DbpediaStanfordCoreNLP

“Don't be afraid of Rasmussen or NATO, this is none of their business. If he ends up sending USS Ronald Reagan aircraft carrier to the coast of Scotland, then he should have done the same to Crimea.”

<http://dbpedia.org/resource/NATO>

<http://dbpedia.org/resource/Scotland><http://dbpedia.org/resource/USS_Ronald_Reagan>

“Don't be afraid of Rasmussen or NATO,this is none of their business. If he ends up sending USS Ronald Reagan aircraft carrier to the coast of Scotland, then he should have done the same to Crimea.”

<http://dbpedia.org/resource/Crimea>

<http://dbpedia.org/resource/Aircraft_Carrier>



6

rdf:type ofNATO?à←dbo:PopulatedPlace

Anentity isaninstanceofsomewell-definedclass

rdf:type?à

←dbo:Shiprdf:type?à←dbo:Location

rdf:type?à←owl:Thing

EntityFiltering



7

Weremoveentities thatdon’thaverdf:type otherthanowl:Thing andowl:Class

rdf:type ofNATO?à←dbo:PopulatedPlace

Anentity isaninstanceofsomewell-definedclass

rdf:type?à

←dbo:Shiprdf:type?à←dbo:Location

Risk:lowerrecall

rdf:type?à←owl:Thing

EntityFiltering


8

Anentitymayappearusingnon-propernames(alias)


Lorem ips um dolor s it amet, consecte turadipisc ing el i t. Ut s usc ipi t lacus ac bibendum mol lis. Pe llenteas dsadas que t incitUnited States

Lorem ips um dolor s it amet, consecte turadipisc ing el i t. Ut s usc ipi t lacus ac bibendum mol lis. Pe llentesquetinc idunt vu lputate ligula a e ffic itur. Aliquam era tv olutpat. Al iquam sedenim et tortor

fringi l la asdasdasdasdas das dqwrqweqweqwr w qeqw eAnders Fogh Rasmussen.

Maecenas vehicula urnaeget m etus imperdie t com modo.

ScotlandRejectsIndependenceinReferendumLorem ipsumdolorsitamet,consectetur adipiscing elit.Ut suscipit lacusacbibendum mollis.Pellentesque tincidunt vulputate ligulaaefficitur.Aliquam erat

…………foaf:givenName,foaf:surname?à

←{‘Anders’, ‘Fogh’, ‘Rasmussen’}

dbpedia-owl:wikiPageRedirects of?à←{‘US’,‘USA’, ‘U.S.’,‘U.S.A.’,…}

NameNormalization


9

Anentitymayappearusingnon-propernames(alias)


Lorem ips um dolor s it amet, consecte turadipisc ing el i t. Ut s usc ipi t lacus ac bibendum mol lis. Pe llenteas dsadas que t incitUnited States

Lorem ips um dolor s it amet, consecte turadipisc ing el i t. Ut s usc ipi t lacus ac bibendum mol lis. Pe llentesquetinc idunt vu lputate ligula a e ffic itur. Aliquam era tv olutpat. Al iquam sedenim et tortor

fringi l la asdasdasdasdas das dqwrqweqweqwr wqeqwe Anders Fogh Rasmussen. Maecenas

v eh icu la urna ege tm etus im perdie t com modo.

ScotlandRejectsIndependenceinReferendumLorem ipsumdolorsitamet,consectetur adipiscing elit.Ut suscipit lacusacbibendum mollis.Pellentesque tincidunt vulputate ligulaaefficitur.Aliquam erat

…………foaf:givenName,foaf:surname?à

←{‘Anders’, ‘Fogh’, ‘Rasmussen’}

dbpedia-owl:wikiPageRedirects of?à←{‘US’,‘USA’, ‘U.S.’,‘U.S.A.’,…}

Risk:lowerprecision

NameNormalization


10

Sometimes,mappingforanaliascannotbefound

AllrelatedentitiesofNATOandtheiraliasesà

←dbprop:leaderName dbpedia:Anders_Fogh_Rasmussen,{‘AndersFogh’, ‘Rasmussen’}

Stringcomparison“Don't be afraid of Rasmussen or NATO,

Context-RelatedEntitySearch


11

Sometimes,mappingforanaliascannotbefound

AllrelatedentitiesofNATOandtheiraliasesà

←dbprop:leaderName dbpedia:Anders_Fogh_Rasmussen,{‘AndersFogh’, ‘Rasmussen’}

Stringcomparison

Risk:lowerprecision

“Don't be afraid of Rasmussen or NATO,

Context-RelatedEntitySearch


12


StanfordCoreNLP

Coreference Resolution


13


StanfordCoreNLP

Risk:lowerprecision

Coreference resolution

Coreference Resolution

EntityExtraction– ExperimentSetup

• 10newsarticlesthatuseDISQUS• 100commentshavingthehighestwordcounts• 5studentsasentity andaspect annotators• Annotateddataasgroundtruth


14

EntityExtraction– ExperimentResults


15

Aspects

• ofproductentities• Theaspectsofanentitye arethecomponents andattributes ofe.(Zhang&Liu,2014)

• ofentitiesonnews• “MoreScots woulddefinitelyhavevoted nothanyes”(voting - action)• “OrlandoBloomisagoodactor”(acting - skill)• “…it’stherightthe rightofScottishPeople”(right - possession)• Other:components,attributes,andmoods

Anaspect isallwhatisarguableaboutanentityEMCLWorkshop2016 Prasojo,Kacimi,&Nutt- EntityandAspectExtractionfor


TypesofAspects

• “…it’stherightthe rightofScottishPeople.”(explicit)

• “Tesco is large.”(implicit)

• “Scotlandisavery beautiful country.”(semi-implicit)

• “TheScotscan vote howevertheywant.” (semi-implicit)


17

aspect::right

aspect::employer

aspect::beauty

aspect::voting

ExtractionofExplicitAspects:ExploitingDependency

“…it’stherightthe rightofScottishPeople.”


18

StanfordCoreNLP Annotation

Extractallnounphrasesthathaveannmod:of,nmod:in,nmod:on,ornmod:atrelationtowardstheentity inthesentence.

PrepositionalDependency

• Specifically,thecoreference resolutionpart


19

ExtractionofExplicitAspects,CombinedwithEntityExtraction

“Don't be afraid of Rasmussen or NATO, this is none of their business.”

possessivedependency

ExtractionofImplicitAspects:Adjective-to-aspectMappingIdeafrom(Zhang&Liu,2014)

“Tesco islarge”

Inothercomments,wefoundasaspectsofTesco,qualifiedas“large”:• employer(2x)• backoffice(1x)• callcenteroperation(1x)

Weconclude:mostprobably,theemployer aspectofTesco wasmeant.Resultisfurtherimprovedby(1)takingintoaccountfrequentcontextwordsand(2)lexicalrelationsofadjective.


20

ExtractionofSemi-ImplicitAspects(1)

• Semi-implicitaspects:implicitaspectsthatdon’thavemapping.• “Scotlandisavery beautiful country.”

beautiful beautyWesearchforanounphrasethatisconnectedtotheentityandthewordindicatingtheaspectusingalexicalrelation.EMCLWorkshop2016 Prasojo,Kacimi,&Nutt- EntityandAspectExtractionfor


WordNet:attribute


Lexicaldatabasesearch Weconsiderfollowinglexicalrelations:attribute>pertainym>participleofverb>derivationallyrelatedform(drf)>seealso


• Generallycanbeusedtoidentifyaspects fromverb,adjective,ornoun.• Otherexamples:

beautiful beautyinstrumental instrumentvote votinginexcusable excusable justifiable

justification justify• Iftherearemultiplepossibleaspects forasingleword,weuseWordNet::Similaritytodecideforthebestone.


22

WordNet:attribute

WordNet:pertainym

WordNet:derivationally relatedform

WordNet:antonym WordNet:similar to

WordNet:drfWordNet:drf

AspectExtraction–ExperimentandResults


23

Visualization


24

REPrasojo,MKacimi,FDarari.IEEEInfoVis.Chicago,25-30October2015.

demoisnowavailableatorcaestra.inf.unibz.it

Conclusion• Generalcontribution:aframeworkfororganizingnewscomment usingentityextractionandaspectextraction.

• Ourentityextraction: unsupervisedtools+entityfiltering,namenormalization,entitysearch,andcoreference resolution.

• Weextractexplicit,implicit,andsemi-implicit aspectsusinggrammaranalysisandlexicaldatabasesearch.

• Experimentshowsimprovementonbothentity andaspect extractioncomparedtobaselinetechnique.


25

FutureWorks

• Addressinglimitations:• Conceptextractions• Idiomsandothermetaphoricalexpressions• Difficultcoreference (e.g.demonstrativepronouns)• Experimentsonmore,variousdata

• Completethemissingpieces:• Sentimentanalysisfornewscomments


26

• EuropeanMaster’sProgramme inComputationalLogic

• To-KnowProject

• SIGIRStudentTravelGrant


27

Acknowledgements

Thankyou!


28

Extraslides


29

Anentity canbe…

aperson,alocation,anorganization,oranywell-definedconcept suchasnationalities,languages,orwars.


30

Entity Extraction Tasks

1. Recognition– throughpropernames/rigiddesignators(Coates-Stephens,1992)(Thielen,1995)(Nadeau&Sekine,2006)

2. Disambiguation – bymappingtoarepresentationinaKB


31

“Scotland can vote however it wants, it's the Scottish peoples right.”

“If he ends up sending USS Ronald Reagan aircraft carrier to the coast of Scotland, then he should have done the same to Crimea.”

<http://dbpedia.org/resource/USS_Ronald_Reagan_(CVN-76)>

<http://dbpedia.org/resource/Crimea><http://dbpedia.org/resource/Scotland>

SupervisedvsUnsupervisedApproachestoEntityExtraction

AspectofEE

Prominent tools

Recognitionability

Disambiguation ability

Running time

Supervised

StanfordNLP

dependsonthetrainingset

limited

fast

UnsupervisedZemanta,AlchemyAPI,NERD,AidaYAGO

domain-independent

providedbyKBs

slow


32

Thereisaneedfororganizingcommentsto help users to:(1) have a better understandingof the viewpoints related to each topic(2) facilitate the participation in discussions and thus increase the

chance of acquiring new viewpoints

Our contribution is a comment organization framework:


33

Entityextraction Aspectextraction

Sentimentanalysis

Visualizationandorganization

ExtractionofImplicitAspects:Adjective-to-aspectMappingIdeafrom(Zhang&Liu,2014)

“Tesco islarge”

Inothercomments,wefoundasaspectsofTesco,qualifiedas“large”:• employer(2x)• backoffice(1x)• callcenteroperation(1x)

Weconclude:mostprobably,theemployer aspectofTesco wasmeant.EMCLWorkshop2016 Prasojo,Kacimi,&Nutt- EntityandAspectExtractionfor


ExtractionofImplicitAspects:ContextWordsSupposenowweneedatiebreaker!

“Tesco islargeandthepayisgood” contextwords:{large,pay,company}

Inothercomments,wefoundasaspectsofTesco,qualifiedas“large”:• employer(2x)contextwords:{employer,large,salary,job,people}• backoffice(2x)contextwords:{back,office,large,headquarter,building}• callcenteroperation(1x)

Weusecontext-words difference,computedusingWordNet:Similarity.Weconclude:mostprobably,theemployer aspectofTesco wasmeant.EMCLWorkshop2016 Prasojo,Kacimi,&Nutt- EntityandAspectExtractionfor


rdf:type

contextwords:{employer,large,salary,job,people}contextwords:{back,office,large,area,building}

contextwords:{large,pay,company},company

ExtractionofImplicitAspects:ExperimentandResult(1)


36

Adjective-to-aspectmappingData Precision RecallAllNews 76.87 26.12

withcontextwordsPrecision Recall88.65 30.03

ExtractionofImplicitAspects:LexicalMapping

“Tesco islarge”

Inadditionto“large”,countalsoaspectsqualifiedwithsimilaradjectives(“huge”,“big”,…)

Similarityismeasuredusing1. lexicalrelationship<synonym>and<similarto>inWordNet2. WordNet::Similarity


37

ExtractionofImplicitAspects:ExperimentandResult(2)


38

Lexicalmapping(1-step)Data Precision RecallAllNews 87.07 32.72

2-stepsPrecision Recall83.33 32.76


• Ifwedon’tfindtheaspectwithin1stepoflexicalsearch:• “OrlandoBloomisagoodactor”

act actoracting

Wesearchfortheclosestnountothepseudo-aspect.EMCLWorkshop2016 Prasojo,Kacimi,&Nutt- EntityandAspectExtractionfor


WordNet:derivationally relatedformLexicaldatabasesearch Tofind intermediatesynsets,weconsider

following lexicalrelations:synonym>antonym>similarto>derivationallyrelatedform(drf) >seealsoWordNet:derivationally relatedform


FindingAspects

1. Findwordsthatrepresenttheaspectjob,beautiful,large,acted<noun>,<adjective>,or<verb>

2. Identifytheaspectjob,beauty,employer,acting


40

Technique:grammaranalysisTool:StanfordCoreNLP

Technique:frequency-basedmapping (implicit)lexicaldatabasesearch(semi-implicit)

Tools:WordNet,DBpedia

Entity and Aspect Extraction for Organizing News Comments · analysis and lexical database search....

Documents

Transcript of Entity and Aspect Extraction for Organizing News Comments · analysis and lexical database search....