An informatics perspective on argumentation mining - SICSA 2014-07-09
-
Upload
jodischneider -
Category
Documents
-
view
368 -
download
1
description
Transcript of An informatics perspective on argumentation mining - SICSA 2014-07-09
![Page 1: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/1.jpg)
Evidence Informatics: An Informatics Perspective on Argumentation
Mining
Jodi Schneider
SICSA Workshop on Argument Mining: Perspectives from Information Extraction, Information Retrieval and Computational LinguisticsDundee, Scotland9-10 July 2014.
![Page 2: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/2.jpg)
Argumentation mining today
• No unified vision of the field. Multiple:– Interrelated problems– Application domains– Tools handling one aspect of annotation
• Few corpora• Need for– Common definition(s) of argumentation– "Challenge problems"– Shared corpora– Applications
![Page 3: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/3.jpg)
Argumentation mining today
• No unified vision of the field. Multiple:– Interrelated problems– Application domains– Tools handling one aspect of annotation
• Few corpora• Need for– Common definition(s) of argumentation– "Challenge problems"– Shared corpora– Applications
![Page 4: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/4.jpg)
Evidence Informatics
• How do we support people in making evidence-based decisions?
• This includes supporting their arguments!
![Page 5: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/5.jpg)
Informatics Perspective
1. Analyze requirements2. Consider which argumentation models to use3. Build a prototype support tool4. Evaluate and iterate
![Page 6: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/6.jpg)
Two examples of evidence informatics
• Evidence about article quality in Wikipedia (PhD)
• Evidence about pharmaceutical drug interactions (ArgDiaP 2014)
![Page 7: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/7.jpg)
http://purl.org/jsphd
![Page 8: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/8.jpg)
Which content belongs in Wikipedia?
![Page 9: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/9.jpg)
Tasks using evidence & arguments
• Convince others of your position, using community norms
• Determine the overall consensus decision
![Page 10: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/10.jpg)
Compare two argumentation theories
• Walton’s Argumentation Schemes (Walton, Reed, and Macagno 2008)
– Informal argumentation (philosophical & computational argumentation)
– Identify & prevent errors in reasoning (fallacies)– 60 patterns
• Factors/Dimensions Analysis (Ashley 1991; Bench-Capon and Rissland, 2001)
– Case-based reasoning– E.g. factors for deciding cases in trade secret law,
favoring either party (the plaintiff or the defendant).
![Page 11: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/11.jpg)
Walton’s Argumentation Schemes
Example Argumentation Scheme: Argument from Rules – “we apply rule X”
Critical Questions1. Does the rule require carrying out this type of action?
2. Are there other established rules that might conflict with or override this one?
3. Are there extenuating circumstances or an excuse for noncompliance?
Walton, Reed, and Macagno 2008
![Page 12: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/12.jpg)
“Rule” Argumentation Scheme
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
![Page 13: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/13.jpg)
“Evidence” Argumentation Scheme
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
![Page 14: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/14.jpg)
Evidence + Rule -> Conclusion
“Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups” CSCW 2013
![Page 15: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/15.jpg)
Supporting Tasks with Walton
• Convince others of your position, using community norms– To win an argument, use popular schemes:
• Argument from Evidence to Hypothesis (19%)• Argument from Rules (17%)
• Determine the overall consensus decision– Ask critical questions to check others' arguments
![Page 16: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/16.jpg)
Example factors analysis (Aleven 1997)
Aleven 1997
![Page 17: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/17.jpg)
Wikipedia Factors AnalysisFactor Example (used to justify 'keep') Example (used to justify 'delete'Notability Anyone covered by another
encyclopedic reference is considered notable enough for inclusion in Wikipedia.
There is simply no coverage in reliable sources to establish notability.
Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band.
There are no independent secondary sources (books, magazine articles, documentaries, etc.) about her.
Maintenance …this article is savable but at its current state, needs a lot of improvement.
Too soon for a page likely to be littered with rumour and speculation.
Bias It is by no means spam (it does not promote the products).
The article seems to have been created by her or her agent as a promotional device.
**Other I'm advocating a blanket "hangon" for all articles on newly-drafted players
it appears to be original research by synthesis
Deletion Discussions in Wikipedia: Decision Factors and Outcomes. WikiSym 2012.
![Page 18: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/18.jpg)
Supporting Tasks with Factors
• Convince others of your position, using community norms– To win an argument, talk about the right topics
• Notability, Sources, Maintenance, Bias
• Determine the overall consensus decision– Group messages by factor– summarize prevalence
![Page 19: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/19.jpg)
Factor-based Summarization
![Page 20: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/20.jpg)
Argument Schemes vs. Factors?
• Argument SchemesDetails of how to put together an argument– Could support WRITING detailed arguments– Critical Questioning
• FactorsTopics of discussion– Basic support for writing arguments– Summarization supports decision-making
![Page 21: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/21.jpg)
Modeling Arguments in Scientific Papers to Support Pharmacists
Jodi Schneider, Carol Collins, Lisa E Hines, John R Horn, and Richard Boyce
12th Argumentation, Dialogue, Persuasion conference (ArgDiaP 2014) Warsaw, Poland2014-05-25
![Page 22: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/22.jpg)
Goal: Support evidence-based updates to drug-interaction reference DBs
• Make sense of the EVIDENCE– New clinical trials– Adverse drug event reports– Drug product labels– Updates to regulatory
information (U.S. FDA,…)– …
• Significant discrepancies between different drug-interaction reference DBs
http://jama.jamanetwork.com/article.aspx?articleid=183454
![Page 23: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/23.jpg)
Drug Interaction Knowledge Base (DIKB) - Boyce 2007-2009
– Hand-constructed knowledge base– Safety issues when 2 drugs are taken together– Focus is on EVIDENCE
![Page 24: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/24.jpg)
• Evidence
![Page 25: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/25.jpg)
Micropublications
![Page 26: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/26.jpg)
Direct Annotation with Domeo
http://swan.mindinformatics.org/ Paolo N Ciccarese
![Page 27: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/27.jpg)
Textual quotes
![Page 28: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/28.jpg)
Micropublication: Claim + Support (e.g. Attribution)
Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical CommunicationsTim Clark, Paolo N. Ciccarese, Carole A. Goblehttp://arxiv.org/abs/1305.3506
![Page 29: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/29.jpg)
Constructs claim-argument network across scientific papers
Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical CommunicationsTim Clark, Paolo N. Ciccarese, Carole A. Goblehttp://arxiv.org/abs/1305.3506
![Page 30: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/30.jpg)
Model Data, Methods, Materials, References
Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical CommunicationsTim Clark, Paolo N. Ciccarese, Carole A. Goblehttp://arxiv.org/abs/1305.3506
![Page 31: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/31.jpg)
Micropublications Ontology
Micropublications: a Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical CommunicationsTim Clark, Paolo N. Ciccarese, Carole A. Goblehttp://arxiv.org/abs/1305.3506
![Page 32: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/32.jpg)
Methods
Methods section of challenge graph
![Page 33: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/33.jpg)
"escitalopram does not inhibit CYP2D6"
Support graph Challenge graph
![Page 34: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/34.jpg)
From individual documents to a searchable claim-argument network
• "Pay as you go" annotation of source documents with Domeo & Micropublications
• Generates claim-argument network– Supports & challenges– Materials, methods, data– Quotes linked into the graph– … within & across documents
• Query support
![Page 35: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/35.jpg)
Argumentation Mining papersArguing on Wikipedia • “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc
Online Task Groups” CSCW 2013.• “Deletion Discussions in Wikipedia: Decision Factors and Outcomes” WikiSym2012.Arguing in Social Media• “Dimensions of Argumentation in Social Media" EKAW 2012• “Why did they post that argument? Communicative intentions of Web 2.0 arguments.” Arguing on
the Web 2.0 at ISSA 2014Arguing in Reviews• “Identifying Consumers' Arguments in Text” SWAIE 2012• “Semi-Automated Argumentative Analysis of Online Product Reviews" COMMA 2012• “Arguing from a Point of View” Agreement Technologies 2012Structuring Arguments on the Social Semantic Web• “A Review of Argumentation for the Social Semantic Web” Semantic Web – Interoperability, Usability,
Applicability, 2013.• “Identifying, Annotating, and Filtering Arguments and Opinions in Open Collaboration Systems" 2013
Thesis: purl.org/jsphd• “Modeling Arguments in Scientific Papers” at ArgDiaP 2014
http://jodischneider.com/jodi.html
![Page 36: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/36.jpg)
![Page 37: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/37.jpg)
Example: "Stop at a red light"
1. Does the rule require carrying out this type of action?Were you driving a vehicle?
2. Are there other established rules that might conflict with or override this one?Did a police officer direct you to continue without stopping?
3. Are there extenuating circumstances or an excuse for noncompliance?Were you driving an ambulance with its siren on?
Critical Questions from Argument from Rules based on Walton, Reed, and Macagno 2008
![Page 38: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/38.jpg)
How to win an argument (Arucaria)?
Classifying Arguments by Scheme. Vanessa Wei Feng. Master's thesis, Toronto, 2010.
![Page 39: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/39.jpg)
Argumentation mining could be the basis for support tools
• Help participants write persuasive arguments– How: provide personalized feedback on drafts– Requires: knowing which arguments are accepted;
identifying argumentation in a drafts • Find weaknesses in others’ arguments– How: suggest & instantiate relevant critical questions– Requires: identifying argumentation schemes
• Summarize the overall conclusions of the debate– How: identify the winning and losing rationales– Requires: identifying rationales and contradictions
![Page 40: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/40.jpg)
Experts vs. Novices
• Experts were more likely to use – Argument from Precedent
• Novices were more likely to use– Argumentation from Values– Argumentation from Cause to Effect– Argument from Analogy
![Page 41: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/41.jpg)
Unsuccessful arguments from novices
• Emsworth Cricket Club is one of the oldest cricket clubs in the world, and this really is worth a mention. Especially on a website, where pointless people … gets a mention.
• Why just because it is a small team and not major does it not deserve it’s (sic) own page on here?
![Page 42: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/42.jpg)
Newcomers don't understand how to counterargue
![Page 43: An informatics perspective on argumentation mining - SICSA 2014-07-09](https://reader036.fdocuments.in/reader036/viewer/2022062702/554b7418b4c90551118b5575/html5/thumbnails/43.jpg)
Wikipedia Factors Analysis
Factors determined by iterative annotation
4 Factors cover– 91% of comments– 70% of discussions
“Other” as 5th catchall
Factor Example (used to justify `keep')
Notability Anyone covered by another encyclopedic reference is considered notable enough for inclusion in Wikipedia.
Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band.
Maintenance …this article is savable but at its current state, needs a lot of improvement.
Bias It is by no means spam (it does not promote the products).
**Other I'm advocating a blanket "hangon" for all articles on newly-drafted players