An Incremental Parser for Abstract Meaning...
32
An Incremental Parser for Abstract Meaning Representation 1 Marco Damonte 1 Shay B. Cohen 2 Giorgio Satta 1 University of Edinburgh 2 University of Padua EACL 2017 1 / 25
Transcript of An Incremental Parser for Abstract Meaning...
An Incremental Parser for Abstract MeaningRepresentation
1 Marco Damonte 1 Shay B. Cohen 2 Giorgio Satta
1 University of Edinburgh2 University of Padua
EACL 2017
1 / 25
AMR
He described her as a genius
describe-01
he genius she
ARG0 ARG1ARG2
2 / 25
Dependency trees
• Transition-based dependency parsing (Nivre, 2004)
My dog also likes eating
root
poss
nsubj
advmodxcomp
3 / 25
Concept identification
The teacher
person
teach-01
ARG0-of
The proposal
thing
propose-01
ARG1-of
10 January 1989
date-entity
101
1989
daymonth
year
4 / 25
Reentrancy
I beg you to excuse me
beg-01
iyou excuse-01
ARG0ARG1 ARG2
ARG0
ARG1
5 / 25
Reentrancy
I beg you to excuse me
beg-01
iyou excuse-01
ARG0ARG1 ARG2
ARG0
ARG1
6 / 25
Reentrancy
I beg you to excuse me
beg-01
iyou excuse-01
ARG0ARG1 ARG2
ARG0
ARG1
7 / 25
Transition-based AMR Parser
8 / 25
Transition system
The boy wants to believe the girl
STACK
boy
GRAPH
9 / 25
Transition system
The boy wants to believe the girl
STACK
want-01
boy
GRAPH
want-01
boy
10 / 25
Transition system
The boy wants to believe the girl
STACK
want-01
GRAPH
want-01
boy
11 / 25
Transition system
The boy wants to believe the girl
STACK
want-01
believe-01
GRAPH
want-01
boy believe-01
12 / 25
Transition system
The boy wants to believe the girl
STACK
want-01
believe-01
girl
GRAPH
want-01
boy believe-01
girl
13 / 25
Transition system
The boy wants to believe the girl
STACK
want-01
believe-01
GRAPH
want-01
boy believe-01
girl
14 / 25
Transition system
The boy wants to believe the girl
STACK
want-01
believe-01
GRAPH
want-01
boy believe-01
girl
15 / 25
Transition system
The boy wants to believe the girl
STACK
want-01
GRAPH
want-01
boy believe-01
girl
16 / 25
Transition system
The boy wants to believe the girl
STACK GRAPH
want-01
boy believe-01
girl
17 / 25
Oracle
• Given the current configuration (σ, β,A) and the gold-standardgraph G = (Vg ,Ag ):
T (G , σ, β,A) =
LARC(`)RARC(`)RED-REENT(`)REDUCESHIFT
• (English, AMR) ⇒ Transitions to obtain AMR* from English
18 / 25
Evaluation
19 / 25
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
Fine-grained evaluation (cont’d)
• Concepts. F-score on the concept identification task
• Negations. F-score on :polarity roles
• Named Entities. F-score on :name roles
• Wikification. F-score on :wiki roles
21 / 25
Fine-grained evaluation (cont’d)
• Concepts. F-score on the concept identification task
• Negations. F-score on :polarity roles
• Named Entities. F-score on :name roles
• Wikification. F-score on :wiki roles
21 / 25
Fine-grained evaluation (cont’d)
• Concepts. F-score on the concept identification task
• Negations. F-score on :polarity roles
• Named Entities. F-score on :name roles
• Wikification. F-score on :wiki roles
21 / 25
Fine-grained evaluation (cont’d)
• Concepts. F-score on the concept identification task
• Negations. F-score on :polarity roles
• Named Entities. F-score on :name roles
• Wikification. F-score on :wiki roles
21 / 25
Experiments
Metric JAMR (’14) CAMR JAMR (’16) OursSmatch 58 63 67 64Unlabeled 61 69 69 69No WSD 58 64 68 65NP-only 47 54 58 55Reentrancy 38 41 42 41Concepts 79 80 83 83Named Ent. 75 75 79 83Wikification 0 0 75 64Negations 16 18 45 48SRL 55 60 60 56
JAMR: Flanigan et al. (2014)
CAMR: Wang et al. (2015)
22 / 25
Software
• Online demo:http://cohort.inf.ed.ac.uk/amreager.html
• Source code for parser:https://github.com/mdtux89/amr-eager
• Source code for evaluation:https://github.com/mdtux89/amr-evaluation
23 / 25
Demo
24 / 25
Conclusions
• AMREager is a linear-time, left-to-right transition system
• AMR parsing akin to dependency parsing
• Fine-grained evaluation suite to assess AMR parsers
25 / 25
