Statistical Models for Frame-Semantic Parsingmiriamp/fillmore-tribute/slides/Dipanjan_Das.pdf ·...

Statistical Models for Frame-Semantic Parsing

Dipanjan Das*

GoogleFrame Semantics in NLP: A Workshop in Honor of Chuck Fillmore

June 27, 2014

*Thanks to Desai Chen, Kuzman Ganchev, Karl Moritz Hermann, André Martins, Nathan Schneider, Noah Smith and Jason Weston

Frame-Semantic Parsing

I want to travel to Baltimore on Sunday

I want to travel to on Sunday

TRAVEL Encodes an event or scenario

Baltimore

TRAVEL Participant or role for the

frameTravelerTime

GoalBaltimore

TRAVEL Participant or role for the

frameTravelerTime

GoalBaltimore

DESIRING

Experiencer Event

Outline

• Why should we build statistical models for frame semantics?

• Overview of statistical methods

• Future directions

Outline

Motivation

Deeper understanding beyond syntax

Motivation

Grouping of predicates and argumentsinto clusters

Motivation

Grounding of natural language to an ontology

Grouping of predicates and argumentsinto clusters

Motivation

Bengal ’s massive stock of food was reduced to nothing

Motivation

Bengal ’s massive stock of food was reduced to nothingN X A N ADP N V V ADP N

Store or financial entity?

Motivation

Store of what?Of what size?Whose store?

Motivation

Store of what?Of what size?Whose store?

Motivation

What was reduced?To what?

Motivation

Bengal ’s massive stock of food wasreducedto nothingN X A N ADP N V V ADP N

What was reduced?To what?

Motivation

Bengal ’s massive stock of food wasreducedto nothingN X A N ADP N V V ADP N

STOREPossessor

DescResource

CAUSE_CHANGE_OF_POSITION_ON_A_SCALE

Value_2Item

Motivation

James Cameron directed Titanic

BEHIND_THE_SCENES

Artist Production

Motivation

BEHIND_THE_SCENES

Artist Production

James Cameron is Titanic’s director

BEHIND_THE_SCENES

ProductionArtist

Motivation

BEHIND_THE_SCENES

Artist Production

BEHIND_THE_SCENES

ProductionArtist

Grouping across syntactic categories

Motivation

James Cameron filmed Titanic

BEHIND_THE_SCENES

Artist Production

BEHIND_THE_SCENES

ProductionArtist

Grouping across different lemmas

Motivation

film.01

director.01

ARG2ARG0

ARG1from

PropBank

fromNomBank

Motivation

film.01

director.01

ARG2ARG0

ARG1from

PropBank

fromNomBank

Hard to align different lemmas across resources

Motivation

film.01

director.01

ARG2ARG0

ARG1from

PropBank

fromNomBank

Very generic argument labels

Motivation

film.01

director.01

ARG2ARG0

ARG1from

PropBank

fromNomBank

Argument labels do not match up

Motivation

James Cameron filmed Titanic in 1997

BEHIND_THE_SCENES

ArtistProduction

Motivation

BEHIND_THE_SCENES

ArtistProduction

�e.filmed.arg1(e, James Cameron)^filmed.arg2(e,Titanic)^filmed.in(e, 1997)

�e.filmed.arg1(e, /m/03 gd) ^ filmed.arg2(e, /m/0dr 4) ^ filmed.in(e, 1997)

Motivation

BEHIND_THE_SCENES

ArtistProduction

�e.filmed.arg1(e, James Cameron)^filmed.arg2(e,Titanic)^filmed.in(e, 1997)

Motivation�e.filmed.arg1(e, James Cameron)^filmed.arg2(e,Titanic)^filmed.in(e, 1997)

�e.BEHIND THE SCENES.Artist(e, /m/03 gd)^BEHIND THE SCENES.Production(e, /m/0dr 4)^BEHIND THE SCENES.Time(e, 1997)

FrameNet

Applications ofFrame-Semantic Parsing

Stance Classification

• Frame Semantics for Stance ClassificationHasan and Ng (CoNLL 2013)

• Two sided debates in an online forum

• Classification of stance

• Improvement over a baseline that uses bag of words and dependencies

Dialog Systems

• Unsupervised Induction and Filling of Semantic Slots for Spoken Dialogue Systems Using Frame-Semantic ParsingChen, Wang and Rudnicky (ASRU 2013)

• Annotation of dialog transcripts with frame-semantic structures

• Uses only a subset of frames

• Uses these annotations for slot induction

Stock Price Movement

• Semantic Frames to Predict Stock Price MovementXie et al. (ACL 2013)

• Predict the change in stock price from financial news

• Lots of features along with features based on frames and roles

• Shows improvements over other features

Summarization

• Generating Automated Meeting SummariesThomas Kleinbauer (PhD thesis, Saarland University)

• Part of a large system for generating meeting summaries

Outline

Structure of Lexicon and Data

PLACINGAgentCause

ThemeArea

PLACINGAgentCause

ThemeArea

PLACINGAgentCause

ThemeArea

PLACINGAgentCause

ThemeArea

core roles

non-core roles

PLACINGAgentCause

ThemeArea

excludes relationship

core roles

non-core roles

PLACINGAgentCause

ThemeArea

excludes relationship

core roles

non-core roles

archive.V, arrange.V, bag.V, bestow.V bin.V

predicates

PLACINGAgentCause

ThemeArea

DISPERSAL

AgentCause

IndividualsDistance

TRANSITIVE_ACTIONAgentCause

Patient

EventPlace

INSTALLINGAgent

Component

Fixed_locationArea

STORINGAgentLocation

ThemeArea

STOREPossessorResource

SupplyDescriptor

PLACINGAgentCause

ThemeArea

DISPERSAL

AgentCause

IndividualsDistance

Patient

EventPlace

INSTALLINGAgent

Component

Fixed_locationArea

ThemeArea

SupplyDescriptor

PLACINGAgentCause

ThemeArea

DISPERSAL

AgentCause

IndividualsDistance

Patient

EventPlace

INSTALLINGAgent

Component

Fixed_locationArea

ThemeArea

SupplyDescriptor

Structure of Lexicon and Datainheritance

used by

Datasets

Benchmark Dataset(SemEval 2007)

665 frames720 role labels

8.4K unique predicate types

Training set:2.2K sentences

11.2K predicate tokens

Test set:120 sentences

1. 1K predicate tokens

LTH Frame-Semantic ParserJohansson and Nugues (2007)

Frame Identification Argument Filtering Argument Labeling

ambiguous

Find the best among all frames

Taken from an SVM classifier trained on

ambiguous predicates

To increase coverage, potential predicates were extracted from WordNet and automatically frame labels were selected for

Frame Identification Accuracy

F-Measure

Argument Filtering

massive stock

Argument Filtering

massive stock

Bengal ’s Bengal massive stock of food

food massive Bengal ’s massive to nothing

to of ’s

Potential Arguments

Argument Filtering

massive stock

to of ’s

Potential ArgumentsBinary SVM Classification

Argument Filtering

massive stock

Multiclass SVM Classification

Argument Filtering

massive stock

to of ’s

Potential ArgumentsMulticlass SVM Classification

Argument Filtering

massive stock

to of ’s

Potential ArgumentsMulticlass SVM Classification

Possessor Resource

Descriptorø

Full Frame-Semantic Structure Prediction

F-Measure

SEMAFORDas, Chen, Martins, Schneider, Smith (2014)

Frame Identification Argument Identification

SEMAFOR: Frame Identification

Logistic regression with a latent variable

Predicates evoking a frame in supervised data, e.g.cargo.N, inventory.N, reserve.N, stockpile.N, store.N, supply.N

evoke STORE

STORE stock.N

stockpile.N

STORE stock.N

stockpile.N

LexSem = {synonym}

STORE stock.N

stockpile.N

LexSem = {synonym}

If STORE

stockpile.Nsynonym LexSem

STORE stock.N

stockpile.N

LexSem = {synonym}

If STORE

stockpile.Nsynonym LexSem

comes from WordNet!

Datasets

New Data(FrameNet 1.5, 2010)

Datasets

ResultsBenchmark

LTH SEMAFOR

6157.3

F-Measure

New Data

log-linear

ResultsBenchmark

LTH SEMAFOR

6157.3

F-Measure

New Data

log-linear

auto predicates

ResultsBenchmark

LTH SEMAFOR

6157.3

F-Measure

New Data

log-linear

auto predicates

SEMAFOR

Accuracy

log-linear

gold predicates

All Predicates

Accuracy

Unknown Predicates

Accuracy

Frame Identification

SEMAFOR: Handling Unknown Predicates

Knowledge of only 9,263 predicates in supervised data

However, English has lot more potential predicates(~65,000 in newswire English)

Knowledge of only 9,263 predicates in supervised data

However, English has lot more potential predicates(~65,000 in newswire English)

Lexicon expansion using graph-based semi-supervised learning

How can label propagation help?

Example Graph

Seed predicates

Example Graph

Seed predicatesUnseen predicates

Example Graph

Graph Propagation

Example Graph

Graph Propagation

Example Graph

Graph Propagation

Continues till convergence...

Supervised Self-Training Graph-Based

18.923.1

Accuracy

SEMAFOR: Unknown Predicates

SEMAFOR: Argument Identification

Possessor

Resource

Supply

Descriptor

Possessor

Resource

Supply

Descriptor

Bengal ’s

Bengal

massive stock

of food

massive

Bengal ’s massive

massive stock

Possessor

Resource

Supply

Descriptor

Bengal ’s

Bengal

massive stock

of food

massive

Bengal ’s massive

massive stock

Possessor

Resource

Supply

Descriptor

Bengal ’s

Bengal

massive stock

of food

massive

Bengal ’s massive

massive stock

Violates overlap constraints

Other types of structural constraints

PLACINGAgentCause

ThemeArea

Mutual exclusion constraint

PLACINGAgentCause

ThemeArea

If an agent places something, there cannot be a cause role in the sentence

PLACINGAgentCause

ThemeArea

The waiter placed food on the table.

In Kabul, hauling water put food on the table.Cause

SIMILARITYDimension

Differentiating_fact

Entity_1

Entity_2

Degree

Requires constraint

difference.N, resemble.V,

unliike.A, vary.V

SIMILARITYDimension

Entity_1

Entity_2

Degree

Requires constraint

unliike.A, vary.V

A mulberry resembles a loganberry.

second entity

first entity

SIMILARITYDimension

Entity_1

Entity_2

Degree

Requires constraint

unliike.A, vary.V

A mulberry resembles.!

Possessor

Resource

Supply

Descriptor

Bengal ’s

Bengal

massive stock

of food

massive

Bengal ’s massive

massive stock

A constrained optimization

problem

Possessor

Resource

Supply

Descriptor

Bengal ’s

Bengal

massive stock

of food

massive

Bengal ’s massive

massive stock

Possessor

Resource

Supply

Descriptor

Bengal ’s

Bengal

massive stock

of food

massive

Bengal ’s massive

massive stock

A constrained optimization problem

a binary variable for each role, span tuple

a binary vector for all role, span tuples

Uniqueness

Prevents overlap

more structural constraints

An integer linear program (ILP)

Punyakanok, Roth and Yih

(2008)

Often, very slow solutions

Fast ILP solvers proprietary

Dual Decomposition

SEMAFOR: Frame-Semantic Parsing

Final Results

Benchmark

LTH SEMAFOR

46.542.0

F-Measure

New Data

Final Results

Benchmark

LTH SEMAFOR

46.542.0

F-Measure

New Data

auto predicates

Final Results

Benchmark

LTH SEMAFOR

46.542.0

F-Measure

New Data

auto predicates

SEMAFOR

F-Measure

gold predicates

Hermann, Das, Weston and GanchevACL 2014

Frame Identification with Embeddings

Discrete lexical features

Syntactic context words connected via a set of

dependency paths

; massiveprep pobj ; food

nsubjpass auxpass ; was

... ; ...

;prep pobj ;

nsubjpass auxpass ;

Replace context words with off-the-shelf word

embeddings

... ; ... ... ; ...

Frame Identification with Embeddingsamod

;prep pobj ;

nsubjpass auxpass ;

... ; ... ... ; ...

amod nsubj dobj>prep>pobjp<nsubjpass>auxpass...

Input sparse embedding vector with context blocks

Frame instance space

N ⇥ Rd

Joint SpaceRm

N ⇥ Rd

Joint SpaceRm

Frame Instance MapM : Rd ! Rm

N ⇥ Rd

Joint SpaceRm

Set of FrameNet labels

STORESTORING

STINGINESS...

N ⇥ Rd

Joint SpaceRm

STORESTORING

STINGINESS...

Y 2 RF⇥mLabel matrix

N ⇥ Rd

Joint SpaceRm

STORESTORING

STINGINESS...

Y 2 RF⇥mLabel matrix

learned learned

frame instances

frame instancesframe labels

test predicate

Supervised Self-Training Graph-Based Embeddings

46.1542.7

18.923.1

Accuracy

Results on Unknown Predicates

SEMAFOR

Supervised Self-Training Graph-Based Embeddings

83.682.3

Accuracy

Results on All Predicates

SEMAFOR

Supervised Graph-Based Embeddings

64.564.1

F-Score

Final Results

SEMAFOR

Outline

Better Models

• Very little research on argument identification

• More non-local features

• Using distributed representations

• Generalization using PropBank resources

Data• Number of FrameNet annotated sentences ~30

times less than PropBank/Ontonotes.

• Number of argument labels ~30 times more

• To make systems usable, we need annotations

• inter-annotator agreement studies

• annotation guidelines127

Custom FrameNets

• Is a general FrameNet lexicon useful?

• often, FrameNet frames are too general or too specific

• is it possible to quickly build customized FrameNet lexicons for applications?

• is it possible to use PropBank-style frames to induce FrameNet-style frames?

Conclusions

• Lot of exciting work in predicate-argument structure prediction

• Semi-supervised methods improve coverage

• Systems trained on small amounts of FrameNet-style data shown to be useful

• More annotations will result in usable systems

Thank You

Statistical Models for Frame-Semantic Parsingmiriamp/fillmore-tribute/slides/Dipanjan_Das.pdf ·...

Documents

Transcript of Statistical Models for Frame-Semantic Parsingmiriamp/fillmore-tribute/slides/Dipanjan_Das.pdf ·...

NESSTAR: A Semantic Web Application for Statistical Data and Metadata

Enriching Parallel Corpora for Statistical Machine ...dekai/ssst6/slides/WetzelBond_Ssst2012_slides.pdf · Enriching Parallel Corpora for Statistical Machine Translation with Semantic

A frame semantic overview of NLP-based information ...

Marco Rospocher · 2020. 3. 2. · • Semantic Sentiment Analysis Award (Task 3 - Frame entities Identi cation) at the Semantic Web Evaluation of the 12th European Semantic Web Conference

Statistical Motion Information Extraction and ... · 1 Statistical Motion Information Extraction and Representation for Semantic Video Analysis Georgios Th. Papadopoulos, Student

Latent Semantic Indexing Debapriyo Majumdar Information Retrieval – Spring 2015 Indian Statistical Institute Kolkata.

Korean Semantic Role Labeling Using Korean PropBank Frame ...onlinepresent.org/proceedings/vol142_2016/15.pdf · Korean Semantic Role Labeling Using Korean PropBank Frame Files ...

semantic field of the basis of Frame

semantic transformation framework to enable statistical research on medical summaries a thesis

Frame Semantic Control of the Coordinate Structure - George Lakoff

Parallel Search Through Statistical Semantic Spaces ...ceur-ws.org/Vol-1243/paper2.pdfParallel Search Through Statistical Semantic Spaces Leveraging Linked Open Data Alexey Cheptsov

A New Statistical and Verbal-Semantic Approach to Pattern ...

On the statistical sensitivity of semantic similarity metrics

The Large-Scale Structure of Semantic Networks ...The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth Mark Steyversa, Joshua B. Tenenbaumb

Semantic Frame-Based Document Representation …hanj.cs.illinois.edu/pdf/icdm13_hkim.pdfSemantic Frame-Based Document Representation for Comparable Corpora Hyungsul Kim, Xiang Ren,

Corpus-based Induction of an LFG Syntax-Semantics Interface for Frame Semantic Processing

Semantic Retrieval Interface for Statistical Research Dataceur-ws.org/Vol-1091/paper9.pdf · 2013-10-22 · Semantic Retrieval Interface for Statistical Research Data Daniel Bahls,

PASSIVE VIDEO FORGERY DETECTION USING FRAME …studentsrepo.um.edu.my/7137/1/PhD_Thesis_Aminu_Mustapha_Bagiwa... · passive video forgery detection using frame correlation statistical

A Common Processing and Statistical Frame for Label … · Oksana Riba Grognuz: Label-Free Quantitative Proteomics i A Common Processing and Statistical Frame for Label-Free Quantitative

DOES IT MATTER IF STATISTICAL AGENCIES FRAME THE … · Does It Matter If Statistical Agencies Frame the Month’s CPI Reporton a 1-Month or 12-month Basis? Jeffrey A. Frankel and