Centering theory and its direct applications

Lecture 2

Some definitions

Discourse = coherent sequence of utterances

Several sentences following one another do not make a readable text

Defining specific computable measures of coherence is the goal of this seminar

Centering theory ingredients

Deals with local coherenceWhat happens to the flow from

sentence to sentenceDoes not deal with global structuring

of the text (paragraphs/segments) Defines coherence as an estimate of

the processing load required to “understand” the text

Processing load

Upon hearing a sentence a personCognitive effort to interpret the

expressions in the utteranceIntegrates the meaning of the

utterance with that of the previous sentence

Creates some expectations on what might come next

Example

(1) John met his friend Mary today.

(2) He was surprised to see her.

(3) He thought she is still in Italy.

Form of referring expressions Anaphora needs to be resolved “Create” a discourse entity at first mention

with full noun phrase Creating expectations

Creating and meeting expectations(1) a. John went to his favorite music store to buy a

piano. b. He had frequented the store for many years. c. He was excited that he could finally buy a piano. d. He arrived just as the store was closing for the day.

(2) a. John went to his favorite music store to buy a piano.

b. It was a store John had frequented for many years. c. He was excited that he could finally buy a piano. d. It was closing just as John arrived.

Interpreting pronouns

a. Terry really goofs sometimes.

b. Yesterday was a beautiful day and he was excited about trying out his new sailboat.

c. He wanted Tony to join him on a sailing expedition.

d. He called him at 6am.

e. He was sick and furious at being woken up so early.

Basic center definitions

Centers of an utteranceSet of entities serving to link that

utterance to the other utterances in the discourse segment that contains it

Not words or phrases themselvesSemantic interpretations of noun

phraes

Types of centers

Forward looking centers An ordered set of entities What could we expect to hear about next Ordered by salience as determined by grammatical

function Subject > Indirect object > Object > Others

John gave the textbook to Mary. Cf = {John, Mary, textbook}

Preferred center Cp

The highest ranked forward looking center High expectation that the next utterance in the

segment will be about Cp

Backward looking center

Single backward looking center, Cb (U)For each utterance other than the

segment-initial one The backward looking center of

utterance Un+1 connects with one of the forward looking centers of Un

Cb (U+1) is the most highly ranked element from Cf (Un) that is also realized in U+1

Centering transitions ordering

Cb(Un+1)=Cb(Un) OR

Cb(Un)=[?]

Cb(Un+1) != Cb(Un)

Cb(Un+1) = Cp (Un+1) continue smooth-shift

Cb(Un+1) != Cp (Un+1) retain rough-shift

Centering constraints

There is precisely one backward-looking center Cb(Un)

Cb(Un+1) is the highest-ranked element of Cf(Un) that is realized in Un+1

Centering rules

If some element of Cf(Un) is realized as a pronoun in Un+1 then so is Cb(Un+1)

Transitions not equalcontinue > retain > smooth-shift >

rough-shift

Centering analysis

Terry really goofs sometimes. Cf={Terry}, Cb=?, undef

Yesterday was a beautiful day and he was excited about trying out his new sailboat. Cf={Terry,sailboat}, Cb=Terry, continue

He wanted Tony to join him in a sailing expedition. Cf={Terry, Tony, expedition}, Cb=Terry, continue

He called him at 6am. Cf={Terry,Tony}, Cb=Terry, continue

He called him at 6am. Cf={Terry,Tony}, Cb=Terry, continue

Tony was sick and furious at being woken up so early. Cf={Tony}, Cb=Tony, smooth shift

He told Terry to get lost and hung up. Cf={Tony,Terry}, Cb=Tony, continue

Of course, Terry hadn’t intended to upset Tony. Cf={Terry,Tony}, Cb = Tony, retain

Rough shifts in evaluation of writing skills One of the graders of student essays in standardized tests is an

automatic program

ETS researchers have developed a number of applications that use natural language processing technologies to evaluate and score the writing abilities of test takers:

The CriterionSM Online Essay Evaluation Service automatically evaluates essay responses using e-rater and the Critique writing analysis tools.

E-rater® gives holistic scores for essays.

CritiqueTM provides real-time feedback about grammar, usage, mechanics and style, and organization and development.

C-raterTM offers automated analysis of conceptual information in short-answer, free responses.

E-rater features

Syntactic variety Represented by features that quantify the

occurrence of clause types Clear transitions

Cue phrases in certain syntactic constructions

Existence of main and supporting points Appropriateness of the vocabulary content

of the essay What about local coherence?

Ranking forward looking centers

Subject > Indirect object > Object > Others > Quantified indefinite subjects (people,

everyone) > Arbitrary plural pronominals

Essay score model

Human score available E-rater prediction available Percentage of rough-shifts in each

essay: analysis done manually

Negative correlation between the human score and the percentage of rough-shifts

Karamanis’07

Why are we reading this paper? Gives quite complete list of references on

later work on centering• Centering variants

Reminds that entity coherence is not the only factor in text flow

• We’ll be discussing rhetorical structure theory during the next class

Applications---can some aspects of the work be done differently/improved upon?

Information ordering task

Given a set of sentences/clauses, what is the best presentation? Take a newspaper article and jumble the

sentences---the result will be much more difficult to read than the original

Criteria for deciding which of two orderings is better Centering would definitely be applicable

Summarization, question answering, generation

Linear multi-factor regression Approximate the human score as a linear

function of the e-rater prediction and the percentage of rough-shifts

Adding rough shifts significantly improves the model of the score 0.5 improvement on 1—6 scale

How easy/difficult would it be to fully automate the rough-shift variable

Centering variations

Continuity (NOCB=lack of continuity) Cf(Un) and Cf(Un+1) share at least one

element Coherence

Cb(Un) = Cb(Un+1) Salience

The Cb(U) = Cp(U) Coherence is more important than salience Cheapness (fulfilled expectations)

Cb (Un+1) = Cp(Un)

GNOME corpus

20 descriptions of museum artifacts Split into finite unites (clauses) Semi-automatic centering annotation

Item 144 is a torc. Its present arrangement, twisted into three rings, may be a modern alteration; it should probably be a single ring, worn around the neck. The terminals are in the form of goats’ heads.

Rhetorical coherence

Each text can be seen as a hierarchical tree structure

Different spans are related by some rhetorical relation Elaboration (adding more information) Contrast Sequence Purpose Summary etc

Local rhetorical coherence

Applies only locally rather than on the text as a whole Signaled by cue phrases Contrast: but, however, on the other hand Continuation: and, then, later Consequence: because, in order to, so

These local rhetorical relations structure the text

When missing, entity coherence determines the flow 8 out of the 20 texts do not have any explicitly marked

rhetorical relations

Joint centering and local rhetorical coherence In clauses directly marked for a rhetorical

relation Merge the Cf lists of the two clauses

Apply centering transitions on the resulting Cf list rather than the original

GNOME-RR contains 1.58 fewer CF lists compared to the original average number (8.35)

Metrics of coherence

M.NOCB (no continuity) M.CHEAP (expectations not met) M.KP sum of the violations of

continuity, cheapness, coherence and salience

M. BFP seeks to maximize transitions according to Rule 2

Experimental methodology

Gold-standard ordering The original order of the text (object

description, news article) Assume that other orderings are inferior

Classification error rate Percentage orderings that score better than

the gold-standard + 0.5*percentage of the orderings that score the same

Results

NOCB gives best resultsSignificantly better than the other

metrics M.BFP is the second best metric

Adding the local rhetorical relations hurts performance---is this surprising?

Reminders

Select topics you would like to present Should schedule next week now The second time you present one of the

goals will be to relate the papers with previous topics we have covered

Start thinking about the topic of your literature overview About 15 papers 5/6 pages Due Nov 12