Prosody & Information Structure

Discourse & Dialogue

CS 359

October 11, 2001

Discussion Questions

• In Hiyakumoto et al, the prevalence of H* and L+H* accents in the BNRC is used to justify the use of these tunes for rheme and theme? Is this actually justified?

• The authors also observe that WordNet only encodes a small set of relationships. Can we just ignore the others?

• Are there any systems that implement contrastive stress? If not, why not?

• How do systems handle other types of “accent”? Different languages?

Roadmap

• Motivation– Synthetic speech– Experimental evidence

• Key components– Prosody– Syntax

• Contextually “appropriate” speech synthesis

Speech Synthesis

• Generally INTELLIGIBLE– But not NATURAL– Requires high attention to listen to

• “Default” sentence intonation– May be misleading– Speaking of BILL,

• A) JOHN thought he would WIN, but he DIDN’T

• B) JOHN thought he would WIN, but HE didn’t

Accent Assignment: Analysis

• Accent: – Increased loudness, duration, pitch movement

• Basic view:– “available”/Given: no accent; New(er): accent

• Attend to new information

• Questions: – Does accent continue to decrease with repetition?– How does discourse “structure” affect accent?

Accent Assignment: Results

• “Topic” status & First/Later mention vs– De-/Accenting, form of referring expression

• Results:– First,+Topic: Accented, Full NP

• Later,+Topic: De-accented , probably pronoun• Later,+Topic,+Refinement: Accented (even Pron)

– First,-Topic: Accented Full NP• Later,-Topic: Accented Full NP, Implicit• Later,-Topic,+past-topic/+contrast: Accented NP (mod)

“ToBI” Intonation Framework

• ToBI: Tone and Break Indices

• Describe English sentence intonation

• Tones:– Two pitch levels: H(igh) and L(ow)

• * - on stressed syllable, e.g. H*, L*, L+H*

– Types: Pitch accents, Phrase Tones (L-,L%)• Last accent in phrase = ‘nuclear’ accent

– Units: Intermediate and Intonational Phrases

“ToBI” Intonation Framework

• Break indices– Mark groupings in speech– 0 - most closely linked; 5 - most disjoint– 4 = Intermediate phrase boundary (-)

• ~ comma

– 5 = Intonational phrase boundary (%,$)• ~ period - sentence

ToBI Examples

Contrast Examples

Contrast Examples

Contrast Examples

Syntax & Information Status

• Intonation units more flexible than standard syntactic constituents, e.g. subject, predicate

• CCG - Combinatory Categorial Grammar– Allows multiple analyses (parses) to fit– Link syntactic, semantic, and

pragmatic/prosodic function with each unit– Lavage=H*L-L%; NP:lavage’;u:rheme

Generating Appropriate Intonation

• Basic “previous mention” strategy– Accent first mention of content words– De-accent closed class function words– De-accent content words already mentioned

• Inadequate– Need contrastive stress TOO

Generating Appropriate Intonation

• Identify theme (topic: links to previous info)

• Identify rheme (contributes new information)

• Shared propositional content

• Assign appropriate basic intonation contour– rheme:H* L-L%; – theme:L+H* L-H% (at most)

Generating Appropriate Intonation

• Identify focus element in theme/rheme– Word to get accent

• Focus– First mention, and– Contrastive

• What is contrastive????

Contrastive Items: Domain

• For each entity x:– 0: find alternatives in discourse and KB

– 1: RSET= x and alternatives,

– PROPS= features of x

– CSET= features of x to mark for contrast

– 2: For each p in PROPS, r in RSET, • IF p is not property of r, add p to CSET.

– 3: Focus p of x

• E.g. She broke her left LEG, NOT her RIGHT leg.

Contrastive Items: WordNet• WordNet: Semantic KB

– 4 parts of speech: N,V,Adj, Adv

– Category/word: one or more synonym sets

– Hierarchies linked by relations: e.g. IS-A

• Content Word W is new if NOT: – In focus history or history’s equivalence class

• Equiv. Class: reachable by N hypernym/synset links

• Content Word W is contrastive if:– In history’s contrast list

• Contrast: hyponyms of hypernyms of W

Examples

• The X4 is a SOLID-state AMPLIFIER• L+H*L- H* H* L- L$• The X5 is a TUBE amplifier.• L+H*L- H* L-L$• It COSTS EIGHT HUNDRED DOLLARS,• H* H* H* H* L-H%• IT costs NINE hundred dollars,• L+H*L- H* L-H%

Summary

• Assigns contextually based intonation– Uses given/new information status– Extended to fine-grained contrastive status

• Identifies contrast based on– Knowledge base if available– WordNet Lexical DB for greater generality

Conclusions

• Theme/Rheme identification difficult

• Contrast/Similarity measures for WordNet– Still oversimplified

• Evaluation: How do you tell if it’s right?– Many alternatives

• Incorporate in larger discourse structure– Discourse segments, plans, ….