The prosodic marking of the contrast between restrictive and appositive clause in Dutch Vincent J....

The prosodic marking of the contrast between

restrictive and appositive clause in Dutch

Vincent J. van Heuven

With the help of:Crit Cremers, Hanna Gauvin, Constantijn Kaland, Eddin Najetovic,

Marjoleine Sloos, Hanna de Vries

Linguistics Program, Universiteit Leiden

Towards the phonetics of the relative clause in

Dutch

Vincent J. van Heuven

With the help of:Crit Cremers, Hanna Gauvin, Constantijn Kaland, Eddin Najetovic,

Marjoleine Sloos, Hanna de Vries

Linguistics Program, Universiteit Leiden

1-6-2007 van Heuven, NVFW/MPI 3

Introduction

Framework: improving prosody of text-to-speech (TTS) systems

Our parser/generator Delilah computes rich syntactic and semantic structures

Can be interfaced to standard TTS systems for Dutch

Opportunities for phonetic optimalisation of output


Introduction

Grammatical distinction in relative clause (1) Restrictive clause

Within scope of antecedentRel. pronoun is not subject of relative clauseNot preceded by comma in textSuggests relatively shallow prosodic boundary

E.g. No journalist [who signed the petition], was arrested.


Introduction

Grammatical distinction in relative clause (2) Appositive clause

Not within scope of antecedentRel. pronoun is subject of relative clausePreceded by comma in text inputSuggests relatively deep prosodic boundary,

E.g.Michael, [who beat up his girl friend], was arrested.


Introduction

Questions How is the contrast between restrictive and

appositive clause coded in the prosody? Does the listener associate a specific

prosodic structure with each type of clause?

Does correct prosodic marking yield better evaluation scores of text-to-speech system?


Prosodic boundaries

The deeper a prosodic boundary, ... ... the longer the physical silence

(pause) immediately before the boundary

... the stronger the domain-final lengthening

... the more disruptive the intonation pattern


Approach (1)

Perceptual evaluation of prosodic marking of Synthetised speech with restrictive vs.

appositive clauses With / without pause With / without domain-final

lengthening Four different melodies


Approach (2)

Four melodies at utterance-internal boundary

Cohesion marker H*L

No/neutral marker H*+L

Moderate break H* %L

Strong break H*L H% %L


Approach (3)

Two sentence structures that force just one interpretation onto relative clause Restrictive (with downward quantifiers)

No journalist who signed the petition, was arrested Few students who are are fraternity members, dislike a

beer Appositive (with proper names and/or evidentials)

Michael, who beat up is girl friend, was arrested The woman, who – by the way – is wearing a pink dress,

is from Germany


Approach (4)

Judgment/evaluation task for listeners How do you rate the way the speaker

reads the following sentence? Scale from 0 to 10

0: very poor/inappropriate 10: perfect


Hypotheses

Appositive clause correlates with Physical silence Pre-final lengthening Maximally disruptive melodic pattern

Strong break > Moderate break > Neutral > Cohesion

Three factors are additive i.e. better scores as more factors reinforce

one another


Hypotheses

Restrictive clause correlates with Absence of physical silence Absence of domain-final lengthening Minimally disruptive melody

Cohesion > Neutral > Moderate break > Strong break

Factors are additive


Methods (1)

Factors Appositive vs. restrictive clause Two lexically different instantiations With / without pause (of 200 ms) With / without domain-final lengthening

(+40%) Four different melodies

Total 2 x 2 x 2 x 2 x 4 = 64 types Presented twice = 128 tokens per listener


Methods (2)

Materials spoken by native speaker of Dutch

Digital tape splicing for removal/insertion of lengthening and pause

PSOLA for duration manipulation and import of computer-generated melodies (imported from Fluency text-to-speech)


Methods (2)

Listeners 20 native speakers of Dutch

Presentation through loudspeakers 11-point quality judgment scale Demonstration of sound files:

GEEN journalist die de verklaring onderTEkend had, werd OPgepakt

‘No journalist who the statement signed had, was arrested’

No pause, +pause,

No length +length

Disaster: the red condition (strong break) was inadvertently generated as the blue one (neutral). It will therefore be absent from the results section.

My student Constantijn Kaland is now re-running the complete experiment for his BA thesis.


Results

First: main effects Yes / no physical pause Yes / no domain-final lengthening Type of melodic boundary marking

Second: interactions i.e. breaking factors down

Restrictive Appositive

Type of relative clause

0

1

2

3

4

5

6

7

Ap

pro

pri

ate

ne

ss (

0 .

.. 1

0)

66

5

6

PauseNo

Yes

Effect of physical pause

Absence of pause is preferred for restrictive clause

Presence of pause is disfavored for restrictive clause

Difference is significant Both presence and

absence of pause are OK for appositive clause



0

1

2

3

4

5

6

7

Ap

pro

pri

ate

ne

ss (

0 .

.. 1

0) 6 6

66

LengtheningNo

Yes

Effect of domain-final lengthening

Tiny effects in predicted direction

No lengthening preferred for restrictive clause

Lengthening preferred for appositive clause

Effects are statistically insignificant

restrictive appositive


0

1

2

3

4

5

6

7

Mea

n Ju

dged

app

ropr

iate

ness

(0.

.10

Melodycohesion: H*L

neutral: H*+L

moderate break: H* %L

Effect of melodic configuration

Order of conditions reasonably in line with prediction

Indeed: green > yellow > red for restrictive clause

But should be red > yellow > green for appositive clause

Effects are statistically insignificant


Summary of results so far

Only one effect Listeners prefer restrictive clauses

without a physical pause separating clause from antecedent

All other prosodic markings are OK for both types of clause

Now let us look at interactions


Results: interactions

Effects of (i) lengthening and (ii) melody evaluated separately for presence vs. absence of physical pause



0

1

2

3

4

5

6

7

Ap

pro

pri

ate

ne

ss (

0 .

.. 1

0)

LengtheningYes

No

Physical pause absent



0

1

2

3

4

5

6

7

Ap

pro

pri

ate

ne

ss (

0 .

.. 1

0)

LengtheningYes

No

Physical pause present

Interaction of pause and lengthening

Effect of lengthening as predicted (though not always significant) More clearly when pause is absent than when present



0

1

2

3

4

5

6

7

Jud

ge

d a

pp

rop

ria

ten

ess

(0

..1

0

Melodycohesion: H*L

neutral: H*+L




0

1

2

3

4

5

6

7

Jud

ge

d a

pp

rop

ria

ten

ess

(0

..1

0

Melodycohesion: H*L

neutral: H*+L


Interaction of pause and melody

Preference for cohesive H*L with restrictive clause But only when pause is absent

Physical pause absent Physical pause present


Summary of results

Effects of (i) lengthening and (ii) melodic configuration are as predicted

But only if pause is absent Indicates hierarchical structure

physical pause is the primary cue lengthening and melody are subsidiary

cues


Conclusions

Appositive and restrictive relative clauses in Dutch are marked by different prosodic

boundaries, which are implemented by different

phonetic cues More specifically, …


Conclusions

… the evaluation study shows that the syntax-prosody interface should assign different prosodic boundaries to restrictive and appositive clauses: deeper boundary to appositive clause shallower boundary to restrictive clause

Therefore, …


Discussion

… quality of (Dutch) text-to-speech systems can be improved Never insert physical pause before

restrictive clause Optionally accompany by appropriate

secondary cuesDo not lengthen syllable before restrictive

clauseUse cohesive melody before restrictive clause


Discussion

But Implementation of secondary cues is not very

effective Fast and dirty implementation requires

proper use of pause only Rerun needed with strong prosodic break Challenge left for computational

linguists: How to compute status of relative clause

from text input?


The end

Thank you for your attention

The prosodic marking of the contrast between restrictive and appositive clause in Dutch Vincent J....

Documents

Transcript of The prosodic marking of the contrast between restrictive and appositive clause in Dutch Vincent J....