The prosodic marking of the contrast between

restrictive and appositive clause in Dutch

Vincent J. van Heuven

With the help of:Crit Cremers, Hanna Gauvin, Constantijn Kaland, Eddin Najetovic,

Marjoleine Sloos, Hanna de Vries

Linguistics Program, Universiteit Leiden

Towards the phonetics of the relative clause in

Dutch

Vincent J. van Heuven

With the help of:Crit Cremers, Hanna Gauvin, Constantijn Kaland, Eddin Najetovic,

Marjoleine Sloos, Hanna de Vries

Linguistics Program, Universiteit Leiden

1-6-2007 van Heuven, NVFW/MPI 3

Introduction

Framework: improving prosody of text-to-speech (TTS) systems

Our parser/generator Delilah computes rich syntactic and semantic structures

Can be interfaced to standard TTS systems for Dutch

Opportunities for phonetic optimalisation of output

1-6-2007 van Heuven, NVFW/MPI 4

Introduction

Grammatical distinction in relative clause (1) Restrictive clause

Within scope of antecedentRel. pronoun is not subject of relative clauseNot preceded by comma in textSuggests relatively shallow prosodic boundary

E.g. No journalist [who signed the petition], was arrested.

1-6-2007 van Heuven, NVFW/MPI 5

Introduction

Grammatical distinction in relative clause (2) Appositive clause

Not within scope of antecedentRel. pronoun is subject of relative clausePreceded by comma in text inputSuggests relatively deep prosodic boundary,

E.g.Michael, [who beat up his girl friend], was arrested.

1-6-2007 van Heuven, NVFW/MPI 6

Introduction

Questions How is the contrast between restrictive and

appositive clause coded in the prosody? Does the listener associate a specific

prosodic structure with each type of clause?

Does correct prosodic marking yield better evaluation scores of text-to-speech system?

1-6-2007 van Heuven, NVFW/MPI 7

Prosodic boundaries

The deeper a prosodic boundary, ... ... the longer the physical silence

(pause) immediately before the boundary

... the stronger the domain-final lengthening

... the more disruptive the intonation pattern

1-6-2007 van Heuven, NVFW/MPI 8

Approach (1)

Perceptual evaluation of prosodic marking of Synthetised speech with restrictive vs.

appositive clauses With / without pause With / without domain-final

lengthening Four different melodies

1-6-2007 van Heuven, NVFW/MPI 9

Approach (2)

Four melodies at utterance-internal boundary

Cohesion marker H*L

No/neutral marker H*+L

Moderate break H* %L

Strong break H*L H% %L

1-6-2007 van Heuven, NVFW/MPI 10

Approach (3)

Two sentence structures that force just one interpretation onto relative clause Restrictive (with downward quantifiers)

No journalist who signed the petition, was arrested Few students who are are fraternity members, dislike a

beer Appositive (with proper names and/or evidentials)

Michael, who beat up is girl friend, was arrested The woman, who – by the way – is wearing a pink dress,

is from Germany

1-6-2007 van Heuven, NVFW/MPI 11

Approach (4)

Judgment/evaluation task for listeners How do you rate the way the speaker

reads the following sentence? Scale from 0 to 10

0: very poor/inappropriate 10: perfect

1-6-2007 van Heuven, NVFW/MPI 12

Hypotheses

Appositive clause correlates with Physical silence Pre-final lengthening Maximally disruptive melodic pattern

Strong break > Moderate break > Neutral > Cohesion

Three factors are additive i.e. better scores as more factors reinforce

one another

1-6-2007 van Heuven, NVFW/MPI 13

Hypotheses

Restrictive clause correlates with Absence of physical silence Absence of domain-final lengthening Minimally disruptive melody

Cohesion > Neutral > Moderate break > Strong break

Factors are additive

1-6-2007 van Heuven, NVFW/MPI 14

Methods (1)

Factors Appositive vs. restrictive clause Two lexically different instantiations With / without pause (of 200 ms) With / without domain-final lengthening

(+40%) Four different melodies

Total 2 x 2 x 2 x 2 x 4 = 64 types Presented twice = 128 tokens per listener

1-6-2007 van Heuven, NVFW/MPI 15

Methods (2)

Materials spoken by native speaker of Dutch

Digital tape splicing for removal/insertion of lengthening and pause

PSOLA for duration manipulation and import of computer-generated melodies (imported from Fluency text-to-speech)

1-6-2007 van Heuven, NVFW/MPI 16

Methods (2)

Listeners 20 native speakers of Dutch

Presentation through loudspeakers 11-point quality judgment scale Demonstration of sound files:

GEEN journalist die de verklaring onderTEkend had, werd OPgepakt

‘No journalist who the statement signed had, was arrested’

No pause, +pause,

No length +length

Disaster: the red condition (strong break) was inadvertently generated as the blue one (neutral). It will therefore be absent from the results section.

My student Constantijn Kaland is now re-running the complete experiment for his BA thesis.

1-6-2007 van Heuven, NVFW/MPI 18

Results

First: main effects Yes / no physical pause Yes / no domain-final lengthening Type of melodic boundary marking

Second: interactions i.e. breaking factors down

Restrictive Appositive

Type of relative clause

0

1

2

3

4

5

6

7

Ap

pro

pri

ate

ne

ss (

0 .

.. 1

0)

66

5

6

PauseNo

Yes

Effect of physical pause

Absence of pause is preferred for restrictive clause

Presence of pause is disfavored for restrictive clause

Difference is significant Both presence and

absence of pause are OK for appositive clause

Restrictive Appositive

Type of relative clause

0

1

2

3

4

5

6

7

Ap

pro

pri

ate

ne

ss (

0 .

.. 1

0) 6 6

66

LengtheningNo

Yes

Effect of domain-final lengthening

Tiny effects in predicted direction

No lengthening preferred for restrictive clause

Lengthening preferred for appositive clause

Effects are statistically insignificant

restrictive appositive

Type of relative clause

0

1

2

3

4

5

6

7

Mea

n Ju

dged

app

ropr

iate

ness

(0.

.10

Melodycohesion: H*L

neutral: H*+L

moderate break: H* %L

Effect of melodic configuration

Order of conditions reasonably in line with prediction

Indeed: green > yellow > red for restrictive clause

But should be red > yellow > green for appositive clause

Effects are statistically insignificant

1-6-2007 van Heuven, NVFW/MPI 22

Summary of results so far

Only one effect Listeners prefer restrictive clauses

without a physical pause separating clause from antecedent

All other prosodic markings are OK for both types of clause

Now let us look at interactions

1-6-2007 van Heuven, NVFW/MPI 23

Results: interactions

Effects of (i) lengthening and (ii) melody evaluated separately for presence vs. absence of physical pause

Restrictive Appositive

Type of relative clause

0

1

2

3

4

5

6

7

Ap

pro

pri

ate

ne

ss (

0 .

.. 1

0)

LengtheningYes

No

Physical pause absent

Restrictive Appositive

Type of relative clause

0

1

2

3

4

5

6

7

Ap

pro

pri

ate

ne

ss (

0 .

.. 1

0)

LengtheningYes

No

Physical pause present

Interaction of pause and lengthening

Effect of lengthening as predicted (though not always significant) More clearly when pause is absent than when present

restrictive appositive

Type of relative clause

0

1

2

3

4

5

6

7

Jud

ge

d a

pp

rop

ria

ten

ess

(0

..1

0

Melodycohesion: H*L

neutral: H*+L

moderate break: H* %L

restrictive appositive

Type of relative clause

0

1

2

3

4

5

6

7

Jud

ge

d a

pp

rop

ria

ten

ess

(0

..1

0

Melodycohesion: H*L

neutral: H*+L

moderate break: H* %L

Interaction of pause and melody

Preference for cohesive H*L with restrictive clause But only when pause is absent

Physical pause absent Physical pause present

1-6-2007 van Heuven, NVFW/MPI 26

Summary of results

Effects of (i) lengthening and (ii) melodic configuration are as predicted

But only if pause is absent Indicates hierarchical structure

physical pause is the primary cue lengthening and melody are subsidiary

cues

1-6-2007 van Heuven, NVFW/MPI 27

Conclusions

Appositive and restrictive relative clauses in Dutch are marked by different prosodic

boundaries, which are implemented by different

phonetic cues More specifically, …

1-6-2007 van Heuven, NVFW/MPI 28

Conclusions

… the evaluation study shows that the syntax-prosody interface should assign different prosodic boundaries to restrictive and appositive clauses: deeper boundary to appositive clause shallower boundary to restrictive clause

Therefore, …

1-6-2007 van Heuven, NVFW/MPI 29

Discussion

… quality of (Dutch) text-to-speech systems can be improved Never insert physical pause before

restrictive clause Optionally accompany by appropriate

secondary cuesDo not lengthen syllable before restrictive

clauseUse cohesive melody before restrictive clause

1-6-2007 van Heuven, NVFW/MPI 30

Discussion

But Implementation of secondary cues is not very

effective Fast and dirty implementation requires

proper use of pause only Rerun needed with strong prosodic break Challenge left for computational

linguists: How to compute status of relative clause

from text input?

1-6-2007 van Heuven, NVFW/MPI 31

The end

Thank you for your attention