Sp-ToBI (Tones and break indices) Sp-ToBI DIME-Sp ToBI

1

El Proyecto DIME, DCC-IIMAS, UNAM 2006

Sp-ToBI (Tones and break indices) DIME-Sp ToBI2006

� Spanish-ToBI� Five levels

� DIME Labeling

� Labeling experiment

� DAMSL and Intonation

� Conclusions


Sp-ToBI

� Spanish Tones and break indices (Sp-ToBI)� Mary Beckman (1999)

� Referential system to transcribe intonational and prosodic information of any Spanish dialect

� Researchers may use it for :

� Unify linguistic vocabulary

� Make the analysis easier with a phonological transcription

� Label different corpus and develop databases

� Based upon ToBi which is based on the Metric and Auto

Segmental Model


Metric and AutosegmentalModel (AM)

� Phonological analysis intended to identify contrastable elements from an intonational system whose combination produces melodic contours founded on the possible elocutions of language.

� Autosegmental phonology theory assumes that the prosodic level of representation (i.e. melody or tunes) is independent of the segmental and


AM Model

� It is characterized by tones anchored to

the tonic syllables.

� Tones could be :

�Phrase (H high, L Low, HL, LH) (Pitch)

�Boundary (H%, L%)


Sp- ToBI Five levels

Syllables

Words

BI

Tones

Code

Miscelaneous


DIME- ToBI labeling

Analyze if the DIME corpus can be labeled

with Sp-ToBI

�Pick up as much information as possible

�Analyze speech phenomena (natural speech)

�Align the labels and the audio files (time)

�Find the best computational tool that gives the needed information

2


Sosa’s example


DIME’s exampleBefore, Before that, could you move the air extractor

above the, the stove


Problems with Sp-ToBI

� DIME Corpus needs to describe real

language

� Laboratory and controlled examples in

other studies provide ideal intonation paterns

� DIME corpus presents several

spontaneous speech phenomena that need to be solved for tone labeling.


Tools


Tools

� MES (Motif editor for speech-signals) )

� FO without interpretation problems (stops, unvoiced, pauses)

� Good visual alignment


DIME Corpus characteristics:

�Spontaneous speech phenomena:

� pauses

� speech repairs

� repetitions

� interjections

�Non “ideal” recording quality

�Different utts length (1 word to 20 words)

3


Labeling experiment

� Two phases: 1. Tone recognition:

� Preliminary study

� New tones proposal

� DIME conventions

2. Tagging validation: � Manual writing

� Tone labeling by three different taggers

� Analysis of results


Phase I

� Descriptive study of 117 utterance

dialogue (d12) using 5 level (Allophones, allophonic syllables, words (orthographic),

break Indices and tones) in order to describe the DIME corpus and find out

possible solutions.


Allophones, syllables and words

Alophones

Phonetic Sylables

Words


Break Indices

� Break indices

�Original

� “0” vowel coalescence (synalefa)

� “1” ordinary interword juncture

� “4” Intonational phrase, different tonal phrase

�DIME Proposal

� “2” Interruption point of speech repairs

� “3” Mid tones (eg.enumerations)


Break indices (2)

Could you move the, the sink a little toward, toward to the minibar?

2


Break Indices (3)Cuboards, shelves, stoves and air extractors,sinks and diswashers, cabinets, tables, refrigerators,

chairs,

3 3 3 3 33 3

4


Tones1. Phrase

� L+H* � L*+H

� H+L*

� H*+L� H*

� L*

2. Boundary� H%� L%

3. Middle� H-

� L-


DIME Tones

Could you show me the sink and dishwasher catalogue?

Middle

L-

Boundary

H%

Phrase

L+H*

4


Tones1. Phrase

� L+H*

� L*+H� H+L*

� H*+L

� H*� L*

2. Boundary� H%

� L%

3. Middle� H-� L-


Tone asignment

L*+H

Could you show me the shelves ?


TonesPhrase tones

� Original:� L+H* : Accented peak after a valley

� L*+H: Stress followed by a peak

� H+L*: Accented valley after a peak

� H*: high-tone stress

� DIME Proposal:� H*+L: Accented peak followed by a valley

� L*: low-tone stress


Original tones

Accented peak after a valley Stress followed by a peak

Accented valley after a peak High-tone stress

5


DIME proposal

Accented peak followed by a valley Low-tone stress


TonesPhrase tones

� Original:

�L*+H: Stress followed by a peak

� L+H* : Accented peak after a valley



� DIME Proposal:

� H*+L: Accented peak followed by a valley



Example (L*+H)

L*+H L*+H

¿Is that all right?


TonesPhrase tones

� Original:� L*+H: Stress followed by a peak

�L+H* : Accented peak after a valley



� DIME Proposal:� H*+L: Accented peak followed by a valley



Example (L+H*)

L+H*

To the other side of the air extractor


TonesPhrase tones

� Original:



�H+L*: Accented valley after a peak


� DIME Proposal:



6


Example (H+L*)

H+L*

This, right to the stove


TonesPhrase tones

� Original:




�H*: high-tone stress

� DIME Proposal:




Example (H*)

H*H*


TonesPhrase tones

� Original:





� DIME Proposal:

�H*+L: Accented peak followed by a valley



Examples (H*+L)

H*+L

This?


TonesPhrase tones

� Original:





� DIME Proposal:


�L*: low-tone stress

7


Example (L*)

L*

Yes


Tones1. Phrase

� L+H* � L*+H

� H+L*

� H*+L� H*

� L*


3. Middle� H-

� L-


Boundary tones

H%

� F0 rise, after any pitch accent.

L%

� F0 fall, with regard to the last stress.

H%

L%


Boundary H% tone

H%

¿Where do you want me to put it?


Boundary L% tone

L%

To the other side of the air extractor


Tones1. Phrase

� L+H* � L*+H

� H+L*

� H*+L� H*

� L*


3. Middle� H-

� L-

8


Mid tones

� H or L tone preceded with a “-”

� Shows a break in the tonal phrase, that

changes the normal intonation.

� L- o H- :

�Enlargements

�Audible changes


Mid tone (enlargment)Could you show me the sink and dishwasher catalogue?

H-


Mid tone (Change)

Would you like me to move or bring any object tothe kitchen?

L-


DIME conventions

Break Indices

0

1

4

Break Indices:

0

1

4

2

3

Tagging layers:

syllables

words

BI

tones

miscelaneous

Tagging layers:

alophones

syllables

words

BI

tones.

BeckmanDIME


DIME conventions

Tones:

Phrase:

L*+H

L+H*

H+L*

L*

Boundary:

L%

H%

Tones:

Phrase:

L+H*

L*+H

H+L*

H*+L

L*

H*

Boundary:

L% H%

Mid:

L- H-


Phase II

1. Write a Manual of DIME-ToBI principles,

based on the found results of the first phase of the experiment.

2. Train three different labelers the DIME-ToBIconventions.

3. Tone labeling of a dialogue (4 layers were

done before)

4. Measure agreement between the three

taggers.

9


Agreement between taggers.

� Parameters considered:

�Speech phenomena

�Total agreement percentajes between taggers

�Toneme match (last tone ands boundary

tone) according to the tonal intention.


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment

One intonational phrase

Intonational phrases

Speech phenomena


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment



Speech phenomena


Monosyllable

L%L%L%

L*

L*

L*

Yes


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment



Speech phenomena


Bisyllable

Here

H+L*L+H*

L*+H

10


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment



Speech phenomena


Speech repair (Repetition)

L+H*

L+H*

L+H*

L%

L%

L%

By, by the stove


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment



Speech phenomena


Interjection

Eh, mh, below the window

H-

H-

NO


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment



Speech phenomena


Enlargments

What i want now is a dishwasher machine

L-L-

L-

11


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment



Speech phenomena



H+L*

H+L*

H+L*


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment



Speech phenomena


Intonational phrasesArriba, arriba de este mueble…

L+H*

L+H*

L+H*

L*+H

L*+H

L*+H

L%

L%

L%

va_7 ri_7


Intonational phrases…del mueble que acabamos de rotar noventa grados

L%

L%

L%


Speech phenomena:

Monosyllables

Bisylables

Speech repair

Interjection

Enlargment



Speech phenomena

12


Miscelaneous

L*+H L- L*+H

L*+H H- L+H*

L*+H L*+¡H

Puedo, puedo mover, eh, el es que quiero…





�Toneme match (last tone ands boundary tone) according to the tonal intention.


Total agreement percentajesbetween taggers

0

20

40

60

80

100

Labeling agreement percentaje

Between 100% and80%

Less than 80%


Total tone agreement

0

10

20

30

40

50

60

Agreement %

Less tan 60%

From 70% to80%

From 80% to90%

From 90% to100%





�Toneme match (last tone ands boundary tone) according to the tonal intention.


Tonemes. Single word

MatchesAgreementCasesUtt

L+H* H- H%60%5Interrogative

H+L* L- L%

H+L* L- L%

60%21“Okey“

L* L- L%85%28Declarative

13


Tonemes. DeclarativesMatchesAgreementCasesUtt

H+L* L- L%

L* L- L%

L+H* L- L%

85%20With pauses

( + 12 tones)

H+L* L- L%L+H* L- L%

H+L* L- L%

48%29Withoutpauses


Toneme. Interrogatives

L+H* H- H%

L*+H H- H%

91%12Interrogative

(Check)

MatchesAgreementCasesUtt

L+H* H-H%54%11Is that allright?

L+H* H- H%80%10Interrogative

(Genuine)


E.g. “Where do you want me to

place it?”

utt75 : dónde quieres que la

ponga ? L*+H H+L* L+H* H- H%

L+H* H+L* L+H* H- H%

L*+H H+L* L+H* H- H%

utt90 : s: dónde quieres que lo

ponga ? L*+H H+L* L- L+H* H- H%

L*+H H+L* L- L+H* H- H%

L*+H H+L* L- L+H* H- H%

utt129: s: dónde quieres que lo

ponga ? L*+H H+L* L- L+H* H- H%

L*+H H+L* L+H* H- H%

L*+H H+L* L- L+H* H- H%


E.g “Is that all right?”

utt44 : <ruido> <sil> <ruido> <ruido>

<sil> <ruido> <sil> ahí está bien ? + + L* !L* L*+H H- H%

L* !L* L+H* H- H%

L* !L* L*+H L- H%

utt59 : ahí está bien ? L*+H L+H* H- H%

L*+H L+H* H- H%

L*+H L+H* H- H%

utt69 : s: ahí está bien ? + + L* !L* L*+H H- H%

L* !L* L+H* H- H%

L* !L* L*+H H- H%


E.g. “Okey”

utt58 : s: okey H*+L L- L%

H*+L L- L%

H*+L L- L%

utt62 : s: okey H+L* L- L%

H+L* L- L%

H+L* L- L%

utt67 : s: okey H+L* L- L%

H+L* L- L%

H+L* L- L%


DAMSL and intonation

� Original hypothesis

�There is a relation between intonation andDAMSL tagging* and it will be possible to find

behavior paterns between labels.

�The same toneme matches with the same

dialogue act.

14


Action directives

L%L-H+!L*H+L*

L%L-H+!L*H+L*

L%L-H+!L*H+L*utt81 : ahora quiero un estante

L%L-H+L*H-

L%L-L*H-

L%L-L*+!HH-utt120: y <sil> ahora quiero otro gabinete

H%H-L*+HL*

H%H-L+H*!L*

H%H-L*+HH+L*utt25 : ahora quiero un refrigerador

L%L-H*+LL*

L%L-!L*L*

L%L-!H*+LH*+Lutt14 : <ruido> mm mh <no-vocal> <sil> ahora

quiero una campana


Action directives

L%L-H+!L*H+L***** utt102: u: dame la blanca superior doble

L%L-H+!L*H+L*utt81 : ahora quiero un estante

L%L-H+L*H-

L%L-L*H-

L%L-L*+!HH-utt120: y <sil> ahora quiero otro gabinete

H%H-L*+HL*

H%H-L+H*!L*

H%H-L*+HH+L*utt25 : ahora quiero un refrigerador

L%L-H*+LL*

L%L-!L*L*

L%L-!H*+LH*+Lutt14 : <ruido> mm mh <no-vocal> <sil> ahora quiero una campana


Information request

H%H-L*+H!L*

H%H-L+H*!L*

H%H-L*+H!L*utt94 : s: ahí está bien ? + +

L%L-H+L*L*+H

L%L-H+L*L*+H

L%L-H+L*L*+Hutt102: quieres que <sil> sustituya <sil> este mueble por éste ?

H%H-L+H*

H%H-L+H*

H%H-L+H*utt28 : s: éste ?

H%H-L+H*H+L*

H%H-L+H*H+L*

H%H-L+H*H+L*utt49 : s: dónde quieres que la ponga ?


Information request

H%H-L*+HH+!L****utt101: s: quieres alguno en particular?

H%H-L+H*!L*

H%H-L*+H!L*utt94 : s: ahí está bien ? + +

L%L-H+L*L*+Hutt102: quieres que <sil> sustituya <sil> este

mueble por éste ?

H%H-L+H*utt28 : s: éste ?

H%H-L+H*H+L*utt49 : s: dónde quieres que la ponga ?


Commit vs. Acknowledge

L%L-H+L*utt22 : s: okey

L%L-H+L*

L%L-H+L*

L%L-H+L*

L%L-H*+L

L%L-L+H*utt134: s: okey

L%L-H*+L

L%L-H*+L

L%L-H*+Lutt32 : s: okey

L%L-H*+L

L%L-H*+L

L%L-H*+Lutt11 : s: okey


Commit vs. Acknowledge

Acknowledge

L%L-H+L***utt135: s: okey

utt22 : s: okey

L%L-H+L*Commit

L%L-H+L*

L%L-H*+LAcknowkedge

L%L-L+H*utt134: s: okey

L%L-H*+LCommit

utt32 : s: okey

L%L-H*+LCommit

utt11 : s: okey

15


Conclusions

� Most popular tones are L*+H and H+L*

� Mostly declarative intonation is related

to L- L%, and interrogative to H- H%

� Before performing deep research

involving more than two dialogues thetone set proposal could be validated.


Conclusions

� DIME labeling has a good agreement levelbetween taggers, which means thatconventions are understandable.

� The relation between agreement betweentaggers and dialogue acts is notrepresentative.

� No pattern between DAMSL anotation andToBI labeling has been found so far.

� The relation between tonal intention andtones has not been found so far.

Sp-ToBI (Tones and break indices) Sp-ToBI DIME-Sp ToBI

Documents

Transcript of Sp-ToBI (Tones and break indices) Sp-ToBI DIME-Sp ToBI