Prosody as Design, Outcome and Confounding variable in...
Transcript of Prosody as Design, Outcome and Confounding variable in...
Prosody as Design, Outcome and
Confounding variable in
Speech & Language Pathology
Toni Rietveld
Dept. of Linguistics, Radboud University Nijmegen (NL) and
Sint Maartenskliniek, Nijmegen
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010 2
OUTLINE of the talk
1. Definition of prosody, the concepts of Design, Confounding
and Outcome variable and short description of three
pathologies which will be discussed in relation with prosody:
aphasia, dysarthria and hearing loss (CI).
2. Common misunderstandings on prosody.
3. Examples 1: Prosody as design variable: Melodic Intonation
Therapy for aphasia and Intonation as design variable for
perception research with CI-users
4. Examples 2: Prosody as confounding and outcome variables
in dysarthria.
5. Conclusions.
1. Some definitions:
Prosody: encompasses all characteristics of speech
which are not relevant for the recognition of
individual speech segments:
Pitch (intonation), Intensity, Temporal structure.
Speech and Language Pathology (3 pathologies relevant in this talk):
Dysarthria: a neurogenetic disturbance in speech, which results in
articulatory/phonatory deficiencies, like problems with
the realization of stop consonants, reduction of the
vowel space, monotony, nasality. Dysarthria can be the
result of a CVA, a trauma or M. Parkinson.
Aphasia: a neurogenetic language disturbance, resulting in
a variety of language problems: agrammatism,
wordfinding, language comprehension etc.
3Rietveld: Prosody and Speech & Language Pathology; Lecture at the
University of Cambridge, October 19, 2010
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
4
Hearing loss: severe hearing loss is often
compensated by a Cochlear Implant
(CI), a device that functions as a
„bypass‟ of the middle and inner ear.
2 = send coil
3 = receiver
4 = electrodes array
5 = auditory nerve
uitwendig oor = outer ear
middenoor = middle ear
binnenoor = inner ear
Variables: Design, Outcome and Confounding
1) Design variable:
a variable which is under the control of the experimenter;
for instance „singing‟ in the MIT (Melodic Intonation
Therapy) for Aphasia.
2) Outcome variable:
a variable which is the result of a pathology (for instance
M. Parkinson: monotony) or a therapy (for instance
restoration of „normal‟ intonation).
5Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
3) Confounding variable:
a variable which may obscure results of a therapy;
example: goal of therapy is pitch lowering and
increasing Intensity: a variant of the Silverman Therapy
for dysarthria).
If, for instance, before therapy words are spoken as if
in one sentence, and after therapy as a list of short
utterances, quite strong pitch decreasing effects can
be expected (see later).
6Rietveld: Prosody and Speech & Language Pathology; Lecture at
the University of Cambridge, October 19, 2010
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
7
Prosody can be measured in different ways:
1) On the basis of stochastic measures:
mean F0, SDF0, mean duration, SDdur, a Variability Index
(VI), mean intensity, SDintensity, ranges of measures,
maximum values of measures etc.
2) Qualitatively:
For intonation: in terms of symbols used in autosegmental
phonology: H, H*, L, L* etc.
For intensity and tempo: in terms of subjective scale
values assigned to utterances.
2. Common misunderstandings on prosody:
Confusion between stress and (sentence) accent
„permit vs. per‟mit (N vs. V)
- in sentences (I HAven‟t got a „permit; I DON‟t per‟mit..)
permit shows (word) stress, either on the 1st or 2nd
syllable; NO accent here; duration strongest cue.
- as single-word sentences they are accented (by
means of pitch movements).
Role of intensity in stress and sentence accent; perceptually
quasi irrelevant.
Funny thoughts on relation between high
pitch and sentence accent: nobody thinks of L*H:
8Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
F0-contour of the utterance: „Is Noorwegen een DUUR
vakantieland?‟ (Is Norway an exPENsive („duur‟) holiday
country?). Pitch accent: % L L* HH%
9Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
3. Examples of prosody as design variable:
3a) The Melodic Intonation Therapy (MIT)
for aphasia
Purpose of the MIT: to activate the right hemisphere in
aphasics. This is the “non-speech” hemisphere, which is
active in melody generation (and singing).
It is known that contralateral brain structures can be
activated in performing tasks for brain structures which are
damaged (in aphasia: very often the left hemisphere:
Broca and Wernicke regions).
Prosody as Design variable (Task): singing utterances.
10Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
11Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
Prosody is a clear design variable in experiments on
the MIT, as far as brain images are concerned. Are the
contralateral brain structures activated?
Prosody is NOT only a design variable if we look at
other outcome variables: intelligibility, precision of
articulation, word finding.
In the latter case intonation (pitch) may function as
confounding variable: singing elicits slower speech,
which may lead to:
• more time for word finding,
• more precise articulation, thus
• better intelligibility.
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
12
Findings:
• In MIT-task also higher activation in left
hemisphere. Why? Developmental studies suggest
that exaggerating speech prosody may facilitate
recruitment of language related brain areas
(mothers speaking with children, learning poetry).
• Indeed, intelligibility increases, but also on the
basis of rhythmic cues.
3 Examples of prosody as design variable (cont):
3b) Perception of intonation by CI users
13
Citroen (lemon)!
Citroen (lemon)?
Processing F0-information (pitch) is limited in CI-users.
Questions (Straatman, Rietveld et al., JASA (2010):
a) Is performance of CI-users in discriminating between
affirmative utterances and questions lower than that of
„normal‟ listeners? The same question for a same/different
task for speech stimuli.
b) Is bimodal listening (Hearing Aid + CI) of any help?
Ad a) Questions and Affirmations (in Dutch)
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
14
Stimuli used to elicit perception of Questions (Q) or Affirmations
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
15
She doesn‟t know
He does
Responses: pointing to illustrations (sorry for non-political
correctness)
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
16Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
Results: Performance of Normal Hearing (NH) >
Cochlear Implants + Hearing Aid (CI+HA) > Cochlear
Implant (CI) alone.
17
The utterance [ba: ba:] with different pitch heights (ranging from 100 to
358 Hz (male voice)
Ad b) Same/Different task; sequences of bisyllabic
utterances only differing in pitch range. Note: pitch
accent on one-word utterance.
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
18
“equal”
“different”
Responses:
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
19
Results: here too: Performance of Normal Hearing
(NH) > Cochlear Implants + Hearing Aid (CI+HA) >
Cochlear Implant (CI) alone.
Reference:
Straatman, L.V, Rietveld, A.C.M., Beijen,J., Mylanus, E.A.M. & Mens, L.H.M. (2010).
Advantages of bimodal fitting in prosody perception for children using a cochlear implant
and a hearing aid. Journal of the Acoustical Society of America,128(4), 1884-1895.
20
Clearest effects, of course, in tone languages:
mother hemp horse to curse
„hoog‟ = „high‟, „stijgend‟ = „rising‟, „duikend‟ = „high-low-high‟, „dalend‟ = „falling‟
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
21
4) Intonation as confounding (and outcome)
variable
Example in which intonation plays a role both as an
outcome variable and a confounding variable.
The PLVT: Pitch Limiting Voice Therapy for speakers
with M. Parkinson (derived from Lee Silverman V.T.)
Goal: increase loudness (in order to increase articulatory
precision), but limit pitch.
Outcome variable: extent to which pitch stays within
predetermined limits, measured numerically
Confounding variable: prosodic make-up determines
pitch values; qualitative measurement (H*, L, etc.)
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
Time (s)0 2.278
Pitch
(Hz)
0
300
Time (s)0 2.278
Pitch
(Hz)
0
300
Time (s)0 2.278
Pitch
(Hz)
0
300
je kunt van liMOEnen ook limoNAde maken
“you can of LEmons also LEmonade make”
%L L H* L H*L L%
%L L H* H*L L%
%L L H*L H*L L%
22Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
Linked contour
Median pitch: 122 Hz
Mean pitch: 127 Hz
Standard deviation: 16 Hz
Minimum pitch: 100 Hz
Maximum pitch: 179 Hz
Voice Reports given by PRAAT:
Flat hat
Median pitch: 130 Hz
Mean pitch: 146 Hz
Standard deviation: 28 Hz
Minimum pitch: 100 Hz
Maximum pitch: 179 Hz
Two pointed hats
Median pitch: 122 Hz
Mean pitch: 127 Hz
Standard deviation: 16 Hz
Minimum pitch: 100 Hz
Maximum pitch: 179Hz
23Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
24
Experiment:
Speakers with M. Parkinson were submitted to the PLVT
therapy (Pitch Limiting Voice Therapy).
Hypothesis: After therapy (T2), the intensity will have
increased, but the average pitch (F0) decreased.
maandag dinsdag woensdag donderdag vrijdag zaterdag zondag
(days of the week in Dutch)
Time (s)
0 7.687
Pitch
(Hz)
75
200
Time (s)
0 7.687
Pitch
(Hz)
75
200
Time (s)
0 7.687-0.7932
0.5783
0
Time (s)
0 7.687-0.7932
0.5783
0
Time (s)
0 16.03
Pitch
(Hz)
75
300
Time (s)
0 16.03
Pitch
(Hz)
75
300
Time (s)
0 16.03-1
0.9677
0
T1
T2
200 Hz
300 Hz
25Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
Med = 161 Hz
Mean = 154 Hz
SD = 21 Hz
Med = 138 Hz
Mean = 138 Hz
SD = 34 Hz
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
26
However: on the basis of qualitative measurements:
T1 %L H* H*L L% („flat hat‟ )
Prediction: Higher mean F0, lower SD.
T2: %LH*LL% %LH*LL% %LH*LL% %LH*LL% etc.
Prediction: Lower mean F0, higher SD.
Predictions OK, see preceding slide, but not necessarily
due to therapy!
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
27
5) Prosody as confounding variable; degree
of prominence as confounder: M.Parkinson
Relative Intensity of noise in SI of /t/ as outcome
variable: Δ Int (= Int. Vowel – Int. SI, in dB)
Time (s)
0 0.835-0.3299
0.3758
0
0.096316905 0.755674754
ata1
[ a t a ]
Time (s)
0 0.9145-0.3376
0.3541
0
0.457244898
atha1
59 dB 77 dB Δ = 18 dB
120 Hz 159 Hz Δ = 39 Hz
Difference between intensity of noise in silent interval and the
intensity of the following vowel is often seen as an index of
correct realization of /t/. However, promincene may have a
differential effect on noise and the vowel amplitude:
a) Weak pitch accent
28Rietveld: Prosody and Speech & Language Pathology;
Lecture at the University of Cambridge, October 19, 2010
Time (s)
0.004918 0.8153-0.6099
0.6706
0
atha2
60 dB 82 dB Δ = 22 dB
120 Hz 178 Hz Δ = 58 Hz
b) Strong pitch accent
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of
Cambridge, October 19, 201030
The preceding pictures show the following:
In the „weak accent‟ condition, the difference
between the maximum amplitude of the noise in the
SI and that of the amplitude was 18 dB.
In the „strong accent‟ condition, the difference
between the maximum amplitude of the noise in the
SI and that of the amplitude was 22 dB., an increase
of 4 dB, which might be seen as an improvement.
More plausible, however, is that the increase in
amplitude after a therapy resulted in a larger
increase in the amplitude of the vowel than of the
noise in the SI.
Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010
31
Conclusions
• Prosody, both in the temporal and the pitch domain,
can play a role as design, confounding or outcome
variable.
• A thorough knowledge of the effects of tune choice on
voice parameters (mean F0, SD of F0 etc) is a
prerequisite for valid research (ex.: dysarthria)
• The same holds for prosodic effects on segmental
characteristics and processing (ex.: aphasia).