Doğu Erdener
description
Transcript of Doğu Erdener
![Page 1: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/1.jpg)
Doğu ErdenerDoğu Erdener
Auditory-visual speech perception in Auditory-visual speech perception in young children: young children: language and age-language and age-
specific factorsspecific factors
@@
6 December 2007
![Page 2: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/2.jpg)
Development of Speech Perception Development of Speech Perception
First 6 months phonetically-based, language-general
Beyond 1st year phonologically-based, language-specific Consonants 7-11 months (Best, 1995) Vowels 4-6 months (Kuhl et al., 1992) Tones 6-9 months (Burnham & Mattock, 2006)
A working heuristic (Burnham et al., 2002) Acoustic / Phonetic Stage (0-6 months) Phonological Stage (6-12 months) Semantic Stage (12-24 months) Orthographic Stage (6-8 years)
![Page 3: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/3.jpg)
Amodal (e.g.,Schwartz, et al., 1997) Phonetic (e.g.,Burnham & Dodd, 2004) Phonological / Postcategorical (e.g.,Massaro, 1998) none/some/all of the above?
Pre-categorical
Auditory-Visual Speech Perception (AVSP): Integration Auditory-Visual Speech Perception (AVSP): Integration
in early infancy (till ~6 months) is phonetically based beyond first year of life phonological
Recall that Recall that auditory-only auditory-only speech perception …. speech perception ….
AVSP integration research: AVSP integration research: We need differential and ontogentic data!We need differential and ontogentic data!
![Page 4: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/4.jpg)
AVSP: Cross-Language (Differential) StudiesAVSP: Cross-Language (Differential) Studies
Cross-language phonemic differences e.g., Werker et al. (1992) Visual influence from /th/ in the McGurk
stimulus [A]-/ba/ + [V]-/tha/ :• anglophones > francophones• more experienced francophones > less experienced francophones
Amount of visual speech influence across langauges: American > Japanese (Sekiyama & Tohkura, 1993) Japanese > Mandarin (Sekiyama, 1997)
Sekiyama studies Possible reasons: Cross-cultural studies: Less eye/face contact in some cultures +
Japanese Ss tend to look at the eye area whereas American Ss on the mouth area (Yuki et al., 2007).
Linguistic 1: Tonal (e.g. Cantonese) and pitch-accent (e.g., Japanese) information are not visually salient.
Linguistic 2: Fewer visually distinct consonant clusters and vowels in Japanese than in English.
![Page 5: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/5.jpg)
Development of Auditory-Visual Speech PerceptionDevelopment of Auditory-Visual Speech Perception
Infancy Infants match auditory and visual speech information
(Burnham & Dodd, 2004, Rosenblum et al., 1997)
Childhood McGurk & MacDonald (1976) 3 to 8 years + adults Massaro et al. (1986) 4-6 years + adults
Adulthood AV integration adults > children (Sekiyama &
Burnham, in press; Massaro et al.,1986) Late adulthood older Ss > younger Ss on vısual
speech information when auditory sensitivity is diminished (Cienkowski & Carney, 2002).
![Page 6: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/6.jpg)
Focus: Developmental Data from Infants and ChildrenFocus: Developmental Data from Infants and Children
Why is there an age-related increase?
Infancy and childhood are marked with several novel language-relevant and language-specific challenges, such as phonetic phonological semantic vocabulary reading
![Page 7: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/7.jpg)
Japanese and Australian English SpeakersThree child (6-, 8-, and 11-year-olds) groups
and a control group of adults.Method
McGurk stimuli DV: Visual Speech Index (VSI) Score based
on the proportion of auditory-based responses Identification Task: /ba/, /da/, or /ga/?
Sekiyama & Burnham (in press): Sekiyama & Burnham (in press): an ontogenetic + differential study an ontogenetic + differential study
![Page 8: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/8.jpg)
AV+ minus A
Augmentation
A minus AV-
Interference
Visual Speech
Index (VSI)
[AV+] minus [AV-]
Japanese
0
0.25
0.5
0.75
1
-4 4 12 Clear
0
0.25
0.5
0.75
1
1 2 3 4
0
0.25
0.5
0.75
1
-4 4 12 Clear
0
0.25
0.5
0.75
1
-4 4 12 Clear
SN Ratio
Australian English
0
0.25
0.5
0.75
1
-4 4 12 Clear
Prop. Correct Aud Rs
0
0.25
0.5
0.75
1
-4 4 12 Clear
Prop. Correct Aud Rs
0
0.25
0.5
0.75
1
-4 4 12 Clear
Prop. Correct Aud Rs
0
0.25
0.5
0.75
1
-4 4 12 Clear
SN Ratio
Prop. Correct Aud Rs
6-yos
8-yos
11-yos
Adults
![Page 9: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/9.jpg)
Sekiyama and Burnham (in press) found: Developmental increase in visual speech influence with age…
• In English speakers but• Not in Japanese speakers
So…..
What factors modulate the development of auditory-visual What factors modulate the development of auditory-visual speech perception speech perception (measured by VSI), (measured by VSI),
especially especially between 6 and 8between 6 and 8 years of age? years of age?
![Page 10: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/10.jpg)
Experiment 1: Factors InvestigatedExperiment 1: Factors Investigated
Language Specific Speech Perception (LSSP) Effect of phonological experience in native language: Native
Speech Perception minus Non-native Speech Perception (N-NN) (Burnham, 2003).
Reading Reading ability is positively related to LSSP in children
(Burnham, 2003). Evidence for reading-lipreading link:
(de Gelder & Vroomen, 1998; Cavé et al., 2007)
Articulation ability is positively related to visual speech influence in children (Desjardins et al., 1997), and in adults with cerebral palsy (Siva et al., 1995)
![Page 11: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/11.jpg)
Experiment 1Experiment 1
AVSPLanguage-specific Speech Perception
(LSSP) Reading
Articulation
Regression
Correlation
![Page 12: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/12.jpg)
Experiment 1: MethodExperiment 1: Method
Language Specific Speech Perception (LSSP) Test: (Native minus Non-native speech perception N-NN)18 N [voiced vs. voiceless] + 18 NN [voiced vs. prevoiced]
Reading Test WRAT-3 reading subtest: 15 letters + 42 words = 57 items
Articulation Test Queensland Articulation Test (QAT) Naming task: 64 pictures Aus English consonants: initial, medial, & final positions
Auditory-Visual Speech Perception (AVSP) TestAuditory-Visual Speech Perception (AVSP) Test McGurk paradigm: A & V combinations of /ba/, /da/, and /ga/ 48 McGurk stimuli: (24 AV, 12 AO, and 12 VO),
3 consonants x 4 talkers (2 Eng & 2 Jap) x 2 congruence types x 2 background noise (Clear & noisy [+4 dB]) versions
Visual Speech Index (VSI) AV (+) minus AV (-)
![Page 13: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/13.jpg)
Experiment 1: MethodExperiment 1: Method
Participantsn=96; Australian English 5-, 6-, 7-, and 8-year-oldsn=48; Australian English adults
N=144
Procedure Individual testing (~60 minutes) Task orders counterbalanced Clear & Noise (+4dB SN Ratio) Each stimulus presented twice
![Page 14: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/14.jpg)
Exp. 1 Results: AVSP TestExp. 1 Results: AVSP Test
0
0.25
0.5
0.75
1
5 years 6 years 7 years 8 years Adults
Age
Visual Speech Index Score (/1)
Native Speaker Clear VSIForeign Speaker Clear VSINative Speaker Noise VSIForeign Speaker Noise VSI
VSI scores: Linear increase with age (p <.01) + Quadratic between 5 and 6 years (p <.01).
VO scores: Linear increase with age (p <.01).
AO scores: Linear increase with age (p <.01).
0
0.25
0.5
0.75
1
5 years 6 years 7 years 8 years Adults
Age
Correct VO Score (/1) Native VO Scores
Non-native VO Scores
0
0.25
0.5
0.75
1
5-year-olds 6-year-olds 7-year-olds 8-year-olds Adults
Age
Proportion of Correct AO Response (/1)
Native Speaker ClearForeign Speaker ClearNative Speaker NoiseForeign Speaker Noise
![Page 15: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/15.jpg)
Exp. 1 Results: LSSP, reading and articulation Exp. 1 Results: LSSP, reading and articulation
LSSP scores: No age-based differences (p >.01) but native > non-native (p< .01)
Reading scores: Increase with age (p< .01)
Articulation scores: Increase with age (p <.01) + adults > children (p <.01).
0
0.25
0.5
0.75
1
5 years 6 years 7 years 8 years Adults Mean Discrimination Index Score (/1)
Native DI Non-Native DI
Age
0
0.25
0.5
0.75
1
5 years 6 years 7 years 8 years Adults
Age
Reading Test Scores (/1) Reading Test Score (WRAT-3)
0
0.25
0.5
0.75
1
5 years 6 years 7 years 8 years Adults
Age
Articulation Test Scores (/1) Articulation Test Score
![Page 16: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/16.jpg)
Children
Age + AO + VO + LSSP + Articulation + Reading VSI
Adults
Age + AO + VO + LSSP +Articulation + Reading VSI
Exp. 1 Results: Regression Analyses Exp. 1 Results: Regression Analyses (DV = VSI Scores)(DV = VSI Scores)
![Page 17: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/17.jpg)
Exp. 1 Results: Correlation Coefficients Exp. 1 Results: Correlation Coefficients
VSI Age AO VO LSSP Artic. Read.
VSI -
Age .14 -
AO .07 .43** -
VO .33** .32** .26** -
LSSP .21* -.15 -.01 .05 -
Articulation .07 .69** .30** .25** -.15 -
Reading .13 .86** .46** .35** -.06 .68** -
* Sig. at á=.05, **Sig. at á= .01
![Page 18: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/18.jpg)
Experiment 1: DiscussionExperiment 1: Discussion
AVSP and lipreading (VO) increase with age This increase is sharp at around 6 years of age, around the
onset of reading instruction. No age-based LSSP differences
native and non-native difference is largest at 6 years Reading instruction a new challenge new sources of
speech information used? Regression Analyses
Children: lipreading + LSSP VSI Adults : AO VSI
Correlation Analyses Articulation is correlated with AO, lipreading (VO), and reading.
![Page 19: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/19.jpg)
Exp. 1 Discussion Exp. 1 Discussion Experiment 2 Experiment 2
Experiment 1: LSSP & Lipreading AVSP
Schooling and/or new language-specific challenges seem to increase visual speech influence.
Towards Experiment 2 (3- & 4-year-olds): Status of LSSP AVSP link in pre-orthographic
stages?? Are AVSP and basic Cognitive abilities related? Are AVSP and Vocabulary Knowledge related?
![Page 20: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/20.jpg)
Experiment 2: Method Experiment 2: Method
Participants: N=48; 3-year-olds & 4-year-olds AVSP – material, stimuli and procedure
AVSP: AX discrimination task Three conditions AV(36), AO(12), VO(12) AX-VSI (/1): based on different trials
LSSP – material, stimuli and procedure 18 Native [pa-pha] & 18 Nonnative [ba-pa] trials. AX-based category change paradigm.
Vocabulary Test – material, stimuli, and procedure PPVT (Dunn & Dunn,1997) age-based word sets of 12 items on average 5-6 sets.
Cognitive Flexibility – material, stimuli and procedure FIST (Jacques & Zelazo, 2001) 15 items (shapes, sizes & colours)
![Page 21: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/21.jpg)
Exp. 2 Results: AVSP TestExp. 2 Results: AVSP Test
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
3-year-olds 4-year-olds
VSI-AX Score (/1)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
3-year-olds 4-year-olds
Correct Response Rate (/1)
Auditory-Only
Visual-Only
VSI-AX scores: 4-year-olds > 3-year-olds (p <.01)
VO scores: 4-year-olds ≈ 3-year-olds (p =.07)
AO scores: 4-year-olds > 3-year-olds (p <.05)
![Page 22: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/22.jpg)
Exp. 2 Results: LSSP, vocabulary and cognitive flexibilityExp. 2 Results: LSSP, vocabulary and cognitive flexibility
LSSP scores:
4-year-olds > 3-year-olds (p <.01)
No stimulus language effect or contrast/age interaction (p >.05)
Vocabulary scores: 4-year-olds > 3-year-olds (p <.01)
FIST scores: 4-year-olds > 3-year-olds (p <.01)
0
0.05
0.1
0.15
0.2
0.25
3-year-olds 4-year-olds
Discrimination Index Score (/1)
Native DINon-Native DI
0
10
20
30
40
50
60
70
80
3-year-olds 4-year-olds
Receptive Vocabulary Score
0
2
4
6
8
10
12
14
16
3-year-olds 4-year-oldsMean Executive Function Test Score (/15)
Rule Abstraction
Cognitive Flexibility
![Page 23: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/23.jpg)
Exp. 2 Results: Regression Analyses Exp. 2 Results: Regression Analyses
VSI scores as dependent variable
Age + AO + VO + LSSP + Vocabulary + FIST VSI-AX
![Page 24: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/24.jpg)
Exp. 2 Results: Correlation Coefficients Exp. 2 Results: Correlation Coefficients
VSI-AX Age AO VO LSSP Vocab. FIST
VSI-AX -
Age .45** -
AO .54** .42** -
VO .21 .21 .37** -
LSSP .30* .10 .21 .10 -
Vocab. .45** .66** .56** .27* .41** -
FIST .51** .36** .35** .36** .42** .54** -
* Sig. at á=.05, **Sig. at á= .01
![Page 25: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/25.jpg)
Experiment 2: DiscussionExperiment 2: Discussion
AVSP, LSSP, vocabulary and FIST improve with age AVSP improves between 3 and 4 years. Lipreading p=.07 – same age-based trend, but small age
difference? Regression Analyses
AO & FIST VSI-AX similar to adults in Exp. 1 (AO VSI)
Why does AO predict VSI-AX? AVSP product of AO speech perception Three and 4 years: object-word pairing is a challenge, but not as
much as reading – so no need for extra info. AVSP is determined by inherent abilities?
e.g., AO speech perception and cognitive function.
![Page 26: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/26.jpg)
Regression Analyses: Regression Analyses: LSSP, LSSP, aage, language tests regressed onto AVSPge, language tests regressed onto AVSP
5,6,7, 8 years LSSP + VO + AO + Articulation + Reading AVSP
AVSP is related to LSSP and lipreading
AdultsLSSP + VO + AO +Articulation + Reading AVSP
AVSP is related only to auditory speech perception
3, 4 years
LSSP + VO + AO+ Cognitive + Vocab. AVSP
AVSP is related only to auditory speech perception
![Page 27: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/27.jpg)
General DiscussionGeneral Discussion
Visual speech influence develops with age AV: e.g., Massaro et al. (1986) VO: e.g., Desjardins et al. (1997)
(though p=.07 in Exp.2)
AVSP follows a phonetic phonological trend AVSP is amodal and phonetic
(Burnham & Dodd, 2004; Rosenblum et al., 1997) AVSP is phonological (Massaro, 1998) But more importantly, AVSP seems to occur at all
levels of language processing.
![Page 28: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/28.jpg)
AVSP- Articulation Link Vocabulary and articulation challenges in early childhood Tests of AVSP, LSSP, and articulation for children (12-24
months), with and without speech disorder (Dodd et al.,in press). AVSP-Reading Link
Burnham (2003): reading LSSP LSSP AVSP So, integrative ability phonemic/phonological processing This may be the case for English but not for other languages?
Around school age, language-relevant challenges emerge Reading phoneme-grapheme mapping LSSP intensifies (orthographic stage: Burnham et al., 2002) Children use all speech information available
• Hyperarticulated speech styles (Lees & Burnham, 2005).e.g., teacherese (Håkansson, 1987).
• Cognitive skills activated + maturation
General DiscussionGeneral Discussion
![Page 29: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/29.jpg)
Past Past Current Current Future Future
AVSP and reading link (Cave et al., 2007; de Gelder & Vroomen,
1998)
Burnham (2003)
Articulation-lipreading link (Desjardins et al.,1997)
![Page 30: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/30.jpg)
AVSP (integrative module)
Auditory-only speech perception
Lipreading
Speech Production(articulation)
Reading
![Page 31: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/31.jpg)
Some questions awaiting answerSome questions awaiting answer
Speech Production-Perception Link: the perception-production link: a developmental
perspective; e.g., testing children with speech disorders (Dodd et al., in press).
Visual Speech and Reading Link:Research on the link between study strong correlations between visual aspects of speech and reading This study correlations between reading and AO,
lipreading and articulation. Cavé et al. (2007); de Gelder & Vroomen (1998). Cross-language reading x AVSP studies.
![Page 32: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/32.jpg)
Acknowledging…….Acknowledging…….
Prof. Denis Burnham
Prof. Barbara Dodd, Asc.Prof. Chris Davis, Prof.Kaoru Sekiyama
MARCS Auditory Laboratories & MARCS Baby Lab.
College of Arts, University of Western Sydney
Australian Postgraduate Award
Aprica Foundation, Osaka, Japan
Participating schools, teachers, parents and kids in Sydney
![Page 33: Doğu Erdener](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814c52550346895db965ba/html5/thumbnails/33.jpg)