Sound of Silence - Kyushu...
Transcript of Sound of Silence - Kyushu...
04/09/2014
1
Sound of Silence cross-linguistic comparison of perception of
Japanese geminate consonants
Makiko Sadakata1, Mizuki Shingai2, Simone Sulpizio3, Alex Brandmeyer1,4, Loukianos Spyrou1 and Kaoru Sekiyama2
1Radboud University Nijmegen, 2University of Amsterdam 3University of Trento, 4Max Planck Institute Leipzig
5University of Kumamoto
2014.08.07: 2014 Symposium of Across-Channel Processing in Human Audition Niagara-on-the-Lake, Canada
Geminate and Singleton Consonants
Geminates (/kk/ /tt/ /pp/, etc.) Singletons (/k/ /t/ /p/, etc.) • exist in various languages • determined largely by timing
Literatures • Singletons ! short consonants • Geminates ! long consonants
Long consonants?
04/09/2014
2
Acoustics of Japanese geminates
Stop geminate consonants: tt, kk, pp • Silent duration • /akku/
Fricative geminate consonants: ss, ff • Frication • /assu/
75%
25% (Takahashi, 1999)
Writing of Japanese geminate consonants
Japanese: Identical symbol is used to represent the first portion of both stop and fricative geminates • akku "あっく • assu "あっす
Non Japanese (e.g., Italian): Different symbols are used to
represent the first portion of both stop and fricative geminates • asso • rocca
04/09/2014
3
Perception of geminate consonants?
Is it important to perceptually distinguish the two types of geminate consonants?
1. Adults’ behavioural responses 2. Children's behavioural responses 3. Adults’ ERP responses
1: ADULTS’ BEHAVIOURAL RESPONSE SADAKATA, SHINGAI, BRANDMEYER, SULPIZIO, & SEKIYAMA (SUBMITTED)
04/09/2014
4
Perception of Japanese geminate consonants
• How crucial is it for listeners to distinguish acoustical differences (stop / frication) when perceiving geminate consonants?
• Does it vary across different languages?
Hypothesis
" JP: not sensitive for the difference between the two types " Non JP: sensitive for the difference between the two types
Italian: includes geminate consonants Dutch: does not include geminate consonants
N=16 each
04/09/2014
5
Stimuli
assu a_su akku
90 ms 100 ms 120 ms 40ms 100 ms
Identification test
AAAAB test (Dehaene-Lambertz et al., 2000) • A- Standard: 4 female voices • B- Test: 1 male voice
Judge whether Standard and Test belong to the same category It is up to participants to distinguish /ss/ and /_s/ or not
04/09/2014
6
Results: identification
Discrimination (4I2AFC, Gerrits & Schouten, 2004)
Standard tone /assu/ Deviant tone /a_su/ /akku/
time
Participants NEED to distinguish /ss/ and /_s/
04/09/2014
7
Discrimination test
Summary Identification:
• Japanese tended to categorize /_s/ and /ss/ into the same type
• Italian and Dutch tended to separate these two (more than JP did)
Discrimination:
• All participants could hear silence in the material " the confusion was not due to the insensitivity to a silence
Why?
04/09/2014
8
2: CHILDREN’S BEHAVIOURAL RESPONSE
Shingai, 2014
Learning to write the Japanese geminate character
Japanese alphabetic character (hirakana) • Approximately 100 characters (including special characters)
Suzuki & Amano 2003 • Geminate character is reported to be the most problematic
ones to learn for children, because it often corresponds with silence in speech (Fujimoto et al., 2007)
• /u/ /re/ /shi/ /ka/ /tta/ " う れ し か た
Where should I insert /っ/? I
cannot hear it!
っ
04/09/2014
9
Tasks
SOS Identification test* SOS discrimination test* RAN test Writing test Standardized reading / writing test (STRAW) *SOS tests: difficulty was adjusted
Participants
• Total (including pilots): • 9 years old N=59 / 7 years old N=53
• Without specific practice: • 9 years old N=27 / 7 years old N=33
• Better than chance level at the easiest task: • 9 years old N=17 / 7 years old N=16
• Tested in September – December 2013
04/09/2014
10
Predictions
Identification • 7 years old : /ss/ and /_s/ " different • 9 years old : /ss/ and /_s/ " same (same as adults)
Discrimination
• Both groups perform well on the task STRAW, Reading test, RAN may be correlated with the
degree of the SOS effect
SOS Identification task
Standard Test
Same /assu/ /assu/
Different /assu/ /a_su/
Filler /assu/ /akku/
04/09/2014
11
Identification
0
10
20
30
40
50
60
70
80
90
100
110
不一致 一致 ディストラクター
Error R
ate(%)
1年生
3年生
Different Same Filler
7 years old 9 years old
Discrimination test
XAB X = /ss/ A, B = /ss/-/_s/, /kk/-/ss/
Tasks adjusted from Zhow, W. & Broersma, M.
04/09/2014
12
Discrimination
0
10
20
30
40
50
60
70
80
90
100
/_s/ /kk/
Error R
ate(%)
1年生
3年生
Fricative Stop
7 years old 9 years old
Screening Test of Reading and Writing for Japense Primary School Children
• Standardized Reading & Writing tests (kana) for 1st / 3rd year students • Detecting developmental Dyslexia (rather easy)
1st year: reading and writing single Hiragana character (ぴ、ぎゃ、etc.) 3rd year: reading and writing words (あめ、ひこうき、etc.)
04/09/2014
13
Original writing test
Writing test " consisted of 13
items, which include special characters (7 of 13 include geminate consonants)
Rapid Automatized Naming
Name icons as quickly as possible within 60 seconds
Wide range of cognitive skills:
• the child’s ability to quickly retrieve concepts • Used for the diagnoses for reading disabilities • orthographic knowledge • phonological processing
04/09/2014
14
Overview of correlations (all)
ID-Different ID-Same ID-Filler Disc-
Fricative Disc-Stop RAN STRAW WT:Geminate WT:Long-vowel WT:Youon WT:Total
error
ID-Same -.101
ID-Filler -.056 -.015
Disc-Fricative .023 -.195 -.185
Disc-Stop -.074 .241 -.011 -.142
RAN 0.413*** -.018 -.055 -.112 .015
STRAW -.027 .024 -.194 -.190 -.203 -.122
WT:Geminate -0.395* .078 .193 .188 -.042 -0.407*** -0.294*
WT:Long-vowel -.122 .349 -.103 .121 -.239 -.194 -0.294* 0.505***
WT:Youon -0.260 -.246 .166 .181 .057 -.181 .017 .155 -.042
WT:Total error -0.483* .191 .125 .193 -.145 -0.446*** -0.325* 0.803*** 0.645*** .281
Age .075 -.250 .177 -.149 -.097 0.316* -.065 -0.442*** -0.442*** -.097 -0.427**
Shingai, 2014
Writing error (geminate)
7 yo (r=-.726***) 9 yo (r=-.167)
0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8
Err
or R
ate:
Iden
tific
atio
n (d
iffer
ent,
%)
Writing Error
1年生
3年生
Linear (1年生)
Linear (3年生)
7 yo 9 yo 7 yo 9 yo
Shingai, 2014
04/09/2014
15
RAN
1st year students (r=.568*) 3rd year students (r=.181)
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80
Err
or R
ate:
Iden
tific
atio
n (d
iffer
ent,
%)
RAN
1年生
3年生
Linear (1年生)
Linear (3年生)
7 yo 9 yo 7 yo 9 yo
Shingai, 2014
Summary • Negative correlation between the writing error (geminates)
and the critical identification error (/ss/ vs /_s/) " The better they can write, the more they tended to categorize the /ss/ and /_s/ as the same
• Positive correlation between RAN and the critical
identification error (/ss/ vs /_s/) " The more quickly they can name items, the more they tended to categorize the /ss/ and /_s/ as the same
In general, children with more developed writing / phonological skills tend to show similar results as Japanese adults
04/09/2014
16
Summary
• These relationships were more pronounced for the 7 years old but not for the 9 years old
" 9 years olds showed uniform performance on the writing / STRAW / RAN test (ceiling)
" 7 years olds showed diverse performance on the writing / STRAW / RAN test
" what about younger children?
Current action (July – December 2014)
Testing younger children: 5-6 years old Children tend to learn to write even before they “officially
learn” (Shimamura et al., 1994)
• On average, 5 yo 9 months – able to write about 46 hiragana signs
04/09/2014
17
3: ADULTS’ ERP RESPONSES SADAKATA, SPYROU & SEKIYAMA (2013, IN PREP)
Step 1 : Compared ERP responses elicited by /assu/ between Japanese and non-Japanese native speakers
04/09/2014
18
Methods 8 Native speakers of Japanese 8 Native speakers of English /asu/ with different durations of /s/, ranging from 60-240ms with steps of 30ms, /a/=63ms, /u/=90ms EEG recording
• 32 Electrodes + mastoid • All stimuli presented 297 times each, mixed sequence • Participants: watching a self-selected silent movie
Auditory ERPs
P1 75-80 ms N1 100 ms P2 180 ms N2 220-240 ms
P1
P2
N1
N2
+
- time
04/09/2014
19
Auditory Change Complex (ACC)
P1-N1-P2 is sensitive to changes in auditory stimuli e.g. phonemic boundaries Ostroff et al. (1998)
©&Williams&&&Wilkins&1998.&&All&Rights&Reserved.&&Published&by&LippincoP&Williams&&&Wilkins,&Inc.& 4&
Figure&3&Cor,cal*Evoked*Response*To*Acous,c*Change*Within*A*Syllable.*Ostroff,&Jodi;&Mar>n,&BreP;&Boothroyd,&Arthur&&Ear&&&Hearing.&<strong>19</strong>(<strong>4</strong>):290`297,&August&1998.&&
Figure&3&.&Group&mean&waveforms&from&electrode&site&Cz&are&displayed&for&each&of&the&three&speech&s>muli.&The&speech&s>mulus&used&to&evoke&each&of&the&responses&is&located&in&the&insets.&Note&that&the&N1`P2&is&present&for&each&of&the&three&s>mulus&condi>ons.&
ERPs Predicted peak timing (ms)
!! Duration)of)/s/)(ms))!! 60! 90! 120! 150! 180! 210! 240!
/a/)P1! 80!N1! 100!P2! 180!
/s/)P1! 143!N1! 163!P2! 243!
/u/)P1! 203! 233! 263! 293! 323! 353! 383!N1! 223! 253! 283! 313! 343! 373! 403!P2! 303! 333! 363! 393! 423! 453! 483!
04/09/2014
20
Comparison of ERP responses elicited by /asu/ JP vs. EN
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 240ms
P1 N1 P2 N2P1 N1 P2 N2
P1 N1 P2 N2
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 60ms
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 150ms
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 240ms
P1 N1 P2 N2
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 150ms
P1 N1 P2 N2
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 60ms
P1 N1 P2 N2
Comparison of ERP responses elicited by /asu/ JP vs. EN
Cross-group effect: N1 of EN higher than that of JP JP: P2 higher than P1 N1 EN: P1-N1-P2 peaks not significantly different
04/09/2014
21
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 240ms
P1 N1 P2 N2P1 N1 P2 N2
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 150ms
P1 N1 P2 N2P1 N1 P2 N2
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 60ms
P1 N1 P2 N2P1 N1 P2 N2
Comparison of ERP responses elicited by /asu/ JP vs. EN
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 240ms
P1 N1 P2 N2P1 N1 P2 N2
P1 N1 P2 N2
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 150ms
P1 N1 P2 N2P1 N1 P2 N2
P1 N1 P2 N2
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 60ms
P1 N1 P2 N2P1 N1 P2 N2
P1 N1 P2 N2
Comparison of ERP responses elicited by /asu/ JP vs. EN
04/09/2014
22
Summary • Japanese and English ERPs showed significant amplitude
differences, especially at the time point where P1- N1-P2 complex associated with /a/ was expected (80-180ms)
• What is underlying this difference?
• Could this be related with response to /s/?
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 240ms
P1 N1 P2 N2
JP
EN
0 100 200 300 400 500 ms
6
4
2
0
-2
6
4
2
0
-2
Ampli
tude
s (uV
)
S = 240ms
P1 N1 P2 N2P1 N1 P2 N2
Step 2 (De)composing ERPs to /assu/ from individual ERPs to /a/ + /s/ + /u/
Special thanks to Loukianos Spyrou
04/09/2014
23
Auditory ERPs
P1 75-80 ms N1 100 ms P2 180 ms N2 220-240 ms
P1
P2
N1
N2
+
- time
/a/
× wa
/s/
× ws
/u/
× wu
/asu/
+
+
timeAm
plitu
des
+
-
Estimated ERPs Goodness of FitEstimated ERPs - Actual ERPs
Composition approach
Dependent Variables: GOF & W
04/09/2014
24
Predictions
• ERP group differences may (partly) be explained by different degrees of responses to /s/
• According to the SOS effect on the geminates consonants • Japanese " part of long frication is perceptually similar
to silence " smaller weights to /s/ • English " frication is perceived as frication " greater
weights to /s/
• Weights for /s/ may be different for two groups on geminate conditions (EN > JP)
Additional stimuli • /asu/ with different durations of /s/, ranging from 60-240ms with
steps of 30ms, /a/=63ms, /u/=90ms Additional stimuli
• /a/=63ms, /s/=60, 150 and 240ms, and /u/=90ms • /asu/ with louder /s/ (+7.5dB/+15dB, /s/=240ms) • /asu/ with long /a/ (93ms, /s/=240ms, /u/=90ms) • /akku/ (/a/=63ms, silent = 200ms, /k/=40ms, and /u/=90ms)
04/09/2014
25
ERP to /a/, /s/ and /u/
0 0.2 0.4 0.6
-2
0
2
4
6
0 0.2 0.4 0.6
-2
0
2
4
6
0 0.2 0.4 0.6
-2
0
2
4
6
0 0.2 0.4 0.6
-2
0
2
4
6
0 0.2 0.4 0.6
-2
0
2
4
6
0 0.2 0.4 0.6
-2
0
2
4
6
JP
EN
Amplit
udes (u
V)
aRed - s60Green - s150Blue - s240
u
Comparison Real vs. Composed ERP
The goodness of fit: medium was better than the other two No group difference (accuracy of composition was equivalent)
JP EN
Short (s60) Medium (150) Long (240)
0 0.5
4
2
0
-20 0.5
4
2
0
-20 0.5
4
2
0
-2
S=60 S=150 S=240
JP
EN
0 0.5
4
2
0
-20 0.5
4
2
0
-20 0.5
4
2
0
-2
Actual
A+S+U
04/09/2014
26
00.10.20.30.40.50.60.70.80.91
1.11.21.31.41.5
Mean(Weights)
Weights
Weights for s=240 (geminates) was significantly lower for JP than EN*
s = 60 ms s=150 ms s=240 ms
00.10.20.30.40.50.60.70.80.91
1.11.21.31.41.5
Mean(Weights)
00.10.20.30.40.50.60.70.80.91
1.11.21.31.41.5
Mean(Weights)
Overview weights difference
A S U
60 ms
150 ms
240 ms JP < EN*
240 ms (middle) JP < EN*
240 ms (loud)
Long a (93ms) + s (240 ms) JP < EN*
Silence
Less trace of ACC associated /s/ in Japanese, only for GEMINATES (=long s) condition Corresponds with the hypothesized Japanese representation: ss = silence + /s/
04/09/2014
27
Summary of findings
• Combining individual ACCs predicted the real ERPs to some extent
• Goodnewss of Fit did not differ between groups • Weights of /s/ in the critical fricative geminate conditions
differed between groups (expected direction according to the SOS effect, EN > JP)
Summary • Behavioural
• Japanese adults tended not to distinguish /ss/ from /_s/ while non Japanese adults did
• This specific Japanese tendency seems to develop together with learning to write Japanese characters
" Influence of orthography on perception of fricative geminate consonants
• EEG
• Japanese and English ERPs to a range of /asu/ stimuli showed significant amplitude differences
• This may partly be explained by the degree of response to /s/ especially in “long s” conditions