Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate...
Transcript of Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate...
![Page 1: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/1.jpg)
Music Perception with Combined Stimulation
Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3 Bruce Gantz4
School of Music,1 Department of Communication Sciences & Disorders,2 Department of Biostatistics,3
Iowa Cochlear Implant Clinical Research Center,4 Department of Otolaryngology,
The University of Iowa Hospitals and Clinics
![Page 2: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/2.jpg)
Acknowledgements
This study was supported by grant 2 P50 DC00242 from the NIDCD, NIH; grant RR00059 from the General Clinical Research Centers Program, NCRR, NIH; Grant 1R01DC012082-01A1from NIH-NIDCD, and the Iowa Lions Foundation.
![Page 3: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/3.jpg)
Background
• Low-frequency information available through preserved acoustic hearing can assist CI recipients in segregating the target speech signal from background noise.
(Gantz & Turner, 2003; Gantz et al., 2006; Turner et al., 2004)
![Page 4: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/4.jpg)
• Low-frequency information available through preserved acoustic hearing can also assist CI recipients in perception of salient features of music (Dorman et al., 2008; El Fata et al., 2009; Galvin et al., 2009; Gantz et al., 2006; Gfeller et al., 2006, 2007, 2009; Kong et
al., 2004; Turner et al., 2005, 2007)
![Page 5: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/5.jpg)
Salient Features of Music
• Rhythm • Tempo, basic beat • Duration of individual notes
• Timbre • Identification of voices, instruments, blends • Sound quality
• Pitch • Interval size, direction of pitch change = melody • Concurrent pitches=harmony
![Page 6: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/6.jpg)
Salient Features of Music • Rhythm
• Tempo, basic beat • Duration of individual notes
• Timbre • Identification of voices, instruments, blends • Sound quality
• Pitch • Interval size, direction of pitch change = melody • Concurrent pitches=harmony
• Lyrics in vocal music
![Page 7: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/7.jpg)
![Page 8: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/8.jpg)
![Page 9: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/9.jpg)
Technical Issues: Music vs. Speech
• Differences in acoustic input of speech and music
• Frequency range – Speech: 250-8000 Hz – Music: 27-well above 8000 Hz
• Spectrum – Speech: well controlled spectrum – Music: highly variable and rapidly changing spectrum.
• Intensity – Speech: 53-83 dB SPL – Music: 10-120 dB SPL; rapid changes in amplitude
• Music requires better resolution of pitch/spectral shape
![Page 10: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/10.jpg)
Music Perception
Stimuli • Isolated structural
features • Pitch, timbre, duration
• Computer-generated combinations
• e.g., pitch + rhythm
• Real-world combinations • pitch (melody, harmony),
timbre, duration, amplitude, lyrics.
Measure • Perceptual accuracy
• Detection, discrimination, recognition
• Appraisal • Sound quality and
enjoyment
![Page 11: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/11.jpg)
Pitch Ranking Pitch Rank Test
0
100
200
300
400
500
Tota
l Ite
ms
Cor
rect
A+E 6 mo (N=15) A+E most recent (N=25) LE (N=256) Normal Hearing (N=20)
Gfeller et al., 2007
![Page 12: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/12.jpg)
Melody Recognition
Complex Melody Recognition
0
10
20
30
40
50
60
70
80
90
100
523 Hz
% c
orre
ct
A+E 6 mo (N=14) A+E most recent (N=32) LE (N=176) Normal Hearing (N=17)
Simple Melody Recognition
0
10
20
30
40
50
60
70
80
90
100
% c
orre
ct
A=E (N=26 ) LE (N=191) Normal Hearing (N=17)
(Gfeller et al., 2006, 2007, 2010)
![Page 13: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/13.jpg)
Interval Normalization
Modified Melodies Test
0
10
20
30
40
50
60
70
80
90
100
131 Hz 262 Hz 523 Hz
% c
orre
ct
A+E (N=10) Bilateral LE (N=6) Normal Hearing (N=18)
![Page 14: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/14.jpg)
Timbre Recognition
Timbre Recognition Test
0
10
20
30
40
50
60
70
80
90
100
% c
orre
ct
A+E (N=23) LE (N=51) Normal Hearing
Gfeller et al., 2006
![Page 15: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/15.jpg)
Lyrics Recognition Vocals in Background Music
-30.00
-25.00
-20.00
-15.00
-10.00
-5.00
0.00male female
SNR
A+E (N=18, 19) LE No HA (N=60, 62) LE with HA (N=25)Bilateral (N=32) Normal Hearing (N=13)
Gfeller et al., 2009
![Page 16: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/16.jpg)
Music as Background Noise
• Music functions as a masker of speech sounds in many real-life situations (e.g., music at a party, MUZAK, music scores in movies).
![Page 17: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/17.jpg)
Speech Recognition in Background Music Test
(SRBM)
• Recognition of 12 spondees (as described in Turner et al., 2004)
• against 3 types of background music – piano solo – vocal solo + bass and guitar accompaniment – large orchestra
• Participants • 154 long electrode CI recipients (LE); 21
Hybrid recipients (A+E); 49 normal hearing adults (NH)
![Page 18: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/18.jpg)
Speech Recognition in Background Music
Gfeller et al., 2012
![Page 19: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/19.jpg)
Pitch and Speech
• Pitch perception correlated with • Speech in background noise (Gantz & Turner, 2003; Gantz et al.,
2006; Gfeller, Turner, et al., 2012; Turner et al., 2004)
• Recognition of prosody marking linguistic contrasts (See, Driscoll, Gfeller et al., 2013)
• Tone discrimination in Mandarin Chinese (Xu, 200x)
• Talker discrimination (recognition) (Johnson, Driscoll, Gfeller et al., 2011)
![Page 20: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/20.jpg)
Correlations: SRBM X Pitch Ranking, Ripple Test
SRBM
Pitch Ranking (N=165)
-.26 (p<.001)
Spectral Ripple (N=15)
-.60 (p<.02)
Gfeller et al., 2012
![Page 21: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/21.jpg)
SRBN X SNR: Noise, Babble
SNR (N=124)
SRBM
Noise
.72 (p<.001)
Babble .66 (p<.001)
Gfeller et al., 2012
![Page 22: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/22.jpg)
Conclusions
• Preserved residual hearing benefits
• perception of pitch (melody), timbre
• better extraction of a target speaker from background noise or music
![Page 23: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/23.jpg)
![Page 24: Music Perception with Combined StimulationMusic Perception with Combined Stimulation Kate Gfeller1,2,4, Virginia Driscoll,4 Jacob Oleson,3 Christopher Turner,2,4 Stephanie Kliethermes,3](https://reader033.fdocuments.in/reader033/viewer/2022042918/5fa3697b8a0c9269a213e3e7/html5/thumbnails/24.jpg)
Piano solo
Vocal solo + guitar
Orchestra
Spectral Analyses of Musical Stimuli