Psycholinguistics Lecture 7 Speech Perception II.

PsycholinguisticsPsycholinguisticsLecture 7Lecture 7

Speech Perception IISpeech Perception II

AnnouncementsAnnouncements

Midterm, Nov. 1Midterm, Nov. 1stst!!– Covers material up to October 25Covers material up to October 25thth

October 25October 25thth (by 11:59pm): submit (by 11:59pm): submit potential exam questions.potential exam questions.– Form groups of 1-2 studentsForm groups of 1-2 students

list 5 major points covered in the courselist 5 major points covered in the course submit 5 short essay questions. submit 5 short essay questions. the questions you submit will count for the questions you submit will count for

1/3 of your final midterm grade.1/3 of your final midterm grade. October 30October 30thth part new stuff, part part new stuff, part

review.review.

Fre

quen

cy (

Hz)

Time (msec)

ba da ga

What is Categorical What is Categorical Perception?Perception?

Review

Methods for Testing Methods for Testing Categorical PerceptionCategorical Perception

IdentificationIdentification– Randomly play the audio clips and Randomly play the audio clips and

asked to identify the phonemeasked to identify the phoneme DiscriminationDiscrimination

– Randomly play pairs and asked to make Randomly play pairs and asked to make Same-different JudgmentSame-different Judgment Same pairsSame pairs Different pairsDifferent pairs

Continuation on Categorical Perception

IdentificationIdentification

IdentificationIdentification– Randomly play the audio clips and asked to Randomly play the audio clips and asked to

identify the phonemeidentify the phoneme If there is CP, what should the graph If there is CP, what should the graph

look like? look like? – X-axis stimuli arranged in a continuum X-axis stimuli arranged in a continuum with with

very small incremental differencevery small incremental difference between between the stimulithe stimuli

– Y-axis % Identification as the tested Y-axis % Identification as the tested categorycategory


IdentificationIdentification(idealized results)(idealized results)

% Iden

tifica

tion a

s C

ate

gory

X

0

20

40

60

80

100

2 3 4 5 6

Stimulus #

1 7


Discrimination StudyDiscrimination Study

Last lecture:Last lecture: The ba/da/ga study varied transitional The ba/da/ga study varied transitional

state (up, down of F2).state (up, down of F2).

In this example, Varying Voice Onset In this example, Varying Voice Onset Time.Time.


Voice Onset Time Voice Onset Time (VOT)(VOT)

VOT: VOT: time between consonant release and vocal cord vibration

[p][b]

So what is the difference in VOT between So what is the difference in VOT between VOICELESS [b] and VOICED [p]?VOICELESS [b] and VOICED [p]?– SHORT VOT SHORT VOT voiced voiced– LONG VOT LONG VOT voiced voiced


Voice Onset Time Voice Onset Time (VOT)(VOT)

Short VOT = ?Short VOT = ? Long VOT = ?Long VOT = ? Which one is /di/ Which one is /di/

and which one and which one is /ti/?is /ti/?

di ti


Discrimination StudyDiscrimination Study

Same/Different?0ms 60ms



Why is this pair difficult?

(i) Acoustically similar?

(ii) Same Category?


DiscriminationDiscrimination

Same/Different0ms 60ms



A More Systematic Test

0ms

20ms

40ms

20ms

40ms

60ms

D T

D

T T

D

Within-Category Discrimination is Hard


Categorical PerceptionCategorical Perception(Idealized Discrimination Data)(Idealized Discrimination Data)

% C

orr

ect

Dis

crim

inati

on

0

20

40

60

80

100

1-2 2-3 3-4 4-5 5-6

Pairs by VOT


What researchers What researchers found…found…

ContrastContrast Major Acoustic CueMajor Acoustic Cue

Place in oral stops [ba-da-ga]Place in oral stops [ba-da-ga] Start and direction of the Start and direction of the second formantsecond formant

Word initial voicing in oral Word initial voicing in oral stops [ba-pa]stops [ba-pa]

Voice Onset TimeVoice Onset Time

Place in fricative [sa-ša]Place in fricative [sa-ša] Frequency of the turbulent Frequency of the turbulent noisenoise

Word final voicing in oral stops Word final voicing in oral stops [ab-ap][ab-ap]

Duration of preceding vowelDuration of preceding vowel

Voice in final fricative [as-az]Voice in final fricative [as-az] Duration of preceding vowelDuration of preceding vowel

Place in nasal stops [ma-na]Place in nasal stops [ma-na] Start and direction of the Start and direction of the second formantsecond formant

Where did our speech Where did our speech perception abilities come perception abilities come from?from? The Motor Theory:The Motor Theory:

– – perception is informed by our innate perception is informed by our innate knowledge of articulationknowledge of articulation

The Auditory Theory:The Auditory Theory:– – speech perception is based solely on speech perception is based solely on

auditory properties of speechauditory properties of speech

Motor Theory vs. Auditory Theory

Contrasting the two Contrasting the two theoriestheories

The Motor TheoryThe Motor Theory The Auditory TheoryThe Auditory Theory

Perception is based on production (i.e., understanding of articulatory gestures speakers make)

Perception is based on genetic auditory mechanisms

Perception is species-specific

Speech production and perception evolved together

Perception is not species-specific

Production system evolved by making use of existing auditory capacities

Perception is innateTacit knowledge of articulation is given by evolution

Perception could be innate or learned

(slides adapted from J. Snedeker, C. Phillips)


Questions to AskQuestions to AskThe Motor TheoryThe Motor Theory QUESTIONSQUESTIONS

Perception is based on production (i.e., understanding of articulatory gestures speakers make)

Q3. Is speech perceptionaffected by knowledge ofarticulation?

Perception is species-specific

Speech production and perception evolved together

Q2. Is speech perception species-specific?

Perception is innateTacit knowledge of articulation is given by evolution

Q1. Is speech perception innate?

(slides adapted from J. Snedeker, C. Phillips)


Question 1Question 1

Is speech perception innate?Is speech perception innate?– Do newborns have categorical Do newborns have categorical

perception?perception? If CP requires exposure to language If CP requires exposure to language

(e.g., knowledge of minimal pairs in (e.g., knowledge of minimal pairs in one’s language), then NO.one’s language), then NO.

If CP is innate, then YES.If CP is innate, then YES.

– How do we test newborns?How do we test newborns?

Motor Theory vs. Auditory Theory Question 1

High Amplitude High Amplitude Sucking ProcedureSucking Procedure

Infant given a pacifier that Infant given a pacifier that measures sucking ratemeasures sucking rate

HabituationHabituation – Infant sucks – Infant sucks to hear sound (e.g. ba) until to hear sound (e.g. ba) until bored.bored.

TestTest – Play sound (e.g., ba – Play sound (e.g., ba or pa). Is there or pa). Is there dishabituationdishabituation??– Infants will suck to hear sound Infants will suck to hear sound

if the sound is no longer if the sound is no longer boring.boring.http://psych.rice.edu/mmtbn/language/sPerception/video/sucking_h.movhttp://psych.rice.edu/mmtbn/language/sPerception/video/sucking_h.mov

http://www.learner.org/vod/vod_window.html?pid=1620 (2:50 min. into videoclip) http://www.learner.org/vod/vod_window.html?pid=1620 (2:50 min. into videoclip)


Stimuli for Eimas et. Stimuli for Eimas et. al’s Studyal’s Study BA vs. PABA vs. PA Vary Voice Onset Time (VOT): time btw Vary Voice Onset Time (VOT): time btw

consonant release and vocal cord consonant release and vocal cord vibrationvibration

VOT in milliseconds

0 20 40 60 80

PABA


PredictionsPredictions

Between Between CategoryCategory

BABA11-PA-PA

Within Within CategoryCategory

BABA11-BA-BA22

Within Within CategoryCategory

ControlControl

BABA11-BA-BA11

Innate Innate Categorical Categorical PerceptionPerception

dishabituatdishabituatee

remain remain habituatedhabituated


Untuned Untuned SensitivitySensitivity




InsensitiveInsensitive remain remain habituatedhabituated



BA1 = VOT 20ms; BA2 = VOT 0ms; PA = VOT 40ms


Results for Eimas et. al’s Results for Eimas et. al’s StudyStudy

ME

AN

NU

MB

ER

OF

SU

CK

ING

RE

SP

ON

SE

dishab

no

no


Question 1 AnswerQuestion 1 Answer

Q1: Is Speech Perception Innate?Q1: Is Speech Perception Innate? Many other studies since tested:Many other studies since tested:

– Infants (Neonates) on other contrasts.Infants (Neonates) on other contrasts. Consensus: Yes to Innate Q. Consensus: Yes to Innate Q.

– Infants do not discriminate all physically Infants do not discriminate all physically equal acoustic difference; they show equal acoustic difference; they show heightened sensitivity to those that are heightened sensitivity to those that are important for language.important for language.

– BUT… there is language-specific fine-tuning…BUT… there is language-specific fine-tuning…


Motor vs. Auditory TheoryMotor vs. Auditory TheoryScore cardScore card

Predicted by Predicted by Motor TheoryMotor Theory– Speech Speech

perception driven perception driven by innate by innate knowledge of knowledge of articulationarticulation

Consistent with Consistent with Auditory TheoryAuditory Theory– Speech Speech

perception due to perception due to innate structure innate structure of auditory of auditory systemsystem

Q1: Is Speech Perception Innate?Q1: Is Speech Perception Innate?– Answer: YES.Answer: YES.



Is speech perception species-Is speech perception species-specific?specific?– Do other animals show categorical Do other animals show categorical

perception on the same speech sounds?perception on the same speech sounds?

– How do we test animals?How do we test animals?


?

Avoidance Avoidance Conditioning Conditioning ProcedureProcedure Kuhl & Miller (1978): test Kuhl & Miller (1978): test

chinchillas and humans with chinchillas and humans with identical stimuliidentical stimuli

Human Task: identification (d or t)Human Task: identification (d or t) Chinchillas: avoidance Chinchillas: avoidance

conditioningconditioning


Avoidance Avoidance Conditioning Conditioning ProcedureProcedure

Speech sound at one end of the Speech sound at one end of the continuum continuum paired with shockpaired with shock

Other end Other end paired with safetypaired with safety

0 20 40 60 80

VOT

SafetyShock


Avoidance Avoidance Conditioning Conditioning ProcedureProcedure

Animals learn to “avoid” shock.Animals learn to “avoid” shock. What will they do for between What will they do for between

cases?cases?

0 20 40 60 80

VOT

SafetyShock= STAY

= FLEE? ? ?? ? ?


PredictionsPredictions%

Iden

tifica

tion a

s [d

]

0

20

40

60

80

100

2 3 4 5 6

Stimulus #

1 7

Categorical Perception

Graded Perception


Kuhl & Miller (1978)Kuhl & Miller (1978)


Motor vs. Auditory TheoryMotor vs. Auditory TheoryScore cardScore card

Contrary to Motor Contrary to Motor Theory ClaimTheory Claim– That is, only That is, only

humans have humans have (innate or (innate or learned) learned) knowledge of knowledge of articulationarticulation

Consistent with Consistent with Auditory TheoryAuditory Theory– General auditory General auditory

abilities adequate abilities adequate for (some aspects for (some aspects of) speech of) speech perceptionperception

Q2: Is Speech Perception Species-Q2: Is Speech Perception Species-Specific?Specific?– Answer: NO.Answer: NO.


Some species-specific Some species-specific aspects of speech aspects of speech perceptionperception Other primates (e.g., Macaques, vervet Other primates (e.g., Macaques, vervet

monkeys, chimpanzees)monkeys, chimpanzees)– Boundary of ra/la not human like Boundary of ra/la not human like (Sinnott & Brown, (Sinnott & Brown,

1997)1997)

– Fail to make use of vowel length in consonant Fail to make use of vowel length in consonant discrimination discrimination (Sinnott, Brown, & Borneman, 1998)(Sinnott, Brown, & Borneman, 1998)

– Fail to use formant transitions alone in Fail to use formant transitions alone in consonant discrimination consonant discrimination (Sinnott & Williamson, 1999)(Sinnott & Williamson, 1999)

– Fail to categorize two different vowels in the Fail to categorize two different vowels in the same way infants learn to categorize those two same way infants learn to categorize those two vowels vowels (Kuhl, 1991)(Kuhl, 1991)

– Different similarity space for vowels Different similarity space for vowels (Sinnot, Brown, (Sinnot, Brown, Malik, & Kressley, 1997; Kojima & Kiritani, 1989)Malik, & Kressley, 1997; Kojima & Kiritani, 1989)



Is speech perception based on our Is speech perception based on our knowledge of articulation?knowledge of articulation?– Would knowledge of:Would knowledge of:

Visual information of mouth movementsVisual information of mouth movements CoarticulationCoarticulation

influence our speech perception?influence our speech perception?

– What experiments?What experiments? McGurk EffectsMcGurk Effects Co-articulation ExperimentsCo-articulation Experiments


McGurk EffectMcGurk Effect

AudiAudioo

VisuaVisuall

Your Your PerceptiPercepti

onon

baba baba baba

baba vava vava

baba thatha thatha

baba gaga dada

http://www.faculty.ucr.edu/~rosenblu/McGurkcompressB1.mov


McGurk EffectMcGurk Effect

McGurk Finding: phoneme McGurk Finding: phoneme categorization is affected by visual categorization is affected by visual informationinformation– audio BA + visual GA = percept DAaudio BA + visual GA = percept DA

Q3: Is speech perception based on our Q3: Is speech perception based on our knowledge of articulation?knowledge of articulation?– YES. Adult speech perception is affected by YES. Adult speech perception is affected by

visual cues.visual cues.– Are infants?Are infants?


Three Infant StudiesThree Infant Studies

Do infants make use of visual cues to Do infants make use of visual cues to articulation?articulation?– Kuhl & Meltzoff (1982)Kuhl & Meltzoff (1982)– Rosenblum, Lawrence, Schmuckler, & Rosenblum, Lawrence, Schmuckler, &

Johnson (1997)Johnson (1997)– Burnham & Dodd (2004)Burnham & Dodd (2004)


Kuhl & Meltzoff (1982)Kuhl & Meltzoff (1982)

Familiarlization:Familiarlization:– [a] and [i] face[a] and [i] face– NO AUDIONO AUDIO

Test Phase:Test Phase:– [a] and [i] face[a] and [i] face– AUDIO of either AUDIO of either

[a] or [i][a] or [i]– MEASURE: MEASURE:

looking time to looking time to appropriate face.appropriate face.

Preferential Looking Paradigm

Experiment 1Motor Theory vs. Auditory Theory Question 3

Kuhl & Meltzoff (1982)Kuhl & Meltzoff (1982)

Do infants know visual cues to Do infants know visual cues to articulation?articulation?– Kuhl & Meltzoff (1982)Kuhl & Meltzoff (1982)

Show that 4-5 months-old infants can match Show that 4-5 months-old infants can match sound with correct mouth shapesound with correct mouth shape

But not evidence of visual cues influencing But not evidence of visual cues influencing categorization categorization

Kuhl & Meltzoff, 1982


Rosenblum et al. (1997)Rosenblum et al. (1997)

Tested 5-months-old infantsTested 5-months-old infants– Habituated on audio VA & visual VAHabituated on audio VA & visual VA– Tested on Tested on

(1): Audio BA & Visual VA (adults perceive as VA)(1): Audio BA & Visual VA (adults perceive as VA) (2): Audio DA & Visual VA (adults perceive as DA)(2): Audio DA & Visual VA (adults perceive as DA)

– ResultsResults Dishabituate to (1)? (2)?Dishabituate to (1)? (2)?

– Infants dishabituate only to (2)Infants dishabituate only to (2) Show?Show?

– Infants are like adults!Infants are like adults!– They are sensitive to visual cues to articulation.They are sensitive to visual cues to articulation.


Burnham & Dodd Burnham & Dodd (2004)(2004)

Tested 2 groups of ½ months-old infantsTested 2 groups of ½ months-old infants

Test Group:habituationAudio BA Visual GA

Both groups then tested on:Both groups then tested on:– (1) audio BA; (2) audio DA; (3) audio THA(1) audio BA; (2) audio DA; (3) audio THA

Results: Results: Dishabituation? (1)? (2)? (3)?Dishabituation? (1)? (2)? (3)?– Control Group:Control Group: – Test Group:Test Group:

Show: Show: Dishabituated only to BA

Dishabituated to DA, THA, not BA

Again show infants are show the Classic McGurk Effect Audio BA, Visual GA DACaveat: DA ≈THA

Control Group: habituationAudio BA Visual BA


Fre

quen

cy (

Hz)

Time (msec)

Coarticulation Coarticulation RevisitedRevisited

CoarticulationMotor Theory vs. Auditory Theory Question 3

Coarticulation Coarticulation RevisitedRevisited


Coarticulation RevisitedCoarticulation RevisitedCoarticulationMotor Theory vs. Auditory Theory Question 3

Coarticulation Coarticulation RevisitedRevisited [šu][šu] shoeshoe, , [ša][ša] shoshott

[u] oo[u] oo F1: 310F1: 310 F2: 870F2: 870 [a] ah[a] ah F1: 710F1: 710 F2: 1100F2: 1100

[š] has lower frication frequency with [u][š] has lower frication frequency with [u] [š] has higher frication frequency with [š] has higher frication frequency with

[a][a]


Coarticulation & Coarticulation & Identification CurveIdentification Curve

What should What should the the identification identification curve if there curve if there is categorical is categorical perception?perception?


Coarticulation & Coarticulation & Identification CurveIdentification Curve

Contextual Contextual Effect? Effect? (consonant plus (consonant plus vowel) vowel) – [u][u]– [a][a]

We know:We know:– [š] [š] lo freq w/ [u]freq w/ [u]– [š] hi freq w/ [a][š] hi freq w/ [a]


Motor vs. Auditory TheoryMotor vs. Auditory TheoryScore cardScore card Q3: Is Speech Perception based on our Q3: Is Speech Perception based on our

knowledge of articulationknowledge of articulation– Answer: YES. Our compensation for Answer: YES. Our compensation for

Coarticulation EffectsCoarticulation Effects and response to and response to McGurk StimuliMcGurk Stimuli suggest that knowledge of suggest that knowledge of articulation influences speech perception.articulation influences speech perception.

Predicted by Predicted by Motor TheoryMotor Theory– Innate connection Innate connection

btw production btw production and perceptionand perception

Inconsistent with Inconsistent with Auditory TheoryAuditory Theory– Though perhaps Though perhaps

perceptual perceptual learning might learning might explain these explain these effectseffects


Provisional ConclusionsProvisional Conclusions

Speech Perception makes use of Speech Perception makes use of some auditory mechanisms which some auditory mechanisms which evolved prior to languageevolved prior to language– These abilities are innateThese abilities are innate

Speech Perception also makes Speech Perception also makes use of our knowledge of use of our knowledge of articulationarticulation– – These abilities are likely innateThese abilities are likely innate

Motor Theory vs. Auditory Theory General Conclusions

Becoming a Native Becoming a Native ListenerListener Languages differ in their Languages differ in their

inventories of phonemes.inventories of phonemes.

What develops or changes in our What develops or changes in our speech perception abilities?speech perception abilities?

Language Specific Fine Tuning

Japanese vs. EnglishJapanese vs. English(Miyawaki et al. 1975)(Miyawaki et al. 1975)

RA

LA

AMERICANS


Dental Stop – tip of tongue touching back of front teeth

Retroflex Stop – tongue curled so tip is behind alveolar ridge

Hindi (spoken in India)

unvoiced unaspirated retroflex vs. dental stop

(English /t/ is typically somewhere between the two)


Can you hear the Can you hear the difference?difference?

Hindi

dental

retroflex


Uvular – tongue is raised against the velum

Velar – tongue is raised behind the velum

Salish (Native North American language):

glotalized voiceless stops

(they are actually ejectives - ejective is produced by obstructing the airflow by raising the back of the tongue against or behind the velum)


When does changes in When does changes in sensitivity occur?sensitivity occur?


Infancy

Adulthood

… And testing method?

Conditioned Head-TurnConditioned Head-Turn ConditioningConditioning

– Child hears a Child hears a string of sounds.string of sounds.

– Conditioned to Conditioned to turn head when turn head when detects a change detects a change (e.g., bell (e.g., bell whistle) with whistle) with rewardreward

TestTest– Speech sounds Speech sounds

(e.g., da, da, da, (e.g., da, da, da, da, ta,…)da, ta,…)

– Does the child Does the child turn his or her turn his or her head with head with changed from da changed from da to ta?to ta?


Werker: http://www.learner.org/vod/vod_window.html?pid=1630Kuhl: http://www.learner.org/vod/vod_window.html?pid=1631

When does Change When does Change Occur?Occur?

6-8m 8-10m 10-12m 11-12m

6-8m 8-10m 10-12m 11-12m


What is changing? What is changing? Two contrasting views: 1 or 2?Two contrasting views: 1 or 2?

Maintenance or LossMaintenance or Loss– If you don’t use it, you lose it.If you don’t use it, you lose it.– Parallel aspects of early visual Parallel aspects of early visual

development.development. Functional ReorganizationFunctional Reorganization

– Existing architecture reorganized for Existing architecture reorganized for higher level of processing.higher level of processing.


What is changing? What is changing? Two contrasting viewsTwo contrasting views


What is changing? What is changing? 1. Maintenance or Loss View1. Maintenance or Loss View


Structure-changingNon-native boundariesdisappear.Resulting in native language phonetics

Phonetics

Acoustics

Phonology

What is changing? What is changing? 2. Functional Reorganization2. Functional Reorganization


Structure-buildingNative languagephonemesbuilt fromuniversal phones

Phonetics

Acoustics

Phonology

Which view?Which view? Werker (1997) noted some problems for Werker (1997) noted some problems for

the the maintenance or loss viewmaintenance or loss view..1. Many of the uncategorized sounds do appear in the 1. Many of the uncategorized sounds do appear in the

native language but just are not meaningful (e.g., as native language but just are not meaningful (e.g., as allophones), and speakers can be made aware of the allophones), and speakers can be made aware of the difference.difference.

Example:Example:– /p/ is only /p/ is only aspiratedaspirated in “pin” and not “spin’ in “pin” and not “spin’– /p/ in “pin” and “spin” are /p/ in “pin” and “spin” are allophonesallophones in English in English– But could be But could be minimal pairsminimal pairs in some other in some other

languages.languages.


Which view?Which view? Werker (1997) noted some problems for the Werker (1997) noted some problems for the

maintenance or loss viewmaintenance or loss view..1. Many of the uncategorized sounds do appear in the native 1. Many of the uncategorized sounds do appear in the native

language but just are not meaningful (e.g., as allophones), and language but just are not meaningful (e.g., as allophones), and speakers can be made aware of the difference.speakers can be made aware of the difference.

2. Children who fail to show categorical perception for non-native 2. Children who fail to show categorical perception for non-native phonemes can acquire a new language without an accent.phonemes can acquire a new language without an accent.

3. Adults can be trained to make non-native distinctions.3. Adults can be trained to make non-native distinctions.

4. Perceptual distinction is readily available for non-linguistic 4. Perceptual distinction is readily available for non-linguistic tasks.tasks.


Which model?Which model?

Werker (1997): The evidence that poses Werker (1997): The evidence that poses problems for maintenance or loss view problems for maintenance or loss view supports the supports the functional reorganization functional reorganization viewview..

I.e., the view that:I.e., the view that:– Those perceptual categories which are Those perceptual categories which are

meaningful in the native language become meaningful in the native language become speech categories.speech categories.

– The remainder are perceived but not The remainder are perceived but not recruited in speech perception.recruited in speech perception.


Why functional Why functional reorganization?reorganization? Developmentally: 10-12 months-Developmentally: 10-12 months-

olds lose non-native distinctionsolds lose non-native distinctions What is going on?What is going on?

– Analyzing statistical regularities in Analyzing statistical regularities in the input language and working the input language and working towards word learning? towards word learning?

(more to come when we study language (more to come when we study language acquisition)acquisition)


Phonetics Articulatory

innate

Innate componentsInnate components

Phonetics serves as a connectionPhonetics serves as a connection– McGurk effectMcGurk effect

Articulatory rudimentsArticulatory rudiments– Infants babbleInfants babble

Auditory

Auditory abilitiesAuditory abilities– Categorical PerceptionCategorical Perception


Syntactic

Lexical

Phonological

Auditory Phonetics Articulatory

innate

constructed

Constructing Constructing phonologyphonology


Psycholinguistics Lecture 7 Speech Perception II.

Documents

Transcript of Psycholinguistics Lecture 7 Speech Perception II.