Sensation and Perception (II)3270

Speech

Keywords on hearing (introduction)

auditory canal, ear drum, ossicles, oval window, cochlea, helicotrema, basilar membrane, tectorial membrane, hair cells, kinocilium, stereocilia,

amplification (by ossicles & area difference between ear drum and oval window), travelling wave, resonance, tonotopic coding, cochleotopic coding, transduction

auditory thresholds, effect of age, different animals, fundamental, harmonics, timbre, pitch/frequency, loudness/amplitude, pure tone, equal loudness, masking

place theory, periodicity theory, duplicity theory, missing fundamental, goldfish has no basilar membrane - can distinguish freqs, phase-locking, diplacusis,

PHONEMES

“sounds that create meaning”

47 in English

VOICING PLACE OF MANNER OFARTICULATION ARTICULATION

voiced alveloar ridge stop unvoiced labiodental fricative

etc… nasaletc…

http://nationalstrategies.standards.dcsf.gov.uk/node/85357

time

Formant 3

Formant 2

Formant 1

Formant transitions

Formants

SEGMENTATION PROBLEM

Segmentation problem

Where one sound is influenced by what sound came before or after because of CO-ARTICULATION

Same sounds, different spectrographs...

VARIABILITY PROBLEM

IS SPEECH SPECIAL?

1. MOTOR THEORY OF SPEECH

-- mirror neurones?

http://www.teachersdomain.org/asset/hew06_vid_mirrorneurons/

IS SPEECH SPECIAL?

2. CATEGORICAL PERCEPTION

Voice Onset Time (VOT)

ShortVOT

LongVOT

“SAME”“DIFFERENT”

IS SPEECH SPECIAL?

3. SPEECH PERCEPTION IS MULTIMODAL

McGurk Effect

SOUND

VISION

Video demos

1. McGurk effect

video=gasound = bacombo= da

2. Vision helps degraded sound

IS SPEECH SPECIAL?

4. Are there INVARIANTS for phonemes?

-- practical importance-- some hints-- varied by neighbouring sound-- different conditions eg. speed of speech-- different speakers-- but TOP DOWN INFLUENCES

TOP-DOWN INFLUENCES

1. Segmentation influenced by meaning

Anna Mary Candy Lights Since Imp PulpLay Things.

I scream, you scream, we all scream for ice cream!!

TOP DOWN INFLUENCES

1. Segmentation influenced by meaning2. Semantics (meaning) and syntax (grammar)

both influence your ability to shadow (repeat what you have heard) a text.p

erce

nt

corr

ect 89%

79%

56%

normal s

entences

anomalous sentences

ungrammatic

al sentences

TOP DOWN INFLUENCES

1. Segmentation influenced by meaning2. Semantics (meaning) and syntax (grammar) both

influence your ability to shadow (repeat what you have heard) a text.

3. Phonemic restoration.

Time to meet with their respective legi laturess *

cough

time to aves the countryw the flag

Pink noise demo: http://www.youtube.com/watch?v=k74KCfSDCn8

Legislatures: http://www.youtube.com/watch?v=UlJs24j3i8E&feature=related

SUMMARY “Is speech special?”

1. Motor theory of speech2. Categorical perception3. Multimodal aspects4. Invariants

Top down influences(i) Segmentation influenced by meaning(ii) Shadowing influenced by meaning(iii) Phonemic restoration

PHYSIOLOGY OF SPEECH PERCEPTION

Might there be phoneme detectors in the brain?

Voice onset time (ms)0 20 40 60 80

Per

cen

tag

e h

eard

as

“da”

100%

50%

0%

Per

cen

tag

e h

eard

as

“ba”0%

50%

100%

/da/

/ba/

ADAPT TO /ba/

/ba/ /ba/

/ba/

/ba/

/ba/

/ba/

/ba/

/ba/

/ba/

/ba/

/ba/

/ba/

After adapting to /ba/……

Voice onset time (ms)0 20 40 60 80

Per

cen

tag

e h

eard

as

“da”

100%

50%

0%

Per

cen

tag

e h

eard

as

“ba”0%

50%

100%

/da/

/ba/

As if that wasn’t cool enough…..

What happens after adapting to a McGurk effect?

Sound = /ba/

Vision = /ga/

Perception = /da/

acoustic

perception

After adapting to McGurk

Voice onset time (ms)0 20 40 60 80

Per

cen

tag

e h

eard

as

“da”

100%

50%

0%

Per

cen

tag

e h

eard

as

“ba”0%

50%

100%

/da/

/ba/

If /ba/ (acoustic) adapts

If /da/ (perception) adapts

??

It is the ACOUSTIC signal that adapts

Voice onset time (ms)0 20 40 60 80

Per

cen

tag

e h

eard

as

“da”

100%

50%

0%

Per

cen

tag

e h

eard

as

“ba”0%

50%

100%

/da/

/ba/

Acoustic adapts!!!

Broca’s aphasia http://www.youtube.com/watch?v=f2IiMEbMnPM

Wernicke’s aphasia http://www.youtube.com/watch?v=B-LD5jzXpLE

Cells in cortex show:

• Phoneme information carried in a population code

• Special features in auditory cortex (eg, frequency sweeps) compatible with phonemic features

Background resting activity -- note more activity in frontal regions

Green = averageblue = lessred = more

Looking around

Frontal eye fields

Visual cortex

Listening to words

WERNICKE’S AREA

Auditory cortex

Counting out loud

Supplementarymotor area

BROCA’S AREA(speech production)

WERNICKE’S AREA(speech understanding)

Internal speech -- counting in your head

?????

Internal speech -- counting in your head

Frontal regions

NOTBroca or Wernicke’sareas !!!!

olfactory binding protein, olfactory receptors cells continuously regenerate (about every 60 days), cilia (on olfactory receptor cells), glomerulus (contact zones between receptor cells and mitral cells:plural glomeruli), convergence (1,000:1), mitral cell, olfactory tubercle of cortex (part of paleocortex), medial dorsal nucleus of thalamus , orbitofrontal cortex, olfactory neocortex

paleocortex associated with limbic system, limbic system associated with emotions (electrical stimulation causes sham rage), limbic system associated with memories (H.M. had lesions here and lost the ability to memorize things), some hot spots in olfactory tubercle and on olfactory mucosa

KEYWORDS -- SMELL I

odour quality, no primaries identified in olfactory system, poor tuning of receptors (to chemicals or chemical types) (sharpened by lateral inhibition, inhibitory interneurones, granule cells), Henning smell prism, stereochemical theories based on lock and key partially successful, BUT no receptor sites identified, similar shaped molecules can be associated with different smell perceptions

cells broadly tuned (responding to many different chemicals associated with many different smells)

codingintensity= firing rate/recruitment, quality = distributed pattern code, problems in identifying many smells at once, binding problem

KEYWORDS -- SMELL II

odour thresholds, olfactorium; unique technical problems!, humans very sensitive (eg. mercaton can be detected at 1 part per 50,000,000,000), affected by gender; can be affected by menstrual cycle, affected by age

adaptation, thresholds raised (by exposure), masking (by other chemicals), some cross effects: eg. adapting to orange affects smell of lemons

identification, can identify gender from shirt, prefer own odours, odour memories long lasting; associated with emotions (via limbic system) "designed not to forget”, pheromones, releasers (immediate effect), eg. bitch on heat, territorial markers, humans?, McClintock effect (synchronized menstrual cycles), primers (longer term) eg. mice need males around for proper oestrus cycles

KEYWORDS -- SMELL III

PATHWAYS olfactory receptor cells to mitral cells in olfactory bulb to olfactory tubercle in paleocortex THEN

1 to medial dorsal thalamus to olfactory cortex

(ORBITOFRONTAL CORTEX) 2 to limbic system 3 brain stem pathways associated with pheromones

ALSO

inhibitory pathway (via inhibitory interneurone: granule cells) from one olfactory bulb to the other to do with detecting the DIRECTION from which a smell originates

KEYWORDS -- SMELL IV