Facial identity and expression perception in the human visual system Human Vision and Eye Movement...
-
Upload
stewart-wilson -
Category
Documents
-
view
215 -
download
0
Transcript of Facial identity and expression perception in the human visual system Human Vision and Eye Movement...
Facial identity and expression perception in the human visual system
Human Vision and Eye Movement Laboratory
Medicine (Neurology)
Ophthalmology and visual sciences
Psychology
University of British Columbia
NANOS 2011 Vancouver
DISCLOSURES:
NO COMMERCIAL/PROPRIETARY INTERESTS
FUNDING SUPPORT:
NIMH 1R01 MH069898
CIHR MOP-77615, MOP-85004, MOP-102567, MOP-106511
NSERC RGPIN 355879-08
Michael Smith Foundation for Health Research CI-SSH-035(03-1)
Canada Research Chairs 950-202111
Student support from MSFHR, CIHR, Alzheimer’s Foundation
Canada Research Chairs
Bruce and Young (1986) cognitive model
COMMON CONCEPT: perception of identity and expression emerge from divergent streams of face processing
EXPRESSION
Encode facial dynamics
Invariant for static structure
• allow generalization across people
IDENTITY
Encode static structure
Invariant for dynamic changes
•allow stable recognition despite changes in expression and age
COMMON CONCEPT: perception of identity and expression emerge from divergent streams of face processing
• Are expression and identity functionally and structurally independent?
If so, this would imply that the human brain has evolved highly specific systems for extracting different structural properties of the same object.
• 1. Behavioural studies
• 2. Anatomic fMRI studies
• 3. Patient studies
1. BEHAVIOURAL INVESTIGATION
use FACE-ADAPTATION to probe the functional relationship between the perception of facial expression and perception of facial identity
1. Behavioural adaptation
90/10 70/30 50/50 30/70 10/90
fear anger
a. Expression adaptation: are these invariant for the identity of the face?
Method - PERCEPTUAL BIAS TECHNIQUE- create a morph continuum between two
expressions:
- stare at one ‘end-face’ for 5 sec.. ..then flash a 300ms ambiguous morph face..
..and ask a question:
“Did you see fear or anger?”
“EXPRESSION AFTER-EFFECT”: Subjects are more likely to perceive the other expression
ambiguous stimuli
1. Behavioural adaptation
5-second Adapting stimulus
300 ms Probe stimulus
(*three expression pairs: Angry/afraid; happy/sad; disgusted/surprised)
Same person
Different person,Same gender
Different person,Different gender
• Changing identity reduces but does not eliminate the expression after-effect.
• Expression representations may have two components:
1. Behavioural adaptation
}identity-dependent component
}
identity-invariant component
Afte
reffe
ct m
agn
itud
e (%
)
90/10 70/30 50/50 30/70 10/90
“Bob” “Joe”
b. Identity adaptation: is it invariant for the expression of the face?
Method - create a morph continuum between two different people’s faces:
- after adaptation to BOB, subjects are more likely to respond that a following ambiguous face is more like JOE.
1. Behavioural adaptation
Is the identity aftereffect partially dependent on expression?
1. Behavioural adaptation
Same expressionDifferent expression
Answer: NO. In contrast to expression aftereffects, there is no expression-dependent component of identity aftereffects.
Afte
reffe
ct m
agn
itud
e
Would we find expression-dependency in the identity after-effect for more familiar faces?
notfamiliar
minimallyfamiliar
culturallyfamiliar
veryfamiliar
Answer: NO
1. Behavioural adaptation
Afte
reffe
ct m
agn
itud
e
Both contribute to expression adaptation
identity adaptation
There is an interesting asymmetric relationship between identity and expression!
One possible (speculative) interpretation, influenced by network models….
1. Behavioural adaptation
1. STRUCTURAL INVESTIGATION
Use fMRI-ADAPTATION to probe the structural relationship between perception of facial expression vs. identity
Fusiform face area Superior temporal sulcus
Expression morphingIdentity m
orphing2-D Morph Matrix with “categorical boundaries”:
EX
PR
ES
SIO
N B
OU
ND
AR
Y
2. fMRI adaptation
• Ultimately, what mattered most in this experiment was not the stimulus but the perceptual decision - whether the subject said that they saw the face as similar or different.
IDENTITY BOUNDARY
2. Rapid event-related fMRI adaptation study
identical establishes baseline of full adaptation
In all of these, face 2 is 33% morph different from face 1.
Subject sees pairs of faces:
Do FFA and STS differ in adaptation effects induced by:
• physical changes in expression vs identity?
• perception of difference in expression vs identity?
Subjects perform 2 runs with the same stimulus set:
•Expression task
•Identity task
similar expression
different expression
identity is identical (held constant):
similar identity
different identity
expression is identical (held constant):
same samediff diffidentity expression
FUSIFORM FACE AREA (FFA)
* FFA is sensitive to any structural change
2. fMRI adaptation
Baseline = adaptation for repetition of identical face
Release from adaptation (area ‘detects’ a difference):
# FFA adaptation release is more when subject perceives a difference in EITHER identity or expression
(as reported by subject)
SUPERIOR TEMPORAL SULCUS (STS)
identity task
expression task
samediff diffidentity expression
same samediff diffidentity expression
same
• STS is modulated by TASK - differences only seen when subject is actively processing expression
2. fMRI adaptation
DURING EXPRESSION TASK:
* STS is sensitive to any stimulus change,
# STS adaptation release is more when subject perceives a difference in EITHER identity or expression
Conclusions of fMRI experiment
• FFA and STS activity is sensitive to physical properties of face,
• Correlate with perceptual report of subject, for both identity AND expression
• STS is recruited during expression task
2. fMRI adaptation
• Identity and expression specific signals not seen until later in cortical hierarchy:
Identity – precuneus Expression – middle STS
1. NEUROPSYCHOLOGY
Differential impact of LESIONS on the perception of facial expression and perception of facial identity
3. Lesion studyRegion of interest
Subject
prosopagnosic
prosopagnosic
prosopagnosic
fMRI localization data in patients
expression
ide
ntit
y
Using 2-D morph matrix, create 4 oddity paradigm tests:
Which face is the different expression?
Identity varies
Expression varies Expression fixed
Identity fixed
Which face is the different person?
We vary the level of difficulty by choosing morph faces of varying distance in the matrix
3. Lesion study: are expression and identity differentially impaired?
3. Lesion study
Identity task Expression task
expression varying
expression fixed
identityvarying
identity fixed
• Identity but not expression impaired by lesions of:
OFA/FFAAnt temporal lobeMedial fusiform
• Expression but not identity impaired by lesion of:
STS
• Expression constancy of identity judgments impaired by STS lesion!
Other dimension fixed
Impairedexpression only
Impairedexpression ANDImpaired identity
ImpairedIdentity only
normal
Summary
Behavioural adaptation results: • identity representations are expression-invariant• expression representations have identity-dependent and identity-invariant components
fMRI-adaptation results:• effects of perceptual decision for both expression and identity are REFLECTED in the activity of both FFA and pSTS (does NOT mean that they are encoded there).
Lesion results:• “fusiform - anterior temporal stream” lesions impair identity perception but NOT expression perception• STS lesion impaired expression perception and impedes identity judgments when expression must be discounted.
1. Behavioural adaptation
Middle STS
Posterior STS
Insula Amygdala
FFA
Anteriortemporal
Precuneus?
Other fusiform regions
Bicycle tour of Vancouver! WEDNESDAYmeet at registration desk 1200h
Behavioural adaptation: • Fox CJ, Barton JJS. What is adapted in face adaptation? The neural representations of expression in the human visual system. Brain Res 2007; 1127: 80-9.• Butler A, Oruc I, Fox CJ, Barton JJS. Factors contributing to the adaptation aftereffects of facial expression. Brain Res 2008; 1191: 116-26.• Fox CJ, Oruc I, Barton JJS. It doesn’t matter how you feel. The facial identity aftereffect is invariant to changes in facial expression. J Vision 2008; 8(3): 11.1-13.fMRI-adaptation:• Fox CJ, Moon S-Y, Iaria G, Barton JJS. The correlates of subjective perception of identity and expression in the face network: an fMRI adaptation study. Neuroimage 2009; 44: 569-80.Lesions:• Fox CJ, Iaria G, Duchaine BC, Barton JJS. Behavioral and fMRI studies of identity and expression perception in acquired prosopagnosia. Vision Sciences Society, Naples 2008
PERSONNELAndrea Butler Brad Duchaine Chris Fox Giuseppe IariaSo Young Moon Ipek Oruç
3. Lesion study
Desired test properties:
• equivalent perceptual difficulty, • controls perform well but not at ceiling• low variance
CONTROL DATA: