Human Social Interaction perspectives from neuroscience Dr. Roger Newport Room B47 Student Drop-in...
-
Upload
sierra-emery -
Category
Documents
-
view
216 -
download
1
Transcript of Human Social Interaction perspectives from neuroscience Dr. Roger Newport Room B47 Student Drop-in...
Human Social Interactionperspectives from neuroscience
Dr. Roger NewportRoom B47
Student Drop-in Time: Tuesdays 12-2
www.psychology.nottingham.ac.uk/staff/rwn
Understanding Emotion: auditory emotion recognition
1
This lecture:Recognising emotion from prosodyTheories of emotion processing Current Research
Understanding emotion from auditory cues: prosody
Prosody is the melody or musical nature of the spoken voice - conveyed by changes in e.g pitch, syllable duration, volume
We are able to differentiate many emotions from prosody alone e.g. anger, sadness, happiness
Universal and early skill
What are the neural bases for this ability?
Are they the same as for language? Are they the same as for differentiating emotion from visual cues?
2
We can tell the difference between a spontaneous and a mechanical smile
We think we can tell something about size and attractiveness from the sound of someone’s voice
Babies can differentiate between the sounds of voices before they can understand speech
We can understand cartoon characters even though they do not speak
Children can produce the melody or intonation of speech before they can produce two-word combinations
Prosody skills - innate and special abilities 3
Uses of prosody in the film industry
Trombone Toy whistle BadgerWalrusMad DogStarving Bear
4
Emotion from prosody - the same as from facial expression?
Emotion from facial expressions reminder from last week:
Clear evidence for involvement of amygdala in fear recognition and the insula / striatum in the recognition of disgust
Not so clear for other emotions
Does the same hold true for recognising emotions from the expression of the voice?
5
Emotion from prosody, the same as from facial expression?
The insula and disgust
Most emotions identified at 50-60% accuracy (chance = 10-20%)
Disgust is nearly impossible to recognise from prosody
Very very difficult to measure in experimental conditions
Therefore very little successful research on recognition of disgust from prosody
6
Evidence from human neuropsychology - the amygdala and fear
Adolphs et al, 2001Recognition of emotion from prosody in 15 unilateral left, 11 unilateral right amygdala and 50 BD controls.No differences between groups(But bilateral amydala damage is usually necessary to abolish facial fear recognition)
Adolphs and Tranel, 19992 complete bilateral amygdala (inc. SM), 15 BD and 14 NC controls. SM normal emotion recognition from prosody
Amygdala’s role not as critical for prosody as facial emotion
Which regions might be involved? Are prosody and facial skills dissociable?
7
Evidence from human neuropsychology for right hemisphere involvement - Patient KB and amusia
Music activates brain regions that are also associated with emotion processing
Trained musicians are better at identifying emotion from prosody and tonic imitations of prosody
Children who study keyboard vs drama/nothing are also better
KB: amusic following RH stroke unable to discriminate pitch or rhythm patterns in linguistic or musical stimuli. Also impaired on prosodic perception tasks (e.g., discriminating statements from questions)
8
Evidence from human neuropsychology - hemispheric asymmetry
Barrett, et al., (1999) - patient with large left hemisphere lesionNormal emotional prosody yet severe inability to process propositional speech.
Schmitt, et al., (1997)27 RH patients; 25 LH disproportionately impaired recognition of emotion from facial and prosodic cues in the RH group when judging multimodal video clips
Peper and Irle (1997) RH disproportionatelyimportant for processing of emotion from prosody
Pell (2005) RH (mixed damage!) impaired at emotion from prosody; LH impaired at interpreting prosodic code within language content
9
Adolphs et al., 2002:Lesion analysis66 brain damaged subjects
Brain Damaged Patient Groups Human imaging (TMS) 10
Van Rijn et al. 2005
1Hz TMS for 12 minutes
Somatosensory area associated with lips/tongue/jaw
Slowed RTs to ‘withdrawal’ emotions (fear/sadness), but not approach (e.g. happy)
Human imaging of voice selective cortex
Belin et al., 2000
11
BellsHuman non-vocal soundsNoise (amplitude modulated)Vocal soundsScrambled voices
Friederici and Alter (2004) prosodic data adapted from Plante et al. 2001
Using fMRI, Buchanan et al. showed that the detection of emotional prosody is associated with increased activation in the right hemisphere (inferior frontal lobe and right anterior auditory cortex),
Human imaging of voice selective cortex 12
Wildgruber et al., 2005 fMRI experiment
Designed to separate phonetic from affective prosodic components
Emotionally neutral spoken sentences such asDer Gast hat sich für Donnerstag ein zimmer reserviert orDie Anrufe werden automatisch beantwortet
Read with 5 different emotions (happiness, sadness, anger, fear and disgust)
Tested for recognition behaviourally prior to fMRI(all 90-95% accuracy except disgust (77%).
2 tasks: say the emotion; say the vowel after the first ‘a’.
13
fMRI results:Disgust (and fear) recognition dropped to near 50% accuracy
14
* NS
The trouble with fMRI: both tasks involve listening and automatic processing of linguistic, syntactic, phonological and prosodic information as well as motor responses
So get activation of auditory cortex, phonolgical store, supplementary motor areas etc.
15
Use subtraction method:
2 areas associated with emotion identification: Right STS and right inferior frontal cortex. rIFC involved in emotion comprehension of both facial and prosodic cues.
16
Attend to left or right ear. Make gender judgment
Sander et al., 2005
Prosody, anger and attention
17Imaging of specific emotions
The recognition of emotion from prosody is not analogous to the recognition of emotion from facial expression
Recognizing emotional prosody draws on multiple structures distributed between left and right hemispheres
The roles of these structures are not all equal but may be most apparent in processing specific auditory features that provide cues for recognizing the emotion
Despite the distributed nature of the processing, the right hemisphere appears most critical - in particular the right inferior frontal regions, working together with superior temporal region in the right hemisphere, the left frontal regions, and subcortical structures, all interconnected by white matter.
Summary 18
Break
So far we have looked at
Types of emotion expression (facial/prosodic)
Brain regions associated with various emotions/types of expression
So we know what the brain does with emotionally expressive stimuli and where might be important for doing this, but we have not looked at HOW the brain might process emotionally expressive stimuli
Broadly speaking there are 2 main theories of emotion processing:
1. Theory theory2. Simulation theory
20
Theory theory:
Children as young as 5 have extensive causal knowledge in the form of intuitive theories
Screening off task
21
visually obtained knowledge of the facial configuration of the target
semantic knowledge concerning facial configurations
general knowledge concerning a given emotion, i.e. its typical elicitors or behavioural effects
knowledge that facial configuration ‘C’ is paired with emotion label ‘E’
Theory Theory of emotion recognition
information-based account
employs naiive psychological ‘folk’ theory to infer the emotional states of others
22
Simulation theory
We interpret the emotions of others by covertly simulating their response matching the outcome to our knowledge of outcomes
How might this work for emotion recognition?
Person A sees person B pulling facial configuration ‘C’A covertly facsimiles ‘C’ (or what she thinks ‘C’ to be)A attributes resulting emotion label ‘E’ to person B
23
Paired emotion deficits on FaBER tasks
E.g. patients SM and NM (Fear)
SMBilateral amygdala damage, neighbouring areas sparedDid not express fear, could not recognise expression of fear in others
Knew what fear was supposed to beKnew what should cause itEven knew what response might beBut could not show it.Unable to learn the significance of unpleasant situationsDoes not show fear conditioning
Evidence for simulation theory account of emotion recognition
Paired deficits for fear
24
Paired deficits for disgust
Imaging studiesPhilips et al, 1997, 1998 fMRIObserving FaBER disgust activates right insulaInsula known to involved in experience of unpleasant tastes and smells
Wicker et al., 2003. fMRI watching video of facial expressions in response to pleasant or disgusting smell vs. experiencing pleasant or disgusting smellBoth disgust conditions preferentially activated left ant. insula and right ant. cingulate cortex.
Patient studiesNK (Calder et al., 2000) Insula and BG damage. Paired impairment on disgust measures (questionnaire and FaBER)
25
Dopamine system: a neural subsystem involved in the processing of aggression in social-agonistic encounters in a wide variety of species
Dopamine system plays an important role in mediating the experience of the emotion of anger.
Dopamine levels in rats and other species are elevated in social-agonistic encounters.
Administration of dopamine antagonist (e.g. sulpiride) selectively impairs responses to agonistic encounters.
Sulpiride administration leads to selective disruption of FaBER for anger, while sparing recognition of other emotions.
Following sulpiride administration, subjects were worse at recognizing angry faces, but no such FaBER impairments of other emotions.
Paired deficits for anger 26
Generate and Test
A deficit in the production of an emotion (or its facsimile) leads to an impairment in the recognition of that emotion
How is this done?Random? Too slowTheory? How?
How is this done?V to P matchingLearnt or innate?
27
Generate hypothesised emotion
Produce a facial expression
Test expression(does it match expression of
other?)
Classify own emotional state and attribute this to other
No
Yes
Activation of facial musculature precedes emotion
Reverse simulation
28
Visual representation of other’s facial expression
Activation of facial muscles that imitate other’s facial expression
Experience of emotion
Classify own state and attribute that to other
Such imitation is innate+ adults covertly mimic FaBER stimuli (measurable by EMG) (Dimberg et al.)
Engages cognitive processes in reverse.Emotional state and facial expression are bidirectionalSensation of emotion --> facial expressionFacial expression --> mild sensation of emotion
Reverse simulation
A direct link from visual input of other’s face to somatosensory representation of what it would feel like to pull that face. Bypasses muscle activation so gets round Mobius problem, but no details of how link might work. 29
Visual representation of other’s facial expression
Activation of facial muscles that imitate other’s facial expression
Experience of emotion
Classify own state and attribute that to other
As if
Observation of other’s facial expression
Automatic activation (mirroring) of neural systems associated with facial emotion
Shared emotion Labelling of emotion
Unmediated resonance model or shared manifold hypothesis
Direct activationRequires no mediating structures or processes
30
Wild et al., 2003
You are slower to make an incongruent facial movement than a congruent one
Contagious emotions
31
visually obtained knowledge of the facial configuration of the target
semantic knowledge concerning facial configurations
general knowledge concerning a given emotion, i.e. its typical elicitors or behavioral effects
knowledge that facial configuration ‘C’ is paired with emotion label ‘E’
Theory Theory of emotion recognition
Can you lesion this model?
32
Summary
Research into understanding emotions has revealed that deficits in face-based recognition are paired with deficits in the production of the same emotion.
Of theory and simulation approaches the simulation theory seems to offer the best explanation of the data.
The precise mechanisms by which simulation theory might work are still unclear, but reverse models(with as if loop) and the more recent unmediated resonance model can both account for neuroscientific data
But some people still don’t believe it
33
Audiovisual integration of emotional signals in voice and face:An event-related fMRI study
There has been plenty of research into visual emotion recognition
And some research into auditory emotion recognition
But (almost) no-one has studied audiovisual integration of (dynamic) emotional stimuli
Behavioural and Event-related fMRI taskViewed and heard faces and words either A only, V only or AVNeutral + 6 basic emotions (surprise!)Looked for areas of AV overlap that were not A or V (AV>A)∩(AV>V)
Current Research July 2007
Rationale
Methods
34
HappySadFearDisgustAnger?
Results
Imaging results:Bilateral posterior superior temporal gyrus activation and right thalamus (not shown)
Behavioural results:People were better when both A and V information available
This is what they predicted (honest).35
Amygdala damage impairs emotion recognition from music
We know the amygdala is implicated in the recognition of fear (faces)
Patients following temporal lobe removal (inc. amygdala) are impaired at scary music recognition.
Is the amygdala (specifically) necessary for scary music recognition?
Current Research September 2006
Rationale
Methods
Neuropsychology (Patient SM)Music discrimination and emotion recognition /rating tasks
Results
SM ok at discriminationPoor at negative emotion music recognition (sad and scary) 36
Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing. TICS
Current Research September 2006
Review article
37
Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing
Current Research September 2006
Review article
38
Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing
Current Research September 2006
Review article
39
Behold the voice of wrath: Cross-modal modulation of visual attention by anger prosody. Brosch et al. 2008
Current Research 2008
We know the amygdala directs our attention to visual socially relevant stimuli
We also know that the amygdala responds to anger prosody
Does anger prosody direct our visual attention?
Rationale
Methods
Dichotic listening with cueing paradigm
Results
Yes it does. Visual targets detected faster when on same side as anger delivered 40
Emotional prosodic processing in auditory hallucinations
Current Research 2007
Schizophrenics are impaired at prosody recognition
Prosodic cues are important for speaker identity
Could prosodic deficit be responsible for misattribution of voices in auditory hallucinations
Rationale
Methods
Rate emotional (but semantically neutral) spoken sentences from Sad to Happy on Likert scaleGroups: normal, Schz with hallucinations, Schz without
ResultsOnly hallucinating patients were impaired compared to controls
41
Next lecture: revision/FAQ lecture on the 3rd of November.
What to do now.
Don’t panic (I’ll tell you when).
Read some articles and start planning and writing up an experiment
Submit revision questions using feedback page before revision/feedback lecture on November the 3rd