Reading in the brain 2. Masking, subliminal reading, and ... · Our research strategy - The...
Transcript of Reading in the brain 2. Masking, subliminal reading, and ... · Our research strategy - The...
Reading in the brain
2. Masking, subliminal reading, and the mechanisms of
conscious access
Stanislas Dehaene
Collège de France,and
INSERM-CEA Cognitive Neuroimaging Unit
NeuroSpin Center, Saclay, Francewww.unicog.org
Non-conscious
Conscious
Image by Claire Sergent
The visual word form area
A subpart of the occipito-temporal visual system that systematicallyresponds to writtenwords.
A hierarchicallyorganized region
A short demonstration of digit masking
Two central issues in consciousness research
1. The depth of subliminal processing- How deeply can subliminal stimuli be processed into the visual system?
2. The nature of conscious access- What prevents subliminal stimuli from becoming conscious?- What processes occur when a stimulus is consciously accessed, and
when do conscious and non-conscious processing diverge?
Our research strategy- The contrastive method: « … contrasting pairs of similar events, where
one is conscious but the other is not. » (Baars, 1989)- The primacy of the subjective: « …the first crucial step is to take
seriously introspective phenomenological reports. (…) They constitute primary data that need to be measured and recorded along with other psychophysiological observations » (Dehaene & Naccache, 2001)
In summary, look for:objective neurobiological correlates of subjective processes
Part I. The fate of a non-conscious stimulus
• Masked words and digits elicit activity at a series of hierarchical levels, including visual, semantic and motorstages.
• fMRI priming can help decipher how information is codedin a given region.
time
RADIO29 ms
29 ms
271 ms
29 ms
Making a word invisible: the masking paradigm
Dehaene et al., Nature Neuroscience, 2001
Invisible word
Time
radio
RADIO29 ms
29 ms
271 ms
500 ms
29 ms
Left fusiform(-44, -52, -20)
SameCase
DifferentCase
595
600
605
610
615
620
625
Res
pons
e tim
e (m
s)
Same wordDifferent word
Behavioral priming
Same wordDifferent word
0
0.1
SameCase
DifferentCase
Act
ivat
ion
(%)
C
fMRI priming
0
0.1
similar dissimilar
Left fusiform-48, -52, -12
Perc
ent s
igna
l cha
nge
Case-invariant priming independent of letter
similarity
e.g. COUP-coup vs RAGE-rage
Dehaene et al, Nature Neuroscience, 2001; Psychological Science, 2004Invariance for case in the visual word form area
Single letter?
#REFLET#reflet
TREFLE##reflet
#TREFLE#reflet
anagram
REFLET##reflet
#PATERE#reflet
EPATER##reflet
same word different word
same
different
Prime-Target Relation
wordlocation
RADIOradio
What are the coding units underlying orthographic priming?
Bigram?
Morpheme or whole word?
Dehaene et al., Psychological Science, 2004
#REFLET#reflet
REFLET##reflet
#TREFLE#reflet
TREFLE##reflet
#PATERE#reflet
EPATER##reflet
same word anagram different word
same
different
Prime-Target Relation
wordlocations
595
600
605
610
615
620
625
630
sameword
anagram differentword
resp
onse
tim
e (m
s)
same locationdifferent location
Behavior
Dehaene et al., Psychological Science, 2004
#REFLET#reflet
REFLET##reflet
#TREFLE#reflet
TREFLE##reflet
#PATERE#reflet
EPATER##reflet
same word anagram different word
same
different
Prime-Target Relation
wordlocations
LETTERS AT THE SAME LOCATION
same locationdifferent location
both locations
0
0.1
0
0.1
Left posterior fusiform right posterior fusiform
sameword
anag diffword
sameword
anag diffword
fMRI : location-specific priming
Dehaene et al., Psychological Science, 2004
5
0
0.1
0.2
Left middle fusiform (y=-56)
sameword
anag diffword
#REFLET#reflet
REFLET##reflet
#TREFLE#reflet
TREFLE##reflet
#PATERE#reflet
EPATER##reflet
same word anagram different word
same
different
Prime-Target Relation
wordlocations
LETTERS AT ANY LOCATION LARGER UNIT (WHOLE WORD?)
fMRI : location-independent priming
0
0.1
Left middle fusiform (y=-48)
sameword
anag diffword
Dehaene et al., Psychological Science, 2004
Coded units
Local bigrams
Small words and recurring
substrings (e.g. morphemes)
Leftoccipito-temporal
sulcus?(y ≈ -56)
Left occipito-temporal
sulcus?(y ≈ -48)
Putative area
Bank of abstract letter
detectors
Bilateral V8?(y ≈ -64)
Oriented bars
Local contrasts
Local contours(letter fragments)
Letter shapes(case-specific)
BilateralLGN
BilateralV1
BilateralV2
Bilateral V4?
Receptive fieldsize and structure
+- -
Examples of preferred stimuli
En
extentCONTENT
E
Local combination detectors: A model of invariant visual word recognition
Dehaene et al.TICS, 2005
Priming within and across scriptsin Japanese subjects
Design:• Targets and primes can appear in Kanji or in Kana• Task = semantic classification (natural/man-made)
Nakamura, Dehaene et al., JOCN, 2005
Repetition priming in Japanese
Within and cross-script priming in response times
Repetition priming with Kanji primes and Kanji targets
Nakamura, Dehaene et al., JOCN, 2005
VWFA: Script-specific priming
Left middle temporal region:Cross-script priming (semantic?)
Visual word form area
bilateral intraparietal sulci
Motor lateralized readiness potential
Left middle temporal gyrus
Kanji-Kana
Sofa-couch
Evidence for extensive subliminal processing using fMRI priming
Orthographic priming Left fusiform gyrus (Dehaene et al,
2001; Devlin et al, 2004)Semantic priming
Numerical proximity in bilateralintraparietal sulci (Naccache and Dehaene, 2001)
Semantic proximity of words in leftmiddle temporal gyrus (Devlin et al, 2004; Nakamura, Dehaene et al, 2005)
Amygdala activation by maskedemotional words (Naccache et al, 2005)
Motor primingBilateral motor areas (Dehaene et
al., 1998)
Part II. The nature of conscious access
• Conscious access is associated with a sudden activation of a distributed parieto-prefrontal network
time
RADIO29 ms
Making a word visible again
Dehaene et al., Nature Neuroscience, 2001
71 ms
71 ms
71 ms
71 ms
time
RADIO29 ms
71 ms
71 ms
71 ms
Making a word visible again
Dehaene et al., Nature Neuroscience, 2001
71 ms
visible words masked words
2.26 3.33t scale
p value0.02 0.0025
x = -38
29 17
6.3 20.8t scale
p value10-5 3.10-12
x = -38
29 5 1745
left fusiform gyrus(-48, -60, -12)
-0.1
0.3
-5 0 5 10 15
time (s)
% s
igna
l cha
nge
visible wordsmasked words
Consciousness correlates with-Local amplification in the relevant processors
-Global parieto-frontal activity (Dehaene et al, Nature Neuroscience 2001)
Baar’s (1989) theory of a conscious global workspace:
An architecture mixing parallel and serial processing
…the intellectual activity, the will and self-consciousness…are the result of a combined action of a large number of mnemonic spheres
(Ramon y CAJAL, Histologie, p.878)
What is the neural basis of the global workspace?
Prefrontal cortex and temporo-parietal association areas form long-distance networks
V1
TE
PFC
Pat Goldman-Rakic(1980s):
long-distance connectivity of dorso-
lateral prefrontal cortex
Guy Elston (2000)Greater arborizations and spine density
Von Economo (1929):Greater layer II/III thickness
processorsmobilizedinto the
consciousworkspace
hierarchy of modularprocessors
The global neuronal workspace model
Dehaene, Kerszberg & Changeux, PNAS, 1998Dehaene & Changeux, PNAS, 2003; PLOS, 2005inspired by Mesulam, Brain, 1998
automaticallyactivated
processors
high-level processorswith strong
long-distanceinterconnectivity
Dehaene, Sergent, & Changeux, PNAS, 2003
Detailed simulations of the global neuronal workspaceusing a semi-realistic network of spiking neurons
(Dehaene et al., PNAS 2003, PLOS Biology, 2005)
T1 T2
Area A1
Area B1
Area C
Area D
Area A2
Area B2
Feedforward
Feedback
Thalamocortical column
to/fromhigherareas
to/fromlowerareas
Thalamus
Cortex
supragranular
layer IV
infragranular
Spiking neurons
NMDA
AMPAGABA
NaP
KS
Spike-rateadaptation
Leak
neuromodulatorcurrent
Optional cellular oscillator currents
0 100 200 300 400 5000
20
40
60
« Ignition » of the global
workspace by target T1
Failure of ignition by target T2
Three distinguishable states of subliminal, preconscious and conscious processing
Dehaene, Changeux, Naccache, Sackur, & Sergent, TICS, 2006²
GlobalWorkspace
T1Conscious
high strengthand attention
T2Preconscioushigh strengthno attention
T3Subliminal
weak strength
T1 versus T3 : unmasked versus masked stimuli(both attended)
Unmasked words (T1) Masked words (T3)
GlobalWorkspace
T1Conscious
high strengthand attention
T3Subliminal
weak strength
Contrast betweenconscious and subliminal processing
16ms
Delay: 0-150ms
prime
mask+
.
M6MEE
+
..
.
+6.
. .
OBJECTIVE JUDGEMENT
Compare prime to 5
SUBJECTIVE JUDGEMENT
Rate Prime
Visibility
Not seenMaximalvisibility
250 ms
Time
DIGIT MASKING PARADIGM
Antoine Delcul
16ms
Delay: 0-150ms
prime
mask+
.
M6MEE
+..
.
+6.
. .
Time
Exploring the cerebral mechanismsof the non-linear threshold in conscious access
(Delcul and Dehaene, PLOS Biology 2007)
Logic = Use this sigmoidal profile as a « signature » of consciousaccess. Which ERP components show this profile?
Subjective visibility Objective performance
% of responsedelay
1
3
5
7
9 11
13
15
17
19
21
0ms
16m
s
33m
s
50m
s
66m
s
83m
s
100m
s
150m
s
0
10
20
30
40
50
60
70
80
90
100
Subjective ratingDelay
subliminal processing conscious processing
at threshold
Early visual areas Higher visual areas Prefrontal areas
subliminal
conscious
weak masking
strong masking
Masking strength
time following stimulus onset (ms)
Predictions of the global neuronal workspace model
-100 0 100 200-2
-1
0
1
2
3
4
5mask subtracted, aligned on mask onset
N1
P1a
100 ms
33 ms50 ms66 ms83 ms
16 ms
C
Apparent P2after
mask subtraction
Time (ms)
Ampl
itude
(μV)
The Mask-SubtractionMethod
Separation of activation evoked by the mask and by the target
-100 0 100 200-2
-1
0
1
2
3
4raw data, aligned on mask onset
100 ms
maskonly
33 ms50 ms66 ms83 ms
16 ms
A
Alignement on
target onset
Mask subtraction
Mask-evoked N1
Time (ms)
Ampl
itude
(μV)
0 100 200 300-2
-1
0
1
2
3
4
5mask subtracted, aligned on target onset
N1
P1a
D
Time (ms)
Ampl
itude
(μV)
Apparent P2after
mask subtraction
Conclusions:-Masking leaves the P1a and N1 essentially intact (exceptat the shortest SOA)-Masking reduces the P1b (less inter-hemispherictransfer)-Masking interrupts the N2-As the masking delayincreases
-target activation become increasinglystronger and longer lasting-Mask activation decreasescorrespondingly
- Only the P3 shows a non-linearity similar to behavioralreport
Brain mechanisms of masking
m-N2
m-N1
180 ms 0 50 100
180
200
220
240
260
280
16 33 50 66 831000
1
2
3
4
5
Latency
P1a
P1b
N1
N2
P3
100ms
140ms
167ms
227ms
370ms
Left target
-6-5-4-3-2-1012345μV
Amplitude
0 50 100
20
40
60
80
100
120
16 33 50 66 831000
0.5
1
1.5
0 50 100
50
100
150
16 33 50 66 831000
0.5
1
1.5
2
0 50 100
140
160
180
200
220
240
16 33 50 66 831000
1
2
3
4
0 50 100
300
350
400
16 33 50 66 831000
1
2
3
4
5
Target evoked components
Mask evoked components
0 50 100
200
250
300
16 33 50 66 831000
1
2
3
4
200 ms
Right target
0 50 10060
80
100
120
140
160
180
16 33 50 66 831000
0.5
1
1.5
2
0 50 100
180200
220240
260280
16 33 50 66 83100
01
23
45
0 50 100
200250
300
16 33 50 66 83100
01
23
4
100 200 300 4000
2
4
6
8raw data, aligned on target onset
P3: sudden non linear divergencearound 270 ms
Amplitude (microV)
100 200 300 400-10
-8
-6
-4
-2
0raw data, aligned on target onsetFronto-polar electrodes
Suddenonset
100 ms
maskonly
33 ms50 ms66 ms83 ms
16 ms
Central electrodes
Suddenonset
370 ms
100 200 300 400-1
0
1
2
3
4
5
6raw data, aligned on target onset
maskonly
seennot seen
Central electrodes
t-test at t=341 ms post target
+4
0
-4
t value
A second independent criterion for conscious access:Only the amplitude of the P3 distinguishes
seen versus not-seen trials at a fixed delay (50 ms)(9 subjects only)
Note, however, that the unseenstimuli evoke strong activation atSOA=50 ms.
All components showed a significant difference:Not-seen > mask-only
EM M
E
A late non-linearity underlying conscious access during masking(Delcul et Dehaene, PLOS Biology 2007)
100 ms
33 ms50 ms66 ms83 ms
16 ms
First phase: local and linear
9
Variable delay
16 ms
250 ms
Delay:
Second phase: global and non-linear
(amplification)
Parietal
FusiformFrontal
Distributed visual, parietaland prefrontal activity atthe peak of the P3 (conscious trials)
Test of the theory2. with two competing stimuli during AB
GlobalWorkspace
T1Conscious
high strengthand attention
T2Preconscioushigh strengthno attention
T3Subliminal
weak strength
Conscious access and non-conscious processingduring the attentional blink
Sergent, Baillet & Dehaene, Nature Neuroscience, 2005
02 4 6 8 10 12 14 16 18 20
1 2 3 4 6 8
0
10
20
30
40
50
60
70
80
90
Perc
ent o
f tria
ls
Subjective visibility
Lag
Bimodal distribution of T2 visibility
Visibility ratingT1-T2 Lag
Seen
Not seen
HRQFCINQ
Target 2
time
Variable T1-T2 lag
Target1
CVGRXOOX
Sergent et al., Nature Neuroscience, 2005
UNSEEN T2(minus T2-absent trials) DIFFERENCE
Time course of scalp-recorded potentialsduring the attentional blink
SEEN T2(minus T2-absent trials)
Timing the divergence between seen and not-seen trialsin the attentional blink (Sergent et al., Nature Neuroscience 2005)
Seen
Not seen
Unchanged initial processingP1 : 96 ms
-2 μV +2 μV
N1 : 180 ms
-4 μV +4 μV
Abrupt divergence around 270 ms…
Seen
Not seen
N2 : 276 ms
-2 μV +2 μV
N3 : 300 ms
-3 μV +3 μV
Late non-conscious processingN4 : 348 ms
-3 μV +3 μV
All-or-none ignition
P3a : 436 ms
-2 μV +2 μV
P3b : 576 ms
-2 μV +2 μV
Dorsolateral prefrontal
Differencebetween seen and
not seen trials
436 ms
Middle temporal
Inferior frontal
Anterior cingulate
seennot seen
t = 300 ms
t = 436 ms
t = 436 ms
Recording of ‘virtualsources’ at variouscortical locations
fMRI:Marois et al, Neuron 2004
Inferior temporal
MEG:Gross et al, PNAS 2004
Target T1
Target T2
Response 1
Response 2
Test of the theory3. with two competing stimuli during PRP
Sigman and Dehaene, PLOS Biology, 2005
Pashler’s « central bottleneck » model of the psychological refractory period
T1 T2
Workspace state for task 1
R2
R1
Workspace state for task 2
Pashler’s central bottleneck model
Response 1
Response 2
Target 1
Target 2
time
Task 1
Task 2
0
InterferenceRegime
IndependentRegime
T1-T2 SOA
Event-related potentials dissociate parallel and serial stages during dual-task processing
Sigman and Dehaene, submitted
-1000 -500 0 500 1000 1500 20000
0.5
1
1.5
2
2.5
V A
N1 P3 N1 P3
Visual events Auditory events
Separation of ERPs at long T1-T2 delays
0 300 900 1200
600
1000
1400
1800
RT2
RT1
Response times
T1-T2 delay
Task 1: number comparison of a visual Arabic numeral with45, respond with right hand
Task 2: pitch judgment on an auditory tone, respond with lefthand
- 300 ms delay in or outside of the interference regime
-5 0 0 0 5 0 0 1 0 0 0 1 5 0 0-1
-0 .5
0
0 .5
1
1 .5
P3
Event-related potentials dissociate parallel and serial stages during dual-task processing
N1
-5 0 0 0 5 0 0 1 0 0 0 1 5 0 0-1
-0 .5
0
0 .5
1
1 .5
P3-5 0 0 0 5 0 0 1 0 0 0 1 5 0 0
-0 .5
0
0 .5
1
1 .5
-5 0 0 0 5 0 0 1 0 0 0 1 5 0 0-1
-0 .5
0
0 .5
1
1 .5
N1
Sigman and Dehaene, submitted
-1000 -500 0 500 1000 1500 20000
0.5
1
1.5
2
2.5
V A
N1 P3 N1 P3
Visual events Auditory events
Separation of ERPs at long T1-T2 delays
0 300 900 1200
600
1000
1400
1800
RT2
RT1
Response times
T1-T2 delay
Task 1: number comparison of a visual Arabic numeral with45, respond with right hand
Task 2: pitch judgment on an auditory tone, respond with lefthand
- 300 ms delay in or outside of the interference regime
Functional MRI can be sensitive to small delays, in the order of a few hundreds of milliseconds
Sigman, M., Jobert, A., LeBihan, D., & Dehaene, S. (2006). Parsing a sequence of brainactivations at psychological times using fMRI. NeuroImage, 2006
•The phase of the BOLD response can beestimated with high precision even when the TR exceeds 2 seconds• Injected delays of ~200 ms can bedistinguished with whole-brain imaging• Changes in onset, duration or both can beseparated
1 2 3 46
6.5
7
7.5
8
8.5
Pha
se(s
ec)
Injected delay (multiples of 500 ms)
Slope=0.512 sMenon et al, 1998
Separating parallel and serial stages during dual-taskprocessing with high temporal resolution fMRI
Sigman and Dehaene, submitted
Task 1: number comparison of a visual Arabic numeral with45, respond with right hand
Task 2: pitch judgment on an auditory tone, respond with lefthand
- 300 ms delay in or outside of the interference regime
- slow fMRI (one trial = 14.4 s)
z=54 z=6 z=-6 0 1.2
5.2
6.7
0 1.2
4.4
5.9
z=54 0 1.2
5
6.5
z=54 z=6z=30 0 1.2
4.9
6.4
z=46z=34z=20 0 1.2
6
Injected T1-T1 delay (seconds)
Measured BOLD phase (seconds)
Task 1network
Task 2perceptual
Task 2Decision/motor
Shared betweentask 1 and task 2
High-level control
PRP
VSTM/MOT
Att. Blink
Marois & Ivanoff, TICS, 2005review of imaging studies of bottleneck tasks
Locating the sites of processing bottlenecks:parieto-prefrontal networks
Dux, Ivanoff, Asplund & Marois, Neuron, 2007
Conclusion: Towards a neuronal understandingof consciousness
Non-conscious processing is extensive in the human brainBrain activity can remain non-conscious for at least two reasons:– bottom-up strength is insufficient (e.g. masking)– Top-down attention is distracted (e.g. attentional blink)
A representation becomes conscious whenever it wins the central competition and ignites a distributed, self-sustained assembly of neurons in prefrontal, cingulate and other cortical association areasConscious access corresponds to a sharp and relatively late (~270 ms) dynamical phase transition in neural network activity. Although I have only talked about access to consciousness, changes in viligance (intransitive consciousness) relate to neuromodulation of the same network (S. Laureys, P. Maquet).
Vegetative state Coma Slow-wave sleep General anesthesia