How does the brain decide what to look at next? John Findlay University of Durham (acknowledgements...
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
1
Transcript of How does the brain decide what to look at next? John Findlay University of Durham (acknowledgements...
How does the brain decide what to look at next?
John FindlayUniversity of Durham
(acknowledgements to Val Brown)
Saccades are quintessentially voluntary movements.The gaze selects informative detail but eye scans also
appear random and arbitrary
from Yarbus (1967)
Pathways involved in saccade generation
Visual pathways form a massively interconnected neural network
Retinotopic mapping is maintained through to the saccadic generator in the SC
Multiple interconnected maps offer the possibility of selection
by biased competition
“ Some kind of short-term description of the information currently needed must be used to control competitive bias in the visual system, such that inputs matching that description are favoured in the visual cortex.”
(Desimone and Duncan, Annual Review of Neuroscience, 1995)
Biased competition creates salience maps
• A salience map is a two-dimensional map in which a single scalar quantity (salience) is represented at each point.
• Biased competition results in similarity to the search target being represented as salience.
• Instantiated in various modelsItti L and Koch C (2000). A saliency-based search mechanism for
overt and covert shifts of attention. Vision Research, 40, 1489-1506.
Hamker FH (2004). A dynamic model of how feature cues guide spatial attention. Vision Research, 44, 501-521.
Saccade target selection in the superior colliculus
The SC is the main final site for selection of saccade destinations
Activity in the superior colliculus related to saccades
Wurtz R H (1996). Vision for the control of movement. The Friedenwald Lecture Investigative Ophthalmology and Visual Science, 37, 2131-2145.
How does the brain decide what to look at next?
• During visual search, biased competition creates a salience map and processes, probably in the SC, select the point of highest salience to convert to an orienting saccade.
• Supported by detailed studies of saccades during visual search (Findlay, Vision Research, 1997; Motter and Belky, Vision Research, 1998a,b)
Task:
Search for a
red cross
(Look at it)
Findlay J M (1997). Saccade target selection during visual search. Vision Research, 37, 617-631
Properties of first saccades(Findlay, 1997)
Short latency (~ 250 ms) – very similar for saccades to target and to distractor
Properties of first saccades(Findlay, 1997)
Short latency (~ 250 ms) – very similar for saccades to target and to distractor
Frequently (75%) on target when target is in inner ring, occasionally (26%) when target in outer ring.
.
Properties of first saccades(Findlay, 1997)
Short latency (~ 250 ms) – very similar for saccades to target and to distractor
Frequently (75%) on target when target is in inner ring, occasionally (26%) when target in outer ring.
Incorrect saccades go preferentially to distractor sharing a feature with the target.
Monkey visual search(Motter & Belky, Vision Research, 38, 1007-1022; 1885-1815, 1998)
• Monkeys trained to search for a conjunction target (colour and orientation)
Saccade selection in visual search
• The conclusion in both the Findlay and the Motter & Belky studies was that the biased competition/ salience map approach provided the most satisfactory account of saccadic selection.
• In particular, no evidence for a rapid covert attentional scan (favoured by many psychologists).
• This conclusion was reached earlier in physiological studies of single cell responses in the visual system of primates carrying out search tasks
FEF Schall & Hanes (1993)IT Chelazzi, Miller, Duncan & Desimone (1993)
How does the brain decide what to look at next during visual search ?
• During visual search, biased competition creates a salience map and processes, possibly in the SC, select the point of highest salience to convert to an orienting saccade.
• Selection from a salience map is basic
• Supplementary processes
1. Inhibition of return
2. Saccade pipelining
3. Strategies
How does the brain decide where to look next?
4. Implicit learning
5. Contingent learning
6. Neuro-economics (Glimcher)
7. Task specific requirements for
information acquisition
(Land, Hayhoe, Ballard)
• Selection from a salience map is basic
• Supplementary processes
1. Inhibition of return
2. Saccade pipelining
3. Strategies
How does the brain decide where to look next?
NO OTHER ATTENTIONAL
SELECTION
4. Implicit learning
5. Contingent learning
6. Neuro-economics (Glimcher)
7. Task specific requirements for
information acquisition
(Land, Hayhoe, Ballard)
• Selection from a salience map is basic
How does the system avoid ‘salience loops’?
(B is the most salient location when A is fixated, then A becomes the most salient when B is fixated)
Salience map alone would give A > B > A > B . . . . .
Proposed answer - Inhibition of Return (IOR)
An attended location is subject to some form of inhibition
when attention is shifted elsewhere
Klein R M and MacInnes W J (1999). Inhibition of return is a foraging facilitator in visual search. Psychological Science, 10, 346-352
The rings task
Scan order partly specified
Centre - red – free scan through blacks - blue
Count target letters and make Yes/No response
The Rings Task
3 6 9 12
Scan through the rings, starting with the red and ending with the blue
(Scans from 6 individuals)
Rings task - typical eye scan
Deviations from sequential scan
BACKTRACK 1
BACKTRACK 1
BACKTRACK 1
BACKTRACK 2
BACKTRACK 1
OMISSION
Trials with deviations from sequential scan
Omissions Backtrack 1 Backtrack 2 Backtrack > 2
BR 15 15 1 15
JP 9 15 6 9
LS 6 12 3 4
LW 5 29 5 4
PB 5 39 5 10
SL 3 13 7 13
Proportion 0.09 0.26 0.06 0.11Error rate* 47 %
(target omissions)
6 % 15 % 23 %
* Error rate on trials with standard scan 4 %
Backtracking in visual search
BACKTRACK 1
BACKTRACK 1
BACKTRACK 1
Found by other workers
(Motter & Belky, 1998)(Peterson et al. 2001)
• IOR time course may relate to visual processingslower with increased processing demands, so not always immediate
• Backtracking sequences may be pre-planned (pipelined saccades)In A1, B, A2 fixation sequences, the B fixation was normal duration (228 ms) but A1 and A2 were both shorter than normal (~ 170 ms).The saccade following a backtracking sequence tended to follow the
direction of the last saccade in the sequence.
Backtracking in visual search
BACKTRACK 1
BACKTRACK 1
BACKTRACK 1
Found by other workers
(Motter & Belky, 1998)(Peterson et al. 2001)
• IOR time course may relate to visual processing
• Backtracking sequences may be pre-planned (pipelined saccades)In A1, B, A2 fixation sequences, the B fixation is normal duration but A1 and A2 are both shorter than normalThe saccade following a backtracking sequence tends to follow the
direction of the last saccade in the sequence
X
Visually-guided and memory-guided saccades
How does the brain decide which?
Hikosaka et al (2000) argue for basal ganglia pathway (blue route)
Inhibitory effects on SC, others excitatory
Separate sets of cells in caudate and in SNr are active during visually guided and memory guided
movements.
Directional strategies
COUNTTHE
DOTS
(Convex Hull)
Directional strategies
COUNTTHE
DOTS
Directional strategies are one form of memory (Gilchrist & Harvey)
NO OTHER ATTENTIONAL SELECTION
‘Visual attention selects the saccade target’
Statement supported by the finding that visual information at the destination point of a forthcoming saccade receives preferential pre-processing (Deubel and Schneider, 1996; Kowler et al. 1995)
Biased competition is a form of attentional selection but does not operate in a localised region of the visual field.
Visual attention is commonly thought of as selection of a localised region.
‘Visual attention selects the saccade target’
Why I don’t like this statement
1. It’s getting close to a homunculus view
2. Localised visual attention should be able to eliminate distractor interference.
Visual attention should be able to eliminate distractor interference.
nextsaccade ?
Visual attention should be able to eliminate distractor interference.
Attentional spotlight selects next target
Visual attention should be able to eliminate distractor interference.
Attentional spotlight selects next target
Visual attention should be able to eliminate distractor interference.
Attentional spotlight selects next target
No effective spotlight
Visual attention should be able to eliminate distractor interference.
Attentional spotlight selects next target
No effective small spotlight
Global effect
The Global Effect
Saccades to neighbouring target pairs tend to land towardsa centre-of-gravity position.
Findlay, 1981, 1982; Deubel, 1982; Ottes, Van Gisbergen and Eggermont, 1984
Reliably found with onset stimuli: does it occur in free scanning?
How accurate are scanning saccades ?
SACCADETO CENTREOF GRAVITY ?
Is there a global effect in free scanning ?
• Are saccades less accurate when there is a distractor present in the critical sector (as defined by Walker et al. 1997)?
Subject Nodistractor
Withdistractor
JP 16.9 43.3
LW 14.9 36.7
PB 20.4 38.0
SL 22.8 55.1
Percentage of inaccurate saccades
Probability of inaccurate saccadefor distractors in differentlocations relative tothe target
Is accuracy higher follower longer fixations ?
accuracy coding
4 3 2
1
Accuracy is highest following short duration fixations (although distractors still decrease it).
This is the opposite to a speed-accuracytrade off.
Short(< 200 ms)
Medium(200 – 300 ms)
Long(> 300 ms)
Nodistractor
JP 2.00 2.17 1.86 2.73SL 2.60 2.47 2.19 2.95PB 2.66 2.42 2.33 3.01LW 2.55 2.26 2.49 2.84
mean 2.45 2.33 2.22 2.88
Attentional selection and saccades
• Saccades during a free scan of a set of identical elements show the global effect. Thus no evidence here for a spotlight-like attentional selection.
• In most practical situations, elements are not identical; hence biased competition will act to reduce the global effect
How does the brain decide where to look next?
• Selection from a salience map is basic
• Supplementary processes influencing salience
1. Inhibition of return
2. Saccade pipelining
3. Strategies
4. Implicit learning
5. Etc. etc. etc.
THE END
Thank you for your attention
Kowler et al (1995)
Do distractors that have been already scanned reduce accuracy ?
accuracy coding
4 3 2
1
nodistractor
scanneddistractor
new distractor
JP 2.73 2.06 2.13SL 2.95 2.41 2.30PB 3.01 2.48 2.63
LW 2.84 2.37 2.48Mean 2.88 2.33 2.38
Accuracy is reduced both by scanneddistractors and by new ones.
REPLICA TRIALS NON REPLICA TRIALS
How replicable are scanning patterns ? (repeat run with one subject - different trial order)
NEAR REPLICA TRIALS
REPLICAS 26% overall, 68% ring count 3REPLICAS and NEAR REPLICAS 45% overall
Replicability of directional selection
Saccade direction histogramsHeuristic scanners Strategic scanners
Saccade direction change histograms
Saccade Landing Points
Accuracy is largely independent of saccade size
SACCADES WITH NO DISTRACTOR IN SECTOR
Subject PB
Saccade undershoot
1%
5%
10%
Saccade variability : on-axis
10%
5%
1%
10%
5%
2%
Saccade variability : off-axis
5%
2%
1%
Corrective saccades in the multi-element scanning task
Oculomotor capture in the multi-element scanning task