Searching the truth:Searching the truth:Visual search for abstract, well-Visual search for abstract, well-learned objectslearned objects

Denis Cousineau, Denis Cousineau,

Université de MontréalUniversité de Montréal

This talk will be available atThis talk will be available atwww.mapageweb.umontreal.ca/cousinedwww.mapageweb.umontreal.ca/cousined

How do we find a target?How do we find a target?

33

Visual search: a basic Visual search: a basic proficiency…proficiency…

very little understood…very little understood…

44

Two models of visual search…Two models of visual search…

Serial search:Serial search: The famous 2 : 1 ratio of The famous 2 : 1 ratio of

mean slopes;mean slopes; Based on the MEAN Based on the MEAN

response times;response times;

Target absent Target present

Condition

1 2 3 4Display size

320

360

400

440

480

Re

sp

on

se

tim

e

Target absent Target present

Condition

1 2 3 4Display size

320

360

400

440

480

Re

sp

on

se

tim

e

Parallel searchParallel search Flat performance.Flat performance. Unlimited capacityUnlimited capacity

55

Some problems with these Some problems with these models…models…

This dichotomy difficult This dichotomy difficult to conciliate with to conciliate with progressive transitionsprogressive transitions

Mean performances are little diagnosticMean performances are little diagnostic Mimicking (Townsend, 1990)Mimicking (Townsend, 1990) Standard deviations can also be mimicked…Standard deviations can also be mimicked…

2:1 ratio depends heavily on the stopping rule2:1 ratio depends heavily on the stopping rule How do we stop searching?How do we stop searching?

5 10 15 20session

250

500

750

1000

1250

Re

sp

on

se

tim

e

Target absent Target present

Condition

1 2 3 4Display size

320

360

400

440

480

Re

sp

on

se

tim

e

Target absent Target present

Condition

1 2 3 4Display size

320

360

400

440

480

Re

sp

on

se

tim

e

Target absent Target present

Condition

1 2 3 4Display size

320

360

400

440

480

Re

sp

on

se

tim

e

66

Standard model:Standard model:Serial Self-Terminating Search Serial Self-Terminating Search (SSTS)(SSTS)

Get ready

Implicitly: a Random-Order visual search model

Experiment 1Experiment 1

88

Methodology: Methodology: Visual search taskVisual search task

34 34 sessionssessions of training; 10 sessions of test, of training; 10 sessions of test, 4 subjects, consistent mapping:4 subjects, consistent mapping: Targets:Targets: Distractors:Distractors:

Targets had to be learned; Targets had to be learned;

*

Fixation point

Test display

Reaction time measured since stimulus presentation

Circles indicating where the stimuli will appear

99

Mean resultsMean results

A seems to be perfectly serial; B is the least A seems to be perfectly serial; B is the least “serial”“serial”

Yet, we will see thatYet, we will see that B is nearly identical to AB is nearly identical to A None of them are random-order serialNone of them are random-order serial

1010

Results of Target-present RT Results of Target-present RT distributionsdistributions

A and B are the most similar!A and B are the most similar!

1111

Modeling the modes of the Modeling the modes of the distributionsdistributions

The D =1 condition could be modeled with a The D =1 condition could be modeled with a normal distribution with parameters ;normal distribution with parameters ;

The D = 2 condition should be the same as the The D = 2 condition should be the same as the D = 1 condition except shifted by and variance D = 1 condition except shifted by and variance doubled;doubled;

In general, the distributions have parametersIn general, the distributions have parameters

The modes are pooled: a “mixture of distribution”The modes are pooled: a “mixture of distribution”-With parameter-With parameter according to according to

SSTSSSTS

-With free mixture parameter -With free mixture parameter unrestricted model unrestricted model

¹ ;¾2¹ ;¾2

¿¿

¹ +(D ¡ 1)¿;D¾2¹ +(D ¡ 1)¿;D¾2

p1 = p2 = ¢¢¢= pD = 1=Dp1 = p2 = ¢¢¢= pD = 1=D

1212

Results of Target-present RT Results of Target-present RT distributionsdistributions

For all participants, the mixture parameters For all participants, the mixture parameters are not equal to 1/D.are not equal to 1/D.

The last mode is underrepresented. Errors?The last mode is underrepresented. Errors?

1313

Results of Target-Results of Target-absentabsent RT RT distributionsdistributions

B perform early terminationB perform early termination A does not, yet her ps are not equal!A does not, yet her ps are not equal! C does this too often compared to his error rateC does this too often compared to his error rate

1414

In sumIn sum

1.1. Regarding the exhaustivity prediction:Regarding the exhaustivity prediction: The participants sometimes stop earlier than predicted The participants sometimes stop earlier than predicted

by an exhaustive searchby an exhaustive search This predicts errors, but too many errors are predicted.This predicts errors, but too many errors are predicted.

Regarding the random-order prediction:Regarding the random-order prediction: The participants are serial…The participants are serial… ……but they are not randombut they are not random Seriality is one process going on, but there must be a Seriality is one process going on, but there must be a

second process which aims at biasing the search second process which aims at biasing the search itinerary so that targets will be visited earlier than by itinerary so that targets will be visited earlier than by chance.chance.

1515

A new model of visual search:A new model of visual search:m-Sr-STSm-Sr-STS

The The MostlyMostly Serial, Serial, RoughlyRoughly Self-Terminating Search Self-Terminating Search

Fixate

Is it a target

?

LTM or STM

« Yes »

Another location &

won’t give up

?

Memory for location

« No »

Pr

e at

tent

ive

modu

le

?

no

yes

no

¼¼

yes

Essentially a two-stage model (Chun & Wolfe, Essentially a two-stage model (Chun & Wolfe, 1996, Wolfe, 1994, Cousineau & Larochelle, 2004).1996, Wolfe, 1994, Cousineau & Larochelle, 2004).

The pre-attentive The pre-attentive module outputs module outputs probabilitiesprobabilities

1616

Yet, there is still some magic Yet, there is still some magic left…left…

Unbeknownst to the participantsUnbeknownst to the participants was diagnostic:was diagnostic: was irrelevant:was irrelevant:

The pre-attentive module could drive The pre-attentive module could drive attention on the stimuli having those attention on the stimuli having those conjunctions of featuresconjunctions of features

A parallel search for conjunctionsA parallel search for conjunctions It should be an impossible feat according to It should be an impossible feat according to

Treisman (1980), Wolfe (1994) and many others.Treisman (1980), Wolfe (1994) and many others.

1717

Let’s concentrate on the Let’s concentrate on the decision mechanismdecision mechanism

The The MostlyMostly Serial, Serial, RoughlyRoughly Self-Terminating Search Self-Terminating Search

Fixate

Is it a target

?

LTM or STM

« Yes »

Another location &

won’t give up

?

Memory for location

« No »

Pr

e at

tent

ive

modu

le

?

no

yes

no

¼¼

yes

The pre-attentive The pre-attentive module outputs module outputs probabilitiesprobabilities

What is “Recognizing a target”?What is “Recognizing a target”? How does cycling occurs?How does cycling occurs?

Experiment 2Experiment 2

1919

Methodology:Methodology:Same-different taskSame-different task

Well-trained participants (10 hours to reach Well-trained participants (10 hours to reach asymptote then 5 hours of testing).asymptote then 5 hours of testing).

The display size D is fixed at 1;The display size D is fixed at 1; The stimuli are varying in complexity C, e.g. The stimuli are varying in complexity C, e.g.

*

Fixation point

Test display

Target to be memorized

*

2020

Mean “Same” response timesMean “Same” response times

Saying “Same”Saying “Same” is very fast is very fast affected by C (20 ms/spike)affected by C (20 ms/spike)

Linearity is not found using characters Linearity is not found using characters instead of complex stimuliinstead of complex stimuli

Parallel, limited-capacity models complies Parallel, limited-capacity models complies with such resultswith such results e.g. a template matching process?e.g. a template matching process?

SameCondition

1 2 3 4Complexity C

280

300

320

340

360

380

400

Re

sp

on

se

tim

es

2121

Mean “Different” response Mean “Different” response timestimes

A main effect of the A main effect of the number of differencesnumber of differencesbut no effect of complexity!but no effect of complexity!

Suggests that responding “Different” Suggests that responding “Different” requires the localization of at least one requires the localization of at least one difference.difference. Parallel search for a difference benefits from the Parallel search for a difference benefits from the

presence of many differencespresence of many differences

1 2 3 4

Nber of differences

1 2 3 4Complexity C

280

300

320

340

360

380

400

Re

sp

on

se

tim

es

2222

The Revised possible The Revised possible explanationexplanation

There might be two distinct processes:There might be two distinct processes: one for confirming the sameness, one for confirming the sameness, one for establishing the “differenceness”one for establishing the “differenceness”

How do they relate to one another? In succession?How do they relate to one another? In succession?

Fixate « Yes »

« No »

Pr

e at

tent

ive

modu

le

no

yes

¼¼

Has located a difference

?

STM

Is it a target

?

STM

not yet

yes

2323

Slow “Same” vs. fast “Different” Slow “Same” vs. fast “Different” in the in the C = 4 conditionC = 4 condition

The two conditions are very close (mean The two conditions are very close (mean difference of 13 ms). Do they follow in time?difference of 13 ms). Do they follow in time?

Again, let’s look at distributionsAgain, let’s look at distributions

0 1 2 3 4

Nber of differences

1 2 3 4

Complexity C

280

300

320

340

360

380

400

Re

sp

on

se

tim

es

Same Different

1 2 3 4

Complexity C

2424

Distributions of RT in Same and Distributions of RT in Same and (very) Different responses at C = (very) Different responses at C = 44

The slow “Different” responses are faster The slow “Different” responses are faster (by 4 ms) than the slow “Same” responses.(by 4 ms) than the slow “Same” responses.

One process cannot operate *after* the One process cannot operate *after* the other.other.

300 400 500 600Response times

0

25

50

75

100

Eff

ec

tif

300 330 360 390 420diff = 4

270

300

330

360

390

420

dif

f =

0

diff = 0 = -104.65 + 1.25 * diff4R-Deux = 1.00

Same

300 400 500 600Response times

0

50

100

150

Eff

ec

tif

2525

Revised revised-architectureRevised revised-architecture

““No” may not be an option for a neural No” may not be an option for a neural decision mechanism…decision mechanism…

Fixate « Yes »

« No »

Pr

e at

tent

ive

modu

le

yes

¼¼

Has located a difference

?

STM

Is it a target

?

STM

not yet

not sure

yes

In conclusion…In conclusion…

2727

Visual search is a proficiency Visual search is a proficiency (1/2)(1/2)Proficiencies are an amalgam of Proficiencies are an amalgam of processesprocesses Parallel pre attention process outputs Parallel pre attention process outputs

probabilitiesprobabilities Serial deployment of central attentionSerial deployment of central attention Stopping rule which can end prematurelyStopping rule which can end prematurely

Unitary (template matching?) recognition Unitary (template matching?) recognition processprocess

Unitary (find-a-difference) rejection processUnitary (find-a-difference) rejection process

In sum, the SSTS architecture was all wrong.In sum, the SSTS architecture was all wrong.

2828

Visual search is a proficiency Visual search is a proficiency (1/2)(1/2)Processes are Processes are univoqueunivoque (from french: One and only one meaning, one and only one semantic content, but also one and only (from french: One and only one meaning, one and only one semantic content, but also one and only one voice)one voice) As an exampleAs an example

If a “not-face” is presented to a face recognition If a “not-face” is presented to a face recognition module, does it “knows” that it is not a face, or does it module, does it “knows” that it is not a face, or does it remains “silent” by omitting to respond…remains “silent” by omitting to respond…

What would be a brain which detects objects (of many What would be a brain which detects objects (of many kind) and their negation? what would be the EEG of kind) and their negation? what would be the EEG of such a system?such a system?

Negation is not part of the neural process toolboxNegation is not part of the neural process toolbox it is not “To be or not to be” but “To be and to un-be”it is not “To be or not to be” but “To be and to un-be”

““NO” branches should be forbidden in psychology.NO” branches should be forbidden in psychology.

2929

Methodological considerationMethodological consideration

Distribution analyses rocks!Distribution analyses rocks! Mean results can be interpreted in so many ways Mean results can be interpreted in so many ways

that they cannot reject any model at all.that they cannot reject any model at all. We have been stuck with a fruitless dichotomy We have been stuck with a fruitless dichotomy

for over 40 years because we were unable to for over 40 years because we were unable to make the data speak.make the data speak.

Anyone with a serious model should implement it Anyone with a serious model should implement it using distributions or remain quietusing distributions or remain quiet

Distribution modeling and testing is not difficult Distribution modeling and testing is not difficult (it can be learned in 3 hours).(it can be learned in 3 hours).

as long as you know matlab or mathematica…as long as you know matlab or mathematica…

Thank you.Thank you.

This talk will be available atThis talk will be available atwww.mapageweb.umontreal.ca/cousinedwww.mapageweb.umontreal.ca/cousined