1. Chronometric effects of orthographic and semantic distracters on visual search for

isolated words

2. Impact of lexical familiarity on eye movements during visual search for verbal

information

Nicolas Vibert, Jean-François Rouet, Christine Ros, Edward Back, Julien Dampuré

Laboratoire Langage, Mémoire et Développement Cognitif, CNRS UMR 6215 – Université de Poitiers, Poitiers, France

Workshop on computational modeling in information seeking, Nice, July 2007 - ANR “Eye-LSA” and MSH of Nice

A general question : What are the determinants of visual search for verbal material within complex

documents ?

• A complex, finalized activity…• …that is difficult to learn and long to master (in

particular by teenagers and students)• Eye movement recordings (and maybe also

electrophysiological recordings) should give insights into the cognitive bases of these tasks

• “Ecological” experiments, in close to natural conditions…

• …should be complemented by more controlled experiments, using very simple tasks, to assess each of the multiple cognitive processes involved in these tasks

Visual search for target « objects »: general principles

• Research and/or identification of a target object within a display of other, more or less distracting objects

• Daily activity in real life: objects in images, words or expressions in texts or documents, links on Web pages

• Most experiments were done on non-lexical material: geometrical shapes, faces, isolated letters or digits. Very big literature, including neurophysiology and optimality analysis

• What does make the difficulty of a particular visual search?

Visual search for words as an instance of visual search for “targets”

• The typical experimental paradigm gives the slope of increase of the target-finding time as a function of the number of items in the display

• Some authors have run similar experiments with isolated words as targets and distracters

• Example : Harris et col. 2004

Target word: ELBOW

(2, 6 or 12 words displays)

Visual search for words is not reading

Eye movements during visual search (within a paragraph, Rayner & Raney, 1996):

- The “word frequency effect” disappears

- The average size of saccades is bigger

Eye movements during reading: - The “word frequency effect”- Fixations are longer on words that have ambiguous meaning (homographs), or on the last words of sentences- The duration of fixations would reflect the amount of cognitive “treatment” taking place on each given word

Impact of the visual characteristics of the verbal material on visual search

• Eye movement recordings and behavioral experiments show that the search pattern for words strongly depends on the visual characteristics of the material

• Mode of distribution of the words in the display: Ojanpää et al., 2002; Léger et al., 2005.

• Color, size and style of the writing, etc…

Impact of the nature of the task

• In most experiments published to date, the target word was given explicitly: find the word “APPLE”

• In recent papers (e.g. Léger et al., 2005), the task is different: the target word is defined as an exemplar of a category (i.e. a fruit, a bird), and not explicitly given

• In that situation, the duration of the search is strongly increased: 5 to 6 seconds instead of 3 for displays of 32 words

• In that task, performance depends on the semantic distance between the target word and the distracters, and on the typicality of the target word within its category

The semantic activation of words during visual search for a known word: myth or reality?

• The absence of word frequency effect suggests that no semantic treatment is performed

• Several chronometric experiments have addressed that question. Priming and/or familiarity effects have been found for the target words only

• In contrast, semantic treatment of the distracter words would be minimal

• When searching for objects, people remember the parts of the display they explored, but not the exact nature of the objects that were there. They just know that they were not the target object

Chronometric effects of orthographic and semantic distracters on visual search for isolated words

• Goal: to show that visual search for a target word known in advance is essentially a “visual” task, while searching for a word defined by its category is more a “semantic” task

• The two tasks are given to different groups of participants who see the same material (same target words and same distracters)

• Hypotheses:- Searching for a target word known in advance (“visual” task) will be

more difficult when the display includes visually similar distracters, but not when semantic associates of the target word are present

- Searching for an indirectly defined target word (“semantic” task) will be more difficult when the display includes distracters that are semantic associates of the target word, but not when visually similar distracters are present

- In both conditions, searching for an indirectly defined word will take more time than searching for a target word known in advance

Procedure and variables (1)

• Presentation of the target word (“visual” task) or of its category (“semantic” task), then masking

• Search for the word or the exemplar of the category within a vertical list of 15 words including 7 “neutral” distracters, and 7 “variable” distracters)

• 2 or 3 syllables words, frequency > 1/million (lexique.org)

• Independent variables: - task (visual or semantic)- nature of the distracters (1- neutral, 2- orthographically

similar to the target word, 3- orthographically similar between themselves but not to the target word, 4- semantic associates of the target word that are not other exemplars of the category used as a clue)

Procedure and variables (2)

• Dependent variables: time to find the target word (mouse click) and error rate

• Data treatment restricted to the trials where the target word was in the second, bottom half of the list

• 40 participants, first year psychology students

• Reading proficiency test (pipe et rat), vocabulary test (verbal part of the PMA)

Example of experimental material

Category fleur

Target word anémone

Visual distr autochtone, arcane, aubaine, aspirine, amazone, arène, aumône

“Similar” distr paluche, panache, pastiche, peluche, potiche, pouliche, pistache

Semantic distr jardin, balcon, terreau, pétale, parfum, printemps, bouquet

Neutral distr ministre, courtier, retraité, coucher, dépit, outrage, zombi

The lists of 15 words include the target word, 7 neutral distracters and 7 distracters that vary across conditions. The location of the target word in the upper or lower part of the list is counterbalanced over trials

The target word is either directly presented « anémone », either defined by its category « fleur ».

TR moyen des items

0

1000

2000

3000

4000

5000

6000

7000

8000

neutre orthographique contrôle associé

Sémantique

Visuelle

Results: times taken to find the target word

0

5

10

15

20

25

neutre orthographique contrôle associé

Sémantique

Visuelle

Recherche sémantique Recherche visuelle TR Global

Neutre Ortho. Contr. Assoc. Moy. Erreur Assoc. Neutre Ortho. Contr. Assoc. Moy. Moy

Pipe et rat -0,32 -0,36 -0,49 -0,39 -0,52 -0,21 -0,11 -0,11 -0,31 -0,37 -0,3 -0,55

Vocabulaire -0,53 -0,36 -0,52 -0,44 -0,62 -0,17 -0,23 -0,24 -0,13 -0,26 -0,3 -0,63

Results: error rates and inter-individual variables

Conclusions

• Task effect: with the same material, visual search takes longer when the target word is indirectly defined

• In the visual task, the performance is impaired when distracters visually similar to the target words are present

• In the semantic task, the performance is impaired when semantic associates of the target word are present

• Strong inter-individual variability. The level of performance correlates separately both with reading proficiency and vocabulary level

Impact of lexical familiarity on eye movements during visual search for verbal information

• This study deals with the cognitive mechanisms involved in the search for structured, target verbal expressions within lists.

• Vibert, Rouet, Ros, Limam, & Back (2006) demonstrated that when distracters resembling the target are present, the efficiency of target selection strongly depends on the verbal load in working memory. For instance, the impact of distracters is higher when unfamiliar vocabulary is used.

• The experiments were designed to check how the familiarity of vocabulary impacted the eye movements of participants during target reading (and memorization) and subsequent visual search within expression lists. The words used to compose the expressions were either familiar or specific to neuroscience.

Main relevant points in the literature

• Rayner (1998), Wiley & Rayner (2000): Word frequency effect on the duration of eye fixations during reading

• Rayner & Raney (1996): No word frequency effect during visual search for a word within a paragraph

• Harris, Pashler, & Coburn (2004): The meaning of distracter words does not influence search efficiency for an already known target word

Hypotheses

• During target reading and memorization, the duration of fixations will be shorter with familiar vocabulary, but there will be no impact of vocabulary during the visual search phase

• The absence of automatic semantic activation

during visual search will result in shorter fixations during visual search than during target reading

Procedures• 22 adult participants without any expertise in neuroscience • TOBII 1750 oculometer and display screen, coupled to an ASUS

laptop. • Structured, semantically valid expressions including five content

words. The participant could take as much time as wanted to read and learn the target expression. Then, after a fixation cross, a list of 15 expressions was displayed. The participants had to search for and click on the target expression as quickly as possible.

• Independent variables were the vocabulary, either familiar (condition 1) or specific to neuroscience (condition 2), and the task performed, i.e. target reading versus visual search. Six trials were done in each condition.

• Dependent variables were the times spent to read the target and perform the visual search, the average duration of fixations, and the average size of saccades.

Examples of target expressions and search displays

le petit dictionnaire de français illustré en couleur

un petit livre illustré en français et en couleurapprendre le français avec un petit illustré en couleurle petit dictionnaire de français illustré en couleur # un reportage en français et en couleur sur les petits dictionnairesce matin : assemblée générale dans la grande sallele petit dictionnaire de français illustré électroniquele sacrifice des soldats de la première guerre mondialele record battu du tour du monde à la voileutilisation de la couleur dans les petits dictionnaires françaisle livre épais de l'étudiant diplômé en histoireun petit dictionnaire bilingue illustré et sans couleurjeux de société : une activité pour la famille entièrele jardin fleuri de la vieille dame du quartier

Chronometric dataTime spent to read the target expression:

Condition 1 (familiar vocabulary)M = 4,785 ms SD = 1,266Condition 2 (neuroscience vocabulary) t (21) = - 7.41, p < 0.001M = 9,299 ms SD = 3,519

Time spent to search for the target expression within the list:

Condition 1 (familiar vocabulary)M = 5,434 ms SD = 2,078Condition 2 (neuroscience vocabulary) t (21) = - 4.02, p < 0.001M = 8,017 ms SD = 5,048

Eye movement dataThreshold for fixations: 50 ms

Tem ps m oyens des fixations en Lec ture et en R I en fonc tion de C 1 et C 2 (Tps en m s)

C 1 C 2

Lec ture R I150

155

160

165

170

175

180

185

190

195

200

205

Tps

mye

n de

s fix

atio

ns (

en m

s)

Taille m oyenne des saccades en lec ture et en R I en fonc tion de C 1 et C 2

C 1 C 2

Lec ture R I

LEC /R I

86

88

90

92

94

96

98

100

102

taill

e m

oyen

ne s

acca

des

Tem ps m oyens des fixations en lec ture et en R I en fonc tion de C 1 et C 2 (Tm ps en m s ; 100m s)

C 1 C 2

Lec ture R I210

220

230

240

250

260

270

280

290

300

310

Tps

moy

en d

es fi

xatio

ns (

en m

s)

Taille m oyenne des saccades en Lec ture et en R I en fonc tion de C 1 et C 2 (100m s )

C 1 C 2

Lec ture R I85

90

95

100

105

110

115

120

125

130

135

taill

e m

oyen

ne d

es s

acca

des

Eye movement dataThreshold for fixations: 100 ms

Discussion and conclusions

The chronometric data confirm evidence that prior knowledge facilitates verbal information search.

As hypothesized, there is a word frequency effect on eye fixations during target reading. There is no word frequency effect during visual search within structured expressions lists.

The absence of word frequency effect and the fact that fixations are shorter during visual search suggest that visual search for verbal expressions does not involve automatic semantic activation, and relies primarily on visual similarity judgments.

The larger saccades during visual search through unfamiliar vocabulary might reflect problems to recognize unknown words with parafoveal vision that would lead participants to plan their next saccade far from the current fixation point.