Giacomo Veneri 2012 phd dissertation

1

Feature-Based Information Processing of Selective Attention through Entropy

Analysis system

Giacomo VeneriNovember 2012

Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI


2

Objectives

• Study the influence of (eye) motor control on selective attention

• Develop a method to extract motor control parameters during visual search

• Develop a method to extract selective attention features during visual search

Methods Results

Attention FE

Motor Control

FE

TMT

ET

Healthy SubjectsPatients

SCA2,NDC

Psychological Test


3

Selective Attention• Selective attention ( Posner,

1980) is the process to select some region of the scene to be processed in detail; then, selective attention works as filter.

• Top-Down: attentional process that influences sensory processing in an automatic and persistent manner

• Bottom-Up: influence on the nervous system due to extrinsic properties of the stimuli


4

Motor Control and Cerebellum• The neuronal circuitry of the

cerebellum is thought to encode internal models that reproduce the dynamic properties of body parts (Kelly2003,Ito2005,Ito2006a).

• These models control the movement allowing the brain to precisely control the movement without the need for sensory feedback (Barlow2002,Ito2008,King2011)

• SCA2 and NDC Patients


5

Attention and Motor control(Corbetta2001, Osborne2011)


6

Methods1. Veneri, G., Federighi, P., Rosini, F., Federico, A., & Rufa, A. (2010). Influences of data filtering on human-computer interaction by gaze-contingent

display and eye-tracking applications. Computers in Human Behavior , 26 (6), 1555 - 1563. doi: 10.1016/j.chb.2010.05.030 [SCOPUS, ACM]2. Veneri, G., Federighi, P., Rosini, F., Federico, A., & Rufa, A. (2011, Mar). Spike removal through multiscale wavelet and entropy

analysis of ocular motor noise: A case study in patients with cerebellar disease. Journal of Neuroscience Methods , 196 (2), 318–326. doi: 10.1016/j.jneumeth.2011.01.006 [MEDLINE, SCOPUS]

3. Veneri, G., Piu, P., Rosini, F., Federighi, P., Federico, A., & Rufa, A. (2011). Automatic eye fixations identification based on analysis of variance and covariance. Pattern Recognition Letters , 32 (13), 1588 - 1593. doi: 10.1016/j.patrec.2011.06.012 [SCOPUS]

4. Veneri, G., Pretegiani, E., Rosini, F., Federighi, P., Federico, A., & Rufa, A. (2011, Mar). Evaluating the human ongoing visual search performance by eye tracking application and se-quencing tests. Comput Methods Programs Biomed . Retrieved from http://dx.doi.org/10.1016/j.cmpb.2011.02.006 doi:10.1016/j.cmpb.2011.02.006 [SCOPUS. MEDLINE, ACM]

5. Veneri, G., Rosini, F., Federighi, P., Federico, A., & Rufa, A.(2012, Feb). Evaluating gaze control on a multi-target sequenc-ing task: The distribution of fixations is evidence of exploration optimisation. Comput Biol Med , 42 (2), 235–244. Retrieved from http://dx.doi.org/10.1016/j.compbiomed.2011.11.013 doi: 10.1016/j.compbiomed.2011.11.013 [SCOPUS. MEDLINE, ACM]

InProceedings6. Veneri, G., Federighi, P., Pretegiani, E., Rosini, F., Federico, A., & Rufa, A. (2009). Eye tracking - stimulus integrated semi automatic case base system. In

Proceeding of the 13th world multi-conference on systemics, cybernetics and informatics.7. Veneri, G., Pretegiani, E., Federighi, P., Rosini, F., & Rufa, A. (2010). Evaluating human visual search performance by monte carlo methods and heuristic

model. In IEEE (Ed.), 10th ieee international conference on information technology and applications in biomedicine (itab 2010). [SCOPUS, IEEE]8. Veneri, G., Piu, P., Federighi, P., Rosini, F., Federico, A., & Rufa, A. (2010, jun.). Eye fixations identification based on statistical analysis - case study. In

Cognitive information processing (cip), 2010 2nd international workshop on (p. 446 -451). IEEE. doi: 10.1109/CIP.2010.5604221 [SCOPUS, IEEE]Others (posters)9. Veneri, G., Federighi, P., Rosini, F., Pretegiani, E., Federico, A., & Rufa, A. (2009). The role of latest fixations on ongoing visual search: a model to

evaluate the selection mechanism. In Rovereto workshop of attention.10. Veneri, G., Olivetti, E., Avesani, P., Federico, A., & Rufa, A. (2011). Bayesian hypothesis on selective attention. In Rovereto visual attention congress.


7

PSYCHOLOGICAL TESTEye Tracking, TMT, ET

Methods Results

Attention FE

Motor Control

FE

TMT

ET


SCA2,NDC

Psychological Test


8

Eye Tracking

• Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head.

• ASL 3000 (240Hz)


9

Visual (conjunction) Search Test

E Search (Wolfe, 1994) Sequencing (Reitan, 1958)

... and others (Veneri 2010, Veneri 2012)


10

SELECTIVE ATTENTION FEATURES EXTRACT

Psycological Test, Mathematical Method

Methods Results

Attention FE

Motor Control

FE

TMT

ET


SCA2,NDC

Psychological Test


11

Attention Features Extraction 1/2

Common Method• Visited ROI• Reaction Time

Our geometric Method (Veneri, Rosini 2012)

• Distance to nearest Target• Distance to Nearest ROI• Sequencing

DN

DT


12

Sequencing (2/2)

• Look for the best path (Veneri, Rosini 2012)


13

MOTOR CONTROL FEATURES EXTRACTION

Wavelet Entropy

Methods Results

Attention FE

Motor Control

FE

TMT

ET


SCA2,NDC

Psychological Test


14

Motor Control Noise Evaluation

• (Beers2007, Veneri2011) gaze noise may be additive with or multiplicative of the eye movement, and is lost in recording noise (RN) due to blinks or signal loss;

• noise = PN + RN = SDN (signal) + ADN + RN where SDN is physiological signal dependent noise and ADN physiological additive noise.


15

Frequency Analysis

Fourier analysis• A signal is a «sum» of a sine

curve

ECG Example


16

Wavelet and Entropy

Wavelet Multiscal decomposition Wavelet (Mallat, 1989)


17

Decomposed Eye Signal

Original signal

Noise?

Main componet


18

Wavelet Entropy

The idea (Veneri 2011)• After decomposition• We removed spikes• We evaluated Entropy

• Entropy is the measure of the chaos on a system

Algorithm


19

RESULTSHealthy Subjects and Patients

Methods Results

Attention FE

Motor Control

FE

TMT

ET


SCA2,NDC

Psychological Test


20

Despiking

Healthy Subject Patient


21

Despiking


22

Healthy Subjects

Clusters ROC (20% error rate)


23

Patients

P-value Clusters


24

Entropy levels

All levels Last level


25

Variance

Signal Signal on fixations


26

Before conclusions

• Proposed Wavelet Entropy Implementation is NOT noise on fixations or noise of global signal

• Proposed Wavelet Entropy Implementation «catches» motor noise topical featurese of each subject (colored noise)

• Wavelet Type or levels are critical


27

Selective attention• DT provided a indicator to under-

stand the ability of humans to converge to the target.

• ANOVA reported significant difference among groups (F (2, 35) = 9.476, p < 0.01)

• post-hoc Sidak procedure confirmed significant difference between – CTRL-SCA2 (p CTRL−SCA2 < 0.01),– CTRL-NDC (p NDC−SCA2 ≤ 0.01);– no significant dif-ference was

found between SCA2-NDC (p SCA2−NDC = 0.622).


28

Correlation DT-E

• Pearson and Spearman test reported correlation between E and DT for NDC patients (p < 0.05, ρ = 0.892, A), and correlation for SCA2 patients (p < 0.05, ρ = 0.736, B) not confirmed by Spearman (p = 0.18). No correlation was found for CTRL subjects (p = 0.43).


29

CONCLUSIONSTools and Hypothesis


30

Summary

• In the current work two methods have been developed: • Selective attention evaluation• Entropy analysis through wavelet decomposition.

• Both methods are based on eye tracking• Subjects and patients cannot control eye movements or

fixations perfectly, then, analysing eye motor entropy it is possible to extract some important features and conclusions.


31

Tool1. Import Eye gaze data2. Export Eye gaze data3. Fixations recognition

(Veneri, Piu, et al., 2010, 2011; Salvucci & Gold-berg, 2000)

4. Saccades recognition (Fischer et al., 1993)

5. TMT sequencing analysis6. Transition Matrix analysis7. ROI Analysis8. Experiment segmentation


32

Study the influence

• Does the motor control (cerebellum) influence selective attention?


33

Cerebellum could influence selective attention (Top-Down) sending afferent information of noise in order to minimize the functionalcost of energy.

Our hypothesis is systematically supported by recent application of opti-mal control theory; (Najemnik & Geisler, 2005), (Beers, 2007) and (Osborne, 2011) argued that humans’ vision is an optimal mechanism minimizing theeffect of motor or cognitive noise. Our findings are compatible with this hypothesis: patients preferred sparser fixations avoiding saccade directed to thetarget. The non correlation of DN with WS suggested that this mechanismwas a strategy to minimize the effort to control saccade rather than a directinfluence on visual search.


34

THANKS

Feature-Based Information Processing of Selective Attention through Entropy Analysis system


35

Model

Energy Saccade length

Giacomo Veneri 2012 phd dissertation

Health & Medicine

Transcript of Giacomo Veneri 2012 phd dissertation