12/04/2023 1

Cibernetica, modelli computazionali e tecnologie informatiche

per lo studio dei processi di cognizione sociale

Nature, Nurture and Future8 Dicembre 2009

Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science

University of Siena, Italy

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

2

Overview

• Cibernetica• Knowledge Engineering• Biomedical Engineering• Neuronal Machine

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

3

Cibernetica

La cibernetica è la scienza che studia i fenomeni di autoregolazione (controlli automatici e

controlli adattativi) e comunicazione (teoria dell'informazione - IT), sia negli organismi

naturali quanto nei sistemi artificiali.

Norbert Wiener

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

4

Knowledge EngineeringExample

Data … --- …

Information S O S

Knowledge emergency alert

•Biomedical System

•Telecomunication System

•Storage System

Data

•Storage System

•Data Mining & Search Engine

•Image , Signal Processing

Information

•Knowledge Based Storage

•Automatic, Adaptive, Complex Control

Knowledge

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

5

Ingegneria Biomedica (una volta Bio Ingegneria)

Estrarre dati (sistemi

biomedici)

Produrre Informazione

(Signal & Image

Processing)

Modellare sistemi naturali

Emulare sistemi naturali

Costruire sistemi

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

6

Biomedical System

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

7

fMRI

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

8

Eye Tracking

EOG Pupil Tracking

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

9

EEG & Evocated Potential

Signal Decomposition EEG

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

10

Active System

tDCS TMS

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

11

IT & LIFE

Data 2 Information

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

12

Search Engine – Patent DB

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

13

Tele Riabilitazione : Eracle Project

In loco rehabilitation

Tele-rehabilitation

Remote help desk

Social Area

Decision Making - NeuroEconomic

• Game Theory• The subject was asked

to choose between two sign according to previous chooses– Overconfident– Bayesian

04/12/2023 EVA – EYE TRACKING - STIMULUS INTEGRATED SEMI AUTOMATIC CASE BASE SYSTEM 14

[Innocenti & Rufa 2010]

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

15

Knowlede Management nelle Scienze della Vita

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

16

Algoritmi per la Diagnosi

• Ipertensione endocranica idiopatica

[Tornese 2007]

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

17

Biomedical System• “More 60% of our Barin

works on vision …”:

– Image Processing– Attention – Working memory– Memory– Visual-Spatial exploration– Eye Movement– Intention– Decision

The focus of attention is jointly determined by stimulus salience and top-down control signals. IOR is implemented by a mechanism that reduces the salience of stimuli at recently explored spatial locations.

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

18

EVA – EYE TRACKING - STIMULUS INTEGRATED SEMI AUTOMATIC CASE BASESYSTEM

• Test the cognitive system by multi-stimulus

• Diagnosis by Case Base

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

19

EVA – EYE TRACKING - STIMULUS INTEGRATED SEMI AUTOMATIC CASE BASESYSTEM

STIMULUS INTEGRATED SEMI AUTOMATIC CASE BASE SYSTEM

Vision Cognitive Task

Analysis(Indicators)

Case Base

Diagnosis

Rehabilitation

University of Siena Patent

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

20

Mean Sequence

• Diagnosis SCA2• Vp=Vmax (1 – exp(A/C))

[Leigh 2004]

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

21

Nauronal Machine

Model of Brain and ….

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

22

Working Memory & Visual Search• Working memory :

– is a theoretical construct that refers to the structures and processes used for temporarily storing and manipulating information.

• Visual Search– memory maintains the

information in order to allow past experience to guide future behaviors

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

23

Early Model of saccade System

Neural network

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

24

Cerebellum as adaptive control

[2007 T. Yamazaki]

[Ito 2008]

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

25

A real-time spiking cerebellum model for learning robot control

[Carrillo 2008]

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

26

Visual Search

Frontal area

Internal intention

Planning fixations

Priority map

[Itti 2000]

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

27

Eye Movements & Attention• Eye movements

– Sequences of fixation and saccade

– The sequences of saccade-and-fixation enable humans to receive a continuous flow of information from the world.

• Attention– Attention is the process of

selecting a few relevant perceptive stimuli by filtering of negligible aspect of the environment

Overlapped region

Peripheral attentionSaccadic eye movements

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

28

Break Away from Fixations

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

29

Break Away from Fixations Model

100 200 300 400 500 600 700 800 900 1000

100

200

300

400

500

600

700

100 200 300 400 500 600 700 800 900 1000

100

200

300

400

500

600

700

[Veneri 2010]

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

30

Decisione Making• Decision making:

– can be regarded as an outcome of mental processes (cognitive process) leading to the selection of a course of action among several alternatives.

Basic connectivity framework. Dotted arrows represent external inputs to the model. Abbreviations: 5-HT, dorsal raphe serotonergic neurons; ACC, anterior cingulate cortex; AMYG, amygdala; DA, midbrain dopaminergic neurons; DLPFC, dorsolateral prefrontal cortex; OFC, orbitofrontal cortex; VS, ventral striatum.

12/04/2023 Eye tracking & Vision Applications Lab (EVA Lab) Department of Neurological Neurosurgical and Behavioral Science University of Siena, Italy

31

References• Carrillo R, Ros E, Boucheny C, Coenen O. A real-time spiking

cerebellum model for learning robot control. Biosystems. 2008;94(1-2):18-27. Available at: http://linkinghub.elsevier.com/retrieve/pii/S0303264708001226.

• Ito M. Control of mental activities by internal models in the cerebellum. Nature. (Box 1).

• Karl E. Misulis, Toufic Fakhoury (2001), Spehlmann's Evoked Potential Primer, Butterworth-heinemann

• 1975 – Engel - Visual cospicuity, visual search and fixation tendencies of the eye – Vision

• 1976 – Banks et al – Configurational effects in visual information processing. Perception & Psycophysics

• 1993- Fischer, B., Biscaldi, M., and Otto. Saccadic eye movements of dyslexic adults. Neuropsychologia, Vol. 31, No 9, pp. 887-906.

• Zhaoping Li 2002, A Saliency Map in Primary Visual Cortex (2002), Trends in neuroscience

• W. Wolwer 2003 – Trait Characteristic id Impaired Visuomotor Integration during Trail-Making Test B Performance in Schozophrenia – Neuropsycobiology

• 2004 E.Davis – Visual search attention – Spatial Vision• 2004 R.Leigh – Using saccades as a research tool in clinical

neurosciences – Brain• 2005 B. Machner – cerebellar infarction affects visual search -

Neuroreport• 2006 – Administration and interpretation of the Trail Making Test –

Nature Protocols• 2006 – H. Ojanpaa - Visual search and eye movements in patients

with chronic solvent-induced toxic enecelophaty – NeuroToxicology• 2007 – T. Yamazaki,Shigeru Tanaka - The cerebellum as a liquid state

machine – Neural Network• 2008 G. Jacobson, et Al.- A model of the olivo-cerebellar system as a

temporal pattern generator• 2008 M. Ito - Cerebellar circuitry as a neuronal machine – Nat

neuroscience