Arthur Franz 1

Modeling event perception in Modeling event perception in infancyinfancy

Arthur Franz

Frankfurt Institute for Advanced Studies

http://fias.uni-frankfurt.de

FIAS, 2008-4-29

Arthur Franz 2

• Where does knowledge come from?

Study “simple” systems: infants

• Perception and conception of spatial events seem to be crucial

• E.g.: occlusion, launching, object unity and, permanence, continuity, object solidity, support,… ”naïve physics”

• Hypothesis: most of them can be learned from purely statistical properties of visual input.

MotivationMotivation

Arthur Franz 3

How do people investigate what infants know?

Habituation paradigm

Example: Perception of object unity

The habituation paradigmThe habituation paradigm

rod movement baseline

habituationdisplays

testdisplays

habituation test

habituation test

Mean

lookin

g tim

e

(sec)

Arthur Franz 4

Input example for the object unity experiment

7 x 7 pixel retina

BACKGROUND

FOREGROUND

We build a network that learns to represent occluded objects.

How can we model this?How can we model this?

Arthur Franz 5

Assumption:

Neurons tuned to

velocity AND disparity

In MT?

Input codingInput coding

Arthur Franz 6

Neural networkNeural network

Got it?Got it?

Arthur Franz 7

Calculation detailsCalculation details

Learning: backpropagation of error with Learning: backpropagation of error with realreal inputs and outputs. inputs and outputs.

Objective function:Objective function:

Arthur Franz 8

Pre-training corresponds to the infant’s visual experience with the world

Varying the pre-training time allows for modeling infant’s of various ages!

Pretraining with random

moving or stationary rectangles

Pre-trainingPre-training

Arthur Franz 9

What the network What the network “imagines”“imagines”

real inputsreal inputs

real outputsreal outputs

full inputsfull inputs

imagined outputsimagined outputs

Arthur Franz 10

In infant experiments the looking time is measured.

Looking time ~ attention, novelty ~ habituation (“tiring”) of certain neurons in the infant’s brain.

New stimulus => other neurons get active => dishabituation

Dishabituation in the model = difference between the hidden layer activity during habituation and the activity during a test stimulus.

Relation to infant Relation to infant experimentsexperiments

Arthur Franz 11

Modeling object unity (1)Modeling object unity (1)full inputs

rod movement baseline

complete rod broken rod

Rod movement => preference for broken rod

Baseline => no preference

habituationdisplays

testdisplays

Arthur Franz 12

Modeling object unity (2)Modeling object unity (2)

rod occlusion complete rod broken rod control control

habituationdisplays

testdisplays

Rod occlusion => preference for broken rod (age effect!)

Complete rod control => pref. for broken rod

Broken rod control => pref. for complete rod

Arthur Franz 13

Modeling object unity (3)Modeling object unity (3)full inputs

rod occlusion rod pieces

Rod occlusion => preference for broken rod

Rod pieces => preference for broken rod

habituationdisplays

testdisplays

Arthur Franz 14

Modeling object unity (4)Modeling object unity (4) rod moves rod & block move block moves no movement

habituationdisplays

testdisplays

Result: after long pre-training the network shows a preference for the broken rod in each condition!=> Age effect, see adult data

Arthur Franz 15

Modeling object unity (5)Modeling object unity (5)

rod-polygon baseline

complete broken

rod-polygon => preference for broken rod

Baseline => no preference

habituationdisplays

testdisplays

Arthur Franz 16

Adult dataAdult data

Arthur Franz 17

Modeling perception of Modeling perception of occluded trajectories (1) occluded trajectories (1)

habituation

continuoustest

discontinuoustest

thick occluderthin occluder4-month-olds

Arthur Franz 18

Modeling perception of Modeling perception of occluded trajectories (2)occluded trajectories (2)

habituationdisplays

testdisplays

exp. condition baseline long pre-training

exp. condition baselineshort pre-training

Exp. condition => preference for discontinuous display

Exp. condition => preference forcontinuous display

Baseline => no preference

Arthur Franz 19

Modeling perception of Modeling perception of occluded trajectories (3)occluded trajectories (3)

thin

thick

• Natural explanation for data• Model explains how and why preferences change• Object permanence develops in the network!

2 mo 4 mo 6 mo

preference

Pre-training time / 1000

Arthur Franz 20

• The neural network provides a model of infant’s perception of occluded objects, object unity and object permanence.

• In total 9 fundamental experiments from 2 different laboratories have been explained.

• The network is a developmental model and can reveal the mechanisms of change. Especially, the how and why questions can be adressed.

• It demonstrates that much of infants’ perception can be learned and explained solely on the basis of statistical regularities of raw visual input. No innate principles or modules need to be postulated.

SummarySummary

Arthur Franz 21

• Backpropagation of error => Andrea’s network?

• Dishabituationmeasurment is done only with first hidden layer. What not the second? Why not the whole network? Habituation with intrinsic plasticity?

• Stimuli are “flat” on the screen in the lab => no bottom-up disparity-based separation possible!

• Evidence for neurons tuned to both velocity AND disparity?

• In some experiments the prediction error is more suitable as a dishabituation measure. How to combine?

• Imagined outputs are noisy. The calculation of the full inputs is too “constructed”.

Drawbacks and open Drawbacks and open questionsquestions

Arthur Franz 22

• Predictions of the model

• Elaborate the relation of this model to the experimenters verbal accounts

• Include other event categories into pre-training (blocked motion, launching). Many other experiments can then be explained. continuity, solidity, object permanence,…

Future workFuture work

Arthur Franz 23

Thank you!Thank you!

Arthur Franz 24

• Kellman, Spelke (1983). Perception of Partly Occluded Objects in Infancy. Cognitive Psychology, 15, 483-524

• S.P. Johnson, J.G. Bremner, A. Slater, U. Mason, K. Foster, A. Cheshire (2003). Infants' perception of object trajectories. Child Development, 74, 94-108

ReferencesReferences

Arthur Franz 25

• Hidden+context layer doing everything?

• Feedback from im. Layer to hidden/context?

• Bayesian / optimization approach

• Disparity cells not present before 4 months

• Modeling with Kalman filters

Talk feedbackTalk feedback