Constructivist approachCSCTR Session 11

Dana Retová

Start bottom-up Create cognition based on sensori-motor

interaction

◦ Cohen et al. (1996) – Building a baby◦ Cohen & Lederle (1998) - Dynamic maps as

representations of verbs

Constructivist approach

agent acquiring conceptual knowledge by sensorimotor interaction with its environment

Uses notion of image-schemas Goal

◦ Produce conceptual structures from sensorimotor interactions

◦ Show that very little prior structure is sufficient do to this (nature vs. nurture)

Building a baby

Pattern detectors or filters that map sensory streams onto redescriptions or partial representations.◦ E.g .

Image schemas

ANIMATE MOTIONANIMATE MOTION

Simulated agent◦ Lives in simulated environment – BabyWorld

Implements Neo’s sensations, mental representations, mental and physical activities nad the behavior of objects.

1. Neo – implements everything that Neo does Learning, moving, mouthing, looking, crying,…

2. StreamsWorld – represents Neo’s environment, implements events that happen around Neo and in response to Neo’s actions

◦ Neo senses its environment through a collection of streams in discrete time steps.

NEO

Streams◦ Affect, pain, hunger, somatic, haptic, visual,

auditory,… Simple and probabilistic Only random actions

◦ E.g. If Neo’s eyes alight on a rattle then Neo will grasp the rattle with some probability

At the beginning experience has no apparent structure

BabyWorld

1. Changes in token values: Tokens in streams are augmented by noticing when they change value.

2. Scopes: Neo finds pairs of correlated streams called scopes.

3. Base fluents: Neo finds common token-value pairs within scopes.

4. Context fluents: Neo finds base fluents that tend to follow each other in time.

5. Chains: These temporal dependencies are combined into temporal chains, which represent activities. Chains are used for activity-based categorization.

5 levels of redescription of Neo’s experience

Agent was able to learn basic concepts◦ It learned base fluents corresponding to the shape

and color of most objects in its envirnoment◦ It learned activities

Grasping, mouthing, movintgits arm, seeing its arm move

Results

Context fluents

Context chains

Classes of objects are differentiated by how we interact with them,

Concepts are abstractions over those classes

Meaning of concepts are in large part predictive models of how interactions with objects will unfold.

In the interactionist view, category distinctions are based on activity

Interactionist account

Thank you!!

DiscussionSix lessons from babies (Smith & Gasser. 2005)

It is claimed, that systems should be build incrementally. (not by selecting random x samples for training, and n-x for evaluation). Can you think of problems, where this does not work? [Martin]

It is said that it seems to be counter intuitive that children fail to locate an object in a room if the model that should help with this task is too similar to the room. Isn’t this just because a very similar model has to be very complex and is therefore harder to understand? [Thomas]

Incremental learning

How could multimodal associations be established in an infant? How does a baby know to look in the direction of the radio when it hears a sound? (In case of a moving car, this would be easy, but if there are no obvious visual clues?) How does it know to look in the direction of its parents when they are talking (prior to it being able to focus well enough to perceive the mouth motions)? [Thomas]

Multimodal associations

It is mentioned that babies start with "motor babbling", and then discover activities which trigger interesting events. Can you think of a way to better define what could be "interesting" for a baby? [Christian]

The time babies spend on an activity seems to depend on how much "fun" this activity is (examples in the paper are the mobile attached to a babies ankles, which jingles when the baby moves; and mothers in general, who smile if babies smile and coo if babies coo). In your opinion, what does this "fun" factor depend on? Can we define it?

[Thomas]

Babbling and goal directedness

Why should we assume that solving the reaching task starts from random, non-goal directed actions? Isn’t it more plausible that, while the babies’ movement might be mostly random, the reaching itself is always goal directed but erroneous? [Simon]

How would you try to come up with the "algorithm" the baby uses in order to explore its world? What kind of strategy does it use?

[Christian]

Babbling and goal directedness

Must a artificial system be equiped with language? [Martin]

Since a lot of interesting things must be learned by the baby before being able to understand language, or even talk, do you think it is a mistake to try and build language understanding systems, without trying to solve earlier developmental problems first?

[Christian]

Language

Must it be social? [Martin]

It is said that the lesson to be learned from facial-imitation-behavior is that intelligent systems have to be raised in a social world. But isn’t the main function of this facial imitation to allow for empathy in human interaction? Is this therefore really something we should be especially concerned with at the moment? [Simon]

Social learning

It is said that it seems to be counter intuitive that children fail to locate an object in a room if the model that should help with this task is too similar to the room. Isn’t this just because a very similar model has to be very complex and is therefore harder to understand? [Simon]

Arbitrary symbols