learning3 23 09 - UCSD Cognitive Sciencecreel/COGS101b/COGS101b_files/... · 2012. 2. 21. · Bad...

2/1/12

1

Learning I

Overview

•  Biological background •  Types of learning •  Modern versions of learning

What is learning?

•  What goes with what •  Profiting from experience* •  Storing information in memory

– Without [necessarily] acting on it immediately

Non-learning influences on behaviors

Fixed-action patterns

•  Non-learned behaviors •  Organism does reflexively •  Initiated by releasers •  Once initiated, run from start to finish

Fixed action patterns

•  Babies: – Grasping, head-turning, sucking – All aid in nursing

•  Adults: yawning? •  Birds:

– Greylag goose and egg-rolling – Oystercatchers and larger eggs

•  “supernormal” stimulus

2/1/12

2

Critical periods

•  Limited time for learning

•  Extra plasticity •  Found in humans,

animals •  Language •  Also, vision

From Johnson & Newport

From Banks & Aslin

Critical periods

•  Imprinting – Halfway between built-in behaviors and

flexible learning – Baby duck: Follows moving object – Mama duck, person, wooden duck

•  Very quickly learns to prefer it

Critical periods

•  Birdsong learning – Some birds are ‘born with’ song – Others learn their species’ song

•  E.g. white-crowned sparrow

– Often compared to (human) language acquisition

Critical periods

•  Birdsong learning – White-crowned sparrow (Marler, 1970) – Must hear species’ song between 10 & 50d – Doesn’t actually sing till 150-200d – Raised in lab:

•  Can learn from recording of w.-c. s. song •  Can’t learn from song sparrow song (diff.

species)

Critical periods

•  Birdsong learning – Marler, 1970:

•  Crude ‘template’ of species-typical song •  Hooked into intrinsic feedback/reinforcement

system •  Like human speech, which also isn’t linked to

an extrinsic reward –  I.e., it must be internally rewarding

Critical periods

•  Language acquisition – Critical period for learning language[s]?

•  The island experiment •  Umm, illegal.

– Test case: late learners of ASL

2/1/12

3

Critical periods

•  Language acquisition –  Late ASL learners (Newport & Suppalla, 1989)

•  ASL first language •  Acquired at different ages •  10+ years experience using ASL

–  Better performance the earlier it was acquired –  Johnson & Newport (1990)

•  Similar results with L2 learners of English

Critical periods

•  Language acquisition – Newport (1990): “Less-is-more” hypothesis

•  Language [sign] elements a, b, c •  Meaning elements m, n, o •  Many ways to combine (7 * 7 = 49) •  Kids have worse working memory so they can

only consider small # of combinations – Much more likely to be right

•  Cochran, McDonald, & Parault (1999) – Concurrent task = better generalization

Other constrained learning

•  Some things very difficult to condition to some stimuli (Bolles 1970) – Run away from shock, press bar for food – Sweetness & nausea, click & pain – Vice versa

•  Species-specific constraints – Pigeons learn visual cues to illness

Behaviorism & learning

•  Very quantitative •  In the end, couldn’t fully account for all

real-world learning •  Still useful in some contexts today

– Drug treatment – Chemotherapy & food aversions – Learning rules in many computational

models

Behaviorism & learning

•  Habituation •  Classical conditioning •  Operant conditioning

– Also “trial and error learning” or “instrumental learning”

Habituation

•  Decline in organism’s reaction as stimulus loses novelty

•  Don’t react to stable, consequence-free event

•  May also contribute to ease of learning the right associations – Unusualness heuristic

2/1/12

4

Classical conditioning

•  Pavlov •  Terms:

– Unconditioned stimulus – Unconditioned response – Conditioned stimulus – Conditioned response


Food

Drool

Bell

Drool Drool Drool Drool Drool

US

UR

CS

CR

What’s ‘set in stone’ What’s learned


Toxin

Nausea

Scapegoat food

Nausea Nausea Nausea Nausea Nausea

US

UR

CS

CR



Reading important

email

Heightened pulse

Bold font in email inbox

Heightened pulse

Heightened pulse

Heightened pulse

Heightened pulse

Heightened pulse

US

UR

CS

CR



•  Extinction –  CR can go away if US/CS relationship broken

•  Blocking –  Kamin, 1969

•  Two CS’s •  Simultaneous: both elicited CR •  One conditioned first: only first one elicited CR


•  Predictiveness of the CS is crucial for C.C. to occur (not simple cooccurrence) –  Rescorla (1967)

Group Prob. US follows CS

Prob. US by itself

Learning?

1 .80 .80 N 2 .80 .40 Y 3 .40 .40 N 4 .40 0 Y

2/1/12

5


•  Stimulus substitution? – CS takes place of US – Too simple

•  Adaptation – Sometimes the CR is opposite of UR – Compensatory response model

•  Dinitrophenol: + O2, + temp •  Conditioned response: -O2, -temp


•  Compensatory response model –  Explanation for drug tolerance? –  Need increasing doses to get effect

•  CR in opposite direction counteracts drug

–  Heroin: Gutierrez-Cebollada et al. (1994) •  Addicts admitted to hospital for OD

–  50% had injected normal dose in unusual environment •  Addicts admitted for unrelated stuff (controls)

–  All in familiar environment •  No CS -> lower tolerance


Altoid

Good taste

Computer noise

Bad taste

US

UR

CS

CR


Overview

•  Biological background •  Types of learning •  Modern versions of learning

Operant conditioning

•  Other names –  Instrumental learning – Trial-and-error learning

•  Behavior that acts (“operates”) on the environment

•  Voluntary response is encouraged by rewarding the response


Behavior Consequence (Reinforce/Punish)

Outcome Type of reinforcement

X Appetitive (“good”) stimulus occurs

+ behavior Positive reinforcement

X Appetitive stimulus ends

- behavior Negative punishment

X Aversive (“bad”) stimulus occurs

- behavior Positive punishment

X Aversive stimulus ends

+ behavior Negative reinforcement

2/1/12

6


•  Reinforcers – Primary (food, water, mating) – Secondary

•  Strong association with primary •  E.g. money


•  In the lab –  “Skinner box”

•  Behavior: keypeck, bar press –  Pigeon/rat will do sometimes randomly

•  Reinforcement: food •  (Or punishment, e.g. footshock or nausea) •  Directly manipulate frequency of behavior by

changing contingencies of behavior


Beg

ging

beh

avio

r Likelihood of receiving ice cream for begging

high

medium

low

0 # of trials

A lot

Operant conditioning Ju

mp

on fu

rnitu

re

Likelihood of being squirted with water

low

medium

high 0

# of trials

A lot


Ala

rm c

lock

sla

ps

Likelihood of alarm ceasing

high

medium

low 0 # of trials

A lot


Rud

e co

mm

ents

Likelihood of party invitations

high

medium

low 0

# of trials

A lot

2/1/12

7

Paired associates

•  Analog to instrumental learning •  Pair words, flashcard-style

(“car”-”coche”) •  Anticipate word 2 till it’s readily recalled

– Bidirectional (word 2 --> word 1)

Paired associates

•  Each time a pair is learned, strengthened associations

•  Learn nonwords--”new” associations – But depends greatly on existing knowledge

Conditioning wrap-up

•  Weaknesses of ‘conditioning’ approach –  Ignore biological influences on behavior –  Ignores role of top-down info (knowledge)

•  Nonethless, still useful in many practical contexts, including…

Infant methods •  Definitely cognitive stuff going

on. •  But how to investigate in non-

verbal organisms? •  By looking at behavior.

Also used with

xkcd.com

Habituation

•  Bore kids to death with something (e.g. sound, picture).

•  Then present something new. •  Do they dishabituate?

•  If so, suggests they can tell the difference. •  Do brains dishabituate?

–  fMRI adaptation –  Petkov et al., 2008:

•  Present vocalizations to macaque in fMRI •  Change to vocalizations from another animal •  BOLD signal increases!

Learning

•  Figuring out what things go together

•  (And also what things don’t go together)

2/1/12

8

•  Lookattheprettybabyeveryone •  Whereisthebabynowprettygirl •  Thisroastedbabyisprettygood


•  ty->ba: unlikely (.33 transition probability)

•  ba->by: likely (1.0 transition probability) •  by->ev: unlikely (.33)


•  ty->ba: unlikely (.33 transition probability)

•  ba->by: likely (1.0 transition probability) •  by->ev: unlikely (.33)

Other cues to word boundaries

•  Stress – English: most words are stress-initial – ThePREttyBAbyWANTSaBOTtle


•  Jusczyk, Houston & Newsome – 7.5. Months:

•  Familiarize infants with stress-initial words –  KINGdom

•  Play passages with KINGdom vs. HAMlet •  Infants listen longer to kingdom >hamlet

passages •  Recognize words



•  Stress-initial OK •  Familiarize with guiTAR •  Test with guiTAR, surPRISE •  No difference in looking times

2/1/12

9



•  Stress-initial OK •  Stress not initial XX •  Familiarize with tar, prize •  Play guiTAR/surPRISE--get it


•  Jusczyk, Houston & Newsome – 7.5. months:

•  Stress-initial (kingdom) OK •  Stress not initial (guitar) XX •  Stressed syllable (tar) OK

– 10.5 months: •  Stress not initial (guitar) OK

But…

•  Not all languages have this kind of stress pattern.

•  (Similar to the segmentation problem in object recognition--how do you know what the properties of objects/words are if you don’t know what the objects/words are themselves???)

Statistical learning

•  Saffran, Aslin & Newport (1996) – 8-month-old infants

golabubidakutupiropadotitupirobidakupadotigolabupadotibidakugolabutupiro golabupadotitupirobidakupadotibidakutupirogolabupadotibidakugolabutupiro …







2/1/12

10





bi da ku

pi ro bi

tu pa go

… ti ro bu

tu da

.33 .33

.33

1.0 1.0

1.0 1.0 1.0


•  Play for 2 minutes •  Present kids with bidaku or piro.bi type

words •  What do they listen to longer?

bidaku < piro.bi –  (novelty preference)

•  At 8 months infants can segment words based on statistical cues

But what about stress?

•  BRILligSLIthyTOVES.


•  BRILligSLIthyTOVES. •  Johnson & Jusczyk (2001)

– 8-month-olds – Stress vs. statistics: Stress wins




•  Thiessen & Saffran (2003) – Statistics wins @ 7 mos – Stress wins @ 9 mos

2/1/12

11




•  Thiessen & Saffran (2003) – Statistics wins @ 7 mos – Stress wins @ 9 mos

•  Segmentation precedes stress.


•  What’s getting learned? – Frequency

•  Thedog > hassock •  But “thedog” isn’t a word, and hassock is (srsly)

– Conditional probability •  Given X, Y is __ likely to happen •  Predictiveness

•  Aslin, Saffran & Newport (1996)


•  What’s getting learned? – Aslin, Saffran & Newport (1996)

•  golabu/padoti 2x as often as tupiro/bidaku •  So ti.gola is as frequent as bidaku (45x each) •  But

bi->da->ku

ti->go->la

1.0 1.0

.33 1.0


•  What’s getting learned? – Aslin, Saffran & Newport (1996)

•  golabu/padoti 2x as often as tupiro/bidaku •  So tiba.go is as frequent as bidaku (45x each) •  3 min of listening; then test

bi->da->ku

ti->go->la

1.0 1.0

.33 1.0

Frequency:

=

Cond. prob.:

<


•  Infants are indeed learning conditional probabilities, not just frequency.

Language specificity?

•  Works for language •  Works for tone sequences (C-C#-G…) •  Works for visual stuff •  Tamarin monkeys do it (Hauser et al.)

–  (or not?)

•  Parallel results: language and music •  Slightly differing: sound and sight

2/1/12

12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

Always vs impossible

Always vs possible

Possible vs impossible

Prop

ortio

n co

rrec

t

***

*** ***

Statistical learning: your data pre.tty

vs pre.ba

pre.tty vs

ty.ba

ty.ba vs

pre.ba

(some word analogs of the visual stimuli [prettybaby])

Nonadjacent dependencies

•  Phenomenon in language •  Not only…but also

– Either…or – Who…with? – Got _____ed

•  Dependent on linguistic structure, or learnable from input?


•  Nonadjacent dependency learning:


Nonadjacent dependencies Nonadjacent dependencies

Syllables: No (Newport & Aslin, 2004)

2/1/12

13


Consonant->consonant Vowel->vowel

(Newport & Aslin, 2004)

0

500

1000

1500

2000

2500

3000

0 500 1000 1500 2000 2500

Time (ms)

Fre

quency

(Hz)

A B C

⇓ ⇓ ⇓

0

500

1000

1500

2000

2500

3000

0 500 1000 1500 2000 2500

Time (ms)

Fre

quency

(Hz)

A B C

2/1/12

14

0.5

0.5

1.0

1.0

1.0

1.0

0.5 0.5 0.5 0.5 0.5 0.5

1.0 1.0 0.5...

1.0 1.0 0.5...

0.5 0.5 0.5 0.5 0.5 0.5 0.5...

chance

2/1/12

15

Is high v. low the only grouping cue?

Redo different-pitch condition, but with complex tones of two different timbres

– Perceived fundamentals of each tone set in same frequency range – Actual harmonics don’t overlap

0

500

1000

1500

2000

2500

3000

0 500 1000 1500 2000 2500

Time (ms)

Fre

quency

(Hz)

A B C

All-or-none auditory effect? – Could be a low-level perceptual phenomenon – Not related to language-learning

–  Speech doesn’t “fly apart” like that

New timbre difference: flute vs. violin –  Discernibly different but harmonics overlap

0 500 1000 1500 2000

10000

5000

0

Freq

uenc

y (H

z)

Time (ms)

Learned both structures!

2/1/12

16

Vision vs. audition

•  Temporal sequences: – Audition better

•  Simultaneous cooccurrence: – Vision better – But some auditory cooccurrences learned

learning3 23 09 - UCSD Cognitive Sciencecreel/COGS101b/COGS101b_files/... · 2012. 2. 21. · Bad...

Documents

Transcript of learning3 23 09 - UCSD Cognitive Sciencecreel/COGS101b/COGS101b_files/... · 2012. 2. 21. · Bad...