Post on 28-Mar-2015
LearningLearning
Psychology Summer School 2005Psychology Summer School 2005
Peter WardPeter Ward
Reference:Reference:
Gleitman: Chapter on learningGleitman: Chapter on learning
Any other basic psychology textbook Any other basic psychology textbook should cover much of the materialshould cover much of the material
Overview of LectureOverview of Lecture
1.1. Why is learning important?Why is learning important?
2.2. Classical conditioning Classical conditioning
3.3. Instrumental conditioningInstrumental conditioning
4.4. Cognitive views on learningCognitive views on learning
Why Learning?Why Learning?
Humans are born with a number of ‘built-in Humans are born with a number of ‘built-in programs’programs’
» Muscle reflexesMuscle reflexes» The Universal GrammarThe Universal Grammar
But much of what humans do and think is a result But much of what humans do and think is a result of learning. of learning.
The things that make individuals, societies and The things that make individuals, societies and nations different are almost all due to learning.nations different are almost all due to learning.
Obviously a complex process, so psychologists Obviously a complex process, so psychologists have tried to understand it from the most basic have tried to understand it from the most basic level upwardslevel upwards
Learning and MemoryLearning and Memory
The simplest form of learning: HabituationThe simplest form of learning: Habituation
HabituationHabituation
(loud noise)
HabituationHabituation
Goodness me! What a surprise!
HabituationHabituation
That’s still quite surprising…
HabituationHabituation
Okay, I’m less surprised now
HabituationHabituation
Boring.
HabituationHabituation
The point being…The point being…– Leaning involves REMEMBERINGLeaning involves REMEMBERING– Strong link to another crucial area of Strong link to another crucial area of
psychologypsychology
– Moving on…Moving on…
Part 2Part 2
Classical ConditioningClassical Conditioning
Classical ConditioningClassical Conditioning
First type of learning studied in depthFirst type of learning studied in depth Focused on learning Focused on learning associations associations between events.between events.
Russian physiologist, Ivan Pavlov (1849-1936), Russian physiologist, Ivan Pavlov (1849-1936), worked on digestive reflexes in dogs worked on digestive reflexes in dogs
» Realised he could study learned reflexes as well as innate onesRealised he could study learned reflexes as well as innate ones
» Discovered he could actually train, or Discovered he could actually train, or conditioncondition reflexes reflexes (hence the name)(hence the name)
How did he do it?How did he do it?
Pavlov and his amazing drooling dogs.Pavlov and his amazing drooling dogs.
How did he do it?How did he do it?
The man himselfA dog
Some Russians
How did he do it?How did he do it?
Dogs in the lab salivated when you put Dogs in the lab salivated when you put meat powder in their mouths. meat powder in their mouths.
But they would also salivate whenBut they would also salivate when– They saw the They saw the meatmeat– They saw the They saw the dishdish– They saw the They saw the person who brought itperson who brought it– They heard the They heard the person’s footstepsperson’s footsteps
Classical Conditioning MethodClassical Conditioning Method
Food = salivatingFood = salivating Food + Bell = salivatingFood + Bell = salivating
……do it enough times…do it enough times…
Bell = salivatingBell = salivating
Classical Conditioning TheoryClassical Conditioning Theory
Different types of response, different types of Different types of response, different types of stimulusstimulus– The innate bit:The innate bit:
» Unconditioned stimulus = meatUnconditioned stimulus = meat
» Unconditioned response = salivatingUnconditioned response = salivating
– The learned bitThe learned bit» Conditioned stimulus = bellConditioned stimulus = bell
» Conditioned response = salivating (to bell alone)Conditioned response = salivating (to bell alone)
In pictures:In pictures:
In pictures:In pictures:
In pictures:In pictures:
Classical ConditioningClassical Conditioning
Same pattern of learning found in Same pattern of learning found in – CrabsCrabs– AntsAnts– Ant eaters Ant eaters – CatsCats– PEOPLEPEOPLE
Fundamental type of learning!Fundamental type of learning!
The important parts of The important parts of Classical ConditioningClassical Conditioning
1.1. Acquisition of Conditioned responsesAcquisition of Conditioned responses– CS (bell) and UCS (meat) must be paired CS (bell) and UCS (meat) must be paired
several times. The CS must be presented just several times. The CS must be presented just before the UCS for the association to be before the UCS for the association to be strongest.strongest.
– These pairings are called These pairings are called reinforced trialsreinforced trials, , because they reinforce the connection. Trials because they reinforce the connection. Trials with only the CS are called with only the CS are called unreinforced unreinforced trials. trials.
The important parts of The important parts of Classical ConditioningClassical Conditioning
Measuring the strength of the conditioned Measuring the strength of the conditioned response:response:
1.1. Response Response amplitudeamplitude, or strength, or strength– E.g. amount of salivaE.g. amount of saliva
2.2. Probability of response (when CS is Probability of response (when CS is presented alone)presented alone)
3.3. Response latency (how long a gap is there Response latency (how long a gap is there between the CS and the CR?between the CS and the CR?
The important parts of The important parts of Classical ConditioningClassical Conditioning
The learning Curve:
The important parts of The important parts of Classical ConditioningClassical Conditioning
Second order conditioningSecond order conditioning Bell => salivating.Bell => salivating.
– Fair enough, but…Fair enough, but…
– Black square + bell => salivatingBlack square + bell => salivating– ……– Black square => salivatingBlack square => salivating
The important parts of The important parts of Classical ConditioningClassical Conditioning
2.2. ExtinctionExtinction ‘‘Undoing’ the Undoing’ the
connectionsconnections Important so an Important so an
animal doesn’t get animal doesn’t get locked into locked into behaviours that behaviours that aren’t useful.aren’t useful.
The important parts of The important parts of Classical ConditioningClassical Conditioning
3.3. GeneralisationGeneralisation In the real world, two In the real world, two
stimuli are rarely the stimuli are rarely the samesame
– E.g. voices at different E.g. voices at different intonationsintonations
– footsteps at a different footsteps at a different pacepace
The more different the The more different the stimulus from the original stimulus from the original CS, the weaker the CS, the weaker the responseresponse
The important parts of The important parts of Classical ConditioningClassical Conditioning
4.4. DiscriminationDiscrimination To avoid ‘over-generalisation’To avoid ‘over-generalisation’ It may take many trials for an animal to It may take many trials for an animal to
tell the difference between e.g. a black tell the difference between e.g. a black square and a grey square, but if only the square and a grey square, but if only the black square is reinforced, it will black square is reinforced, it will eventually tell them apart perfectly.eventually tell them apart perfectly.
Extensions of classical Extensions of classical conditioningconditioning
HungerHunger Emotions Emotions
– How we feel about certain peopleHow we feel about certain people» Daddy + Disney world => happinessDaddy + Disney world => happinessHopefully … daddy => happinessHopefully … daddy => happiness
– Phobias and their treatmentPhobias and their treatment Drugs e.g. insulin needlesDrugs e.g. insulin needles Drug addictionDrug addiction
» Sight of needle => body’s compensatory response => toleranceSight of needle => body’s compensatory response => tolerance» Danger of overdose if compensatory reaction is not elicited by a relevant Danger of overdose if compensatory reaction is not elicited by a relevant
CS such as location.CS such as location.» CS, (such as needle) but without the US (the drug) means the CS, (such as needle) but without the US (the drug) means the
compensatory response just carries on. This causes cravings.compensatory response just carries on. This causes cravings.
Important point:Important point:
In classical conditioning, psychologists In classical conditioning, psychologists thought that an animal learns to respond in thought that an animal learns to respond in a reflexive manner to some new stimulus.a reflexive manner to some new stimulus.– There was no thinking involvedThere was no thinking involved– The animal was basically like a machine The animal was basically like a machine – It matched its old response to a new stimulus It matched its old response to a new stimulus
for no good reason.for no good reason.– Learning is a bit ‘stupid’Learning is a bit ‘stupid’
Classical Conditioning QuizClassical Conditioning Quiz
1.1. What is the simplest form of learning called?What is the simplest form of learning called?2.2. Give an example of something humans don’t have to Give an example of something humans don’t have to
learnlearn3.3. Who accidentally discovered classical conditioning?Who accidentally discovered classical conditioning?4.4. In the experiments, was the bell the conditioned stimulus In the experiments, was the bell the conditioned stimulus
or the unconditioned stimulus?or the unconditioned stimulus?5.5. To get the best effect, should the bell come before or To get the best effect, should the bell come before or
after the meat?after the meat?6.6. What is it called when an association ‘wears off’?What is it called when an association ‘wears off’?7.7. What is it called when an animal cal tell the difference What is it called when an animal cal tell the difference
between two different bells?between two different bells?8.8. Why can taking drugs in an unusual place be dangerous?Why can taking drugs in an unusual place be dangerous?
AnswersAnswers
1.1. HabituationHabituation2.2. The rules of speaking their own language, The rules of speaking their own language,
jumping at a loud noise etc.jumping at a loud noise etc.3.3. Ivan Pavlov, Russian PhysiologistIvan Pavlov, Russian Physiologist4.4. The conditioned stimulusThe conditioned stimulus5.5. BeforeBefore6.6. ExtinctionExtinction7.7. DiscriminationDiscrimination8.8. The body doesn’t get the conditioned stimulus The body doesn’t get the conditioned stimulus
(the location) to prepare for the drugs(the location) to prepare for the drugs
Part 3Part 3
Instrumental ConditioningInstrumental Conditioning
What is it?What is it?
When a seal does a When a seal does a trick and gets a fish, it trick and gets a fish, it learns an instrumental learns an instrumental response – the trick is response – the trick is instrumentalinstrumental to to getting the fish…getting the fish…
What is it?What is it?
Different from classical conditioningDifferent from classical conditioning NOTNOT an improvement or replacement of classical an improvement or replacement of classical
conditioning.conditioning.– This time the reinforcement (e.g. food) depends This time the reinforcement (e.g. food) depends
on making the right response. Pavlov’s dogs on making the right response. Pavlov’s dogs could not influence when they were fed by could not influence when they were fed by making a particular response.making a particular response.
– The response the animal makes has to be The response the animal makes has to be deliberately deliberately selectedselected – Pavlov’s dogs had no – Pavlov’s dogs had no choice whether or not they salivated.choice whether or not they salivated.
Instrumental ConditioningInstrumental Conditioning
Thorndike and the Law Thorndike and the Law of Effect (1898)of Effect (1898)
Studied animals’ Studied animals’ behaviour to see if behaviour to see if they showed signs of they showed signs of ‘human’ thinking and ‘human’ thinking and reasoningreasoning
Thorndike’s not-so-clever catsThorndike’s not-so-clever cats
Thorndike’s catsThorndike’s cats
• The wrong responses led to failure, and the right response led to success. The wrong ones had to be ‘stamped out’, the right ones had to be ‘stamped in’
• The cats did learn to make the right response, but it was a slow process. No evidence of thinking!
The law of effectThe law of effect
If a response is followed by a reward, it will If a response is followed by a reward, it will be strengthened.be strengthened.
If a response is not followed by a reward, or If a response is not followed by a reward, or is followed by punishment, it will be is followed by punishment, it will be weakened.weakened.
The strength of a response is adjusted The strength of a response is adjusted according to the response’s according to the response’s consequencesconsequences
Skinner and Skinner and Operant ConditioningOperant Conditioning
B.F. Skinner (1904-1990)B.F. Skinner (1904-1990) Insisted Instrumental conditioning was Insisted Instrumental conditioning was
different from classical conditioning as an different from classical conditioning as an animal’s response is basically animal’s response is basically voluntaryvoluntary
He called instrumental responses He called instrumental responses operantsoperants because they operate on the environment.because they operate on the environment.
Developed better puzzle boxesDeveloped better puzzle boxes
The main features of The main features of Instrumental ConditioningInstrumental Conditioning
1.1. ReinforcementReinforcement– In classical conditioning, reinforcement In classical conditioning, reinforcement
strengthens a response. strengthens a response. – In Instrumental conditioning, this is done by In Instrumental conditioning, this is done by
presenting some stimulus presenting some stimulus AFTERAFTER the correct the correct response has been maderesponse has been made
– Two ways of doing this…Two ways of doing this…
1(a) Presentation of 1(a) Presentation of ‘Appetitive’ Stimulus‘Appetitive’ Stimulus
Correct response => Nice juicy fish
1(b) Removal of aversive 1(b) Removal of aversive stimulusstimulus
Evil Psychologist Gigantic rat
Shock + correct response = no more shock
ReinforcementReinforcement
As in CC, the more reinforcements, the As in CC, the more reinforcements, the more likely the response will be to occurmore likely the response will be to occur
As in CC, the response will suffer As in CC, the response will suffer extinction if it is not reinforced.extinction if it is not reinforced.
2. Generalisation and 2. Generalisation and DiscriminationDiscrimination
The responses might be voluntary, but The responses might be voluntary, but outside stimuli can affect it, e.g. outside stimuli can affect it, e.g. discrimination.discrimination.
Red light vs. Green light
Pigeon poo
Generalisation and Generalisation and DiscriminationDiscrimination
The responses might be voluntary, but The responses might be voluntary, but outside stimuli can affect it, e.g. outside stimuli can affect it, e.g. discrimination.discrimination.
The difference is, in instrumental The difference is, in instrumental conditioning, the stimulus tells the animal conditioning, the stimulus tells the animal what to dowhat to do, not just , not just what’s going to happenwhat’s going to happen..
Generalisation and Generalisation and DiscriminationDiscrimination
Generalisation is just the same: if the Generalisation is just the same: if the stimulus is quite similar to the training stimulus is quite similar to the training stimulus, the animal is quite likely to make stimulus, the animal is quite likely to make the same responsethe same response
Training stimulus
Quite likely to respond
Less likely to respond
3. Shaping3. Shaping
The way to train animals…The way to train animals…
Water-skiing squirrel
Kung fu hamster
?
How?How? The method of The method of successive approximationssuccessive approximations
– Little bit at a timeLittle bit at a time– Takes a looooonnnnngggggg timeTakes a looooonnnnngggggg time– Requires perfect timingRequires perfect timing– Someone trained a pig to turn on a radio, eat Someone trained a pig to turn on a radio, eat
breakfast at a kitchen table, put dirty clothes in breakfast at a kitchen table, put dirty clothes in a wash basket, hoover the floor, select a certain a wash basket, hoover the floor, select a certain type of pig food from lots of alternativestype of pig food from lots of alternatives
AndAnd
Dogs playing the pianoDogs playing the piano Pigeons playing ping pong…Pigeons playing ping pong… Etc. etc. etc…Etc. etc. etc…
4. Reinforcers4. Reinforcers
Many different types (food, money, less Many different types (food, money, less pain etc.)pain etc.)
Not just things that meet immediate needs.Not just things that meet immediate needs.
ReinforcersReinforcers
It seems a reinforcer is anything that It seems a reinforcer is anything that motivates behaviourmotivates behaviour
Reinforcers will motivate behaviour Reinforcers will motivate behaviour differently depending on what other differently depending on what other rewards are available. rewards are available.
» If a kid gets no pocket money, £5 is a good If a kid gets no pocket money, £5 is a good reinforcerreinforcer
» If they get £30 pocket money, suddenly £5 is not so If they get £30 pocket money, suddenly £5 is not so interesting. interesting.
ReinforcersReinforcers
Some behaviours have their own reinforcers Some behaviours have their own reinforcers ‘built-in’ (‘built-in’ (intrinsic motivationintrinsic motivation))– ArtArt– SportSport– SexSex
5. Schedules of Reinforcement5. Schedules of Reinforcement
In the lab and in real life, reinforcement is In the lab and in real life, reinforcement is usually usually partialpartial
Reinforcement usually comes in patterns or Reinforcement usually comes in patterns or after a delay.after a delay.– The rules for reinforcing are known as The rules for reinforcing are known as
schedules of reinforcementschedules of reinforcement
5(a). Ratio Schedules5(a). Ratio Schedules
Fixed ratio: Fixed ratio: – E.g. 10 responses for each rewardE.g. 10 responses for each reward
Variable ratio:Variable ratio:– Roughly Roughly 10 responses for a reward.10 responses for a reward.– The uncertainty produces a high response rateThe uncertainty produces a high response rate– This is how gambling machines make moneyThis is how gambling machines make money
5(b). Interval Schedules5(b). Interval Schedules
Reinforcement only comes at a fixed time Reinforcement only comes at a fixed time since the last one.since the last one.– Responses slow down after a reward, and then Responses slow down after a reward, and then
start to pick up near the end of the 5 minute start to pick up near the end of the 5 minute period. So you can train an animal to respond period. So you can train an animal to respond every 5 mins.every 5 mins.
– Example: you only check for mail in the Example: you only check for mail in the morning, because that’s when the effort is morning, because that’s when the effort is rewarded.rewarded.
6. Punishment6. Punishment
The opposite of rewardThe opposite of reward Timing is crucialTiming is crucial Escape and avoidanceEscape and avoidance The point is to The point is to
weaken weaken responses.responses.
In the real worldIn the real world
Table mannersTable manners PrisonsPrisons Token economiesToken economies Mental health treatmentsMental health treatments
Instrumental ConditioningInstrumental Conditioning
1.1. Why do we use the term ‘instrumental’?Why do we use the term ‘instrumental’?
2.2. Why do we use the term ‘operant’Why do we use the term ‘operant’
3.3. Are instrumental responses voluntary or reflexive?Are instrumental responses voluntary or reflexive?
4.4. Does the reinforcer come before or after the response?Does the reinforcer come before or after the response?
5.5. Can an instrumental response become extinct?Can an instrumental response become extinct?
6.6. What is the correct term for the way you train animals?What is the correct term for the way you train animals?
7.7. How do we define a reinforcer?How do we define a reinforcer?
8.8. Give an example of intrinsic motivationGive an example of intrinsic motivation
9.9. What reinforcement schedule do gambling machines What reinforcement schedule do gambling machines use?use?
AnswersAnswers
1.1. The response is The response is instrumentalinstrumental to getting some reward to getting some reward
2.2. The response is a way of operating on the environmentThe response is a way of operating on the environment
3.3. VoluntaryVoluntary
4.4. AfterAfter
5.5. YesYes
6.6. Shaping, or Successive ApproximationsShaping, or Successive Approximations
7.7. Anything that motivates behaviourAnything that motivates behaviour
8.8. DrinkingDrinking
9.9. Variable ratioVariable ratio
Part 4Part 4
Cognitive LearningCognitive Learning
Cognitive learningCognitive learning
‘‘Cognitive’ = information processingCognitive’ = information processing Psychology today, in this university, is Psychology today, in this university, is
cognitive psychology.cognitive psychology. More concerned with what goes on in the More concerned with what goes on in the
mind than simple reflex actions.mind than simple reflex actions. Earlier psychologists treated people almost like Earlier psychologists treated people almost like
machines – they believed you couldn’t study machines – they believed you couldn’t study mental processesmental processes because you couldn’t see them. because you couldn’t see them.
Cognitive learningCognitive learning
Learning cannot just be a change in Learning cannot just be a change in behaviour – it involves new behaviour – it involves new KNOWLEDGEKNOWLEDGE
The proof ?The proof ?– Carry a rat round a maze and it will learn the Carry a rat round a maze and it will learn the
layout of the mazelayout of the maze» Without doing anything!Without doing anything!
Cognitive views of classical conditioning.Cognitive views of classical conditioning.
The conditioned stimulus (bell) doesn’t become a The conditioned stimulus (bell) doesn’t become a substitute for the unconditioned stimulus (meat). substitute for the unconditioned stimulus (meat). Often the CR and the UR can be different, as in Often the CR and the UR can be different, as in the drug example.the drug example.
– The dog learns that the bell means the food is The dog learns that the bell means the food is on it’s way, so it responds appropriately.on it’s way, so it responds appropriately.
– It learns the It learns the relationshiprelationship between the two between the two stimuli.stimuli.
Cognitive views of classical conditioning.Cognitive views of classical conditioning.
– It learns the It learns the relationshiprelationship between the two between the two stimuli.stimuli.
Ah, I see…Ah, I see…
Why does conditioning work?Why does conditioning work?
Pavlov thought it was just because the the bell and Pavlov thought it was just because the the bell and the meat occurred close to each other.the meat occurred close to each other.
But we now understand in terms of signals: the But we now understand in terms of signals: the dog hears the bell and knows it signals dog hears the bell and knows it signals approaching food.approaching food.
Presenting the stimuli the other way around is Presenting the stimuli the other way around is poor at producing the response, and having them poor at producing the response, and having them at the same time is pretty useless…at the same time is pretty useless…
Why does conditioning work?Why does conditioning work?
Think about road signsThink about road signs Signs have to come Signs have to come beforebefore the thing they the thing they
warn us about. It’s no good having them at warn us about. It’s no good having them at the same time, or after the thing itself!the same time, or after the thing itself!
Why does conditioning work?Why does conditioning work?
The important thing is that the bell The important thing is that the bell predictspredicts the food, not just that they happen closely the food, not just that they happen closely together in time.together in time.
Storm clouds – we look at them and predict Storm clouds – we look at them and predict storms. We don’t immediately dive for storms. We don’t immediately dive for cover.cover.– Storm clouds are not a substitute for an actual Storm clouds are not a substitute for an actual
storm; they serve as a signalstorm; they serve as a signal
The role of surpriseThe role of surprise
The CS – US pairing has to be NEW, it has The CS – US pairing has to be NEW, it has to make an animal sit up and pay attention. to make an animal sit up and pay attention. – If the dog was already expecting the food, it If the dog was already expecting the food, it
would ignore the bell because it doesn’t tell it would ignore the bell because it doesn’t tell it anything new.anything new.
» Animals tend not to make new connections when Animals tend not to make new connections when the old ones are good enoughthe old ones are good enough
» Temperature exampleTemperature example
Cognitive view of Instrumental ConditioningCognitive view of Instrumental Conditioning
An animal doesn’t just learn to perform a response – it An animal doesn’t just learn to perform a response – it learns the relationship between a response and it’s learns the relationship between a response and it’s outcomeoutcome– This is called an This is called an act-outcome representationact-outcome representation
Ah, I see…**Deep thoughts**
Latent learningLatent learning
Rats know their way around a maze, but Rats know their way around a maze, but only bother demonstrating this when it gets only bother demonstrating this when it gets them some food!them some food!– So what they have learned is So what they have learned is knowledgeknowledge, not , not
just just behaviourbehaviour
Cognitive view of Instrumental ConditioningCognitive view of Instrumental Conditioning
Animals can learn which action leads to Animals can learn which action leads to which reward.which reward.– Press lever => foodPress lever => food– Pull chain => sweet waterPull chain => sweet water– ……– Poison the water…Poison the water…– No more chain pullingNo more chain pulling
» Points to quite complex knowledgePoints to quite complex knowledge
Cognitive view of Instrumental ConditioningCognitive view of Instrumental Conditioning
Animals (and humans) know when there is Animals (and humans) know when there is a relationship between two events and when a relationship between two events and when there isn’tthere isn’t– And they seem to prefer it when there is one!And they seem to prefer it when there is one!
Cognitive view of Instrumental ConditioningCognitive view of Instrumental Conditioning
Learned Learned helplessnesshelplessness– When we fail to When we fail to
learn the relationshiplearn the relationship– How to make a dog How to make a dog
depresseddepressed
Link to depression in humansLink to depression in humans
People may be in situations when they People may be in situations when they really were helplessreally were helpless– Losing jobLosing job– Losing a court caseLosing a court case– Someone dyingSomeone dying– IllnessIllness
» They end up learning that there is no relationship They end up learning that there is no relationship between what they do, and the things that happen to between what they do, and the things that happen to themthem
SummarySummary
In the cognitive view, animals and people actually In the cognitive view, animals and people actually gain knowledge when they learn.gain knowledge when they learn.
They don’t just learn new reflex actions, or new They don’t just learn new reflex actions, or new complex behaviours. They don’t just act like complex behaviours. They don’t just act like machines.machines.
Learning about relationships is highly useful, but Learning about relationships is highly useful, but can also be damaging, as in the case of depression.can also be damaging, as in the case of depression.
Psychologists only work with dogs, pigeons and Psychologists only work with dogs, pigeons and rats because they’re too scared to work with tigersrats because they’re too scared to work with tigers
The End.The End.
Thanks for listening!