Post on 17-Jan-2018
description
Extinction of Conditioned BehaviorChapter 9
• Effects of Extinction• Extinction and Original Learning• What is learned during Extinction
Effects of Extinction
Extinction involves omitting the US or reinforcer In Pavlovian conditioning, this means repeated presentations of the CS alone In operant conditioning, this means not giving the reinforcer after the response
Two main effects of extinction procedures on behavior the rate of responding decreases response variability increases
Extinction Increases Variability in Behavior
Three levers present and Var group had to vary sequence
Neuringer, Kornell, & Olufs (2001)
All groups increased variability during extinction
Extinction and Original Learning
• Spontaneous Recovery• Renewal• Reinstatement• US-devaluation
Spontaneous Recovery
Rescorla (1996)Rats trained to make 2 different responses for food
Lever-press – foodNose-poke – food
At end of training phase, rats were making about 17 resp/minAfter acquisition, the rats were given 2 20-min extinction sessionswith each responseFor one of the responses, the extinction phase occurred shortlybefore the recovery test (R-No rest)For the other response, the extinction phase occurred 7 daysbefore the recovery test (R-Rest)
At end of extinction phase, rats were making about 2 resp/min
Spontaneous Recovery
Shows importance of passage of time
RenewalBouton & King (1983)Rats were first trained to bar-press for food
Then, Context AT – Shock
Conditioning resulted in Tsuppressing BP response
During extinction, the rats were divided into 3 groupsGroup A: CS alone in Context AGroup B: CS alone in Context BGroup NE: did not receive extinction
After extinction, all rats received test trials in Context A
Renewal
Reinstatement
Could be due to context conditioning
In initial studies, the excitatory conditioning, extinction and subsequent US exposures were all given in same context
Recent research indicates that context is important but notbecause it permits summation
The role of context appears to be to disambiguate the significanceof a stimulus that has a mixed history of conditioning and excitation
Context has little effect on stimuli that have a history of onlyconditioning
Reinstatement
Bouton (1984)Conditioned suppression with rats
Phase 1CS – weak shockCS – weak shockCS – strong shockCS – strong shock
Phase 2No treatmentNo treatmentExtinctionExtinction
ReinstatementShock-sameShock-differentShock-sameShock-different
TestCSCSCSCS
Reinstatement was introduced when the level of responding was similar in all subjects
However, some rats got to that point by receiving only conditioning,whereas for others the CS was both conditioned and extinguished
Reinstatement
ReinstatementFor rats that did not receive extinction, it did not matter whetherthe reinstatement shocks were given in the same context or adifferent context
- shows that contextual conditioning did not summate with thesuppression elicited by the CS
For rats that did receive extinction, reinstatement shocks given in the same context produced more suppression than shocks given ina different context
- shows that contextual conditioning facilitates the reinstatementeffect
According to Bouton, reinstatement US presentations given in thetest context serve to restore the excitatory properties of the contextual cues and enable those cues to be more effective inreactivating the memory of excitatory conditioning of the CS
US-Devaluation
Rescorla (1993)Used devaluation procedure in operant experiment to show thatthe original R-O association was not abolished by extinction
Phase 1R1 – O1
R2 – O2
R3 – O1
R4 – O2
Phase 2R1 extinguishedR2 extinguished
US-devaluation
O1 – LiCl
TestR1 vs R2
andR3 vs R4
The extinguished responses were re-trained with a third reinforcerbefore the devaluation procedure to make sure there would be some responding during the final test
Effect of Devaluation - Operant
Shows that extinction does not eliminate the R-O association
What is learned in extinction?
The experiments on spontaneous recovery, renewal, reinstatementand US-devaluation all suggest that extinction does not abolish theoriginal learning
So why does extinction produce a decrease in responding?
Extinction may produce an inhibitory S-R association
The non-reinforcement of a response in the presence of a specificstimulus produces an inhibitory S-R association that suppressesthat response whenever S is present
Inhibitory S-R Association
Paradoxical Reward Effects
• Overtraining Extinction Effect• Magnitude of Reinforcement Extinction Effect• Partial Reinforcement Extinction Effect
Partial Reinforcement Extinction Effect
In addition to behavioral effects of extinction, there are also emotional effects (i.e. frustration response)
The most important variable that determines the magnitude of both the behavioral and emotional effects of an extinction procedure is the schedule of reinforcement that was in effect before extinction was introduced
The dominant schedule characteristic that determines extinction effects is whether the response was reinforced every time it occurred (CRF) or only some of the time it occurred (partial or intermittent reinforcement)
The general finding is that extinction is slower, and involves fewer frustration reactions, if a partial reinforcement schedule, rather than CRF, was in effect before extinction — PREE
The PREE was previously thought to be counterintuitive- if a reward strengthens a response, then CRF should
produce a stronger response and slower extinction
Partial Reinforcement Extinction Effect
Mechanisms of the Partial Reinforcement Extinction Effect
• Discrimination hypothesis• Frustration hypothesis• Sequential memory hypothesis
Discrimination Hypothesis
• introduction of extinction is easier to detect after CRF than partial reinforcement
• with CRF, easier to notice when reinforcementis absent during extinction
• with PRF, not as easy because reinforcement was sometimes absent during training
2 groups: Phase 1 Phase 2Group 1: CRF CRF ExtinctionGroup 2: partial CRF Extinction
Discrimination Hypothesis
While this idea makes intuitive sense, it was discarded a long time ago.Jenkins (1962)
Extinction procedure should be equally discriminable in the 2 groups
However, group 2 was slower to extinguish
These results suggest that the PREE is not just a discrimination problem but that subjects actually learned something during partial reinforcement that affects the rate of extinction
Frustration Theory
SF: R SAppetitive With F = frustration
Early in training, after nonreward, the organism is frustrated and expects not to be rewarded on the next trial.
But on some trials following nonreward, the organism is rewarded
Thus the stimuli associated with nonreward (frustration) become associated with reward, and the organism learns to respond in their presence – they become SDs for responding.
SF is present during extinction, thus the organism makes many responses.
SN: R SAppetitive With N = memory of nonreward
Thus the animal learns to make the response when the memory of nonreward, including sequences of nonreward, is present.
Sequential Theory
Early in training, when the organism is rewarded on a trial after one or more nonreward trials, the organism remembers not being rewarded on those previous trials. That memory then becomes associated with reward on the present trial.
During extinction, the stimulus conditions are always those of non-reward.
Animals that were on PR schedules should make many responses in extinction. They have learned to respond in the presence of stimuli associated with nonreward (either frustration or memory of nonreward).
In contrast, animals that were on CRF, should make few responses during extinction. They have never experienced nonreward, and never learned to respond in its presence. They quickly become frustrated and give up responding.