Extinction of Conditioned Behavior

Effects of Extinction the rate of responding decreases response variability increases experiment by Neuringer, Kornell, & Olufs (2001)

Extinction does not eliminate the original learning

Spontaneous Recovery Renewal Reinstatement US-devaluation

What is learned during extinction?

The non-reinforcement of a response in the presence of a specific stimulus produces an inhibitory S-R association that suppresses that response whenever S is present

Partial Reinforcement Extinction Effect

Extinction is slower if a partial reinforcement schedule, rather than CRF, was in effect before extinction

Possible explanations: Discrimination hypothesis Frustration hypothesis Sequential memory hypothesis

Stimulus Control of Behavior

Stimulus control Stimulus discrimination Generalization Gradient

Stimulus and Response Factors in Stimulus Control

sensory capacity relative ease of conditioning various stimuli

• overshadowing type of reinforcement

• certain types of stimuli are more likely to gain control over the instrumental response with positivereinforcement than with negative reinforcement

type of instrumental response

• experiment by Foree & LoLordo (1973)

• experiment by Dobrzecka, Szwejkowska, &Konorski (1966)

Foree & LoLordo (1973)2 groups of pigeons trained to press a foot treadle in the presence of a LT compound stimulus

For 1 group of pigeons, reinforcement was foodFor the other group of pigeons, reinforcement was theavoidance of shock

5

10

15

Meanresponses

Tone

Shock-avoidancereinforcement

Foodreinforcement

Light Tone + Light

Foree & LoLordo (1973)

Pigeons trained with food reinforcement responded muchmore when tested with the light alone than when tested with the tone alone

Pigeons trained with shock-avoidance reinforcement responded much more when tested with the tone alone than when tested with the light alone

Selective association?

Dobrzecka, Szwejkowska, & Konorski (1966)

Buzzer

Raise left leg

Metronome

Raise right leg

Buzzer

Raise leg(go)

Metronome

Do notraise leg(no go)

Group 1(right/left discrimination)

Group 2(go/no go discrimination)

Training

TestingBuzzer

Raised left leg

Metronome

Raised right leg

Buzzer

Did notraise leg

Metronome

Raised leg

Dobrzecka, Szwejkowska, & Konorski (1966)

Two components to the auditory cues: - quality - location

Dogs trained on the left/right discrimination respondedmainly on the basis of the location of the auditory cues

Dogs trained on the go/no go discrimination respondedmainly on the basis of the quality of the auditory cues

Selective association?

Stimulus elements versus configural cues

So far, we’ve assumed that subjects treat stimulus elements as distinct and separate features of the environment (i.e., quality and location of sounds)

However, subjects could treat a complex stimulus as an integral whole that is not divided into parts or elements

Stimulus-element approach

Configural-cue approach

Learning Factors in Stimulus Control

Effects of Training on Generalization and Discrimination

- Intradimensional Training: * reinforced in the presence of S+ and not reinforced in the presence of

S-

Nondifferential Training :

- S+ always present. Differential (or Discrimination) Training: - Presence/Absence Training:

* reinforced in the presence of S+, not in its absence.

More peaked gradient

Flat gradient

Most peaked gradient

Less generalization; more discrimination

Least generalization; most discrimination

Non-Differential

Presence/Absence

Intradimensional

What is learned in discrimination training?

respond during the S+ do not respond during the S- or both (i.e., Spence)

Contextual Cues and Conditional Relations

Control of behavior by discrete stimuli (i.e., L or T)- discrete cues are those which are present for a

brief time and have a clear beginning and end

Contextual cues are the visual, auditory, and olfactorycues of the room or place where the discrete stimuli are presented

Contextual cues can provide an additional source ofcontrol of learned behavior

Conditioned Place Preference

Control by Contextual Cues

barrier

Rats given drugand put in Black side

Rats given salineand put in White side

On test:Barrierremoved

Rats placed in the center

On test:

Rats spend more time in drug paired context

Contextual cues can control behavior if they serve as a signal for a US or reinforcer

- experiment by Akins (1998) described on p. 243

Do contextual cues control behavior when they serve asbackground rather than explicit signal for reinforcer?

Experiment by Thomas, McKelvie, & Mah (1985)

Trained pigeons on a line-orientation discrimination in context 1

Context 1:S+ = vertical line (90º)S- = horizontal line (0º)

After the discrimination was learned, the context was changed andthe discrimination training contingencies were reversed

Context 2:S+ = horizontal line (0º)S- = vertical line (90º)

After learning the second problem, pigeons given generalizationtests in which lines of orientation between 0º and 90º were presentedin the two contexts

10

20

30

0 15 30 45 60 75 90

Responses

Line angle(degrees)

S+ S-Context 1 90 0Context 2 0 90

Results:

Thomas et al (1985) showed that: contextual cues can come to control instrumentalbehavior contextual control can occur without one context being more strongly associated with reinforcementthan the other direct associations between context and reinforcercannot explain the pattern of results each context associated with a different S+/S-contingency

Conditional Relations

Control by Conditional Relations

Binary relations:

S-R S-O R-O

In some cases the nature of a binary relation isdetermined by a third event = modulator

S-(R-O)

This is termed a conditional relation

S+/S- discriminations are conditional relations

Conditional Control in Pavlovian Conditioning

Rescorla, Durlach, & Grau (1985)

Reinforced trialsNoiseKey light - food

Nonreinforced trialsNo NoiseKey light – no food

20

40

60

80

Trials

Responsesper min

Element Compound

Conditional Control in Pavlovian Conditioning

In instrumental conditioning, modulators (i.e., S+ and S-)are called discriminative stimuli

In Pavlovian conditioning, modulators are called facilitators or occasion setters

A positive occasion setter signals that a CS-US contingency is in effect

A negative occasion setter signals that a CS-US contingency is not in effect

Positive Occasion Setting (Facilitation)

Training

If Occasion Setter is present:then CS USIf Occasion Setter is absent:

then CS no USResult

Thus CS CR only when Occasion Setter is present

Distinction between excitation and modulation

1. CS signals US; BUT occasion setter signals the relationship (if - then) between CS and US.

2. CS elicits CR; BUT occasion setter does not elicit a CR

3. CS does not facilitate responding to a new CS+; BUT an occasion setter can.

4. CS can support second order conditioning, BUT occasion setter does not.

5. CS presented alone results in extinction of CR; BUT occasion setter presented alone, no extinction of CS CR

Training

If Occasion Setter is present:then CS no USIf Occasion Setter is absent:then CS US

ResultThus CS CR only when Occasion Setter is absent – is this familiar?

Negative Occasion Setting

Conditioned Inhibition may be opposite of facilitation, not excitation