Operant or Instrumental Operant or Instrumental ConditioningConditioning

Psychology 3306Psychology 3306

IntroductionIntroduction

Thorndike and his Thorndike and his puzzle boxespuzzle boxes

Guthrie and HortonGuthrie and Horton Superstitious Superstitious

BehaviourBehaviour InterimInterim TerminalTerminal AdjunctiveAdjunctive Not exactly Not exactly

superstitious or superstitious or randomrandom

ShapingShaping

Successive approximationsSuccessive approximationsGet closer and closer to behaviourGet closer and closer to behaviour

Secondary reinforcersSecondary reinforcersFeeder click for exampleFeeder click for example

Behaviour modificationBehaviour modification

Freddie!Freddie!

Coined the phrase Coined the phrase ‘Operant conditioning’‘Operant conditioning’

The animal operates on The animal operates on the environmentthe environment

Unlike ‘respondent Unlike ‘respondent conditioning’ (Pavlovian)conditioning’ (Pavlovian)

Pioneered the use of free Pioneered the use of free operantsoperants

Pioneered the use of Pioneered the use of respone rate respone rate

The Skinner BoxThe Skinner Box

Basically this allowed Basically this allowed the researcher to walk the researcher to walk awayaway

Allowed for a Allowed for a dependent variable dependent variable that could be easily that could be easily measured and measured and compared across compared across species toospecies too

Criticisms of the Skinner boxCriticisms of the Skinner box

Is it artificial?Is it artificial?Well duh…Well duh…ButButMany species can be testedMany species can be testedReal world applicationsReal world applicationsTherapyTherapyWho cares?Who cares?

Key concepts and termsKey concepts and terms

Discriminative stimulusDiscriminative stimulusThree term contingencyThree term contingencyAcquisitionAcquisitionExtinctionExtinctionSpontaneous recoverySpontaneous recoveryGeneralizationGeneralizationConditioned reinforcementConditioned reinforcementResponse chainsResponse chains

ConstraintsConstraints

Instinctive drift and the BrelandsInstinctive drift and the BrelandsAutoshaping (Brown and Jenkins, 1968)Autoshaping (Brown and Jenkins, 1968)

Superstitious behaviour?Superstitious behaviour?Form of response depends on reinforcer Form of response depends on reinforcer

(Jenkins and Moore, 1973)(Jenkins and Moore, 1973)Wasserman’s chicks (1973)Wasserman’s chicks (1973)Timberlake’s behaviour systems approachTimberlake’s behaviour systems approach

Schedules of ReinforcementSchedules of Reinforcement

You could give a reinforcement after each You could give a reinforcement after each behaviour you are interested inbehaviour you are interested in

This is called CRF or Continuous This is called CRF or Continuous reinforcementreinforcement

However this is rarely usedHowever this is rarely usedDoes not maintain behaviour very wellDoes not maintain behaviour very well

Schedules of ReinforcementSchedules of Reinforcement

Fixed IntervalFixed Interval First response after a First response after a

given interval is given interval is rewardedrewarded

FI ScallopFI Scallop

Variable IntervalVariable Interval Like FI but varies with Like FI but varies with

a given averagea given average Scallop disappearsScallop disappears

Schedules of ReinforcementSchedules of Reinforcement

Fixed RatioFixed Ratio Reinforcement is given Reinforcement is given

after a given number after a given number of responsesof responses

A little less smoothA little less smooth

Variable RatioVariable Ratio After a varying number After a varying number

of responsesof responses

Schedules and their propertiesSchedules and their properties

Variable schedules are more robustVariable schedules are more robustPREE, Partial reinforcement extinction PREE, Partial reinforcement extinction

effecteffectHarder to extinguish responding on VI, FI, VR Harder to extinguish responding on VI, FI, VR

and FR than on CRFand FR than on CRFDRL, Differential reinforcement for low DRL, Differential reinforcement for low

rates of respondingrates of respondingDRH, High ratesDRH, High rates

Schedule this….Schedule this….

Concurrent and chained schedulesConcurrent and chained schedulesBehaviour follows the schedule in effect at Behaviour follows the schedule in effect at

the timethe timeAllowed people to determine that the post Allowed people to determine that the post

reinforcement pause in FR schedules is reinforcement pause in FR schedules is due to the present schedule and not the due to the present schedule and not the previous oneprevious one

ApplicationsApplications

Work with autistic kidsWork with autistic kidsPromptsPromptsFadingFadingSecondary reinforcersSecondary reinforcers

Token economiesToken economies I/O applicationsI/O applicationsBehaviour therapyBehaviour therapy

These ideas are nothing new…These ideas are nothing new…

Most folks are unaware of schedules and Most folks are unaware of schedules and contingenciescontingencies

Systematic application thereofSystematic application thereofWho cares?Who cares?

Punishment and avoidancePunishment and avoidance

Positive Positive

reinforcementreinforcement

punishmentpunishment

Negative Negative

reinforcementreinforcement

OmissionOmission

Behavior

increases decreases

Stimulus Presented

StimulusRemoved oromitted

AvoidanceAvoidance

Shuttle boxShuttle boxGo from escape to avoidanceGo from escape to avoidanceAvoidance paradoxAvoidance paradoxTwo factor theoryTwo factor theoryAvoid by escaping CSAvoid by escaping CSAnimals will avoid a CS that predicts Animals will avoid a CS that predicts

shock in another contextshock in another context

But….But….

Does the CS induce fear?Does the CS induce fear?Equivocal at bestEquivocal at bestMaybe avoidance itself is reinforcingMaybe avoidance itself is reinforcingThis is the one factor theoryThis is the one factor theoryThe Sidman test shows that this is true, The Sidman test shows that this is true,

avoidance itself is reinforcingavoidance itself is reinforcingBut, temporal conditioningBut, temporal conditioning

Cognitive theoriesCognitive theories

Selligman and JohnstonSelligman and JohnstonExpectationsExpectationsAnimal expects:Animal expects:

No shock if it respondsNo shock if it respondsShock if it does not respondShock if it does not respond

This explains the slow extinctionThis explains the slow extinctionShock avoidance response blocking, Shock avoidance response blocking,

remove the ability to escape, you get remove the ability to escape, you get extinctionextinction

More on avoidanceMore on avoidance

Bob Bolle’s idea about SSDRsBob Bolle’s idea about SSDRsLearned helplessnessLearned helplessness Is it depression?Is it depression?

Suggestive, but not quite I don’t thinkSuggestive, but not quite I don’t think

PunishmentPunishment

Opposite of reinforcement?Opposite of reinforcement?SortaSortaBut, to be effective it must be:But, to be effective it must be:

Introduced at full intensityIntroduced at full intensityGiven immediatelyGiven immediatelyAfter every behaviourAfter every behaviour

Motivational effectsMotivational effectsOther contingencies and behavioursOther contingencies and behaviours

Bad boys bad boys, watcha gonna Bad boys bad boys, watcha gonna do?do?

Maybe a punisher is an SD?Maybe a punisher is an SD?All that said punishment CAN control All that said punishment CAN control

behaviourbehaviourSo, what’s the down side?So, what’s the down side?

punishmentpunishment

Fear and anger are bad for learningFear and anger are bad for learningGeneral suppressionGeneral suppressionConstant monitoring neededConstant monitoring neededAvoidanceAvoidanceReluctance to use itReluctance to use itBad consequencesBad consequences It is just plain meanIt is just plain mean

OmissionOmission

The avoidance of punishmentThe avoidance of punishmentEasily learnedEasily learnedWith all this stuff on punishment, With all this stuff on punishment,

remember morality and data are two remember morality and data are two different thingsdifferent things