Chapter Outline Klein

8/6/2019 Chapter Outline Klein

1/32

ChapterOutline

I. A DEFINITIONOF LEARNINGLearningissaid to occur when experience causesa relatively permanent change inbehavior. Changesin behavior that are caused by temporary states (e.g.,drug states or

fatigue), maturation, or innate (instinctive) response tendenciesare not considered to belearned. Learning allowsanimals to behave in flexible ways,depending upon changesinenvironment and experience.

II. HISTORICAL ORIGINSOF BEHAVIOR THEORYA. Functionalism: Many early psychologists (late 1800'sand early 1900's) were less

concerned with how learning influences behavior than with how reflexesand"instinctive" behaviors promote adaptive responses that keep organismsaliveand healthy.Functionalismultimately faded from psychology because (1) theleaders could not agree on how to viewinstincts, nor on how many instinctsexist,and (2) this confusion prevented true scientific progressin the study ofbehavior.

B. Behaviorism: With the failures of functionalism andinstinct theory, psychologybegan to focus on learning. Beginning in approximately the 1920's,a

"behaviorist" perspective emphasized how behavior is greatly determined byexperience.

1. BehaviorismBehaviorismwasan early school of psychology, founded on theassumption that the best way to understand most human (andnonhuman) behavior isin the careful analysis of how experience altersbehavior.

2. AssociationismFollowing some of the suggestions by Aristotle, later philosophers(mostly British,in the 1700's) speculated that the human mind contains

knowledge in the form ofassociations.That is, one fact or idea or imageis linked with other facts or ideas or images. Locke argued that allhuman knowledge accrues from experience, firstfromsimpleideas(sensory impressions) and then to morecomplexideas(associations).Hume suggested that associations may be baseduponresemblance(similarity),contiguityin time (temporal) or in place(spatial), or the perception ofcause and effect.

3. ThorndikeOne of the first scientific analyses ofassociations was by Thorndike whostudiedanimalssuch as catsin "puzzle boxes." Hungry cats repeatedlylockedin a box became more efficient in making a response thatpermitted escape andaccess to a foodreward.In hisview, behaviorchanged because the satisfying rewardstrengthenedastimulus-response (S-R) association.Hislaw of effectsuggests that any responsethat leads to asatisfying state (reward) will strengthen the connectionbetween the response and the stimuli that immediately precede theresponse.Hislaw of exercisestates that repeated use of the associationcan also strengthen the S-R connection,and that not using it will weakenthe connection.Thelaw of readinesswas less clearly described, butrefers to the suggestion that associations can be acquired or displayedonly when there is underlying motivation for learning or responding.Hisprinciple ofassociativeshiftingdescribes how a previously acquired


2/32

response (associated with one stimulus) can be transferred to anotherstimulus.

4. PavlovAlthough Thorndike believedS-R associationsdeveloped through trial-

and-error learning,Pavlov outlinedspecific "rules" by which associationsdevelop intoconditionedreflexes.Unconditionedresponses(URs) areunlearned (innate) reactions to specificunconditioned stimuli(UCSs).Pavlovshowed that animals can learn to makeaconditionedresponse(CR) to a previously neutralconditionedstimulus(CS),ifduring learning the CS occursin contiguity with theUCS.Once the CSis capable of eliciting the CR,animals will respond withasimilar CR to stimuli that are similar to the original CS,a phenomenoncalledgeneralization.Ifa previously trained CSis presented repeatedlywithout the UCS,extinctionof responding occurs. Another "rule" ofconditioning known ascounterconditioningisdemonstrated by pairing apreviously trained CS with a UCS that elicitsa reflex that isantagonisticto the originally trained CR.Pavlovian, or "classical," conditioningcontinues to have a profoundimpact on psychology,asdoesThorndike's

work that generally represents "instrumental conditioning."

5. WatsonWhat came to be known as "behaviorism" receivedits greatest impetusfrom Watson, whose work wasinformed by Pavlov but also Bechterev.Watson provided evidence for CS-UCSassociative learning,including theconditioning of emotional responsesin humans.One of his most famousstudies was of Little Albert,an infant who developeda fear ofa white rat(CS) by itsassociation with a loud noise (UCS). Jones later showed thatan acquired fear response can be counterconditioned,and thisdemonstration paved the way for a learning-based therapy for phobias.

III. THE ETHICSOF CONDUCTING RESEARCHWatson's early research with Little Albert raises ethical questions for psychology. Formany years, modern psychological research with humansand nonhumans may beconducted only by adhering to very specific guidelines.

A. Conducting Research With Humans: The American Psychological Association(APA) and other organizations have publishedsuggestions for how to conductethically responsible research.Important aspects of ethical researchincludeinformedconsentbefore the research begins, minimization of risksandstressduring the study,anddebriefingafter the research is completed.TheAPA's guidelinesare foundin a book, Ethical Principlesin the Conduct ofResearch with Human Participants.

B. The Use ofNonhuman Animalsin Research: Nonhuman animalsare usedinlearning research more than humans, partly because it is often more convenientto reach confident conclusionsabout associative conditioning when studyinganimalssuch as birds, rodents,and fish than it is when studying humans. Also,some important research cannot ethically be done with humans,so nonhumansprovide a more acceptable way to conduct such studies.Guidelines for ethicalresearch with nonhuman animalsare published by local,state,and federalgovernments.


3/32

ChapterOutline

I. THE INSTINCTIVE BASISOF BEHAVIORLorenz proposed that instinctive systems promote adaptation to the environment.Moreover, the ability to learn is programmedinto species by their evolutionary histories.

Therefore, theabilitytolearnisaninnatecharacteristic.

A. The Search for Knowledge: Lorenz emphasized that organismsare "built"through evolution to process environmental information that isimportant fortheir survival.

B. The Interaction of Energy and Environment: Lorenzand his colleague Tinbergenspent many years observing animal behavior in natural habitats. From theseobservations, Lorenzdevelopedan energy model ofinstinctive behavior.

1. EnergyModelLorenz proposedaction-specific energyas the internal force thataccumulates within the organism and provides the motivation forbehavior.It isaction-specific in the sense that each behavior (action)requiresits own specific energy. As the energy builds,it is preventedfrom being released by an innate releasing mechanism (IRM).Theincreasing pressure stimulatesappetitive behaviorand energizesoverall behavior,increasing the likelihood that an animal will encounteran environment that containsasign stimulus.Thisstimulusisa"trigger" that releases the accumulated energy.The result isa fixedaction pattern, which might be consideredan instinctive behavior.

2. EnvironmentalReleaseThe releaser (sign stimulus) for afixed action pattern may be simpleor complex.The probability that a fixedaction pattern will occur dependsupon several factors,including (1) the intensity of the sign stimulusand(2) the accumulated level ofaction-specific energy.

3. HierarchicalSystemAlthough some simple behaviorsinvolve only a few movements, many"higher-level" instincts,such as reproduction, require many fixedactionpatterns.In such cases, Lorenzassumed that energy,in the form ofneural/hormonal events, builds up at this high level,andis released tolower levels.The specific fixedaction pattern that occursat any momentdepends upon which sign stimulusis present.Presumably,separatebrain areas control each action.Thus, complicatedsequences ofbehaviors,such asin reproduction,are controlled by both brain andenvironmental eventsin a coordinated fashion.

4. Conflicting MotivesBecause separate fixedaction patterns have their own action-specificenergy andsign stimuli, conflicts can arise when two opposing systemsare triggered by the two sign stimuli.In this case, energy may spill overand release some other behavior, calleddisplacement.

C. The Importance of Experience: Lorenz recognized that instinctive behavior is notnecessarily rigidand unmodifiable.He suggested that experience (learning) canalter the form ofappetitive behaviors,and/or the sensitivity of the IRM to the


4/32

sign stimulus. Although there are many parallels between Lorenz's energy modeland the way environmental cues can release behaviors, the model is not aperfect one of how brain controls behavior.

II. HABITUATION AND SENSITIZATIONHabituationandsensitizationare two fundamental learning processes.In each case,animals change their reactions to astimulus with repeatedstimulation.Habituation is

definedasadecrease in responsiveness to astimulus,asisseen iningestionalneophobia (decreasedavoidance ofa novel food over time).Sensitizationrefers to an increase in reactivity to the stimulus.

A. The Nature ofHabituation andSensitization: GrovesandThompson suggest thathabituation involvesdecreased reactivity ofinnate reflexes,so that withrepeatedstimulation, the animal's nervoussystem reacts less to the stimulus.Conversely,sensitization may occur because ofincreasing nervousactivity withrepeated exposures to the particular stimulusand perhaps to all stimuli.

B. The Conditions Affecting Habituation andSensitization: Not only are habituationandsensitization different behaviorally, they differ in how various conditionsaffect them.Some of these factorsinclude (1) the intensity of the stimulusand(2) the consistency of the stimulus (e.g., whether itsshape changes from

stimulation to stimulation). Both habituation andsensitization can be transientphenomena.If habituation lasts over a long time,it suggestssome learning hasoccurred.Sensitization appears to result from an increase in overall arousallevel.

C. Dishabituation: Support for increasedarousal asa cause ofsensitization is thephenomenon ofdishabituation,definedas the recovery ofan habituatedresponse asa result of encountering asensitizing stimulus.Thus, even thoughan animal has habituated to astimulus,some other arousing stimulus canreinstate the response to the habituatedstimulus.It isadaptive for our nervoussystems to habituate to irrelevant stimulation.It isalso adaptive to be morealert to our environment when some sensitizing (arousing) stimulusappears.

III. OPPONENT-PROCESSTHEORYSolomon and Corbit proposedan opponent-process theory of emotion to describe how

we react to emotional stimuli,and how our responses change with repeated exposure tothe stimuli.

A. Our Initial Reaction: An emotional stimulus can elicit a primary emotionalresponse (A state), which then instigatesan opposite emotional response (Bstate).The A state may be pleasant (thus evoking an unpleasant B state), or itmay be unpleasant (thus evoking a pleasant B state).The intensity of A is tied tothe intensity of the stimulus,anda given stimulus will always produce the sameA state.The intensity of B isdetermined by the intensity of A. Compared to theA state, B isinitially lessintense than A,issluggish in intensifying,andisalsosluggish in terminating.The overall emotional response to astimulusis thesummation of A and B over time.

B. The Intensification of the Opponent B State: The theory suggests that repeatedstimulation (and thus repeated A states) causes changesin B that include its (1)

more rapid onset, (2) greater intensity,and (3) slower offset.This may helpexplain toleranceandwithdrawalin drug addiction.Tolerance involvesreduced reactivity to adrug with repeated exposure to the drug. Withdrawalrefers to the adjustment of the body andadrug craving when the drug is notpresent.The development of tolerance is the weakening of the A state (becauseB increases). Withdrawal is the intensification of the B state, especially when nodrug is present.

C. The Addictive Process: Addiction does not alwaysdevelop in people who usedrugs. But when it does,Solomon saysit is because addicts recognize that drugabstinence produces withdrawal symptoms (B state).Thusaddictsseek the drug


5/32

(which increases the A state) simply to overcome the unpleasant feelings ofwithdrawal.

D. The Influence ofOther Aversive Events: The theory may help explain whyaddictionsare difficult to cure.The addict's B state isa generalizeddiscomfort.Thus,any other situation that arousesasimilar general discomfort mightmotivate the addict to seek out the drug again to decrease the discomfort.

E. The Search for Pleasure: Addiction to most drugsis based on the drug producinga pleasant A state, which is followed by an unpleasant B (withdrawal) state. Butsome behaviors,such as thrill-seeking, may represent asituation that producesan unpleasant A state with a pleasant B state that intensifies with repeatedstimulation.

ChapterOutline

I. THE ACQUISITIONOF THE CONDITIONED RESPONSEA. TheConditioningParadigm

1. Basic ComponentsIn classical (or Pavlovian) conditioning, there are four importantelements.In a normal conditioning experiment, these include the pairingofaconditioned stimulus (CS)followed by the unconditionedstimulus (UCS).The UCSinnately elicitsan unconditioned response(UCR), which is known as the unconditioned reflex. After CS-UCSpairingsin training, the CS comes to elicit the conditioned response(CR), which is evidence for the conditioned reflex.The CR becomesstronger with more CS-UCS trials until it reachesasymptotic level.

2.

TheConditioning ofH

unger

Conditioned hunger develops when a CSis followed by the occurrence ofadeprivation condition,such as hunger, the CSacquires the ability toelicit hunger and feeding responses.

3. TheConditioning of FearWhen a CSis followed by the occurrence ofa painful event, the CSacquires the ability to elicit fear and motivates the avoidance ofadversity.

4. OtherExamples of Conditioned ResponsesThe conditioning processisso pervasive that virtually any response ofthe body can be elicited by CSs that have been paired with UCSs.Inmanysituations, more thanone CR isconditioned.

B. Conditioning Situations: Several techniquesare used to measure the level ofclassical conditioning.

1. Signtracking


6/32

Brown and Jenkins conducted the first sign tracking experiment knownasautoshaping.The researchers found that pigeons would peck a keylight presentedshortly before the delivery of food.The response to thekey light occurred even though no activity was required to produce thefood.

2. EyeblinkConditioningEyeblink conditioningdevelops when rabbits receive repeatedpresentations ofa tone CSandan air puff UCSdelivered to the cornea ofthe eye, the tone CS will elicit a conditioned blink response.Strength ofconditioning can be measured by computing the CR probability overtrials.

3. FearConditioningFear conditioningisdemonstratedin the conditioned emotionalresponse (CER) paradigm.In thissituation,a CS previously associatedwith shock,is presented while a rat is performing an instrumentalresponse for food.The suppression of the instrumental behavior is

assumed to reflect the influence of fear conditioning attached to the CS,andasuppression ratio can be computedas the number of responsesduring the CS compared to the number of responsesduring the CS plusthe number of responses without the CS. A suppression ratio of 0.5indicates that fear has not been conditioned,anda ratio of 0.0 indicatescomplete suppression of behavior or complete fear.

4. FlavorAversionLearningAnimals can learn to associate a flavor with illness. For example,if ratstaste saccharin and then are made sick, they acquire aflavor aversionandsubsequently avoid the taste of the saccharin.The strength of theaversion is reflective of the classical conditioning process.

C. Conditioning Paradigms: There are five different paradigms or procedures thatare commonly usedin Pavlovian conditioning.They differ with respect to how aCSis paired with a UCS.

1. DelayedConditioningIn delayed conditioning, the onset of the CS precedes the onset of theUCSand termination of the CS occurs either with the onset of the UCS orduring UCS presentation.

2. Trace ConditioningIn trace conditioning, the CSis presentedand terminated prior to the

onset of the UCS.

3. SimultaneousConditioningIn simultaneous conditioning, the CSand the UCS have onsetsat thesame time.

4. BackwardConditioning


7/32

In backward conditioning, the UCSis presentedand terminatedbefore the onset of the CS.

5. Temporal ConditioningIn temporal conditioning, the UCSis presentedat regular time

intervalsallowing the timing of the UCS to serve as the CS eliciting theCR.

D. Conditions Affecting the Acquisition ofa Conditioned Response: CSand UCSpairingsdo not automatically insure that a CR will be acquired.Several variablesdetermine whetherconditioningwill be successful.

1. ContiguityContiguity refers to the closenessin time between the deliveries of theCSand the UCS.

2. TheOptimal CS-UCSInterval

The optimal CS-UCS interval. For each response system, there isanoptimal time between the onsets of the CSand UCS that supportsconditioning.

3. A Bridge Between the CSand the UCSIfasecondstimulusis presented between the onset of the CSand theonset of the UCS, conditioning will occur despite a long temporalseparation between the CSand UCS onsets.

4. Long-delayLearningIn flavor-aversion learning,an aversion develops to a CS paired with

illnessdespite long intervals between the onset of the CSand the onsetof the illness.

E. TheInfluence ofIntensity1. CSIntensity

In general,increasesin the intensity of the CSsupport higher levels ofconditioning. Also, presentations of loudandsoft auditory CSs can raisethe level of conditioning to the loud CSand reduce the level to the softCS.

2. UCSIntensityThe strength of the CR increases with more intense UCSs.

3. TheSalience of the CSSeligman proposed that evolution hasshaped how sensitive speciesareto various CSs. With some CSs, learning occurs rapidly,demonstratingpreparedness. For other CSs, learning is essentially impossible,demonstrating contrapreparedness.Salience refers to the tendency


8/32

for a CS to be associated with a particular UCS.

4. ThePredictiveness of the CSThe ability ofa CS to elicit a CR depends not only on its pairing with theUCS, but also itsability to predict the UCS.Cue predictivenessis

degradedif the CSsometimesis not followed by the UCS or if there isasecond, more predictive CS.

5. UCS-AlonePresentationsSimilarly,if the UCSsometimes occurs without the CS, learning isdegraded.

6. CS-AlonePresentationsEven when groupsare given the same number of CS-UCS trials, learningisdegradedif the CSis presented without the UCSduring training.

7. TheRedundancy of the CSIfa CSis first trained to elicit a CR,and then that CS plusanother CSare simultaneously presented before the UCS, the first CS can blockconditioning to the second.This occurs because the second CS providesno new information for the animal.

II. EXTINCTIONOF THE CONDITIONED RESPONSE ACR that is elicitedin avariety of environmental situations can come to occur in only onesituation,a process calledstimulus narrowing,andinvolvesextinction of theconditioned response.

A. Extinction Paradigm: Extinction isa method that can eliminate CRs. After firstconditioning a CS to elicit a CR,subsequently delivering the CS without the UCSextinguishes the CR.

B. How Rapidly Doesa Conditioned Response Extinguish?1. TheStrength of the CR

Three factorsinfluence the extinction of the CR.One is the strength ofconditioning.In general, the stronger the CS-CR bond, the slower theextinction of the CR.However, the correlation between the CRacquisition level and the resistance to extinction is not perfect.

2. TheInfluence ofPredictivenessWhen the CSis occasionally presentedalone during acquisition,resistance to extinction isincreased.

3. Duration of CSExposure As the duration of the CS exposure in extinction increases, thestrength of the CR weakens.

C. Spontaneous Recovery: Pavlov proposed that the extinction of the CR is caused


9/32

by an inhibition process. Furthermore, the inhibitory state is only temporaryallowing the return of the CR following extinction (spontaneous recovery).However, long-term extinction prevents the occurrence ofspontaneous recovery.

D. Other Inhibitory Processes: The inhibition of the CR can become permanent,aprocessPavlov called conditionedinhibition.

1. ConditionedInhibitionConditioned inhibitionis the ability ofastimulus (CS-) associated withthe absence of the UCS to suppress responding to the CS+.

2. ExternalInhibitionExternal inhibition occurs when a novel stimulusis presentedduringconditioning andinterferes with the occurrence of the CR.

3. LatentInhibitionLatent inhibition retards conditioning ofa CS.The procedure involvespresenting the CSalone before the CSis paired with the UCS. Butbecause CS preexposure also retards the development of conditionedinhibition,it is not clear just what is learnedduring CS preexposure.

4. Inhibition of DelayThe absence of responding to the CS until the end of the CS-UCSintervalwas called by Pavlovinhibition of delay.

5. DisinhibitionDuring extinction trials, the CR isstronger when a novel CSis presented,a process calleddisinhibition.

III. A CR WITHOUT A CS-UCSPAIRING?Several procedures exist that allow a CS, never directly paired with an UCS, to elicit aCR.These proceduresinclude higher-order conditioning,sensory preconditioning,andvicarious conditioning.

A. Higher-OrderConditioning1. TheHigher-OrderConditioningParadigm

A CS1 that has previously been paired with a UCS can supportconditioning to asecond CS (CS2) when the CS2 and CSl are pairedtogether.Thus CS2, never directly paired with the UCS,still elicitsa CR.Thisishigher-orderconditioning.

2. ResearchonHigher-OrderConditioning Higher-order conditioning is weaker than original (first-order) conditioning. Rescorla's research suggests that during second-order conditioning, the CS2-CS1 pairings not only condition CS2 to elicitthe CR (excitation), but they also condition inhibition of the CR becausethe UCSisabsent.Thus,assecond-order conditioning trials proceed,there isinitial excitatory conditioning of CS2, but eventually there is


10/32

increased conditionedinhibition.

B. SensoryPreconditioning1. TheSensoryPreconditioningParadigm

The procedure for sensory preconditioning requires three phases.In

phase 1, CS2 and CS1 are paired together.In phase 2, CS1 is pairedwith an US for conditioning.In phase 3, CS2 is presentedalone andelicits the CR previously evoked by CSl.

2. ResearchonSensoryPreconditioning Reliable sensory preconditioning occursif,during thepreliminary CS2-CS1 trials, the two CSsare separated by severalseconds,andif there are only a few CS2-CS1 trials.

C. VicariousConditioning1. ResearchonVicariousConditioning

A CR can develop to a CSifa person observesanother personexperiencing the CS-UCS pairing.Thisis termedvicariousconditioning,and hasalso been observedin monkeys.

2. TheImportance of ArousalVicarious conditioning requires that the observer be at least somewhatemotionally aroused while observing the CS-UCS pairing in the otherperson.

IV. APPLICATIONSOF PAVLOVIAN CONDITIONINGA. Systematic Desensitization: Systematic desensitizationisa therapy based on

Pavlovian conditioning procedures that can be effective in the elimination of

conditioned fear and the reduction of phobic behavior.1. Original Animal StudiesA central feature ofsystematic desensitization isa counterconditioningprocedure, which involves the elimination ofa CR when the CSis pairedwith an opponent or antagonistic UCS. According to Wolpe, the extinctionissuccessful due to reciprocal inhibition, which assumes that only oneemotional state can be experiencedat a time. Wolpe first trained cats tofear an environment where a buzzer CSsignalledshock. By letting thecats eat foodin similar environments, the animals eventually lost theirfear.

2. ClinicalTreatmentTherapy involvescue-controlled relaxationin which the relaxationresponse is conditioned to aspecific environmental event.Therapy alsoinvolves the construction ofthematic hierarchies which refers to alisting of fear-inducing situations that are related to a basic theme.Additionally,aspatial-temporal hierarchy, which isa listing ofactivities related to a phobic object on the basis of time or physicaldistance,isalso used. By remaining relaxed while imagining the lowestitem in the hierarchy, the fear to that situation or object iscounterconditioned. By then counterconditioning situations higher in the


11/32

hierarchy,all situations may be counterconditioned.

3. ClinicalEffectivenessTo test the effectiveness of the treatment, the individual must encounterthe feared object.Treatment is effective to the extent that the person

can tolerate the once-feared object or situation. For the therapy to work,patients must be able to imagine feareditems on their hierarchy offearedsituations. Virtual reality technology may enhance the success ofsystematic desensitization.

B. ExplorationsfortheFuture1. An Intense Craving

A drug addict experiences withdrawal after a period ofabstinence.Additionally, environmental cues (CSs) that were previously paired withdrug use can elicit conditioned withdrawal reactions.Thus,curingaddictionsmightrequireextinction of these responses.

2. The Conditioning ofImmune System SuppressionAder and Cohen have demonstrated that Pavlovian procedures canproduce conditioned immune system suppression.Their findingshave important implications for the treatment ofautoimmune disorders,such as lupus,in which the immune system is overactive.In otherdisorders,such as AIDS,in which the immune system is underactive,itmay be possible to produce conditioned immune systemenhancement.

ChapterOutline

I. THE ACQUISITIONOF AN APPETITIVE RESPONSEA. Skinner's Contribution: Skinner demonstrated that reinforcement hasa

significant impact on behavior. Central to Skinner's theory is the concept ofcontingency which expresses the specific relationship between a targetbehavior anda reinforcer. A reinforcerisany event used to increase thefrequency of behavior.

B. The Distinction Between Instrumental andOperant Conditioning: Skinnerconsideredinstrumental conditioningasasituation requiring a response forreinforcement.However, the environment of the conditioning situation constrainsthe opportunity to make the instrumental response.In contrast,in operantconditioning there are no constraints placed upon the subject regardingresponse execution.Skinner developed the operant chamber to study

conditioning more conveniently.C. Types of Reinforcers: Skinner identifiedseveral types of reinforcers. A primary

reinforcer hasinnate reinforcing properties;asecondary reinforcerdevelopsits reinforcing properties through itsassociation with primary reinforcement.Theability ofastimulus to function asasecondary reinforcer depends upon (1) thestrength of the primary reinforcer with which it has been paired, (2) the numberof pairing of the secondary and primary reinforcers,and (3) the contiguity of thepresentations ofsecondary and primary reinforcers.Skinner also distinguishedbetween positive and negative reinforcers. A positive reinforcerisan eventthat has the ability to increase the frequency ofa behavior which precedes the


12/32

event. A negative reinforcerisan event whose termination after theoccurrence ofa behavior increases the frequency of the behavior.

D. Shaping: Shapingisa technique for training an animal to emit adesiredbehavior by first selecting a high-frequency operant behavior and then slowlychanging contingencies until the desired behavior is learned.Shapingisalsoknownas thesuccessiveapproximationprocedure.

1. Training a Rat to Bar PressA rat's operant rate of bar pressing is not at zero, because the animal,through exploration, might engage in bar pressing.However,shapingensuresrapidacquisition of thedesiredbehavior.

2. Shaping Social DiscourseParents can employ shaping to effectively teach social skills to theirchildren.

E. Schedules of Reinforcement: The contingency relationship between an operantresponse and the probability of reinforcement is known asaschedule ofreinforcement. Many such schedules have been studied,including two major

classes: (1) ratio schedules of reinforcement that require a certain numberof responses to produce reinforcement,and (2) interval schedules ofreinforcement that reinforcesa response only after a certain amount of timehas elapsedsince the last reinforced response.

1. Fixed-Ratio SchedulesA fixed ratio schedule of reinforcement requiresa fixed number ofresponses to be made to obtain reinforcement.In other words, every nthresponse is reinforced, regardless of how much time has elapsedsincethe last reinforcement.Thisschedule is characterized by apostreinforcement pause, which refers to a temporary cessation ofbehavior following reinforcement, followed by a resumption ofresponding at the level characteristic of that ratio schedule.

2. Variable-Ratio SchedulesA variable ratio scheduleis one in which an average number ofresponses produces reinforcement.It is exactly like a fixed ratioschedule except that the actual number of behaviors required to producereinforcement changesafter each reinforcement.

3. Fixed-IntervalSchedulesThe fixed interval scheduleinvolvesa contingency in whichreinforcement for a response isavailable only after aspecified period oftime has elapsedsince the previous reinforced response.Thisschedule

produces the scallop effect, which refers to a pattern of behaviorcharacteristic of the fixedinterval schedule. Response rate slowsafterreinforcement and then slowly increasesas the time approaches whenreinforcement will be available.

4. Variable-IntervalSchedulesThe variable interval scheduleis the same asa fixedintervalschedule, except that the interval changesafter each reinforced


13/32

response.The scallop effect typically does not occur in thisschedule.

5. DifferentialReinforcementSchedulesWhen reinforcement depends on both time and number of responses, thecontingency is calledadifferential reinforcement schedule.

6. Differential Reinforcement ofHigh Responding SchedulesThe differential reinforcement of high responding (DRH) schedulerequiresa high rate of behavior over aspecified time period before aresponse is reinforced.

7. Differential Reinforcement of Low Responding SchedulesIn the differential reinforcement of low responding (DRL)schedule,a certain interval of time must pass without a response.Thefirst response following the end of the time periodis reinforced.

8. Differential Reinforcement ofOther BehaviorsSchedulesA differential reinforcement of other behaviors (DRO) scheduleprovides reinforcement only when there isa total absence ofa particularresponse in aspecified period of time.

9. CompoundSchedulesIn acompound schedule, two or more of the above schedulesare ineffect, one at a time,andanimals must complete whatever schedule isinoperation at any one time.

F. How Readily Isan Instrumental Response Learned?1. TheImportance of Contiguity

Reward leads to the conditioning of the response when it followsimmediately after the response.Delayingtherewardinterfereswithconditioning.

2. TheEffect of DelayA delay between the operant response andits reinforcement hasapronounced negative effect on learning rate.

3. Delay of Rewardand Conditioning in HumansAsis true for nonhuman animals, humansalso learn more slowly as thedelay between response and reinforcement increases.

G. TheImpact of RewardMagnitude1. The Acquisition ofan Instrumental Response

Many studies have shown that the greater the reward magnitude, thefaster the acquisition of the instrumental response.


14/32

2. The Performance ofan Instrumental ResponseThe greater the reward magnitude, the higher is the performance level ofan operant or instrumental response.

3. TheImportance ofPastExperienceThe shift from high to low reward magnitude momentarily reduces thelevel ofinstrumental performance below that which occurs when the lowreward magnitude is used consistently. Crespi (1942) called thisoutcome the depression effect. When the shift movesin the otherdirection, from low to high reward magnitude, the level ofinstrumentalbehavior is momentarily higher than found when the high rewardmagnitude is used consistently. Crespi used the term elation effect todescribe this outcome. Later,Zeaman (1949) substituted the termpositiveandnegative contrast for the elation anddepression effect.Contrast effectsare more likely the result of changesin motivation oremotion (e.g., frustration in negative contrast) than in learning.

4. The Influence of Reward Magnitude in HumansAs they do in nonhuman animals,increasesin reward magnitude tend tohave a positive effect upon performance in humans.

II. EXTINCTIONOF ANOPERANTOR INSTRUMENTAL RESPONSEA. The Discontinuance of Reinforcement: Omitting reinforcement after an operant

or instrumental response leads to extinction.B. Spontaneous Recovery: Ifan operant response is extinguished by removing the

reinforcer,asucceeding "rest" period might be followed by spontaneousrecovery of the response.This outcome parallelsspontaneous recovery inPavlovian conditioning.Presumably, extinction occurs because nonreinforcedtrialsin extinction increase inhibition.

C. The Aversive Quality ofNonreward: Amsel claimed that nonreward elicitsanaversive frustration state. Cues that are associated with frustration becomeaversive,and can motivate escape behaviors to terminate those cues.

D. Activation ofan Instrumental Behavior: Omitting an expected reward can notonly lead to extinction of the response that previously produced the reward,itcan also facilitate other instrumental appetitive responding.

E. Resistance to Extinction: The following three factorsinfluence the resistance toextinction ofan operant or instrumental response.

1. TheInfluence of RewardMagnitudeWhen the level ofacquisition training is low,a large reward producesgreater resistance to extinction than doesasmall reward.However, withextendedacquisition,asmall rewardduring acquisition produces greaterresistance to extinction.

2. The Influence of Delay of RewardWhen a constant delay of rewardis experiencedduring acquisition,resistance to extinction is not affected by the length of the delay.However,variable delay of rewardduring acquisition increases resistanceto extinction in some situations.

3. The Importance of Consistency of Reinforcement


15/32

Compared to continuous reinforcement,intermittent reinforcement inacquisition produces greater resistance to extinction.This outcome is thepartial reinforcement effect (PRE).Two theories that explain the PREreasonably well are Amsel's frustration theory and Capaldi'ssequentialtheory. Amsel suggested that nonreward elicitsa frustration state.Continuously reinforcedanimals quickly stop responding in extinctionbecause frustration isaversive andinhibits responding. But animals that

are intermittently reinforced often are rewarded for responding whilethey are experiencing frustration from prior nonreward,and thus learn torespond when those cuesare present.Thus,during extinction,nonreward elicits responding in partially reinforcedanimals, butsuppresses responding in continuously reinforcedanimals.

According to Capaldi,if reward followsa nonrewarded trial,an animalwill associate the memory of nonreward (SN) with the response leadingto reward.The PRE occurs because during extinction, continuouslyreinforcedanimals have more novel cues present (SN) than partiallyreinforcedanimals.Thus,in continuously reinforcedanimals, there arefewer cues present in extinction that are associated with responding.

III.

APPLICATION: CONTINGENCY MANAGEMENT

Skinner proposed that faulty contingencies often produce behavioral problems.Behavior modificationincludesa broadarray behavioral treatments.One of these iscontingency management, which uses reinforcement and nonreinforcement toincrease the frequency ofappropriate behaviorsand eliminate maladaptive behaviors.Contingencymanagementinvolvesthreephases:

A. The Assessment Phase: In the initial stage of contingency management,adetermination of the behavioral problem is made.

B. The Contingency Contracting Phase: In the secondstage,adesired response isspecifiedand the precise relationship between response and reinforcement isdetailed. Reinforcement can be dispensed by another or by the individualperforming the behavior.

C. The Implementation of the Contingency Management Program: Contingencymanagement programs have been found to be very successful in a wide varietyofsituations.

ChapterOutline

I. THE AVERSIVE EVENTS AROUND USMany unpleasant situationsare encountered throughout life.Some events can beescaped but not avoided.Other events can be avoidedifa certain response is learnedthat precludes the occurrence of the aversive event. Finally, punishment is oftenexperienced following the display ofan unwanted behavior. Because the environmentcontainsaversive events,it isimperative that organisms learn adaptive ways to deal

with adversity.

II. ESCAPE CONDITIONINGAn escape responseisan instrumental behavior that is motivated by an aversive eventandis rewarded by the termination of the event.

A. The Escape from Aversive Events: Several factorsinfluence the acquisition of theescape response.


16/32

1. The Intensity of the Aversive EventThe intensity of the aversive event controls the rate of escapeconditioning. For example,Trapoldand Fowler (1960) trained rats toescape electric shock.The rats received either 120, 160, 240, 300, or400 volts of electric shock in the start box ofan alley.The results of theexperiment indicated that the greater the shock intensity, the shorter the

latency to escape from the start box.

2. TheAbsence of RewardThe strength of learned escape behavior isinfluenced by the degree towhich the aversive event is reduced by the escape response. Forinstance, Campbell and Kraeling (1953) gave ratsa 400-volt electricshock in the start box ofan alley. When the escape response wascompleted, the shock was reduced to either 0, 100, 200, or 300-volts.Thus, greater reductionsin the intensity of the shock enhanced escapeperformance.

3. TheImpact of DelayedRewardDelaying the termination of the aversive event following the escapebehavior negatively affects performance. Asan example, Fowler andTrapold (1962) delayed the termination ofan electric shock by either 0,1, 2, 4, 8, or 16 seconds following the escape response. Longer delaysinterfered with the acquisition of the escape response.

B. The Elimination ofan Escape Response: Eliminating a learned escape responsecan be accomplished by no longer presenting the aversive event or by no longerterminating the aversive event following the escape response.

1. TheRemoval ofNegativeRewardAn escape response may be extinguishedif the aversive event continues

to occur even after the escape response has been made.

2. TheAbsence of AversiveEventsElimination ofan escape response also occurs when the aversive eventhas been discontinued.However, the escape response may occur manytimes before disappearing due to the continuing motivational influencesupplied by conditioned fear.

3. Vicious-CircleBehaviorBrown and hisassociates observedaspecial situation in which escaperesponding continuesdespite the fact that the subject is punished for

making the response.The persistence of escape responding despite thedelivery of punishment is calledvicious-circle behavior. Vicious-circlebehavior is presumed to occur from the motivational influence of fearconditioned to environmental cues.The resulting punishment serves toreinforce the fear thus maintaining the motivation for the escaperesponse.

III. TheAvoidance of AversiveEvents


17/32

Avoidance conditioning involves the learning ofa response that serves to prevent theoccurrence ofan aversive event.

A. Types of Avoidance Behavior: An active avoidance responseis one thatprevents the occurrence ofan aversive event. A passive avoidance response,on the other hand,is the withholding of responding to prevent an aversiveevent.

1. Active AvoidanceLearningThe paradigm for active avoidance conditioning wasdeveloped byMowrer (1938, 1939).In active avoidance conditioning,anenvironmental stimulus (CS) is paired with the delivery ofa painful event(UCS).Subjects first learn to escape from the aversive event.Eventually, the escape response begins to occur when the CSispresentedand before the arrival of the UCS. Under these conditions, theresponse prevents the delivery of the aversive event.

2. PassiveAvoidanceLearningSubjects can also learn to avoidan aversive event if by withholding a

response the aversive event does not occur.

B. How Readily is Avoidance Behavior Learned?1. The Severity of the Aversive Event

One important factor determining how well a learned response occursisthe intensity of the aversive event.In many situations,increasesin theintensity of the aversive event produce higher levels ofavoidanceconditioning.However, the opposite relationship holds true in the two-way avoidance situation.

2. PassiveAvoidanceBehavior

Increasesin the intensity of the aversive event enhance conditioning inthe passive avoidance situation.

3. One-Way Active AvoidanceBehaviorIn a one-way active avoidance situation,increasesin the intensity of theaversive event also enhance conditioning.

4. Two-Way Active AvoidanceBehavior

In a two-way active avoidance situation, the subject (usually a rat) ispresented with a CSand must run to a chamber (side B) before anaversive event is presented.Then,after ashort period of time, the CSispresentedagain and the subject must return to the original chamber(side A) to avoid the aversive event.Increasesin the intensity of theaversive event produce decrementsin performance in thissituation.Thisoutcome may be due to the subject experiencing conflict because theaversive event has been associated with chambers that also serve asafety function.Increasesin the intensity of the aversive event mayenhance the level of conflict.


18/32

5. The Delay Between the CSand the UCSLonger temporal intervals between the presentation of the environmentalCSand the aversive UCSinterfere with conditioning of the avoidanceresponse.Presumably, longer CS-UCSintervalsare detrimental to theconditioning of fear to the CS, which isimportant for motivating theavoidance response.

C. Application: Response Prevention or Flooding: Response prevention orfloodingisa form of behavior therapy in which a phobiais eliminated by forcedexposure to the fearedstimulus without any aversive consequences. A largenumber ofstudies have reported positive results using flooding to eliminatephobias, panic disorder,and obsessive-compulsive behavior.

IV. PUNISHMENTPunishmentisa form of conditioning that can eliminate unwanted behavior by the useofan aversive event contingent upon the occurrence of the unwanted response.

A. Types ofPunishers: Positive punishment refers to the delivery ofa physicallyor psychologically painful event following the occurrence ofan unwanted

behavior.Negative punishment (Omission training)involves the loss ofpositive reinforcement due to the occurrence ofan unwanted behavior.There aretwo types of negative punishment.One type is calledresponse cost whichinvolves the withdrawal of positive reinforcement when an unwanted responseoccurs.The other type of negative punishment is calledtime-out fromreinforcement which involvesa period of time during which reinforcement forresponding is unavailable.

B. The Effectiveness ofPunishment: Skinner (1938) concluded that punishmentsuppresses behavior on a temporary basis.However,subsequent research hasshown that a number ofvariablesdetermine whether punishment is effective insuppressing behavior.

C. When isPunishment Effective?: Three factorsdetermine the effectiveness ofpunishment in suppressing behavior.

1. TheSeverity ofPunishmentMild punishment produces little,ifany,suppression of the punishedresponse. An extremely strong punishing event is more likely to producecomplete suppression ofa punished behavior in both humansandnonhumans.

2. TheConsistency ofPunishmentFor nonhumansand humans, punishment is more effective when it isadministeredin a consistent manner than when it isadministeredintermittently.

3. Delay ofPunishmentIn nonhumansand humans, punishment is more effective when it occursimmediately after the completion of the unwanted behavior than if thepunishing event isdelayed for a time.

D. The Negative Consequences ofPunishment: Punishment procedures can produceseveral undesirable effects.

1. Pain-InducedAggression


19/32

Punishment often leads to aggressive behavior. At least in humans,suchpain-induced aggressionapparently occurs because punishment elicitsanger,andanger leads to aggression.

2. TheModeling of Aggression

Experimentsshow that children often mimic other peoples' behavior,aphenomenon calledmodeling. After observing a model'saggressivebehavior, children'saggressivenessincreases. More specifically, childrentend to use the same type of punishment that they have received.Inaddition, children often model punishment that isdelivered to them,andthere isa correlation between the use of punishment by parentsand thelevel ofaggressive behavior in their children.

3. The Aversive Quality ofaPunisherHumansand nonhumans learn, through classical conditioning,aboutcues that accompany punishment,and those cues become capable ofeliciting fear.Such cues, which may include a parent who deliverspunishment to a child, may motivate behaviors to escape those cues.

4. AdditionalNegativeEffects ofPunishmentDelivery ofaversive stimulation (punishment) can be problematic forother reasons. First,suppressing one behavior through punishment canlead to suppression ofsimilar behaviors through generalization.Second,subjects may not perceive the contingency between the punishing eventand the behavior it isintended to suppress,and this can lead tohelplessnessanddepression.

E. Application: The Use ofPunishment: Punishment is commonly usedin humansociety,including spanking of children by parents,disapproval ofstudentbehavior by teachers,and fines for violating laws.Several proceduresare known

to be effective in suppressing unwanted behaviors.1. Positive Punishment

Positive punishmentis the presentation ofa painful event following anunwanted response.Positive punishment has been used clinically tosuppress the occurrence ofseveral types of maladaptive behaviors.However, one problem with the therapeutic use of positive punishment isthe possibility that the effects of punishment will generalize to situationsoutside the clinical setting.

2. Response CostResponse costisanegative punishment technique in which the

occurrence ofan undesired behavior resultsin the withdrawal or thefailure to obtain positive reinforcement. Response cost techniquesareeffective in controlling the occurrence ofseveral maladaptive responses.

3. Time-OutfromReinforcementTime-out from reinforcement,another negative punishmentprocedure, refers to a program in which the occurrence ofaninappropriate behavior resultsin the loss ofaccess to positivereinforcement for adesignated period of time.This procedure also has


20/32

been shown to be effective in controlling the occurrence ofseveral typesof maladaptive behaviors.

4. TheEthical Use ofPunishmentBecause punishment can lead to unwanted outcomes, the administration

and use of punishment must always be guided by law and ethics.Stateand Federal laws concerning the appropriate use of punishment havebeen adopted,and the American Psychological Association's EthicalPrinciples ofPsychologists (1992) guide therapistsand researchersin theuse of punishment.

ChapterOutline

I. TRADITIONAL LEARNINGTHEORIESTwo types of theories explain the learning process.Historically older,S-R (stimulus-response) theoristsadopt assumptions that learning involves more or lessautomatic"mechanistic" processes that permit adaptive changes to the environment. More

recently,cognitive theoristssuggest that higher-order (mentalistic) processesareinvolvedin learning.

II. S-R ASSOCIATIVE THEORIESThe S-R theorists can be dividedinto two major theoretical approaches to how learningoccurs.One classassumes that learning occurs only when rewardis provided.The otherassumes that S-R contiguity is the only requirement for learning to occur.

A. Hull's Drive Theory: Based on Woodworth's earlier assumption that drivemotivates behavior,Hull proposeda theory of learning that intended to create amathematical equation for predicting behavior.The equation predictedbehavioral potential,and predictor variablesincludeddrive,incentive motivation,habit strength,andinhibition.Prediction of behavior depends upon knowing thevariousvaluesin the equation.

1. UnconditionedSources of DriveSome drivesare triggered by internal deprivation conditionssuch asalack of food or water for a period of time.This type ofdrive is unlearnedandis termedSUR.

2. Acquired DrivesOther drives can develop through Pavlovian conditioning. Environmentalcues that reliably predict adeprivation condition can come to elicit anacquired drive, or conditioneddrive,state.This type ofdrive learned,is calledhabit strength,andis termedSHR.

3. The Reinforcing Function of Drive ReductionHabit strength develops through learning,anddepends upon how often aresponse decreasesdrive.Thus, learning depends upon the reduction ofthe drive state.

4. TheElimination of UnsuccessfulBehavior


21/32

Ifdrive persists,all behaviorsare inhibited for a time,a process calledreactive inhibition. After time, the habitual behavior occursagain andif not reinforced (ifit does not reduce drive),it leads to conditionedinhibition of that particular response. But because drive isstill high,other behaviorsin the habit hierarchyare activated until one of themreducesdrive andis thus reinforced.

B. Incentive Motivation: Hull recognized that different reinforcers have differentmotivational value.That is,some large or rich reinforcers have greaterincentive motivation than smaller or poorer reinforcers. Further,animals canlearn about cues that predict different reinforcers,and these learned cues thusacquire conditioned properties that can motivate behavior.

C. Spence's Acquired Motive Approach: Hull assumed that drive reduction andreward were synonymous.However,studies of the rewarding properties ofelectrical stimulation of brain (Olds & Milner, 1954) and ofdrive induction(Sheffield, 1966) posed problems for Hull's theory.Spence refinedHull'sS-Rapproach to account for these situations.

1. TheAnticipation of RewardSpence suggested that a reward elicitsan unconditioned goal response

(RG),an internal response, which then producesan internal stimulusstate (SG) that increases motivation andissimilar to drive. Early inlearning, the cues present at the time of rewardare associated withrewardand create a conditioned or anticipatory goal response (rG),which causesinternal stimulus changes (sG) that motivate approachbehavior, which increasesarousal. By assuming that rewardintensitydeterminesa more intense rG than lessintense rewards,and thatanimalsassociate variousstimuli with responses that reduce drive,Spence's theory extendedHull'sS-R theory to explain how behaviorchangesin predictable ways with different rewards.

2. Avoidance of FrustratingEventsAmsel suggested that the absence of expected reward generates

frustration, which motivatesavoidance behavior,andalso suppressesapproach behavior.The internal, unconditioned frustration state (RF)motivates the animal and producesinternal stimulusstates (SF). Cuespresent during frustration become conditioned to produce ananticipatory frustration response (rF),and produce internalfrustration stimuli (sF) that motivate avoidance of frustrating situations.

D. The Avoidance ofPainful Events: Hull's theory sufferedin the 1930's because ofresearch showing that animals learn to avoidaversive events, perhaps becauseof cognitive processes. Mowrer extendeddrive-based theory to account for thisresearch.

1. Two-Factor Theory of Avoidance Learning

Mowrer proposedatwo-factor theory of avoidance learning.Thetheory assumes that subjectsare motivated to escape fear andare notperforming on the basis of the future expectation ofan aversive event.Thus, the first factor of the theory assumes that fear is conditioned toenvironmental cues that precede the occurrence of the aversive event.The conditioned fear motivates the occurrence ofan escape responsethat serves to terminate the CS.The second factor of the theory holdsthat the removal of the cue eliciting fear serves to reinforce thebehavior.Thus, escaping from a CS that elicits fear servesas the meansto avoid the aversive event.


22/32

2. Criticisms ofTwo-Factor TheorySeveral problems have been noted with Mowrer's two-factor theory ofavoidance conditioning. First,avoidance responding can be extremelyresistant to extinction.Second, fear isapparently absent when theavoidance response is well practiced. Finally,avoidance behavior can belearnedin situationssuch as the Sidman avoidance task, where there

is no external CS preceding the delivery of the aversive event.

3. D'Amato's View of AvoidanceLearningD'Amato's theory assumes that the prevention of the aversive event isimportant in avoidance conditioning. According to D'Amato, the aversiveevent elicitsan unconditioned pain response(RP)that hasastimulusconsequence (sP).The painful stimulus consequence serves to motivateescape from the aversive event. When the aversive event terminates,the subject experiencesan unconditioned relief response (RR) which alsohasastimulus consequence (SR).Through conditioning, environmentalCSs come to elicit an anticipatory pain response (rP) with astimulusaftereffect (SP).The rP-sP mechanism motivatesan escape response

from the environmental CSs.Other environmental CSs becomeassociated with the termination of the aversive event leading to theconditioning ofan anticipatory relief response(rR) with its rewardingstimulus consequence (rs).The cuesassociated with conditioned reliefprovide asecond motivational base for avoidance learning.

E. Nature of Anticipatory Behavior: Spence suggested that conditionedanticipatoryresponses (rG) were peripheral nervoussystem events, but RescorlaandSolomon suggestedan important role for the central nervoussystem.

F. Guthrie's Contiguity View: Guthrie suggested that animals learn to associatestimuliand responses merely through their contiguity. Learning,in otherwords,depended on a response occurring close in time to particular stimuli.Rewardwasnotnecessary.

1. Impact of RewardThe function of rewardaccording to Guthrie is not to strengthen the S-Rconnection asHull proposed, but rather simply to change the stimulus(S) situation. When an animal is rewarded, the stimulusis changed,thereby preserving the previousS-R connection.

2. TheFunction ofPunishmentAccording to Guthrie, punishing stimuli elicit other responses,any one ofwhich may become associated with the preceding stimuli.

3. TheImportance ofPracticeGuthrie thought that learning ofS-R connections occursin one trial only.The fact that learned behavior changes with practice isdue, he said, toanimalsattending to different aspects of the stimulus environment oneach trial,and/or to associating different components ofa complicatedresponse with the stimuli on each trial.

4. AnEvaluation of ContiguityTheory


23/32

Guthrie did not provide much empirical evidence for his theory. Laterresearch confirmed the importance of contiguity and other aspects ofGuthrie's theory, but other research has failed to confirm otherpredictions of the theory.

III. COGNITIVE APPROACHESTO LEARNINGA. Tolman'sPurposive Behaviorism: Tolman proposeda more cognitive theory of

learning that was not accepted well in the 1930'sand 1940's when Hull's theorywas predominant.Tolman'sapproach representsa forerunner to current ideasabout learning foundin cognitive psychology.

1. Flexibility of BehaviorIn contrast to Hull who believed that learning wasautomatic andinvolvedS-R associations,Tolman believed that organisms' behaviorsare purposive and goal-directed.He emphasized the role of expectanciesin guiding behavior and the ability of cues to convey information aboutwhere our goalsare located.

2. MotivationProcessesTolman proposed two motivational principles that are parallel toprocessesin Hull's theory.One principle was based on deprivation(drive) that producesan internal drive state that increasesdemand forthe goal object.Second, environmental events can be associated withthisdemand.This processis calledcathexis,and can motivate eitherapproach behavior (positive cathexis) or avoidance (negative cathexis).Tolman'sequivalence belief principle,analogous to Spence'santicipatory goal concept, explains how animals learn about and respondto secondary reinforcers (e.g., money).

3. Is RewardNecessary for Learning?According to Tolman, rewardis not necessary for learning to occur;

however, rewardis necessary asa motivating condition for theoccurrence of learned behavior.

4. AnEvaluation ofPurposiveBehaviorismHull andTolman were contemporariesandHull's theory enjoyed morepopularity.However,Tolman forcedHull to make theoreticalmodifications that attempted to account for the purposive nature ofbehavior in mechanistic terms.In more modern times, the cognitiveapproach inspired by Tolman has gained wide appeal in theunderstanding of the learning process.

B. Expectancy-Value Theory: Tolman's theory was the basis for Rotter'sexpectancy-value theory.1. Basic Tenets

There are three main ideasin Rotter's theory. First is the assumptionthat preference for a particular event isdetermined by its rewardvalue,which isitselfdetermined by comparison with experience with otherrewards.Second, each individual hasasubjective expectationconcerning the probability of obtaining a particular reward.Third,previous experiences with the rewardin varioussituations govern ourexpectation that it can be obtainedin a particular environment.


24/32

2. Locus of ControlLocus of control refers to an individual's beliefsabout how rewardsareobtained. An internal expectancy refers to beliefs that one's ownactions/abilitiesare important, whereasan external expectancy refersto beliefs that outside forces/situationsdetermine when rewards will bedelivered.

3. An Evaluation of Rotter's Expectancy-Value TheoryBased on Tolman's earlier theorizing, Rotter emphasized that learningand behavior is often determined more by cognitive processes than bythe external and biological events that were emphasized by drive theory.

IV. SKINNER'S BEHAVIORISTIC METHODOLOGYSkinner has been a major influence on learning theory and our understanding of how topredict and control behavior.He believed that the best way to study behavior is tounderstand how the environment,including reinforcers, control responding.Hisapproachto behavior issometimes known asbehavior modification.

A. The Importance of the Environment: Skinner developeda methodology, calledoperant conditioning, to study how environmental conditions control behavior.Skinner was most concerned with determining how reinforcement controlsbehavior.He definedareinforcerasan environmental event that increases theprobability ofa response.The relationship between an operant response andareinforcer is termedcontingency.Skinner suggested that even very complexbehavior,such as language,is controlled by various contingencies.

B. The Role ofTheory?: Skinner eschewed theory.He believed that research shouldbe devoted to identifying the relationships between changesin the environmentand changesin behavior. Many othersdo not agree that theory interferes withthe progress of behavior analysis.

ChapterOutline

I. THE GENERALIZATIONPROCESSGeneralizationis the process that allows responding to stimuli that are similar to thecues usedin original training.Discriminationis the process of responding to somestimuli but not others.Generalization anddiscrimination are studied by plotting ageneralization gradient which shows the relationship between the degree of change inthe original conditioning stimulusand response strength.

A. Generalization Gradients: Generalization gradients have been plotted for stimulibased upon excitatory conditioning.There isalso a limitedamount of researchstudying inhibitory generalization gradients.

1. Excitatory-ConditioningGeneralizationGradientsMuch of the research on excitatory generalization gradients has usedpigeonsassubjects.Guttman and Kalish (1956) performed the classicexcitatory generalization experiment.They traineddifferent groups ofpigeons to peck astimulus key that variedin its wavelength. Aftertraining, each group was given test stimuli,including wavelengths thatwere higher and lower than the original stimulus. Response levels weremeasured for each test stimulus.Guttman and Kalish consistently


25/32

observedsymmetrical generalization gradients with the highest level ofresponding occurring at the original training stimulus.

The same type of excitatory generalization gradients found by Guttmanand Kalish are common in other species,including humans.

Flat generalization gradients can be observed under certaincircumstances.

2. Inhibitory-ConditioningGeneralizationGradientsWeisman andPalmer (1969) studiedinhibitory generalizationgradients.Pigeons were first trained to peck a green disk (S+) toreceive food reinforcement. When a white vertical line (S-) waspresented, the pigeons were not reinforced for key pecking.Inhibitorygeneralization tests were conducted with other lines that differed fromthe original vertical line by variousdegrees ofarc.The resultsshowedthe greater the difference between the original vertical line and the testline, the less was key pecking inhibited.

B. The Nature of the Generalization Process: The Lashley-Wade theory ofstimulus generalization proposes that generalization occurs when the subjectcannot distinguish between the test stimulusand the conditioning stimulus.Severallines of researchsupportthisexplanation.

II. DISCRIMINATION LEARNINGDiscrimination learninginvolves the use of environmental cues to signal theavailability or unavailability of positive and negative reinforcement as well aspunishment.In discrimination training, the symbol SDis used to indicate the availabilityof reinforcement contingent upon the occurrence ofan appropriate response.The symbolSis used to indicate that reinforcement is not available anda response will not beeffective in producing reinforcement. When operant behavior is under the control ofadiscriminative stimulus (either SD or S),Skinner called the behavior a

discriminative operant.

A. DiscriminationParadigms1. Two-ChoiceDiscriminationTasks

In the two-choice discrimination learningsituation, the SDand theSare within the same stimulusdimension.In this task,subjects usuallybegin by responding equally to the SDand the S.However, withcontinued training, the response rate to the SDincreasesand theresponse rate to the Sdecreases.

2. ConditionalDiscriminationTaskIn aconditional discrimination task, the availability of reinforcementto a particular stimulusdepends upon the presence ofasecondstimulus.

B. Behavioral Contrast: In discrimination tasks, the increased responding to the SDanddecreased responding to the Sis calledbehavioral contrast. Flahertysuggested that two types of contrast occur.Local contrastisactivated by thesudden change in stimulation (SD to Sor vice versa) andisdue to a temporaryemotional change (elation or frustration).Sustained contrast (sometimescalledanticipatory contrast) occurs when animalsanticipate an impendingchange to more or less preferred reinforcement contingencies.


26/32

C. OccasionSetting1. The Occasion-Setting Function ofa ConditionedStimulus

The ability ofa CS to enhance responding to asecondstimulusis calledoccasion setting because it can facilitate the effectiveness of thesecondstimulus'sability to elicit a conditioned response.This facilitationeffect (occasion setting) apparently is not due to the excitatory

properties of the CS.That is,it is not due simply to the ability of theoccasion setter to elicit a conditioned response.Instead, the facilitatingor occasion-setting property of the CSisseparate from conditionedexcitation.To have astimulus function asan occasion setter,it mustprecede asecond CS that is then paired with an UCS.

2. The Occasion-Setting Function ofa Discriminative StimulusRescorlaand colleagues (1988) have shown that an occasion setting CScan raise the level of operant responding. Moreover,Pavlovian occasionsettersand operant discriminative stimuliare interchangeable,suggesting a common mechanism ofaction.

3. ConditionedStimuliandOperant/Instrumental BehaviorCSsappear to exert mediational control over instrumental performance.RescorlaandSolomon (1967) proposed that the mediational effects ofCSsare tied to their ability to arouse central motivational states.Research hasindicated that a CSassociated with foodarouses theappetitive motivational state but inhibits the aversive motivational state.Also,a CSassociated with a painful event arouses the aversivemotivational state while inhibiting the appetitive motivational state.Additionally,a CS predicting the absence ofa rewardinhibits theappetitive motivational state but activates the aversive motivationalstate. Finally,a CS predicting the absence ofa painful event inhibits theaversive motivational state but excites the appetitive motivational state.

D. The Nature of Discrimination Learning: How do we discriminate the SD from theS?

1. The Hull-SpenceTheoryThe Hull-Spence theory of discrimination learning providesanassociative description ofsome essential features ofdiscriminationlearning.

2. Development of Conditioned Excitation andInhibitionAccording to the Hull-Spence view,discrimination learning developsinthree stages. First, conditioned excitation develops to the SDas the

result of reinforcement.Second, conditionedinhibition develops to theSas the result of non-reinforcement. Finally, conditioned excitation andinhibition have generalization gradients.The algebraic summation ofexcitatory andinhibitory strength determines the response rate to teststimuli.

3. ThePeakShiftPhenomenonThe Hull-Spence theory predicts the peak shift phenomenon whichrefers to the development of maximum responding to astimulus other


27/32

than the SD that isin adirection opposite to the S.Experimentalsupportexistsforthepeakshifteffect.

4. TheAversiveCharacter ofSTerrace (1964) claims that exposure to the Sproducesan unpleasant

frustration emotion that has the effect ofincreasing the response rate toother stimuli.

5. TheTranspositionEffectKohler developedan alternative view to the Hull-Spence theory.According to Kohler,stimuliare evaluatedin relative, not absolute,terms. Kohler'sview predicts the transposition effect which proposesthat subjects learn the relationships between stimuliin discriminationtraining. Kohler's relational view issupportedin discrimination tasksinwhich the subject must choose between two presentedstimuli.However,the Hull-Spence theory isan adequate explanation when subjectsaretested with only one stimulusat a time.

6. ErrorlessDiscrimination TrainingUnder certain experimental arrangements,adiscrimination can belearned with few or no errors.a result known aserrorlessdiscrimination learning.

7. Training ProcedureErrorlessdiscrimination training wasdemonstrated by Terrace (1963)whose pigeons peckedat a redilluminated key (SD) to receive foodreinforcement. When the pigeons received early and progressiveintroduction to the S(adark key lasting ashort period of time), theylearned to ignore the Seven when it reachedits full effect.

8. Application: Errorless Discrimination Training in HumansErrorlessdiscrimination training hasalso been observed with humansusing the fading technique.

9. Nonaversive SThe behavioral characteristics found with standarddiscrimination trainingare not observed with errorlessdiscrimination training.The peak shiftphenomenon is not foundin errorlessdiscrimination training suggestingthat the Shas not acquiredaversive properties.

10. Sutherlandand Mackintosh'sAttentional ViewSutherlandand Mackintosh developedan attentional view ofdiscrimination training that operatesin two stages.In the first stage, thesubject'sattention to the relevant stimulusdimension isstrengthened.Inthe secondstage, the relevant stimulusisassociated with a response.

11. The Recognition of the Relevant Dimension


28/32

The Sutherland-Mackintosh attentional theory claims that eachstimulus can activate an analyzer that actively scans the environmentsearching for the relevant aspects ofastimulus configuration.Once aparticular analyzer has been aroused, the output from the analyzer isassociated to a particular response.

12. PredictiveValue of DiscriminativeStimuliResults by Wagner andassociates (1968) suggest that the relativepredictiveness of the SD, not the percentage of trials on which the SDisassociated with reinforcement,determinesitsability to control behavior.

13. Continuity versus DiscontinuityThe Hull-Spence view claims that excitation andinhibition graduallyincrease during the acquisition ofadiscrimination.This position is calledthe continuity theory of discrimination learning. Krechevsky (1932)and Lashley (1929) presentedan alternative view that denies thatdiscrimination learning is gradually acquired.Instead,subjectsarepresumed to form hypothesesabout the relevant dimension that predicts

reinforcement.The Krechevsky and Lashley view predicts thatdiscrimination performance is exceptional once the relevant stimulusdimension isdiscovered.Thus, their view has been referred to thenoncontinuity theory of discrimination learning.

ChapterOutline

I. CONTEMPORARY DIRECTIONS FOR LEARNINGTHEORIESSince the 1960's, theories of learning have focused more on specific aspects of learningthan on learning in general.Thisshift in perspective has occurred because the olderglobal theories (1) concentrated on instrumental behavior,assuming that Pavlovianprocesses were simple and rare, (2) assumed that learning processes were the same for

all species,and (3) attempted to reduce all learning processes to simple,automaticassociative mechanisms. Modern theoriesshow that (1) Pavlovian processesare moreimportant and complicated than previously thought, (2) not all species learn in the samefashion,and (3) cognitive explanations of behavior can be more useful than previouslybelieved.

II. THEORIESOF PAVLOVIAN CONDITIONINGA. The Nature of the Conditioned Response: Recent research has tried to clarify the

nature ofPavlovian conditioning and has reexamined the assumptions ofPavlovconcerning the conditioning process.

1. TheStimulus-SubstitutionTheoryPavlov'sview that the pairing of the CSand the UCSallows the CS to

elicit a CR that wassimilar in form to the UCR.Theoretically, the CSexcites the neural centers for the UCS which then triggers the occurrenceof reflexive behavior.Thisisknown as thestimulus-substitutiontheory.

2. The Conditioning ofan Opponent ResponseThe opponent processview attempts to state the conditions when the CRand the UCR are not the same response. A clear example showing thatthe CR and the UCR are opposite reactions comes from the conditioning


29/32

ofdrug reactions.

The phenomenon ofdrug tolerance shows that the CR and the UCR canbe different in character.Tolerance isa reduction in analgesia withrepeated exposure to drugssuch as morphine.Siegel suggests that theconditioning ofan opponent response, hyperalgesia, contributes to thedevelopment of tolerance.

3. SometimesOpponent-Process (SOP) TheoryWagner'ssometimes-opponent process (SOP) theoryattempts toexplain when the CR issimilar to anddifferent from the UCR. Accordingto Wagner, the UCS elicits two unconditioned responses, called A1 andA2.The A1 component hasa rapid onset and offset in contrast to A2which hasaslow onset and offset.

4. The Importance of the Nature of the A2 ResponseWagner claims that conditioning only occurs to the A2 component.Therefore, when the A1 and A2 componentsare similar, the CR and theUCR are similar. When the A1 and A2 componentsare different, the CRand the UCR are different.The conditioned emotional reactionisanexample where the A1 and A2 componentsare different and the A2component becomes conditioned to the CS. Research on hypoalgesicresponsesare also relevant.

Some neural/anatomical evidence for the SOP theory comes fromeyeblink conditioning in rabbits.

5. Backward Conditioning ofan Excitatory CRExcitatory conditioning may occur with a backward paradigm when theCSimmediately precedes the A2 response.

6. Problemswith SOPTheorySOP theory predicts that all A2 response measuresshould follow thesame empirical laws of conditioning.Thisdoes not always occur whichhas led to a revision in the SOP theory.

7. Affective Extension ofSOP, or AESOPWagner proposeda revised theory, the affective extension of SOPtheory (AESOP) that assumes that the A1 and A2 components eachelicit separate sensory andaffective unconditioned response sequences.The parameters of the conditioning situation determine whether thesensory or emotive reaction predominates.

B. The Nature of the Pavlovian Conditioning Process: Several theoriesattempt toexplain the importance of predictiveness of CSsin Pavlovian conditioning.

1. Rescorla-Wagner AssociativeModelThe Rescorla-Wagner associative model of conditioning is basedupon four assumptions that refer to the process by which the CSandUCS gain associative strength, V: (1) a particular US can only support a


30/32

specific level of conditioning, l, (2) associative strength increases witheach reinforced trial, but depends upon prior conditioning, (3) particularCSsand UCSs can support different rates of conditioning, K,and (4)when two or more stimuliare paired with the UCS, the stimuli competefor the associative strength available for conditioning.The model isbased on an important equation

VA = K( - VAX)

2. An Evaluation of the Rescorla-Wagner ModelThe Rescorla-Wagner model accurately describes many research findingsin Pavlovian conditioning,including for example the UCS preexposureeffect.Preexposure to UCS without a CSimpairssubsequent CS-UCSconditioning.

3. Problems with the Rescorla-Wagner Modela. The Potentiation ofa Conditioned Response

When two CSsare presentedsimultaneously andare followed bya UCS, the more salient CSis more strongly conditioned than thelesssalient CS,a phenomenon calledovershadowing.TheRescorla-Wagner model accounts for overshadowing well.However,in some cases, the opposite outcome is observed withthe salient CSactually enhancing the conditioning of the lesssalient CS,a phenomenon known aspotentiation.TheRescorla-Wagner model predicts that the salient cue shouldreduce the conditioning of the lesssalient cue.

b. The CSPreexposureEffectJust as preexposure to the UCS retardsubsequent CS-UCSconditioning,so also does CS preexposure reduce later

conditioning with the CS,a phenomenon known as the CSpreexposure effect.The Rescorla-Wagner model does notreadily account for this effect.

c. TheCueDeflationEffectThe cue deflation effectsometimes occursin asituation wheretwo simultaneous CSs ofdifferent salience are paired with anUCS.The extinction of responding to the more salient(overshadowing) CSsometimes producesincreased CR strengthto the lesssalient CS.The Rescorla-Wagner theory cannotaccount for the cue deflation effect.

4. The Importance of Within-Compound AssociationsMore recently, Rescorlaandassociates have proposed that within-compound associationsare formed between the CS elements.Theassumption that within-compoundassociationsare formedduringconditioning helps the Rescorla-Wagner model to explain phenomenasuch as potentiation and the cue-deflation effect.


31/32

5. A Comparator Theory ofPavlovian ConditioningThe comparator theory maintains that the ability ofa particular stimulusto elicit a CR isdependent upon a comparison of the level of conditioningto that stimulusand other stimuli pairedin compound with the UCS.

6.

Mackintosh'sAttentional View

Mackintosh's attentional viewsuggests that animalsseek informationfrom their environment that predicts the occurrence of biologicallyrelevant events.Thusanimals play an active role in the conditioningprocess.Therefore, the presentation ofa CS without the UCS produceslearnedirrelevance indicating that the CS carries no informational value.Therefore,attentionis removed fromthe CS.

7. TheRetrospectiveProcessing ViewAccording to Baker and Mercier'sretrospective processing theory,animals constantly assess the contingenciesamong different eventsintheir environment and compare those experiences with priorexperiences.This continual activity means that new information receivedcan change the meaning of previously established CSs. Backwardblocking isan example of research supporting the retrospective view.Inbackward blocking, two CSsare simultaneously paired with an UCS.Then one of those CSs continues to be paired with the UCS. Finally,when the other CSis tested,itsassociative strength is reduced(backward blocking) leading to a weakening of conditioning to the otherCS.

III. THEORIESOF OPERANT/INSTRUMENTAL CONDITIONINGA. The Nature of Reinforcement: Although Skinner suggested that theories of

reinforcement are impediments to scientific progress,several empirical andtheoretical explanations have been offered for how reinforcement strengthens

the behavior it follows.1. Premack'sProbability-DifferentialTheory

Premack'sprobability-differential theory claims that an activity willhave reinforcing properties when its probability of occurrence is greaterthan that of the behavior it isintended to reinforce.

2. Application: The Use of Activity ReinforcersThe use ofactivitiesas reinforcers,such asin educational and businesssituations, has been successful.

3. Response DeprivationTheoryTimberlake and Allison proposedaresponse deprivation theory thatsays that when an animal's normal response rate (e.g., eating food) isrestricted (e.g., by fooddeprivation), that behavior becomes morepreferredand therefore reinforcing.

4. BehavioralAllocationThis proposal claims that when an animal'sability to emit responsesis


32/32

restricted,it will emit the minimal number of contingent responsesinorder to obtain the maximum number of reinforcing activities,aphenomenon calledblisspoint.Thisbehavioral allocationview isstudiedin situations which allow the animal unrestricted performance oftwo behaviors.

5. ChoiceBehaviorAt times,an individual can choose from two or more simultaneouslyavailable reinforcement contingencies.Herrnstein'smatching lawpredicts that responding to each contingency will be proportional to thereinforcement available on each schedule.The matching law is based oneconomic principles that assume animalsact asif they computebehavioral costs with the probability of reward benefits.This "law"assumes that animals will match the proportion of their responsesineach of two situations to the probability of rewardin those situations.Onthe other hand, the maximizing lawassumes that animals will behavein such a way to achieve the maximum number/rate of rewards.Otherhypothesessupport amomentary maximization theory that considershow animals perceive their temporary alternatives for reinforcement,and

the delay-reduction theory which suggests that animalsare sensitiveto different delays of reinforcement in choice situations.

Chapter Outline Klein

Documents

Transcript of Chapter Outline Klein