Processing of Expected and Unexpected Events During...

Psychological Review1982, Vol. 89, No.5, 529-572 Copyright 1982 by the American Psychological Association, Inc.

0033-295X/82/8905-O529S00.75

Processing of Expected and Unexpected Events DuringConditioning and Attention: A Psychophysiological Theory

Stephen GrossbergDepartment of Mathematics, Boston University

Some recent formal models of Pavlovian 'and instrumental conditioning containinternal paradoxes that restrict their predictive power. These paradoxes can betraced to an inadequate formulation of how mechanisms of short-term memoryand long-term memory work together to control the shifting balance between theprocessing of expected and unexpected events. Once this formulation is strength-ened, a unified processing framework is suggested wherein attentional and ori-enting subsystems coexist ina complementary relationship that controls the adap-tive self;organization of internal representations in response to expected andunexpected events. In this framework, conditioning and attentional constructscan be more directly validated by interdisciplinary paradigms in which seeminglydisparate phenomena can be shown to share similar physiological and phar-macological mechanisms. A model of cholinergic-catecholaminergic interactionssuggests how drive, reinforcer, and arousal inputs regulate motivational baseline,hysteresis, and rebound, with the hippocampus as a final common path. Ex-tinction, conditioned emotional responses, conditioned avoidance responses, sec-ondary conditioning, and inverted U effects also occur. A similar design in sensoryand cognitive representations suggests how short-term memory reset and atten-tional resonance occur and are related to evoked potentials such as N200, P300,and contingent negative variation (CNV). Competitive feedback properties suchas pattern matching, contrast enhancement, and normalization of short-termmemory patterns make possible the hypothesis testing procedures that search forand define new internal representations in response to unexpected events. Long-term memory traces regulate adaptive filtering, expectancy learning, conditionedreinforcer learning, incentive m'otivational learning, and habit learning. Whenthese mechanisms act together, conditioning phenomena such as overshadowing,unblocking, latent inhibition, overexpectation, and behavioral contrast emerge.

Mackintosh (1971), Rescorla and Wagner(1972), and Wagner and Rescorla (1972).The great heuristic value of these articlesstimulated new developments in such articlesas those of Dickinson, Hall, and Mackintosh(1976), Frey and Sears (1978), Hall andPearce (1979), Mackintosh (1976), Mackin-tosh, Bygrave, and Picton (1977), Mackin-tosh and Reese (1979), Sutton and Barto(1981), and Wagner (1976, 1978). The otherstream is found in a series of my own articles(Grossberg, 1968, 1969a, 1969c, 1971, 1972a,1972b, 1974, 1975, 1976a, 1976b, 1978a,1978b, 1980, 1981b, 1982a, 1982b).

This is a good time to make this compar-ison because ideas from the two streams havegradually converged over the years. Oncetheir remaining differences are resolved, bothstreams may be merged into a theoreticalframework wherein conditioning, cognitive,

Internal Problems of SomeConditioning Models

1. Merging Parallel Streams of Theory onConditioning and Attenrion

This article compares and contrasts twoparallel streams of theoretical progress in theconditioning and attention literature since1968, using the article of Pearce and Hall(1980) as a basis for discussion. One streamwas energized by such seminal articles asthose of Estes (1969), Kamin (1968, 1969),

The preparation of this article was supported in partby the National Science Foundation (NSF 1ST -80-00257)and the Air Force Office of Scientific Research (AFOSR82-0148).

Requests for reprints should be sent to Stephen Gross-berg, Department of Mathematics, Boston University,264 Bay State Road, Boston, Massachusetts 02215.

529

530 STEPHEN GROSSBERG

motivational, psychophysiological, and phar-macological data can be discussed in a uni-fied fashion. In this framework, theoreticalalternatives and predictions can be studiedusing interdisciplinary paradigms that canprobe interactions that are opaque to moreconventional experiments. Some experi-ments of this ~ype will be summarized below.

2. The Processing of Expected andUnexpected Events in Short-Term Memoryand Long-Term Memory

I suggest that various difficulties faced bythe first stream are due to the fact that it doesnot adequately probe how mechanisms ofshort-term memory (STM) and long-termmemory (L TM) influence the shifting bal-ance between the processing of expected andunexpected events. These difficulties takeseveral related forms: (a) Internal paradoxesexist within the theories. (b) No one theorycan explain all the relevant data. In fact, noone theory can explain all the data explicableby any of the other theories. (c) The theoriesprovide formal, as opposed to physical, mod-els of the data. These models have no veri-fiable properties outside of the conditioningexperiments they are constructed to explain.When some of these formal' properties areinterpreted as physical mechanisms, they arefound to be either paradoxical or to have noexternal experimental support.

To clarify these assertions I will reviewconcepts from Pearce and Hall (1980) as asource for this first stream. To illustrate howmy approach overcomes these difficulties, Iwill review concepts from Grossberg (1980)as a source for the second stream, althoughthe main concepts and mechanisms that Iwill need appeared in Grossberg (1971, 1972a,1972b, 1975).

access to the processor only when it. ..hasbeen followed by a surprising event" (p. 540).One consequence of this position is that anunconditioned stimulus (US) that is an ex-cellent predictor of food will not be processedeven if no conditioned stimulus (CS) is pres-ent. Despite this implication, Pearce and Hallstate that "stimuli such as the USs used intypical conditioning procedures are alwayslikely to gain access to the processor" (p.538). Pearce and Hall need this assumptionbecause "conjoint processing of the CS andUS representations results. ..in an increasein the ability of the CS ~o excite what we maycall a 'US memory'" (p 542). This is themain conditioning event of their theory.

In an effort to embed their hypotheticalprocessor into a broader theoretical perspec-tive, Pearce and Hall analogize the processorto the limited-capacity STM system of hu-man information-processing models. Giventhis processing interpretation, the Pearce andHall model simultaneously implies that anexpected US will not be stored in STM be,.cause it is expected and will be stored in STMbecause it is a US. One might try to escapethis contradiction by claiming that the pro-cessor somehow knows the difference be-tween a CS and a US. Even if one couldovercome the problem of showing how theprocessor knows this difference, one wouldthen be faced by the harder problem of show-ing how the processor changes its mind abouta cue when the cue switches from CSto USstatus as a result of prior conditioning, andis thereupon used as a US in a secondaryconditioning paradigm.

One can summarize this internal contra-diction within the Pearce and Hall theory bysaying that these authors have emphasizedthe processing of events that have unexpectedconsequences at the cost of implying para-doxes about the processing of events thathave expected consequences.

Mackintosh (1975) developed a theorythat emphasizes the processing of events thathave expected consequences at the cost offalling into difficulties when explaining theprocessing of unexpected events. Pearce andHall (1980) summarized Mackintosh's posi-tion as follows: "He suggested that the as-sociability (a) of a stimulus will increase ifit predicts reinforcement more accurately

3. Some Internal Paradoxes

First I will review an internal paradox thatlies at the heart of the Pearce and Hall (1980)theory. Pearce and Hall assert that "stimulithat fully predict their consequences will bedenied access to the processor. ...A stim-ulus is likely to be processed tq the extentthat it is not an accurate predictor of its con-sequences" (p. 538). Or, "a stimulus will gain

CONDITIONING AND ATTENTION 531

than other stimuli present in the situationbut will decrease if it predicts reinforcementless accurately" (p. 536). This hypothesis wasmade to explain how conditioning of a CueX is blocked if X is presented on compoundtrials AX after prior conditioning of Cue Ahas occurred. Mackintosh's hypothesis ex-plains blocking by claiming that X does notcondition well because A is a better predictorof the US due to its prior conditioning trials.

Mackintosh's hypothesis is, however, in-compatible with the fact that the Cue X con-ditions normally on the first compound trial(Kamin, 1968; Mackintosh et al., 1977; Res-corIa & Wagner, 1972). This experimentalresult contradicts the hypothesis because onthe first compound trial, Cue X is a worsepredictor of the US than it is on the secondcompound trial. Why does the Cue X con-dition normally on the first trial if on thistrial it is the worst possible predictor of theUS, h~ving never before been correlated withthe US?

To escape this contra.diction Mackintoshsimply assumes that only the intensity of X,not its predictability, influences its associa-bility on the first compound trial. Whetherone considers this an internal paradox of histheory or an ad hoc restatement of the datais a matter of taste.

Despite this difficulty, Pearce and Hall(1980) write that "the success of Mackin-tosh's model. ..convinces us that the prin-ciple it embodies-the modification of CSassociability as a result of the consequencesof one trial influencing conditioning on thenext-must be a part of any successful the-Ory" (p. 537).

Wagner (1976, 1978) has attempted to giveMackintosh's ad hoc assumption a physicalbasis by assuming "that the associability ofa CS is inversely related to the strength of anassociation between the CS and the context"(Pearce & Hall, 1980, p. 549). Pearce andHall (1980) criticize Wagner's concept bynoting that "it is very difficult to see how asurprising shock omission. ..or shock in-crease. ..after a CS can reduce the strengthof the association between the CS.and thecontext and thus restore associability" (p.549). Their criticism does not distinguishbetween the processing of an event that isunexpected within a given context and the

processing of an event that is followed by anunexpected US. I will suggest below howsuch distinctions tend to be blurred withinthese formal models and how they can beclarified using a physically based model

4. The Needjor Behaviorally UnobservableMechanisms

The seriousness of the dilemma into whichconditioning data have driven the formalmodels can be appreciated from the follow-ing considerations: Mackintosh (1975) saysthat events that have expected consequencesare processed, whereas Pearce and Hall (1980)say that events that have unexpected con-sequences are processed. Both viewpointsare, moreover, supported by unimpeachabledata. If the data support the idea that bothexpected and unexpected events are pro-cessed, then why have the formal theoriesavoided this conclusion?

The statement that both expected and un-expected events are processed can easily be-come predictively vacuous in a formal model,because such a model cannot easily distin-guish the sense in which expected and un-expected events are processed in differentways. The alternative conclusion-that ex-pected and unexpected events are processedin the same way-is dangerousiy close to say-ing that all events are processed in the sameway, which is patently false.

Conditioning data need a theory that canavoid these fatal pitfalls. Such a theory mustexplain the sense in which expected and un-expected events are processed by differentmechanisms. It must carefully delineate theproperties of these mechanisms to avoid be-coming vacuous or patently false. It mustshow how these properties can be empiricallytested. Because expected and unexpectedevents are all just events on the behaviorallyobservable level, such a theory needs to es-tablish a link with the behaviorally unob-servable structures within which these pro-cessing distinctions can be physically inter-preted and validated.

5. Causality Violation on the Behaviorally \

Observable Level

The step toward theories that invoke be"'haviorally unobservable processes in a sub-


stantive way runs against the grain for manypsychologists today. This is true despite thefact that as a body of psychological data be-comes more mature and quantitative, it bearsmore sharply on the behaviorally unobserv-able mechanisms that generate these data.

An important experiment that furtherdemonstrates that conditioning data havereached this level of maturity was conductedby Mackintosh et al. (1977) and is reviewedby Pearce and Hall (1980). I will indicatebelow that an interpretation of these datausing only behaviorally observable variablesviolates the causality of the conditioning pro-cess. To reject causality is tantamount todenying the very existence of a predictiveconditioning theory. For my purposes, I willsummarize only one aspect of this experi-ment.

In Part 1 of the experiment, all rats ex-perienced four trials on which a light (CS)was followed by a shock (US). In Part 2 ofthe experiment, two groups of rats receivedan additional single compound light-tonetrial. In one group (Group 1) the light-tonecompound was followed by a single shock.In the other group (Group 2), the light-tonecompound was followed by two successiveshocks that were presented 10 sec apart. Arecall trial with the tone alone showed essen-tially identical fear conditioning to the tonein both groups. In other words, the secondshock seems not to have affected tone con-ditioning.

The remarkable feature of the experimentbecomes apparent when one considers twoother groups of rats tested in Part 3 of theexperiment. One of these groups (Group 3)received the same training as Group 1 didplus an additional compound light-tone trialfollowed by a single shock before a recall trialwith the tone CS. The other group (Group4) received the same training as did Group2 plus an additional compound light-tonetrial followed by a single shock before a recalltrial with the tone CS. In other words, Part3 of the experiment ~imi>ly added an iden-tical learning manipulation onto the Group1 and 2 learning paradigms, which by them-selves elicited the same reaction to the tone.Remarkably, the tone exhibited better fearconditioning for Group 4 than for Group 3.

This is a fascinating experimental finding.

How can a test after identical second com-pound trials have different effects if tests afterdifferent first compound trials had identicaleffects? This experiment seems to violate cau-sality on the behaviorally observable leveland forces us to turn to behaviorally unob-servable mechanisms for an explanation.

6. Some Unpredicted Data

The internal contradictions within the for-mal models are associated with predictivelimitations. Pearce and Hall (1980) note thatthey cannot explain the following phenom-ena: (a) If two CSs differ markedly in theirintensity or salience, the more salient cue canovershadow the less salient cue, but not con-versely (p. 541; Mackintosh, 1976). (b) Over-shadowing effects can sometimes occur whenonly a single trial of compound conditioningis given (p. 541; Mackintosh, 1971; Mack-intosh & Reese, 1979). (c) The associabilityof a stimulus followed by surprising food re-mains high for future conditioning involvingfood but not for future conditioning involv-ing shock (p. 550; Dickinson & Mackintosh,1979). (d) Low associability of a stimulus canbe restored by presenting it in a novel context(p. 550; Dexter & Merrill, 1969; Lantz, 1973;Lubow, Rifkin, & Alek, 1976). (e) The oc-currence of a surprising event soon after aconditioning trial can influence learning onthat trial (p. 550; Kremer, 1979; Wagner,Rudy, & Whitlow, 1973). All of these phe-nomena can be explained by my theory. SeeSections 33-45.

7. Formal Versus: Physical Concepts: ASecond Type of LTM?

Formal models have an advantage overheuristic data analysis in that they commitone's thinking to' a more precise, and there-fore disconfirmable, set of concepts. Formalmodels also need to be compared with otherrelated concepts in the theoretical literatureto check their tenability as physical con-structs. The main conditioning equations ofthe Pearce and Hall (1980) theory lead to amajor qualitative conclusion when this com-parison is made, namely, that "associability"is controlled by a form ofLTM distinct fromthe L TM that is encoded by associativestrength.


Equations 4 and 5 for the following reason:On the A+B+ trials, VA and VB are each as-sumed to approach X even before compoundAB trials begin. Because, by definition, V 2:= VA + VB, this cannot happen because assoon as V2: ~ X, ~VA ~ 0 and ~VB ~ O. Inother words, the Wagner and Rescorla (1972)model, when consistently applied, is incon-sistent with the fact that more suppressioncan occur after A+B+trials than after AB+trials.

Rescorla and Wagner (1972) clearly in-tended that VB should be irrelevant whenonly Cue A is presented and that V A shouldbe irrelevant when only Cue B is presented.Such a concept is needed to eliminate theeffect of VB on the V A asymptote during Atrials, and of V A on the VB asymptote duringB trials. The switching on and off of anevent's momentary relevance occurs rapidlyon a trial-by-trial basis, whereas the growthof the associative strengths V A and VB isslowly varying across trials. Let us call thedistinction between the rapid modulation ofan event's activity and the slow changes inits associative strengths the difference be-tween STM and L TM.

Had Wagner and Rescorla (1972) explic-itly faced this processing implication of theirown data, they might have redefined V 2: as

V2: = SAV~ + SBVB, (7)

where SA and SB are the signals elicited bythe active STM representations of Cues Aand B, respectively. Then, If only A is pre-sented, SA = 0 and V2: depends only on VB,whereas if only B is presented, SB = 0 and V 2:depends only on VA.

Term SA V A in Equation 7 can be physi-cally interpreted as follows: The STM-acti-vated signal SA reads out the L TM trace V Avia a gating, or multiplicative, action SA VA.Such an L TM gating action appears promi-nently in my work. It is, for example, crucialin my approach to serial and paired-associateverbal learning (Grossberg, 1 969b; Grossberg&Pepe, 1971), free recall (Grossberg, 1978a,1978b), and cognitive development (Gross-berg, 1976a, 1976b, 1978b, 1980), as well asin my studies of conditioning and attention(Grossberg, 1968, 1969a, 1969b, 1969c, 1971,1972a, 1972b, 1974, 1975, 1976a, 1976b,1978a, 1978b, 1980, 1981b, 1982a, 1982b).

Once the modification in Equation 7 isaccepted, it becomes clear that the modelmust undergo a more major revision. Thisis true because the left-hand side and theright-hand side of Equation 7 are not di-mensionally the same. The right-hand sidecan fluctuate rapidly through time with SAand SB, whereas the left-hand side is a slowlyvarying associative strength. This observa-tion could have already been made about theterm A -.V 2; in Equations 4 and 5, becauseUS intensity A is a rapidly varying (STM)quantity, whereas V2; is a slowly varying(LTM) quantity. Replacing V2; by SAV A +SB VB in A -V 2; avoids the problem of mixingapples with oranges if we interpret A -SA VA -SB VB as the amount by which a com-monly shared STM representation is acti-vated by the combined effects of A and theLTM-gated signals S"V A and SBVB.

This commonly shared STM representa-tion cannot be the separate representationsof either Cue A or Cue B. Moreover, acti-vationofthis new representation depends onthe choice of reinforcer, because an associ-ative strength learned with respect to a shockis not the same as an associative strengthlearned with respect to' a food reinforcer.Even within the Rescorla-Wagner model, V2;feels the influence of a particular US's inten-sity A. In my theory this new type of STMrepresentation is called a drive representa-tion.

An internal analysis of the Rescorla-Wag-ner equation has hereby distinguished sen-sory representations for cues such' as A andB from drive representations correspondingto distinct reinforcing actions such as food,fear; and sex. Because each Cue A might beconditioned to any of several drive represen-tations, we need to study how the pattern ofL TM associative strengths Vjj, leadi:ng fromthe ith sensory representation to the jth driverepresentation, evolves through time. Oncewe accept the fact that the ith sensory rep-resentation can read out a pattern of LTM-gated signals Sj V ij across several drive rep-resentations (indexed by i), we need todiscover an STM decision rule whereby in-compatible drive representations can gener-ate consistent observable behavior. We alsoneed to discover a law for the selective changeof Vij due to the STM signal Sj of the, ith CSand the intensity A; of the ith US.


9. Secondary Conditioning Implies ThatCue and Drive Representations Are Distinct

/

Now that we have in mind sets not onlyof CSs but also of USs, we can use the factthat prior conditioning can transform a CSinto the US of a later secondary conditioningexperiment to constrain this law. In partic-ular, the asymmetric role of US intensity Ajand of CS intensity Sj in the modified Res-corIa-Wagner equation,

~Vij = ajj(Aj -L SkVkfl, (8)k

shows that this equation cannot be strictlycorrect. I suggest that this problem of theRescorla-Wagner framework is the reasonwhy Pearce and Hall need to assume that twotypes of LTM exist (Section 33). Equation8 also includes the Widrow-Hoffequation onwhich Sutton and Barto (1981) build.

The same argument shows that a second-ary US representation is not a drive repre-sentation, because a CS representation is nota drive representation. Hence, conditioningfrom a CS to a drive representation is not thesame process as conditioning from a CS toa US representation. This conclusion runscounter to the Pearce and Hall (1980) asser-tion "that the amount of learning is deter-mined by the amount of simultaneous pro-cessing that the representations of the CS andUS receive in the processor" (p. 550).

In light of the above argument, it is notclear what "the processor" might physicallyrepresent, because CS representations anddrive representations are qualitatively dis-tinct concepts. In fact, Pearce and Hall(1980) note their model's inability to explainthe Dickinson and Mackintosh (1979) dataon selective effects of distinct reinforcers onassociability and go on to say, "One way forour model to accommodate this result is topropose that there are separate processors f01learning about different reinforcers such asfood and shock" (p. 5.50). A large body ofdata other than that of Dickinson and Mack-intosh (1979) also suggests such a concept.My articles (Grossberg, 1971, 1972a, 1972b,1975, 1982a) review some of these data inlight of the drive representation concept.

This type of internal analysis of the Res-corIa-Wagner framework can be continued,but the breakdown of Equation 8 indicates

that some new theoretical principles areneeded to go much further. The above theo-retical exercise nonetheless clarifies my con-tention that the demand for explicit process-ing descriptions-within any theoreticalframework-rapidly leads either to impor-tant new concepts or to unforeseen contra-dictions.

Some GeneralPsychophysiological Concepts

10. An Alternative Processing Framework:Complementary Attentional and OrientingSubsystems

Having summarized some difficulties ofone theoretical stre~m, I will compare thetwo streams-notably their explanations ofexpectancies, extinction, and STM prim-ing-after my review of the second streamis complete. To start the review, I will sketchin broad strokes the general framework ofmy theory. Then I will review in more preciseterms the several design principles and mech-anisms that I need to quantify this frame-work.

In my theory an interaction between twofunctionally complementary subsystems isneeded to process expected and unexpectedevents (Grossberg, 1975). A precursor of thisconcept is developed in the distinguishedpsychophysiological article of Routtenberg(1968) on the "two-arousal hypothesis." Myconception of these two subsystems will beseen to deviate from Routtenberg's view inseveral basic ways (Section 27).

Expected events are processed within aconsummatory, or attentional, subsystem.This subsystem establisheS ever more preciseinternal representations of and responses toexpected cues. It alSQ builds up the learnedexpectations that are used to characterizeoperationally the sense in which expectedcues are expected. The attentional subsystemis, however, incapable of adapting to unex-pected environmental changes. Left to itsown devices, it would elicit ever more rigid,even perseverative, reactions to the environ-ment, much as hippocampectomized rats donot orient to a novel stimulus while they areindulging in consummatory activity, such asrunning toward a reward. Such rats cannot"shift attention during the presentation of anovel stimulus or in a mismatch situation"


(O'Keefe & Nadel, 1978, p. 250). The secondsubsystem is an orienting subsystem thatovercomes the rigidity of the attentional sub-system when unexpected events occur andenables the attentional subsystem to adaptto new reinforcement and expectational con-tingencies.

Part of the difficulty in understanding con-ditioning and attentional data is due to thefact that these two subsystems interact in asubtle fashion. I will review in the followingsections how both expected and unexpectedevents start to be processed by the attentionalsubsystem. When an unexpected event mis-matches an active expectancy within thissubsystem; the orienting subsystem is disin-hibited. The orienting subsystem acts to rap-idly reset STM within the attentional sub-system as it simultaneously energizes an ori-enting response.

By contrast, an expected event matches anactive expectancy within the attentional sub-system. This matching process amplifies theSTM activity patterns that are currently ac-tive within the attentional subsystem. Theseamplified, or resonant, STM activities inhibitthe orienting subsystem as they simulta-neously drive adaptive L TM changes, in-cluding the learning of new expectancies, in-ternal representations (chunks), and habits.

11. .The Stability-Plasticity Dilemma andEvoked Potential Correlates

The complementary attentional and ori-enting subsystems, indeed all the mecha-nisms that I will use, arise as the solution toa fundamental design problem concerningthe self-organization (e.g., development,learning) of new internal representations(Grossberg, 1976a, 1976b, 1978b, 1980,1982a, 1982b). I call this problem the sta-bility-plasticity dilemma.

The stability-,plasticity dilemma concernshow internal representations can mainta:inthemselves in a stable fashion against theerosive effects of behaviorally irrelevant en-vironmental fluctuations yet can nonethe-less adapt rapidly i~ response to environ-mental fluctuations that are crucial to sur-vival. How does a network as a whole knowthe difference between behaviorally irrele-vant and relevant events even though its in-

dividual cells, or nodes, do not possess thisknowledge? flow does a network transmutethis knowledge into the difference betweenslow and fast rates of adaptation, respec-tively? Classical examples of the stability-plasticity balance are found in the work ofHeld and his colleagues on rapid visual ad-aptation to discordant visuomotor data inadults (Held, 1961, 1967; Held & Hein;1963) and in the work of Wallach and hiscolleagues on rapid visual adaptation to dis-cordant cues for the kinetic depth effect andcues for retinal disparity (Wallach & Karsh,1963a, 1963b; Wallach, Moore, & Davidson,1963).

Because of the fundamental nature of thestability-plasticity dilemma, the mechanismsfrom which the two complementary subsys-temsare built have properties that imply psy-chophysiological, neurophysiological, andpharmacological predictions. For example,on the psychophysiological level, the disin-hibition of the orienting subsystem due to anexpectancy mismatch is suggested to corre-spond to the mismatch-negativity compo-nent of the N200 evoked potential complex.The STM reset in the attentional subsystemis suggested .to correspond to a P300 evokedpotential. The origin of the mismatch-nega-tivity component in the orienting subsystemand its role in generating a P300 suggests arelatiol:tship between the P300 and the ori-enting reaction. The resonant STM activitythat derives from an expectancy match in theattentional subsystem is suggested to corre-spond to the processing-negativity compo-nent oftheN200 evoked potential complex.

This psychophysiological interpretationleads to a number of interdisciplinary pre-dictions (Grossberg, 1982a). For example, inSection 35, I suggest that the tone on thesecond compound trial in Group 4 of theMackintosh, Bygrave and Picton (1977) ex-periment is more unexpected than the toneon the second compound trial in Group 3,and should therefore elicit a larger P300evoked potential.

Sections 12-25 review the concepts thatI use to mechanize the attentional and ori-enting subsystems. Then Sections 26-51 usethese concepts to explain conditioning andattentional data and to compare my theorywith the formal models.


12. Gated Dipoles

The gated dipole design is needed to resetSTM. In the present theory the term STMrefers collectively-to the suprathreshold ac-tivities of STM traces. An STM trace is com-puted at a network node where it equals theaverage potential of the cell, or cell popula-tion, that is represented by the node. AnSTM trace can passively decay at a node, butthe important operations in the theory trans-form these traces in ways other than by pas-sive decay. In particular, STM reset refers toa rapid change in STM, notably the rapidshutting off of activity at a subset of previ-ously active nodes and the rapid turning onof activity at a subset of previously inactivenodes. The STM activities at different typesof nodes represent different psychologicalprocesses. The gated dipoles to be discussedbelow are, for example, assumed to occurboth in cognitive and in motivational net-works. Activity of a gate4 dipole node in acognitive network may represent the occur-rence of a certain perceptual feature or a cer-tain temporal ordering of events in an ex-periment. Activity of a gated dipole node ina motivational network may, by contrast,measure the level of perceived fear or relief.The theory suggests that similar formal prop-erties obtain in gated dipoles wherever thesedipoles are placed in a network. The theoryalso suggests how to place these mechanismsin different types of networks and how tointerpret their formal properties in these dif-ferent contexts.

The gated dipole design shows how slowlyaccumulating'transmitter substances gate thesignals in parallel network pathways beforethese pathways compete to elicit net on-cellor off-cell STM responses (Figure 1). Theseresponses include a sustained on-response tocue onset and a transient antagonistic off-re-sponse, or rebound, to either cue offset or toarousal onset. The off-reactions drive theSTM reset.

One way to motivate the antagonistic re-bound concept is to ask, How does offset ofan event act as a cue for a learned response?For example, suppose that I wish to press alever in response to the offset of a light. Iflight offset simply turned off the cells thatcode for light being on, then there would be

,

l.l,,

no cells whose activity could selectively elicitthe lever-press response after the light wasturned off. Light offset must also selectivelyturn on cells that will transiently be activeafter the light is shut off; The activity of theseoff-cells (the cells that are turned on by lightoffset) can then activate the motor com-mands leading to the lever press. Let us callthe transient activation of the off-cell by cueoffset antagonistic rebound.

In a reinforcement context I claim thatsuch an antagonistic rebound is the basis fora relief reaction (Denny, 1971) upon offsetof a sustained fear-eliciting cue. In a percep-tual context I claim that such an antagonisticrebound is the basis for a negative aftereffectupon offset of a sustained image (Brown,1965, p. 483; Helmholtz, 1866, 1866/1962).

13. Antagonistic Rebound to Cue Offset

I will now describe a minimal model ca-pable of eliciting a sustained on-response toonset of a cue and a transient antagonisticrebound to offset of the cue. The intuitivepostulates that led to the model's originalderivation are given in Grossberg (1972b).An alternative derivation is given in Gross-berg (1980, Appendix E). An extended dis-cussion and mathematical analysis of thegated dipole is found in Grossberg (1981 b,1982a). Herein I will merely provide an in-tuitive description of a gated dipole.

Consider Figure 1. In Figure 1 (a), a non-specific arousal input I is delivered equallyto both the on-channel and the off-channel,whereas a test input J (e.g., light or shock)is delivered only to the on-channel. Theseinputs activate the potentials XI and X2,which create signals 81 and 82 in the on-chan-nel and off-channel respectively. BecauseI+J>I, XI>X2, and consequently, 81>82. What happens next is crucial.'

The square synapses are assumed to con-tain chemical transmitters 21 and 22, re-spectively. Each transmitter slowly accu-mulates to a target level. The slow accumu-lation rate is essential to the model'!properties. The target level is achieved by aconstant trans~ittet production rate that i!reduced by feedback inhibition proportionalto the transmitter concentration. When a sig-nal 81 reaches the synaptic knobs containin~


lU OFF Lil

TONICCELLS

Figure 1. Two examples of gated dipoles. In Panel a thephasic input J and the arousal input I add in the on-channel, thereby activating the short-term memory traceXI. The arousal input I also perturbs the short-termmemory trace X2 in the off-channel. Consequently, XI >X2. Then XI and X2 elicit signalsj(XJ andj(Xv in theirrespective pathways. Because XI > X2,j(XJ > j(Xv also.Each signal is gated (multiplied) by an excitatory trans-mitter Z. or Z2 (in the square synapses) before the gatedsignalsj(XJZI andj(XvZ2 activate their target cells. Theshort-term memory traces Xs and X6 then satisfy Xs >X6. Each short-term memory trace elicits an excitatorysignal down its own pathway, and an inhibitory signalto the other pathway. The net effect after competitiontakes place is an output from the on-channel. The textdescribes how a rapid offset of J triggers an antagonisticrebound that transiently excites the off-channel. In Panelb another version of a gated dipole is depicted. Hereeach excitatory gating pathway is replaced by a two-stagedisinhibitory pathway that is constructed from two suc-cessive inhibitory links. The cells that receive these trans-mitter signals are assumed to be tonic (internally andpersistently activated). The net effect of an input to thetwo-stage disinhibitory pathway is to release its outputcell from tonic inhibition and thereby excite it.

2" transmitter is released at a rate propor-tional to T, = 8,2" The multiplicative effectof 2, on 8, to yield T, is called transmittergating of the signal 8,. The gating law just

ON1 Xs Xe'

t~.{~l21

82

I X.'

IVI , TEST AROUSALL.L--1- INPUT INPUT

t (J) (1)

COMPETITION

GATE

SIGNAL

(a)

Bt

""+

"(\I)

says that 8\ and 2. interact via mass actionto elicit T\.ln particular, if either 8\ = 0 or2. = 0, then T. = O.

One proves that if 8\ > 82, then T. > T2.That is, transmitter is released at a faster rateby larger signals. Consequently, potential X3exceeds potential X4. These potentials thenemit competing signals. Potential Xs wins thecompetition over X6 and emits output signalsthat are the on-reaction of the network.

So far everything seems quite elementary.Only now do we exploit the slow accumu-lation rate and the transmitter gating law toshow how a transient antagonistic reboundis generated by rapid offset of J.

The faster transmitter depletion rate in theon-channel than in the off-channel when Jis on implies that 2. < 22, despite the factthat 8\2. > 8222. When J is shut off, bothchannels receive the equal arousal input [.The potentials X. and X2 rapidly equalize,as do the signals 8. and 82. By contrast, theinequality 2\ < 22 persists because trans-mitter accumulates slowly. Thus, right afterJshuts off, 8.2\ < 8222, X3 < X4, and the off-channel wins the signal competition. An an-tagonistic rebound is thereby initiated.

The rebound is transient because the trans-mitters gradually respond to the equal signals[by reaching a common level 2\ = 22. Then8\2\ = 8222, and the competition shuts offthe rebound.

There exist many variations on the gateddipole theme. Figure 1 (b) points out-that theslow transmitters can be inhibitory trans-mitters within a two-synapse disinhibitorypathway rather than excitatory transmitterswithin a one-synapse pathway. I interpretdopamine or noradrenaline to be the slowinhibitory transmitters in motivational andcognitive dipoles. The other inhibitory trans-mitter is often interpreted as gamma ami-nobutyric acid (Groves, Young, & Wilson,1978). The disinhibitory concept rationalizesmany effects of drugs such as amphetamine,chlorpromazine, 6-hydroxydopamine, andmonoamine oxidase inhibitors on behavior(Grossberg, 1972b, 1982a). A single cell,rather than an intercellular network as inFigure 1, can also act like a gated dipole.Such a dipole is suggested to exist in verte-brate photoreceptors (Carpenter & Gross-berg, 1981) wherein calcium is suggested to


act as the gating chemical. A full understand-ing of the gated dipole concept requires thatwe be able to distinguish the varied anatom-ical substrates of gated dipoles from theircommonly shared functional properties.

14. Antagonistic Rebound to Arousal Onset

A surprising fact about gated dipoles is thata sudden arousal increment AI can triggeran antagonistic rebound despite the fact thatboth the on-channel and the off-channel re-ceive equal arousal inputs (Figure 2). Thismathematical property forced me to realizethat an unexpected event, by triggering aburst. of nonspecific arousal, could discon-firm anon-reaction by tapidly and selectivelyinhibiting it, thereby resetting STM.

I should be more precise about this prop-

J

I

t

ON-CELLACTIVITY

OFF-CELLACTIVITY

t

Figure 2. On and off responses of gated dipoles. After a gated dipole's transmitters equilibrate to an on-channel phasic input J, an antagonistic rebound or off-response can be generated by either rapidly shuttingoff the phasic input J or rapidly turning up the level of nonspecific arousal I. The latter type of reboundcan reset short-term memory in response to an unexpected event that triggers a momentary arousal burstfrom the orienting subsystem.

erty of arousal, because this precision has im-portant implications for my explanation ofovershadowing. The following remarkableproperty holds in the gated dipole of Figure}. (a) if the signals 81 and 82 are linear func-tions of their respective inputs. The off-re-bound size in response to a sustained inputJ and a sudden arousal increment of sizeill above the previous arousal level I is

ABJ(ill -A)Off = (A + 1+ J)(A +1) , (9)

where A and B are positive constants. Notethat a positive off-reaction occurs only ifill > A. This criterion is independent of J,which means that an arousal increment illthat is sufficiently large to rebound any dipolewill be large enough to rebol,lnd all dipoles


c--- 1~ node in p(2) is capable of sending feedbackvj2J J (2) signals to a subset of nodes acrossp(.I). Before

.these signals reach their target cells, they aregated by L TM traces that exist at the endsof the signal pathways (Figure 3). This gatingaction multiplies the signal and the LTMtrace, just like the gating action of the trans-mitters in a gated dipole. In more preciseterms, denote the feedback signal from thejth node Vj(2) in p(2) to the ith node VII) inp(l) by Sj. Denote the L TM trace in this feed-back pathway by Zft. Then the net signal re-ceived by vII) from Vj(2) is SjZft. All thesesignals add up to VII) to generate a total sig-nal

J'FigUre 3. Short-term memory activity at a populationJ'j(2) in F(2) releases a signal Sjo This signal is gated bythe long-term memory trace Zft on its way to a popu-lation Vp> in F(I). All these gated signals add at Vp> togenerate a total input E; = Lj Sj Zft. The pattern E =(E.o E2' ...0 Em) of feedback signals is an expectation.

Ej = L SjZft (10)j

at VII). The pattern

E = (EI, E2, ..., E'1I) (11)

of total feedback signals from F(2) to F(I) isidentified with an expectation. An expecta-tion is not an L TM trace or a family ofL TMtraces. It is not defined with respect to a singlecell. It is a feedback pattern derived fromLTM-gated signaling across the entire net-work. Because the signals Sj depend on themomentary STM activities of the nodes inF(2), the expectation can quickly change evenif there are no changes in the L TM traces.The signal Sj is an STM probe that reads outthe L TM pattern (Zjl, Zj2, ..., Zjm) as partof the expectation E.

The pattern E is called an expectation be-cause the L TM trace Zft can change whenthe signal Sf from Vj(2) and the STM activityXj(l) of vII are actIve long enough for theslowly varying L TM trace to respond tothem. When this happens, the pattern (Zjl,Zfl, ..., ~m) of L TM traces can .learn thepattern {XI I), X!I), ..., Xm(I» ofSTM activ-ities (Grossberg, 1967, 1 969c, 1 972c, 1974).As the STM pattern across F(I) is encodedby J'j(2),S L TM traces, it becomes the patternthat J'j(2) expects to find at F(I) when it isactive. Later STM activation of J'j(2) readsthis L TM pattern into the expectation E viathe gating action of the L TM traces Zft onthe signal Sj. This gated signal pattern equalsE only when J'j(2) is the only active node inF(2). When more than one node is activeacross P(2), E is a weighted average of the

in a field. More precisely, if the arousal in-crement is large enough, then all active di..poles will simultaneously be rebounded. Thisis because the size of the rebound is an in-creasing function of the on-input J andequals 0 if J = O. Thus, rebound size is se-lective despite the fact that all active dipolescan be rebounded at once. I identify acti-vation of the arousal source which resets theSTM of a dipole field with the mismatch-negativity component of the N200 evokedpotential complex. The STM reset event it-self is identified with the P300 evoked po-tential.

Given that a sudden burst of nonspecificarousal can selectively reset a field of on-cellsand off-cells, we need to consider how thisarousal level' is regulated. The theory expli-cates the idea that surprising events arearousing.

15. What Is an Expectation?

To discuss a surprising or unexpectedevent, we need to define what an expectationis and how it is computed. In my theory sev-erallevels of network processing interact tobuild up new internal representations. To fixideas, consider two successive levels F(I) andF(2) in this network hierarchy. Each active

541CONDITIONING AND ATTENTION

L TM patterns of all the active cells (Gross-berg, 1968, 1976a, 197(jb, 1980).

Neurophysiological evidence for the exis-tenceof such feedback expectancies, or tem-plates, can .be found in the distinguishedwork of Freeman (1975,1980,1981) on theolfactory system.16. Unexpected Events Trigger aMismatch-Modulated Arousal Burst

Having defined an expectation, I can nowmore easily describe how an unexpectedevent atp(l) triggers an arousal burst to p(2),

(8)

FEEDFORWARD INPUT

(bi

AROUSAL

MISMATCH

(=~==~:::)

lC) ld)

Figure 4. Mismatch-modulated arousal. In Panel a, a subliminal feedback expectancy E from F(2) to F(I)is maintained by short-term memory signaling from F(2). In Panel b, Ii feedforward input pattern U isalso registered at F(I) as it si~ultaneously activates the arousal branch a. In Panel c the mismatchbetween the two patterns across F(I) attenuates activity at this level. In Panel d inhibition at F(I) removesinhibition from F(I) to a, thereby permitting a to unleash an arousal burst across F(2). Had the feed-forward input m~tched the feedback expectancy, the amplified activity at F(I) would have inhibited the

arousal source a.

which thereupon resets STM across F(2) viaselective antagonistic rebounds.

At a moment when the feedback expec-tation E is active across F(I) (Figure 4 [aD,suppose that an external event causes a feed-forward input pattern U to be received byF(I) (Figure 4 [bD. Suppose that U mis-matches E (in a sense that is defined in Sec-tion 24). In my theory such a mismatchrapidly inhibits STM activity across F(I)(Figure 4 [cD. Attenuating the STM activityin F(I) eliminates the inhibitory signal thatF(I) delivers to the orienting, or arousal, sub-


system a when p(l) is active. Because a alsoreceives excitatory input due to the externalevent, the arousal subsystem a is disinhibitedand releases a burst of nonspecific arousalacross p(2) (Figure 4 [dJ).

This description of the disinhibition of aassumes that an external event excites boththe attentional and the orienting subsystems.but that the orienting subsystem can onlyrelease signals when a mismatch within theattentional subsystem occurs. An early pre-cursor of this idea is the Hebb (1955) hy-pothesis that every event has a cue and avigilance, or arousal, function, A more recentcorrelate is my interpretation of a as the gen-erator of the mismatch-negativity evokedpotential (Na8:tanen, Hukkanen, & Jarvi.lechto, 19,80). A subtle aspect of this inter-pretation is that the mismatch occurs in theattentional subsystem, whereas the disinhib-ited mismatch negativity is elicited in theorienting subsystem.

17. STM Reset Versus STM Resonance

Before explaining how a pattern mismatchacross p(J) can attenuate STM activity there,I need to discuss what happens if a patternmatch occurs across p(J). By definition, sucha match means that the external input thatcauses the feedforward pattern U is expectedwith respect to the feedback pattern E. Theeffect of this, match is to amplify the STMactivities of the matched pattern across p(J).Thus, a mismatch attenuates all activityacross P(J), whereas a match amplifies pat-terned STM activity across p(J). I call theamplification of STM activity in a feedbackmodule such as p(J) and p(2) an STM reso-nance (Grossberg, 1976b, 1978b, 1980).

Neurophysiological evidence for the exis-tence of STM resonance has been describedin ~he olfactory system by Freeman (1975,1980, 1981) and in the visual system by Sin-ger (1977, 1979). On a psychophysiologicallevel, I identify this amplification of STMactivity due to a pattern match with the pro-cessing-negativity or match-detector compo-nent of the N200 evoked potential complex(Grossberg, 1982a).

We have hereby been led to distinguishtwo functionally distinct actions of expectedand unexpected events. Expected events can

18. Chunking and Expectancy Learning

The concepts of STM reset and STM reso-nance become fully meaningful when theyare used to describe how the stability-plas-ticity dilemma is solved, notably how newinternal representations (chunks) and expec-tancies are learned and remembered. .

To complete this description, I assumethat F(I) can send feedforward signals to F(2).The feedback expectancy signals from F(2)to F(I) are thus part of a reciprocal exchangeof signals between successive levels in thenetwork hierarchy. Anatomical correlates ofthis reciprocal exchange are the reciprocalthalamocortical connections that seem tooccur in all thalamoneocortical systems(Macchi & Rinvik, 1976; Tsumoto, Creutz-feldt, & Legendy, 1976). Psychological cor-relates of this reciprocal exchange are theprocesses of recognition and recall that, whenregulated by STM reset operations, lead tor&cpid hypothesis testing, or search, throughassociative memory (Grossberg, 1978a,1978b).

The signals from F(I) to F(2) are gated byL TM traces before the signals reach their tar-

generate an STM resonance, whereas unex-pected events can trigger selective STM reset.A subtle aspect of these complementary STMtransactions is that both of them occur withinthe attentional subsystem, although only oneof them is mediated by the orienting subsys-tem. In other words, the organization of thebrain into structurally complementary sub-systems does not correspond in an obviousway to the Junctional complementarity in theprocessing of expected and unexpectedevents.

A deeper subtlety of this functional inter-action is implicit in the previous discussionand will be rendered explicit in Section 26.There I will conclude that the very STM res-onance that represents paying attention to anevent actively prepares the attentional sub-system to be reset by the orienting subsystem.An STM resonance -does this by selectivelydepleting, or habituating, the transmittergates in the active channels of the gated di-poles from which F(2) is constructed. If anarousal burst perturbs these dipoles, thenSTM will be, rapidly reset.


get cells. Then the gated signals are summedat each target cell, just as in Equation 10.The reader with some engineering back-ground will recognize that the transforma-tion from an output signal pattern S = (Sl,S2, ...Sm) to an input signal pattern T =(Tl' T2,..., Tn), where ~= LjSjZij,definesa linear filter. Because the L TM traces Zij canchange as a function of experience, this filteris called an adaptive filter. Both the adaptivefilter due to F(l) -F(2) feedforward signalingand the learned expectation due to F(2)-F(l) feedback signaling obey the same gatinglaws and laws of associative learning. Theirdifferent intuitive interpretations are due totheir different locations within the networkas a whole.

When learning occurs in the L TM tracesof the F(l) -F(2) pathways,1he same inputpattern U to F(l) will elicit a different STMpattern across F(2). Speaking intuitively, theinternal representation of U across F(2) haschanged. Just as an expectation is influencedby the F(2) -F(l) LTM traces, but is notidentical with these traces, an internal rep-resentation is influenced by the F(I) -F(2)L TM traces, but is not identical with thesetraces.

19; The Code of Adaptive Resonances

With these comments in hand, I can nowexpand the notions of STM reset and STMresonance to include the self-organizationprocess. If a feedback expectation E mis-matches a feedforward input pattern U, thenSTM reset occurs before the L TM traces inthe F(l) -F(2) pathways and the F(2) -F(l)pathways can change. Consequently, mis-matched or erroneous interpretations of theenvironment cannot cause adaptive changesin L TM. The L TM traces can gate presentlyactive signals prior to the reset event and canthereby alter network activity based on theirpast learning. However, the LTM traces areadaptively blind to present network activity,including the signals that they gate, becausethe L TM traces are slowly varying relativeto the rapid time scale of filtering, mismatch,and reset.

By contrast, if a feedback expectation Ematches a feedforward input pattern U, thenSTM is amplified across F(l). The signals

20. The Noise-Saturation Dilemma

Before turning to a consideration of con-ditioning data, we still need to understand

from,F(I) to F(2) are thereby amplified so thatSTM activity across F(2) is also amplified.Then the signals from F(2) to F(I) are am-plified, and the entire network locks into anSTM resonance. This global STM event de-fines the perceptual or attentive moment.

Resonant STM activities can be encodedin L TM because they ,are large enough andlast long enough for the L TM traces to re-spond to them. An STM resonance is thusa context-sensitive global interpretation ofthe input data that signifies that the networkas a whole considers this interpretation wor-thy of being adaptively incorporated into thenetwork's memory structure. I call the dy-namic process whereby L TM adapts to res-onant STM patterns an adaptive resonance.

Using the notion of adaptive .resonance,it is now easy to state what the perceptual orcognitive code of a network is, although thesimplicity of this statement hides numeroussubtleties and the need for much future sci-entific work. The code of a netW(>rk is the setof stable adaptive resonances that it can sup..port in respC5nse to a prescribed input envi-ronment.

James Gibson's lifelong ingenuity as a stu-dent of perceptual phenomena led him toconclude that the perceptual system "reso-nates to the invariant structure or is attunedto it" (Gibson, 1979, p. 249). Gibson hasbeen criticized for emphasizing the phenom-enal immediacy of perception at the cost ofunderemphasizing its processing substrates(Ullman, 1980). We can now understandthat Gibson's emphasis was based on a cor-rect intuition. The many processing steps,such as adaptive filtering, STM activation,readout offeedbackexpecta~cies, STM mis-match, disinhibition of orienting arousal,STM reset, and so forth, all involve perfectlygood neural potentials, signals, and trans-mitters. However they are not accessible toconsciousness. The conscious experience is,I suggest, the resonant or attentive moment,which seems to be immediate because it isa global event that energizes the system asa whole.


5. In Figure 5 (a), a differentiated pattern ofinputs Ii is poorly registered in the activities,or STM traces, Xi because the overall inputintensity is too small to override internal cel-lularnoise. In Figure 5 (b), all the inputs areproportionally amplified to escape the cells'noisy range without destroying relative inputimportance (as when the reflectances of apicture remain constant despite its illumi-nation at successively higher light intensi-ties). Because all cells have only finitely manyexcitable sites, the smallest inputs are nowlarge enough to turn on all the sites of theirreceptive cells; hence, the larger inputs mightnot be able to differentially excite their re-ceptive cells, because there might be no moresites to turn on in these cells. The input dif-ferences are thereby lost because the activitiessaturate as all the cell sites are activated.Thus, in a cellular tissue, sensitivity loss canoccur at both low and high input intensities.As the inputs! fluctuate between these ex-tremes, the possibility of accurately register-ing input patterns is imperiled.

I proved (Grossberg, 1973) that mass ac-tion competitive networks can automaticallyretune their sensitivity as inputs fluctuate toregister input differences without noise orsaturation contaminants. See Grossberg(1980, Appendices C and D) for a review. Ina neural context these systems are calledshunting on-center off-surround networks.Otherwise expressed, a network whose cellsobey membrane equations (not additiveequationsrand that interact via an anatomywith a narrow excitatory focus and broadlydistributed inhibition can automatically re-tune its sensitivity due to automatic gain con-trol by the inhibitory signals.

I.

I

X.I

(a)

\'-./A,-

(b) I

Figure 5. The noise-saturation dilemma. Panel a: At lowinput intensities, the input pattern (I" 12. ..., I,,) ispoorly registered in the short-term memory activity pat-tern (X" X2. ..., X,,) bec~use of internal noise in thecells. Panel b: At high input intensities, the input patternis poorly registered in the short-term memory activitypattern because all of the cells' finitely many excitablesites get saturated.

how a pattern mismatch attenuates STM ac-tivity, how a pattern match amplifies STMactivity, and how an STM resonance depletesthe on-cell transmitter gate in a gated dipole.All of these properties follow from a studyof competitive interactions between the cellsat each of the several levels P(I), p(2), ...,p(n) of the network hierarchy.

The need for competitive interactions fol~lows from a basic processing dilemma-thenoise-saturation dilemma-that is faced byall cellular tissues, not only nerve cells, andmust be solved before continuously fluc-tuating input patterns can be registered at all.

The noise-saturation dilemma is easy tostate because it is so basic. Consider Figure

21. STM Contrast Enhancement andNormalization: Hypothesis Testing andOvershadowing in a Limited-CapacitySystem

Because of the noise-saturation dilemma,co~petitive networks are ubiquitous wher-ever input patterns are accurately registeredand transformed by cells. If these input pat-terns also need to be stored beyond the offsettimes of the inputs, as in STM representa-tions that can remain active until later re-inforcements can act upon them to influence


VI.1

r

II

LTM encoding, then the competitive net-works must also be feedback networks (Fig-ure 6) whose positive feedback .loops can re-verberate the SiM activities after the inputscease.

Competitive feedback networks also existfor a deeper processing reason that is relatedto the stability-plasticity dilemma. Thesenetworks need to possess the properties of I 81

contrast enhancement and normalization tocarry out hypothesis testing operations lead-ing to the self-organization of new internalrepresentations after an u.nexpected event -(;I '-occurs. The property of contrast enhance- ."'--=- .-.::;~~. ~.ment is the capability of the network to at-tenuate small inputs and amplify large inputsbefore storing the contrast enhanced inputpattern in STM. The property of normali-zation is the tendency of the total supra-threshold STM activity across the network Ito be conserved tbrough time. I b 1

These properties enable the network tosolve a problem that could prevent it from Figure 6. Two types o~ ~ri1~titive net:-",orks. Panel ~:adaptively reacting to an unexpected event. A feedforw~rd .c~m~~ltlve net~ork dehv~rs both ex~-

.(I) tatory and inhibitory Inputs to Its receptive cells. ThisMIsmatch across F causes an S~M reset com~titive input distribution allows the cells to respondacross F(2). If the nodes that are actIvated by to input patterns of widely varying background intensityF(I) -F(2) signals are hereby inhibited, then without saturation. Panel b: A feedback com~titivehow does F(2) get activated at all by these ~etwork gener.ates excitatory and inhibitory fe~back. al d . h ..? signals among ItS own cells. When these feedback signals

sIgn s un~g t e next time mterval. Why are sigmoid, or S-shaped, functions of cell activity, thedoes the entire network no~ shut down? How network can contrast-enhance the input pattern beforedoes an unexpected input to F(l) ever get storing it in short-term memory. This contrast-enhance-encoded if a mismatch shuts down the whole ment pro~rty follows from the sigmoid signal's abilitysystem? to suppress, rather than amplify, noise through the net-

.work's excitatory feedback loops. The network also tends.The contrast-enhancement property sup- to conserve its total suprathreshold activity throul?;h

plIes part of the answer. Not all of the nodes time. This normalization pro~rty dynamically explainsin F(2) that receive inputs from F(I) have their the limited capacity of short-term memory as a conse-activities stored in STM. Only the nodes that quence of solving the noise-saturation dilemma.

receive relatively large inputs have their ac-tivities stored in STM. Only those nodeswhose activities get stored in STM are resetby the mismatch-modulated arousal burst.The nodes that receive smaller inputs are notreset because they did not reverberate inSTM. These latter nodes can still respond tothe signals from F(I) to F!2) in the next timeinterval.

The contrast-enhancement property thusshows how some input-activated nodes canbe spared by the STM reset process. How-ever, this property is not sufficient becausethese nodes, after all, receive such small in-puts that they could not previously rever-berate in STM. Why can they reverberate in

"'"

STM after the nodes that received large in-puts are inhibited by dipole rebounds?

The normalization property now comes tothe rescue. The total suprathreshold STMactivity tends to be conserved. Because thedipole-inhibited nodes can no longer com-pete for this conserved activity, the remain-ing input-excited nodes inherit it (Figure 7).Thus, the nodes that fared poorly in the com-petition for STM activity in the original fieldF(2) fare mucbbetter in the "renormalized"field wherein their strongest competitorshave been inhibited by dipole rebounds. Thesuccessive renormalization of the field F(2)by rapid reset events can be viewed as a type


-L~~:=~ ~ ~l (a)

(b)

(c)Figure 7. Renonnalization. The input pattern in Panela elicits short-tenD memory activity pattern X;(2) in Panelb by suppressing small inputs and contrast-enhancinglarge inputs. After Xf) is suppressed by dipole rebounds,the small input activities inherit nonnalized activityfrQm the suppressed populations to elicit the distinctshort-tenD memory activity pattern in Panelc.

of parallel hypothesis testing, or principal-component analysis, whereby the field zoomsin on nodes capable of generating an STMresonance.

The way in which STM normalization in-fluences the processing of unexpected eventsis relevant to my analysis of overshadowing.Because of STM normalization, increasingthe STM activity of one representation forcesa decrease in the STM activities of the rep-resentations with which the enhanced rep-resentation competes. Otherwise expressed,STM normalization provides a dynamicalexplanation of why STM is a limited capacitysystem.

It is important to realize tb,at althoughcompetitive feedback networks possess thenormalization property, competitive feedfor-ward networks do not. An input is small ina feedforward network because it has already

fared badly in the feedforward competitiveinteraction. The normalization property mustoccur after the stage of input delivery, notbefore or at this stage.

22. Overshadowing and Sigmoid SignalFunctions

The discussion of contrast enhancementand normalization indicates some propertiesthat are needed to accomplish hypothesistesting in response to an unexpected event.This discussion does not, however, show howthese properties are obtained or whetherother important properties coexist with them.We now need to review this issue in a moredetail~d fashion, because an important prop-erty that we need is still not available.

The need is clarified by considering a stan-dard overshadowing experiment (Kamin,1968, 1969). Suppose that an animal receivesa series of CER conditioning trials whereina Cue A (such as a light) is followed by astandard shock US. Let these trials be fol-lowed by a series of CER conditioning trialsto the compound cue AB (such as a light-tone combination) followed by the sameshock US. A later test of Cue B's ability tosuppress bar pressing for food shows that itsfearfulness has been blocked by the priorconditioning of Cue A to the shock. By con-trast, if the compound conditioning trials usea different level of shock as a US than wasused during the conditioning of Cue A, thenthe conditionability of Cue B is restored.

In the light of the previous theoretical dis-cussion, one might wish to say that the un-expected change of US at the onset of com-pound trials causes an STM reset that some-how amplifies Cue B's STM representationand thereby frees it from blocking. This wishis, however, incompatible with Equation 9,which shows that all active representationsare reset if any representation is reset. Is thepresent theory incompatible with basic factsof overshadowing?

At this point one must soberly face the factthat the theory would collapse without thebenefit of mathematics. Indeed, it is quiteimpossible tb understand the STM transfor-mations during conditioning experimentswithout a suitable mathematical tool.

Equation 9 was derived under the hypoth-esis that a linear signal function transmutes

CONDITIONING AND ATTENTION

the arouSal signal I and the test signal J intosignals SI and S2 to the dipole gates. I willnow say why a linear signal function cannever be used in a competitive feedback net-work within a perceptual or cognitive pro-cessor. In fact, a linear signal function doesnot imply the contrast-enhancement prop-erty that is needed to reset STM in responseto an unexpected event. The right kind ofsignal function gives all the properties thatare needed, including the overshadowingproperty. This signal function is a sigmoid,or S-shaped, signal function. Such a signalfunction acts like a threshold at low cell ac-tivities and levels off at high cell activities.

23. Noise Suppression and SelectiveEnhancement, or Dishabituation, ofOvershadowed Representations byUnexpected Events

A sigmoid signal function is needed for abasic processing reason. The positive feed-back signaling in competitive feedback net-works can be a mixed blessing if it is notproperly designed. Positive feedback can sub-serve such desirable properties as STM stor-age and normalization. It can also, if im-properly designed, flood the network withnoise generated by its own activity. Thisndise amplification property will have disas-trous effects on network processing wheneverit occurs, and it can occur informal network"seizures" ~nd "hallucinations" (Ellias &Grossberg, 1975; Grossberg, 1973; Kaczma-rek & Babloyantz, 1977; Schwartz, 1980).

A proper choice of signal function pre-vents noise amplification (Grossberg, 197.1).The simplest physically plausible signal func-tion that prevents noise amplification is the

-A -J(J+ J) + (A + J2f(2[A + (J + J)2]1!2g(J, J) --2J+J

due to transmitter depletion can be disha-bituated by an unexpected event. An elec-trode that is too coarse to distinguish betweenon-enhancements and off-rebounds near itsrecording site could record dishabituation ofthis type in response to an arousal incrementat all electrode placements near previouslyactive cells, whereas a more sensitive elec-

Because g(I, J) is a decreasing function of J,it is easier to rebound an intensely activateddipole (J ~ 0) than a weakly activated dipole(J~ 0).

If M < g(I,J), then the arousal incrementM can cause an enhanced on-reaction ratherthan an off-rebound. These properties illus-trate how the habituation of cell responses

sigmoid signal function. The opposite ofnoise amplification is noise suppression. Thisnoise suppression property attenuates smallinputs to the network. By normalization, itthereby amplifies large inputs to the network.A competitive feedback network's contrastenhancement property is thus a variant of itsnoise suppression capability.

We can now reanalyze the response of agated dipole to an arousal burst when a sig-moid signal, as opposed to a linear signal, isused. When this is done, one finds that. thesame arousal burst that rebounds the on-re-sponses of very active dipoles will enhancethe on-responses of weakly active dipoles. Inother words, overshadowed representationsin a dipole field can actually be enhanced bythe same surprising event that inhibits moresalient representations via antagonistic re-bound. That all the properties that are neededoccur automatically as a result of basic pro-cessing constraints like noise suppression iswhat I call a minor mathematical miracle.Minor mathematical miracles should not betaken lightly. They usually mean that the in-tuItive ideas that they reify contain a lot oftruth.

To illustrate this new rebound property,let the sigmoid signal function be f(w) =w2( 1 + w2)-I. Because we are interested in thesmallest, or threshold, increment M that cancause a rebound, we can approximate f(w)by w. Then, in the gated dipole of Figure 1,a rebound occurs to an arousal incrementM given a previous arousal level I and on-input .( only if

M>g(I,J), (12)

where the function g(I, J) is defined by


berg (1980 or 1981a) for mathematical de-tails. '

II x.

(

I181

M+l6L1=LIbl

~~6=M.ICI

Figure 8. Pattern matching. In Panel a the noise suppres-sion property converts a uniform (or zero spatial fre-quency) input pattern (IJ into a zero activity pattern(XJ. In Panel b two mismatched input patterns add togenerate an approximately uniform input pattern, whichis suppressed by the mechanism of Panel a. In Panel ctwo matched patterns add to yield a total input patternthat is more active than either pattern taken separately.

trade could record both on-enhancementsand off-rebounds as dishabituation reactionsat some cell locations, as well as off-reactionsat other cell locations.

24. Noise Suppression and PatternMatching

A variant on the noise suppression themeimplies a mechanism of pattern matching atp(I). A shunting competitive network caneasily be designed to suppress uniform inputpatterns (Figure 8 [aD. Such patterns repre-sent noise in the sense that no node is dif-ferentiated from any other by the input. Ifa network can suppress a uniform input pat-tern, then it can suppress a sum of two mis-matched input patterns (Figure 8 [bD be-cause the mismatched peaks and troug4s ofthe input patterns add to produce an almostuniform total pattern. By contrast, a sum oftwo matched input patterns yields an am-plified network reaction (Figure 8 [cD be-cause the network's shunting mechanismreacts to the sum of two patterns with a largergain than to a single input pattern. See Gross-

25. Sigmoid Signals and Tuning: MultipleEffects of a us

One final consequence of sigmoidal sig-naling in a competitive feedback network canhelp us to understand conditioning experi-ments. Noise suppression due to sigmoid sig-naling also implies the following interestingproperty. A competitive feedback networkundergoing sigmoidal feedback signalingpossesses a parameter called a quenchingthreshold (QT). The QT defines the noiselevel of such a network, because the networkinhibits activities that start out l~ss than theQT and contrast-enhances activities that ex-ceed the QT before storing them in STM.Any network with a QT can be tuned; byvarying the QT the network's ability to storeor suppress inputs can be altered throughtime.

For example, if an arousal ~ource nonspe-cifically increases the level of shunting inhi-bition acting on a competitive network'sfeedback inhibitory interneurons, then thenet disinhibitory action will cause the net-work's QT to decrease. The network's STMstorage of input patterns will thereby be fa-cilitated. This type of arousal event shouldnot be confused with orienting arousal. It isthe type of arousal that lowers and heightensthe sensitivity of the attentional subsystem,as during sleep and wakefulness.

I will now briefly indicate how the conceptof an attentional QT can be implicated dur-ing a conditioning experiment. To do this Iwill use some concepts intuitively that willbe precisely defined in the next sections. Sup-pose that any cue that can activate a driverepresentation can also redu,ce the atten-tional QT. Then an unexpected US can havethree distinct effects on attentional process-ing. As an unexpected event that mismatchesactive feedback expectancies, the US can re-ttlove some STM representations from over-shadowing by differentially amplifying them.As a US per se, it can activate a drive rep-resentation and thereby further abet the STMstorage of overshadowed cues by lowering theQT. This effect sensitizes the processing ofall cues that survive STM reset and occurs


RECURRENT

PATHWAY

later in time than STM reset. Finally, as agenerator of conditioned incentive motiva-tional feedback, the US can differentiallystrengthen STM activities of motivationallycompatible cues. By contrast, an unexpectedCS is only capable of eliciting differentialSTM reset because of mismatch with feed-back expectancies.

All of the three effects itemized above de-pend on learning, but in different neuralpathways and on different time scales. Thefirst effect involves expectancy learning, thesecond effect involves conditioned reinforcerlearning (which enables a cue to turn on adrive representation), and the third effect in-volves incentive motivational learning. Ifsuch a QT effect of a US exists, its genecratorwill probably be turned on by signals that areelaborated within the hippocampus, whichI identify as the final common pathway ofthe drive representations (Grossberg, 1971,1975, 1980).

4 = CHOLINERGIC

.= CATECHOLAMINERGIC

Figure 9. Slow gated feedback and long-tenD memoryinteraction. A conditionable pathway that is excited bya cue's internal representation feeds into a gated feed-back loop. Because the gaie occurs within a feedbackloop, the conditionable pathway can achieve two distinctprocessing objectiveS; It can sample and store in its long-tenD memory (L TM) trace the ~ted output of the loop.It can also alter the short-tenD memory activity in theloop by changing its own signals through time. The L TMtrace in the conditionable pathway is assumed to be partof a cholinergic interaction. The gate in the feedbackloop is assumed to be part of a catecholaminergic in-teraction. The text and Figure 10 show how these com-ponents can be embedded into a gated feedback dipole.

Conditioning and Attention

26. Gated Feedback Dipoles

We now have all the conceptual threadsthat we need to discuss conditioning and at-tention. Some loose threads still need to betied, however. Then we will find that a largebody of data falls into place with little dif-ficulty. Because a deep 'theoretical under-standing comes from knowing how particu-lar mechanisms generate particular constel-lations of data properties, I will build up themechanisms in stages and will supply datamarkers at each stage.

Two distinct design principles coexist atthe same cells: gated dipoles and shuntingcompetitive feedback networks. I will nowshow how to join these two pnnciples intoa single network. This can be done by makingexplicit a property that was mentioned inSection 22 without pursuing its implications.There I suggested that a nonspecific arousalburst could reset STM even after the cuesthat initiated STM storage had terminated.I used this property to begin an explanationof overshadowing. In order for this to hap-pen, the STM feedback loops must containthe transmitter gates so that STM activity candifferentially deplete the active STM loopsand thereby prepare them for rebound. Fig-

ure 9 summarizes this structural arrange-ment by depicting a conditionable inputpathway abutting a gated feedback pathway.I will henceforth assume that the transmitterin a conditionable input pathway is cholin-ergic and that the transmitter in a gated STMfeedback loop is catecholaminergic (Gross-berg, 1972b), because a large body of data iscompatible with this suggestion (Butcher,1978; Epstein, Kissileff, & Stellar, 1973;Friedhoff, 1975a,' 1975b).

There is another way to derive Figure 9even when- no arousal-initiated rebound ex-ists. This alternative derivation holds whenthe off-cells represent features or behavioralcategories that are complementary to thoserepresented by the on-cells: foiexample, fearversus relief, hunger versus satiety, verticalred bar on green field versus vertical greenbar on red field, and so forth. The derivationproceeds in three steps (Grossberg, 1972b).


; (

1. Sudden offset of a conditioned cue in-put can cause a rebound, much as offset ofa conditioned source of fear can elicit relief(Denny, 1971; Masterson, 1970; McAllister& McAllister, 1971). To accomplish this re-bound the cue input is delivered to the net-work at a stage before the transmitter gate.Only in this way can the cue deplete the gateso that its offset can drive a rebound.

2. Onset of a cue input can elicit samplingsignals capable of encoding a rebound inLTM, much as a tone that is turned on con-tingent on shock offset can become a sourceof conditioned relief (Dunham, 1971; Dun-ham, Mariner, & Adams, 1969; Hammond,1968; Rescorla, 1969; Rescorla & LoLOrdo,1965; Weisman & Litner, 1969). Thus, thecue input is delivered to the network at astage after the transmitter gate, where therebound can be sampled.

3. Properties 1 and: 2 are true for all cuesthat can be conditioned to these categories,because whether a given cue will be condi-tioned to the onset or to the offset of anyparticular category is not known a priori.Every cue input is delivered both before andafter the transmitter gating stage. The trans-mitter gate thus occurs in a feedback path-way, as in Figure 9.

The existence of two distinct derivationsleading to a similar network design is im-portant, because not every recurrent networkthat can be reset by offset of a cue need pos-sess a mismatch-contingent arousal source,even though an arousal source per se is re-quired. These derivations suggest that theanatomical design in Figure 9 is basic andthat the input mechanisms that control re-bound in this common design can be adaptedto satisfy specialized processing constraints.

One further constraint can readily be satis-fied by this design. The cue inputs arrive be-fore the stage of dipole competition so thatat most one of the complementary outputs(on-cell versus off-cell) can be positive at anytime. The next section depicts the minimalanatomy that joins together gated dipolefeedback pathways and conditionablecue in-put pathways that terminate before the di-pole competition. stage. This microcircuit isneeded to build up both the sensory and thedrive representation networks that are acti-vated by conditioning and arientional ma-nipulations.

27. Drives, ConditionedReinjorcers,Incentive Motivation, and CNV

Figure to depicts such a minimal anatomyand assigns to its pathways the motivationalinterPretation that turns the anatomy into adrive representation. In Figure 10 the specificinputs to the gated dipole are of two kinds:internal drive inputs and external cue inputs.In the case of eating, the positive drive inputincreases with hunger, and the negative driveinput increases with satiety, owing ejther togastric. distension or to slower metabolic fac-tors (Anand & Pillai, 1967; Janowitz, Han-son, & Grossman, 1949; Le Magnen, t972;Sharma, Anand, Dua, & Singh, 1961). Letthe drive inputs and the nonspecific arousalinput be gated by a catecholaminergic trans-mitter in both the on-channel and the off-channel to calibrate the proper relative sizesof dipole on-responses arid off-responses.

Let each external cue input send a con-ditionable pathway to both the on-channeland the off-channel of the dipole. Each cuecan become a conditioned reinforcer of ei-ther positive or negative sign, depending onwhich of its L TM traces in the on-channelor the off-channel is larger. To calibrate therelative sizes of these L TM traces in an un-biased fashion, I assume that the transmittersystem that subserves L TM encoding in ~othbranches is the same and is cholinergic.These chemical interpretations may even-tually have to be changed, but the process-ing requirements of accurately calibratingrelative rebound or conditioned reinforcerstrength across competing channels are' ro-bust.

The cells at Which external cue, drive, andarousal inputs converge are assumed to bepolyvalent: These cells only fire vigorouslywhen both their external cue and their in-ternal drive inputs are sufficiently large. Theoutputs from these polyvalent cells competebefore generating a net dipole output in ei-ther the on-cell or the off-cell channel, butnot both; These dipole outputs play the roleof incentive motivation in the theory. Theexistence of polyvalent cells is compatiblewith the intuition that incentive motivationneed not be released even when drive is highif compatible cues are unavailable. For ex-ample, a male animal left alone will busilydo the many things characteristic of his spe-


~

CUE INCENTIVE MOTIVATIONAL PATHWAY

(LTM TRACE)

STM LOOP

(LTM TRACe)

T-GATE~. -/ .

POSITIVE AROUSAL NEGATIVEDRIVE INPUT DRIVEINPUT INPUT

Figure 10. A motivational network: network with a gated feedback dipole hooked up to conditionablepathways from the internal representations of cues. The text describes how'the feedback loops betweenexternal cue representations and internal drive representations, and between internal drive repreSentationsand themselves, join together mechanisms of reinforcement, drive, incentive motivation, competition,arousal, and short-term memory (STM). L TM = long tenrt memory.

28. Extinction, Conditioned EmotionalResponses, Conditioned AvoidanceResponses, and Secondary Conditioning

cies, such as grooming, eating, exploring. Hedoes not look sexually motivated. However,if a female animal. is presented to him, hisbehavior can dramatically change (Beach,1956; Bolles, 1967, chapter 7). Incentivemotivation need not be released even whencompatible cues are available if drive is low..Seward and his colleagues (Seward & Proc-tor, 1960; Seward, Shea, & Elkind, 1958;Seward, Shea, & Davenport, 1960) foundthat if an animal is not hungry, then noamount of food will be adequate to reinforceits behavior.

Figure .1 0 suggests a different relationshipbetween drive and incentive than is fQund inRouttenberg (1968). Routtenberg makes in-centive and drive the complementary con-cepts of his two-arousal hypothesis. In Figure10 drive and incentive are both part of theattentional subsystem, which is complemen-tary to the orienting subsystem. In my theorydrive and arousal are not the same concept.

To indicate how a gated feedback dipoleworks, the next two sections summarizesome of its formal properties using reinforce-ment and motivation terminology. This sec-tion reviews how each cue's L TM samplingof both the on-channel and the off-channelcontributes to the explanation of some basicconditioning processes (Grossberg, 1972a,1972b) and thereby indirectly supports theclai~ that both L TM pathways need to bebuilt up from similarly calibrated transmittermechanisms. The simplest L TM law saysthat L TM encoding occurs only when a cuepathway and a contiguous polyvalent cell aresimultaneously active (Grossberg, 1964, 1968;Hebb, 1949).

Suppose that during early learning trials,a cue occurs just before a large unconditional


~

~

signal, such as a shock, turns on the on-chan-nel. This unconditional signal elicits the in-centive output of the on-channel, which trig-gers a fear reaction, even before learning oc-curs. By associating the cue with shock onlearning trials, its L TM trace abutting the on-channel grows much larger than its L T,Mtrace abutting the off~channel. If the cue isthen presented by itself, the LTM-gated sig-nal to the on-channel is much larger than theL TM -gated signal to the off-channel. The on-channel wins the dipole competition, so thecue elicits fear. The cue has to thereby be-come a conditioned reinforcer that can elicita conditioned emotional response, or CER(Estes, 1969; Estes & Skinner, 1941).

Now suppose that after the cue has becomean elicitor of a CER, environmental contin-gencies change. Suppose that the cue no lon-ger reliably predicts future events and thatan unexpected event occurs while the cue ison. Suppose that the unexpected event trig-gers an antagonistic rebound in the off-chan-nel. Because the cue is on, its L TM traceabutting the off-channel will grow. If thisoccurs sufficiently often, the off- L TM tracewill grow as large as the on-LTM trace. Afterthis happens, presenting the cue will generatecomparable LTM-gated signals to both theon-channel and the off-channel. After thesesignals compete, the net incentive motivationoutput will be very small. the cue no longerelicits a CER. It has been rapidly extin-guished by unexpected events.

This cue is extinguished because it remainson both before and after the unexpectedevent. It is an irrelevant cue with respect tothe contingency that triggered the unex-pected event. By contrast, a cue that turnson right after the unexpected event occurswill sample only the off-reaction of the di-pole. Only its LTM trace abutting the off-channel will grow. Later presentation of thecue will thereby elicit a large off-reaction. Ifthe off-reaction corresponds to a relief reac-tion, then the cue has become a source ofconditioned relief by being paired with offsetof a source of fear. Although the cue hasnever been paired with a positive reward, itthereafter can be used as a positive reinforceror source of consummatory motivation. Thismechanism helps us to understand howavoidance behavior can be persistently main-

tained long after an animal no longer exper-(iences the fear that originally motivated theavoidance learning (Grossberg, 1972a, 1972b,1975; Maier, Seligman, & Solomon, 1969;Seligman & Johnston, 1973; Solomon, Ka-min, & Wynne, 1953).

A similar argument shows how secondaryconditioning can occur. For example, offsetof a positive (or negative) conditioned rein-forcer SI can drive an antagonistic reboundthat conditions a cue S2 whose onset is con-tingent upon the offset event to be a negative(or positive) conditioned reinforcer. Thismechanism uses the feedback in the gateddipole in a major way. Offset of the reinforcerS I can elicit a rebound because it occurs ata stage prior to the gate, whereas samplingof the rebound can occur because Cue S2delivers it signals at a stage subsequent to thegate.

29. Motivational Baseline, Switching, andHysteresis

The positive feedback loops in the gateddipole of Figure 10 turn this network into afeedback competitive network. The slowlyvarying transmitter gates do not alter the factthat a feedback gated dipole shares manyproperties with other feedback competitivenetworks. For example, the dipole now hasan STM storage capability, which means thatit can defend a motivational decision againstsufficiently small momentary fluctuations incue or arousal inputs. This hysteresis prop-erty helps to provide the inertia needed tocarry out a sustained, motivated act duringirrelevant environment perturbations. TheSTM normalization property refines this ca-pability by maintaining a temporally stablebaseline of incentive motivation. The con-trast-enhancement property due to sigmoidalfeedback signaling helps to control sharpmotivational switching between behavioralalternatives when the net balance of inputssucceeds in overcoming the hysteretic inertiaof the previous motivational choice.

Frey and Sears (1978) ,have suggested aformal model to explain some of these prop-erties. Their model builds sigmoid and hys-teresis properties into a cusp catastrophemodel of conditioning and attention. Al-though their model provides one way to vi-

553CONDITIONING AND A1TENTION

sualize sudden switches, the catastrophe vari-ables do not correspond to physical variables,and the model provides no physical expla-nation of why sigmoid and hysteresis prop-erties appear in the data. The gated dipoletheory provides a physical explanation thatdoes not correspond to a cusp catastrophe,and it implies a large body of data and pre-dictions that are invisible to the cusp picture.For example, when sigmoid feedback signalsare used in a gated dipole, this network alsopossesses inverted U properties that are re-flected in a large body of data about normaland abnormal behavior (Grossberg, I 972b,1982a). These inverted U properties are partof the minor mathematical miracle.

Because of the importance of the gateddipole concept to my motivational theory,I will also discuss how the same mechanismswork in the case of hunger and satiety, ratherthan shock. In the case of hunger, a positivedrive input increases with hunger, whereasa negative drive input increases with satiety.An increase in the positive drive input dis-inhibits the polyvalent cell that is two inhi-bitory synapses away. Suppose that this poly-valent cell also receives a' large conditionedsignal from a cue representation. In otherwords, the cue is a conditioned reinforcerwith respect to this drive representation, andthe cue is active in STM. Then the polyvalentcell can vigorously fire. Suppose at this mo-ment that the negative drive input is small(e.g., the hunger level is high) and/or onlyweak conditioned signals reach the polyva-lent cell of the negative drive representation.Then this polyvalent cell does not fire vig-orously. Thus, after competition takes place,the positive drive channel wins. It can there-fore emit incentive motivation signals to thecue representations. These conditionable sig-nals help to regulate attention by modifyingthe total excitatory input pattern to the cuerepresentations.

The positive drive channel can also deliverexcitatory feedback to the cells that receivepositive drive input. This excitatory feedbackcan sustain the activity of the positive driverepresentation. It can thereby store a moti-vational decision in STM against small inputperturbations (hysteresis), maintain a steadymotivational baseline (normalization), andregulate sharp motivational switching (con-

trast enhancement). All of these propertiesare STM properties. The sustained STM re-verberation also allows contiguous L TMtraces of active cue representations to encodethe large activity of the positive drive rep-resentation at a rate that increases with theSTM activity of the cue representation. Theseactive cues can thereby become positive con-ditioned -reinforcers.

If the incentive motivation from the pos-itive drive representation supports sustained,motivated behavior (e.g., eating), then thenegative drive input slowly grows (e.g., sa-tiety increases). The increase in the negatived~ve input shuts off STM at the positivedrive representation via the competitive in-teraction. The motivated behavior therebyloses its incentive motivation support (e.g.,eating stops).

If positive conditioned reinforcer input israpidly withdrawn before the negative driveinput increases, then an antagonistic re-bound can be elicited in the negative drivechannel. This rebound rapidly terminates themotivated behavior. An antagonistic re-bound can occur because a sudden reductionof positive conditioned reinforcer input re-duces the signal within the feedback loop ofthe positive drive representation. The totalsignal to the transmitter gate in the positivedrive channel is thereby reduced, and a re-bound is elicited just as in Figures I and 2.A cue whose L TM traces sa~ple the antag-onistic rebound can become a negative con-ditioned reinforcer. A cue whose L TMtracessample both the positive drive representationand the negative drive representation is ex-tinguished (irrelevant) with respect to thisdrive, because its positive and negative gatedsignals to the gated dipole inhibit each otherat the competitive stage before any net in:-centive motivation can be released.

An issue of some importance concernshow strict the polyvalent constraint is on thefiring of cells where external cue inputs anddrive inputs converge. To illustrate the issue,suppose that a satiety input grows becauseof sustained eating. If the polyvalent con-straint is strict, then the polyvalent cell thatreceives the large satiety input cannot fire atall unless it also receives a large cue input.If not, the satiety input cannot inhibit thepositive incentive motivation that was sup-


EXTERNAL CUEINPUTS compete in that network before they influ-

ence the polyvalent cells. Once the conceptof polyvalence is before us, we can begin toclassify which networks best marry pQlyva-lence to other important processing con-straints.

~:OMPETITION FOR CUE~ IN SENSORY STM REPRESENTATIONS

CONDITIONEDINCENTIVE

MOTIVATIONALSIGNALS

(LTMI

\\ \

CONCm(>NEDREINFORCER

SIGNALSILTMI

DRIVECOMPETITION FOR STORAGE IN REPRESENTATIONS

MOTIVATIONAL STM

INTERNAL DRIVEINPUTS

Figure II. Adaptive resonance between dipQle fields.When external cues excite the short-term memory(STM) traces of their internal representations, these in-ternal representations elicit signals that are distributednonspecifically across the various internal drive repre-sentations. During conditioning the pattern of reinforc-ing and drive inputs to the drive representations canalter the long-term memory (L TM) traces within certainof these signal pathways, as Figure 10 has illustrated.The corresponding external cues thereby acquire con-ditioned reinforcer properties. On recall trials the con-ditioned reinforcer signals from external cues combinewith internal drive inputs at the drive representationsto determine which drive representations will fire. Out-put from the drive representations plays the role of in-centive motivation in the network. Incentive motivationis released from a given drive representation only if themomentary balance of conditioned reinforcer signalsplus drive inputs competes favorably against these fac-tors within the other drive representations. The incentivemotivational pathways are also nonspecific and condi-tionable. External cue representations that receive largeincentive motivational feedback signals are favored inthe competition for storage in sensory short-term mem-ory, as Figure 13 describes in greater detail.

porting eating. If the polyvalent constraintis not strict, then small background cue in-puts will suffice to permit polyvalent cell fir-ing in response to sufficiently' large satietyinputs. This problem is overcome by the net-work of Figure 12, because the drive inputs

30. Adaptive Resonance Between DipoleFields

Now that we have a clearer view of howto design the microcircuitry of a gated feed-back dipole, we need to build these micro-circuits into a global processing scheme. Thegated f~edback dipoles are part of dipolefields wherein on-cells are joined by shuntingcompetitive feedback networks, off-C(lls arejoined by shunting competitive feedback net-works, and on-cells are joined to off-cells viagated dipoles. The dipole fields themselvesinteract via adaptive filters, much as the fieldsF(I) and F(2) interact in Section 18.

Figure 11 depicts the macrocircuit that willbe most prominent in my discussion of con-ditioning data. It describes a feedback mod-ule wherein sensory and drive representa-tions send signals to each other via nonspe-cific excitatory conditionable pathways(adaptive filters). These representati()ns areorganized into dipole fields. Each dipole fieldis capable of STM contrast enhancement,normalization, hysteresis, and rebound. Theinterfield conditionable pathways sendbranches to both the on-cells and the off-cellsof the dipole fields, just as they do to explainextinction and secondary conditioning inSection 28.

The conditionable pathways from sensoryrepresentations to drive representations en-code the conditioned reinforcer properties or.external cues. The conditionable pathwaysfrom drive representations to sensory repre-sentations encode the incentive motivationproperties of internal drives. An adaptive res-onance occurs within this network when thereinforcing properties of active external cuessufficiently match the motivational proper-ties of active internal drives to lock STM intoa global interpretation of the data.

In the theory that I developed in Grossberg(1971,. 1972a, 1972b, 1975), the final pro-cessing stage in the external cue representa-tions is assumed to be cortical, and the final


processing stage in the drive representationsis assumed to be hippocampal. Gabriel, Fos-ter, Orona, Saltwick, and Stanton (1980)summarize recent data that support a qual-itatively similar conclusion. They write, "thehippocampal formation is a region criticalfor encoding or 'modelling' of stimulus-re-inforcement contingencies" (p. 189). Theynote that the hippocampus is reciprocallyconnected with cingulate cortex and with theanteroventral nucleus of the thalamus andsummarize data suggesting that cortical"codes inappropriate to the stimulus itembeing presented would create mismatch with

the hippocampal model, thereby elicitingcode-suppression in cortex and thalamus"(p. 189).

31. A Motivational Dipole Field: Drive-Reinforcer Matching and MotivationalCompetition

Now we need to fill in the microcircuitryof the dipole fields using Section 27 as aguide. Figure 12 depicts an anatomy thatpossesses the minimal drive representationproperties that .I will need. In: this anatomyeach motivational channel possesses a posi-


tive feedback loop that runs through a gateddipole. These positive feedback loops are theon-centers of a competitive feedback net-work that joins together motivational chan-nels. The competitive feedback network pro-vides a matching interface that runs acrossthe motivational channels. At this interface,spatial patterns of (conditioned) reinforcersignals are matched with spatial patterns ofdrive signals. Only a sufficiently good matchcan trigger sustained polyvalent cell firing(Sections 17 and 27). If this network is tuned(Section 25) so that only a single winningchannel can reverberate in STM, then sharpmotivational switching will occur. Such asetting of the network defines the channelsas being motivationally incompatible. A lowersetting of. the QT can permit compatiblecombinations of drive representations to besynergistically activated. Possible settings de-pend on the choice of numerical networkparameters and can vary across species andindividuals without changing the underlyingdesign principle.

To understand in greater detail how amotivational dipole field works, I will sum-marize the processing stages in Figure 12 stepby step. Pathways 1 and 2 carry specific, butcomplementary, drive inputs (e.g., hunger vs.satiety) to a single dipole. Pathways labeled3 carry nonspecific arousal to this dipole.Cells 4 and' 5 add these inputs and thereuponinhibit the tonically active Cells 6 and 7.(Tonic cells have open symbols; phasic cellshave closed symbols.) Pathways 4 --+ 6 and5 --+ 7 contain slow transmitter gates (squaresynapses), assumed to be catecholaminergic.If Input 1 exceeds Input 2, then the trans-mitter in Pathway 4 --+ 6 is depleted morethan the transmitter in Pathway 5 --+ 7, there-upon calibrating the dipole for a possible an-tagonistic rebound later on.

The tonic Cells 6 and 7 equally inhibiteach other until Input 1 exceeds Input 2.Then Cell 6 is inhibited more than Cell 7.This imbalance disinhibits tonic Cell 8 andfurther inhibits tonic Cell 9. Both Cells 8 and9 are polyvalent, meaning that all their ex-citatory inputs must be active for these cellsto vigorously fire. (Triangles denote polyva-lence.) The polyvalent cells are assumed tobe pyramidal cells. Because Cells 8 and 9 arepolyvalent, a larger input to Cell 1 than Cell

2 cannot fire these cells. However, such animbalance can prevent Cell 9 from firing.

To see how Cell 8 can fire, we consider thepolyvalent cells, 8 and 10, of two differentmotivational channels. Cells 8 and 10 com-pete via the inhibitory (interneuronal) Path-ways 13. The polyvalent Cells 8 and 10 alsoreceive inputs from external cue represen-tations via the conditionable Pathways 11and 12, respectively, whose L TM traces(within the filled hemicircles abutting Cells8 and 10) encode conditioned reinforcerproperties of their respective external cues.These L TM traces are assumed to be cholin-ergic.

The conditioned reinforcer inputs Com-bine with drive and arousal inputs at theirrespective polyvalent cells, which begin tofire if their thresholds are exceeded. The poly-valent cells thereupon compete among them-selves via the "intrinsic" feedback inhibitoryPathways 13, as they simultaneously try toexcite themselves via positive feedback path-ways such as 8 -4 -6 -8.

If, for example, Cell 8 wins this competi-tion, then the transmitter gate in Pathway4 -6 is depleted as the suprathreshold re-verberation bursting through Cell 8 via Path-way 8 -4 -6 -8 drives L TM changes inPathway 11. The reverberation thus inducesconditioned reinforcer changes even as it pre-pares the network for motivational reset byrapid offset of Pathway 11. or a rapid incre-ment in Pathway 3.

32. A Sensory Dipole Field: TheSynchronization Problem and DC PotentialShifts '

Figure 13 depicts the minimal anatomythat I will need to join together external cuerepresentations. This dipole field possessesadditional structure compared to Figure 12because it solves a specialized design prob-lem, which I call the synchronization probler.nof classical conditioning. The synchroniza-tion problem recognizes that without spe-cialized network buffers, Pavlovian associa-tions could rapidly extinguish whenever a CSand US were presented with different inter-stimulus delays on successive learning trials.The synchronization problem was solved inGrossberg (1971) and provided the impetus


~:=::==:::?Figure 13. Interaction of external cue and incentive motivation signals at polyvalent cells. Let a set ofnodes, or cells, in the dipole field be activated by an external scene. A pattern of short-term memoryactivity across the nodes represents the scen~. Each such node sends an excitatory signal to its polyvalentnode, or cell. Signal size is an increasing function of short-term memory activity. These specific signalsare insufficient to fire the polyvalent cells. Sufficiently large incentive motivational signals from a driverepresentation must simultaneously converge on the polyvalent cells to fire them. The incentive moti-vational pathways are conditionable. A drive representation will therefore preferentially activate thosepolyvalent cells whose cues were paired with this drive representation in the past. The drive representationcan thereby fire a subset of the polyvalent cells that are activated by the external sCene. The relative rateof firing of each polyvalent cell will depend jointly on the short-term memory activity of its trigger cuesin the scene and on the relative size of its long-term memory trace in the conditioned reinforcer pathway.When a polyvalent cell fires, it delivers positive feedback signals to the cue-activated cells that supply itwith specific short-term memory signals. This positive feedback from polyvalent cells selectively augmentsthe short-term memory activities of certain cue-activated cells, which thereupon can more strongly inhibitthe short-term memory of other representations in the dipole field using the short-term memory nor-malization property. The incentive motivational properties of certain cues thereby alter the set of cuesto which the network pays attention. The polyvalent cells that can maintain their firing can also readout learned patterns (e.g., motor commands) to other parts of the network.

for my later work on reinforcement and mo-tivation.

For present purposes, I need to emphasizeone difference between Figure 12 and Figure13. The anatomy in Figure 13 separates thefiring of polyvalent cells from the STM re-verberation through gated dipoles. Owing tothis property, a sensory representation can

reverberate in STM and thereby deliver sig-nals to a polyvalent cell, or cells, without fir-ing these cells. A polyvalent cell in Figure 13can fire only if it simultaneously receivesSTM signals from an external cue represen-tation and incentive motivation signals froma drive representation. This property is anal-ogous to John's (1966, 1967) reports that


Figure 14. Path equivalence of CS and US representa-tions. Both the conditioned stimulus (CS) and the un-conditioned stimulu~ (US) activate similar network de-signs. This path equivalence property overcomes theasymmetry of CS and US contributions to the modifiedRescorla-Wagner Equation 8. Firing of the polyvalentcells VI2 and V22 is prevented except when sufficientlylarge specific signals from VII and V2t. respectively, andnonspecific signals from D2 simultaneously converge atthe polyvalent cells. The network stages {VI t. V12} arepart of the CS representation. The stages {V21. V22} arepart of the US representation. The CS and US activatesimilar network anatomies, but the L TM traces in theseanatomies are not identically distributed. In particular,a US-activated signal from V 21 to D2 can fire D2 if thedrive input is sufficiently large. A CS-activated signalfrom VII to D2 cannot fire D2. See the text for how CS-US pairing endows the CS with conditioned reinforcer.incentive motivation, and habit readout capabilities.

US properties to control observable behavior(Grossberg, 1971). This description showshow to overcome the asymmetry between CSand US in the modified Rescorla- WagnerEquation 8, explains the Dickinson andMackintosh (1979) data on selective effectsof distinct reinforcers on associability, andshows that drive representations are func-tionally separate "processors" from cue rep-resentations, in contrast with the Pearce andHall (1980) theory.

Let a CS activate the population VII (Fig-ure 14), which thereupon begins to rever-berate in STM. Then VII sends specific sig-nals to the polyvalent cell population Vl2(among others) and nonspecific signals to thedrive representations. Nothing else happensuntil a US arrives at population V21. This isbecause VI2 can fire only if it receives an in-put from VII and an incentive motivationinput from a drive representation, but thesignal from VII to the drive representationsis initially too small to fire them. When theUS perturbs V21, V21 sends signals to the poly-valent cells V 22 and to the drive representa-tions. These latter signals can fire a certaindrive representation, ifits drive input is largeenough, because the cue firing V 21 is a USfor that drive representation, which I willhenceforth denote by D2. When Di fires, itreleases nonspecific incentive motivation sig-nals to all the external cue representations.Now five things happen.

First, because VII and D2 are both active,the L TM traces in the pathway from VII toD2 are strengthened. When these L TM tracesget strong enough, the CS alone will be ableto fire D2. Second, the nonspecific incentivemotivational signal from D2 combines withthe US-derived signal from V 21 at V 22, therebyfiring polyvalent cell signals from V 22, whichread out an unconditioned response (UR)pattern. Third, because the incentive moti-vation signal is nonspecific, it also combineswith the CS-derived signal from Vll at V12,thereby firing the polyvalent cells V12. Fourth,because D2 and VI2 are both active, the LTMtraces in the pathway from D2 to VI2 arestrengthened. Fifth, the polyvalent cells VI2fire sampling signals to the cells at which theUR pattern is being read out. These signalsencode (a fractional component of) the URin the LTM traces of this pathway. The en-

certain polyvalent cortical cells that are in-volved in cortical conditioning can fire onlyin response to a sum of CS and US signals.The property is also analogous to the effectsof anodal DC potential shifts on cortical con-ditioning (Morrell, 1961; Rusinov, Note 1).In my theory, the anodal DC shift replacesthe requirement of an incentive motivationsignal to fire polyvalent cortical output cells.

33. The Multidimensional Nature ofSecondary Conditioning

The functional separation of STM rever-beration and polyvalent cell firing implies thefollowing description of how a CS acquires

559CONDITIONING AND A 1TENTION

coded pattern will henceforth be read out a$a conditioned response (CR) pattern. The CSthereby acquires US propertit;s owing toL TM encoding in conditioned reinforcer, in-centive motivation, and habit strength path-ways.

This network provides a simple answer tothe synchronization question: How does theUS turn on the CS with just the right timelag to sample read out of the UR pattern?The same incentive motivation burst thatallows V22 to read out the UR also allows VI2to emit sampling signals that read in the CR.

In the remaining sections I will use theproperties of adaptive resonance and reset inthe cognitive circuits (Section 19) and in thecognitive-motivational circuits (Sections 30-33) to suggest explanations of conditioningand attentional data, including data that nosingle formal model can ~xplain.

34. Unblocking, Context, and Habituation

Unblocking is produced by the surprisingomission of a second shock (Dickinson, Hall,& Mackintosh, 1976). In my theory this isbecause the active internal repr~sentations ofpreviously presented cues read out a learnedexpectancy whose mismatch by the unex-pected event triggers dipole reset and con-sequent STM enhancement of previouslyovershadowed cues (Section 23).

Experiments might be designed to test howthe amount of rebound or enhancement de-pends on the number of cues that have beenstored in STM when the unexpected eventoccurs. Due to the limited-capacity restric-tion jrilposed by STM normalization, storingmore cues can reduce the STM activity ofeach cue and thereby reduce the rate of trans-mitter depletion in each dipole gate. The re-bound pattern in response to a fixed arousalburst will therefore change as a function ofthe number of active representations. Thesimplest version of this idea suggests that ifmore cues are simultaneously stored in STMfor a given amount of time, then they willeach be rebounded less by an unexpectedevent. Let us suppose that a less reboundedcue representation retains more STM activitythan a more rebounded cue representationand that this residual activity can summatewith the STM activity ~licited by a later pre-

sentation of the same cue. Let two experi-mental groups differ according to how manycues are stored in STM and expose bothgroups to an unexpected stimulus array thatincludes one of the previously stored cuesand a new cue. Suppose that the unexpectedstimulus array resets STM by rebounding orenhancing the previously stored representa-tions. Other things being equal, a less re-bounded cue may preserve more residualSTM activity for summation with its reoc-currence as part of the unexpected event. Ifthis happens, the cue's total STM activity willbe larger if it was part of a larger set of pre-viously stored items than a smaller set ofitems. Hence, it will be better attended in thepresence of the new cue.

However, other things are not usuallyequal. Storing more cues simultaneouslymay provide each cue with less STM activitydue to the normalization property. Hence,the occurrence of a smaller reset per cue mayprovide no advantage to larger sets of cues.By contrast, storing more cue& might causemore of their STM activities to be enhanced,rather than rebounded, by an unexpectedevent. When this occurs an advantage mayindeed accrue to larger sets of cues~ Both ofthese effects will be sensitive to the durationwith which the cues are stored before theyare reset. Large STM activities deplete theirtransmitter gates faster than small STM ac-tivities. Consequently, .the relative disadvan-tage to smaller sets of stored cues may begreater if the storage duration is shorter. Fi-nally, a switch from more stored cues to anew event that includes only one of thesecues may cause a greater mismatch, andhence be more unexpected, than a switchfrom fewer stored cues to the same newevent, although normalization tends tocounter this effect also. The reset arousalburst that occurs after more cues are storedmay thus be larger and may offset any ad-vantage due to slower transmitter habitua-tion. Parametric studies are needed in whichthe number of cues originally stored, the du-ration of storage, the number of new cues tobe stored, and the amount of overlap be-tween the two sets of cues are varied todisentangle the relative contributions ofSTMnormalization, mismatch, and reset mecha-nisms on the reallocation of attention.


35. Double Shock Experiments: InitialSTM Advantage of Surprising Events

Inhibiting these chunks can eliminate thesubliminal feedback signaling to expectedsituational cues, including the light, and cantherefore decrease the light's STM activity byremoving a source of STM matching. Thisissue also arises on Trial 25 in Group Y ofthe Kamin (1969) experiment reviewed be-low (Section 45).

36. Conditioned Reinforcer Learning

After both the tone STM representationand the light STM representation send sig-nals to the drive representations, the lightrepresentation can supraliminally activatethe negative drive representation that waspreviously activated by shock on condition-ing trials, because the light i~ now it coI:ldi-tioned reinforcer (Sections 27 and 33). Oncethis drive representation is supraliminallyactivated, the tone's sampling signals can ac-quire some negative conditioned reinforcerproperties via L TM encoding during the firstmoments after tone-light onset. This L TMchange can occur whenever the tone's sam-pling signals are contiguous with supralimi-nal activity at the drive representation. Onthe first compound trial, the tone represen-tation emits unusually large sampling signalsbecause it has acquired unusually large STMactivity due to its unexpectedness. Largesampling signals cause their L TM traces toencode sampled STM patterns faster than dosmall sampling signals. The tone can there-fore acquire negative in.centive propertiesmuch more rapidly on a trial when it is un-expected than on a trial when it is expected.This fact explains why the tone can conditionso well-as a conditioned reinforcer-on thetrial when it is first presented.

To explain the interesting experiment ofMackintosh, Bygrave, and Picton (1977)(Section 5), I accept the fact that on the firstcompound trial, the tone is at its worst as apredictor of shock, having never before beenpaired with shock." hereby avoid the internalproblem within Mackintosh's theory (Sec-tion 3). Because the tone has never beforebeen presented, it is most unexpected on thefirst compound trial. More precisely, on thefirst trial the tone possesses its maximal p0-tency for mismatching the learned sensoryexpectancies that are operative in the situa-tion, in particular, the expectancies that arecontrolled by situational cues. Consequently,tone onset triggers a relatively large STM re-set that abets the STM storage of the tone'ssensory representation. This is the first mainpoint: The tone's very unexpectedness canenhance its initial STM storage without re-gard to what its STM representation samples1ater on. On successive tone presentations,this intitial advantage will fade, other thingsbeing equal, as the tone is incorporated intothe pattern of learned feedback expectancies.

After the tone's advantageous initial STMstorage takes place, the tone's STM repre-sentation begins to emit several types of sig-nals. Some of these signals initiate the processwhereby the tone is incorporated into higherorder chunks (Section 18). Others of thesesignals begin to sample the drive represen-tations (Section 30). On the first compoundtrial, the light can also send negative condi-tioned reinforcer signals to the drive repre-sentation with which it was previously as-sociated.

An important issue is: How much are thelight's conditioned reinforcer signals reducedby the occurrence of the tone? For example,such a reduction can occur because of a di-rect STM competition between light andtone representations via the STM normali-zation effect. An indirect reduction can bedue to antagonistic rebound of the activechunks that bind situational cues togetherinto context-sensitive representations. Sucha rebound can be triggered by the arousalburst that is contingent on the tone's unex-pectedness in a given experimental context.

37. Incentive Motivation FeedbackInfluences STM Competition

As the tone begins to acquire negative re-inforcing properties by being conditioned tothe negative drive representation, the firingof this drive representation also releases con-ditioned incentive motivation signals pref-erentially to the light representation (Figure13). More precisely, the light representationactivates its polyvalent cells directly and viathe conditioned reinforcer-incentive moti-


vation loop through the drive representa-tions. Then these polyvalent cells can fire.The polyvalent cells thereupon feed excita-tory signals back to the light representationto further enhance its STM activity. Becauseof STM normalizati~n among the cue rep-resentations, the enhanced STM of the lightrepresentation competitively inhibits theSTM of the tone representation. Whetherthis feedback inhibition can entirely shut offthe tone representation depends on howmany and how intense the prior light-shocktrials were (and thus how large and selectivethe co!'lditioned incentive feedback signalsare) ant on how surprising the tone was (andthus ho\v big an advantage it acquired in itsinitial STM storage).

If the tone representation is entirely sup-pressed, it will have acquired negative con- .

ditioned reinforcer properties but no condi-tioned incentive-motivation feedback to itsown sensory representation. Even this even-tuality should not prevent the tone from elic-iting fearful reactions when jt is later pre-sented alone. Because the light is not thenpresent to suppress it, the STM normaliza-tion property allows the tone to be stored inSTM, whereupon the tone's conditioned re-inforcer signals can elicit a fear reaction.

38. Sali~nce Influences Overshadowing

This explanation also shows why, whenthe two CSs differ markedly in their intensityor salience, the more salient cue can over-shadow the less salient cue (Mackintosh,1976). In my theory the greater cue saliencyor intensity gives it an initially larger STMstrength, larger L TM sampling signals be-tween cue and drive representations, andthus a competitive advantage from the verystart.

39. Overshadowing on a Single Trial

These concepts suggest how overshadow-ing effects can occur when only a single trialof compound conditioning is given (Mack-intosh, 1971; Mackintosh & Reese, 1979). Asseen above, t~e tone's best chance to avoidovershadowing is to achieve strong initialSTM storage. Without an initial advantagethe tone's conditioned reinforcer learning

will be slow at best in the time interval beforethe light's incentive feedback enables it tosuppress the tone's STM. I have also indi-cated above several ways to abet tone con-ditioning.

We are now ready to consider why a sec-ond shock 10 seconds after a first shock onthe first compound trial need not make thetone more fearful. Also, we will see why twoshocks on one trial followed by either one ortwo shocks on a second trial can lead to amore fearful tone than one shock on the firsttrial followed by either one or two shocks ona second trial.

40. The Tone Is More Unexpected AfterTwo Shocks

If the tone is not on when the second shockoccurs, then the tone's STM representationmay not send sampling signals to the driverepresentation when it is activated by the sec-ond shock. Thus, the tone does not acquiremore negative conditioned reinforcer strengthbecause of the second shock on the first com-pound trial. Why then does the tone acquiresignificantly more negative conditioned re-inforcer strength on the second compoundtrial? This can be explained by noting thatthe second shock on the first compound trialoccurs after a series of trials during whichonly one shock occurs. Thus, the secondshock is unexpected. Also, the second shockoccurs after the tone has unexpectedly oc-curred. The tone's unexpected occuuenceinitiates the process whereby the tone rep-resentation is incorporated into the patternof situational expectancies. The occurrenceof the second shock then alters the patternof learned situational expectancies beyondthe alterations already triggered by the tone.Consequently, when a tone occurs on a sec-ond compound trial that follows two shocks,it is more unexpected than a torte that occurson a second compound trial that follows oneshock. Due to the tone's greater unexpect-edness, the tone's initial STM storage on thesecond compound trial is greater, its sam-pling signals to the drive representations arelarger, and its rate of conditioned reinforcerlearning is accelerated. An independent testof the tone's greater unexpectedness wouldbe achieved if the tone elicits a larger P300


might try covarying two experimental prop-erties: the intensity of the second shock andhow surprising the tone is on the next testtrial.

evoked potential when it occurs after twoshocks than after one shock.

This description of how a tone achieves itssuperior STM storage after two shocks thanafter one shock suggests that the light mayalso be better stored in STM after two shocksthan after one shock, because both cues dis-confirm the second shock component of thesituational expectancies. A greater initialSTM storage of the light does no~ prevent thetone from strengthening its negative rein-forcer strength on the second compound trialfor two reasons: (a) The relative advantageof the light is not greater than that. of thetone, so the light will not competitively in-hibit the tone via the normalization propertyduring the phase of initial STM storage. (b)Once both the light and tone representationsbegin to send signals to the drive represen-tations, the larger signals emitted by the lightrepresentation can speed up the tone's con-ditioned reinforcer learning by increasing theactivity of the negative drive representation.One way to test the effect of the second shockon the initial STM storage of the light is tomeasure the P300 evoked potential that istriggered if the light. alone, rather than a light-tone compound, is presented after one or twoshocks.

42. Release From Overshadowing by anUnexpected US

In experiments wherein the tone is re-peated during several compound trials, thetone's initial STM advantage due to its un-expectedness wears off as it is incorporatedinto the pattern of situational expectancies.The tone's conditionability thereby fades. Ifan unexpected US follows a light-tone com;.bination, the tone's STM activity can be am-plifed owing to the differential enhancementof overshadowed representations by the STMreset event (Section 23). The US also acti-vates a drive representation. Because of thesimultaneity of enhanced STM activity in thetone representation and in the drive repre-sentation, the tone can acquire both condi-tioned reinforcer and incentive motivationproperties. These L TM changes are notrapidly terminated by competitive signalingfrom the light representation because theSTM of this representation has been atten-uated by the reset event.

If the unexpected US reoccurs on severaltrials, its unexpectedness also fades. By thetime this happens, however, the unexpectedUS has endowed the tone's representationwith a conditioned positive feedback loop toitself via the drive representation. A shiftgradually occurs as trials proceed from thetone's initial STM advantage due to theshock's unexpectedness-which is mediatedby situational expectancies and the orientingsubsystem-to a more lasting LTM advan-tage due to the ton~'s reinforcing and incen-tive motivation properties-which manifestthemselves as an attentional resonance.

41. Situational Cues

Having come this far, we are now readyto raise an issue that Pearce and Hall (1980)do not mention. When the second shock oc-curs on the first compound trial, it is a sur-prising event whose negative reinforcingproperties will be conditioned to simulta-neously active cue representations. Theserepresentations will include the representa-tions of situational cues that are again presentwhen the tone is presented on the test trial.Why does the tone-plus-situational cue read-out of negative conditioned reinforcer signalsnot create more negative incentive after thesecond shock than it does after the firstshock? My answer is that the surprising oc-currence of the tone on the test trial tendsto suppress the STM of the situational cuesvia antagonistic rebounds. Then the tone'sSTM will tend to control the net conditionedreinforcer readout from attended sensoryrepresentations. To test this explanation, one

43. Modulation of us and Drive InputEffects by Expectancies, ConditionedReinforcers, and Transmitter Habituation

A further remark needs to be made aboutwhich drive representation is activated by theshock US. This is a subtle matter because itdepends on US intensity, the degree of USexpectedness, and the conditioned reinforc-


ing potency of the light representation dueto prior learning.

Were a shock to suddenly turn on out ofcontext, it would certainly activate a negativedrive representation (Section 28). This neednot happen if a shock turns on while the lightrepresentation is on. This is true because thelight representation is already sending con-ditioned reinforcer signals to the negativedrive representation when the shock occurs.The unexpectedness of the shock attenuatesthe STM activity of the light representation.A sudden reduction in conditioned reinforc-ing signals to the negative drive representa-tion is thereby caused. The shock can offsetthis reduction in input by generating its ownunconditional input to the negative driverepresentation. If the shock input is largerthan the prior level of conditioned reinforc-ing signals, then the total input to the neg-ative drive represcentation will increase anda fear reaction will be elicited. If, however,the shock-induced input is smaller than theconditioned reinforcer decrement, then thetotal input to the negative drive representa-tion will suddenly decrease. The shock canthereby cause an antagonistic rebound thatactivates the positive drive representationthat shares a gated dipole with the negativedrive representation (Section 13). The onsetof shock can thereby cause a transient reliefreaction. This argument also indicates howan unexpected increase in a shock US cancause the tone to become a negative rein-forcer, whereas an unexpected decrease in theshock US can cause the tone to become apositive reinforcer (Kamin, 1968, 1969; Res-corIa; 1969; Wagner, 1969; Weisman & Lit-ner, 1969), despite the fact that the shock USactivates a negative drive representation inboth cases. Given a fixed decrease in shockintensity, the rebound size should be an in-creasing function of shock unexpectedness(measured perhaps as a larger P300 evokedpotential) and an increasing function of theconditioned reinforcer strength of the light(measured perhaps by the number ofpreced-ing light-shock trials). The emotional mean-ing of the shock is thus determined by aninteraction of its unconditional input withthe pattern of active expectancies and con-ditioned reinforcing signals at the momentof its occurrence. In Grossberg (1982a) I pro-

pose that a similar argument, wherein hungerdrive input replaces shock input, explainssome paradoxical findings about eating andsatiety. Oropharyngeal signals that are gatedby conditioned reinforcer L TM traces aresuggested to alter the effects of hunger driveinput much as light-induced signals that aregated by conditioned reinforcer L TM tracesare suggested to alter the effect of shockinput.

Varying the suddenness with which ashock is turned on can alter these conclusionsby influencing both the expectancy and thereinforcing properties of shock. One effect ofshock onset rate on reinforcement is the fol-lowing: Suppose that a shock slowly turns onfrom an initial intensity of 0 to a final inten-sity of J. Because the shock increase is grad-ual, the transmitter in the on-channel of thegated dipole is gradually depleted, or habit-uates,at a rate proportional to signal strengthtimes the amount of available transmitter(Section 13). Because the transmitter levelaccumulates slowly, by the time the shockintensity J is reached, ttie amount of trans-mitter 2 can be much smaller than its max-imal amount B. Thus the effcct of intensityJ is proportional to f(J + /)2, where I is thearousal level, and f(w) is the sigmoid signal(Section 22). By contrast, a sudden shockcreates the signal f(J + /)B, where B > 2,because transmitter is fully accumulatedwhen the shock intensity suddenly switchesfrom intensity 0 to intensity J. Sudden shockscan thereby be more negatively reinforcingthan gradually increasing shocks (Church,1969, Miller, 1960). In Grossberg (1982a) Isuggest a similar argument about transmitterhabituation rates in gated dipoles to explaindrug tolerance and withdrawal effects, in-cluding symptoms like rebound insomnia.Many of the expectancy, reinforcing, andtransmitter habituation effects that occurduring conditioning experiments have mech-anistic analogs in other behavioral and clin-ical syndromes.

44. Latent Inhibition

Similar concepts can be used to explainthe following interesting Hall and Pearce(1979) experiment. In Stage 1 of this exper-iment, a tone was paired with a weak shock


in the experimental group. In the controlgroup a light was paired with the same shock.In the next stage the tone preceded a strongershock in both groups. In the experimentalgroup, learning was slower. In my theory thisoccurs because the tone is more unexpectedin. the control group, thereby acquiring agreater initial advantage in STM, and there-fore conditions faster, as in Section 43.

Rather than continue to explain other datathat Pearce and Hall (1980) mention, I willsuggest an explanation of some classical ex-periments that seem to go beyond the ca-pabilities of all the formal theories. I will alsosuggest an interdisciplinary paradigm to testmy explanations.

.45. Interaction of STM Reset, STMNormalization, and LTM Gating in CERExperiments

The STM normalization property oftenholds only partially, due to the fact that feed-back inhibitory interactions can decrease asa function of intercellular distances. Thesedistance-dependent interactions help to de-fine the generalization gradients that deter-mine which cues mutually interact duringconditioning experiments (Grossberg, 1975,1981 b). A possible instance ,of a partial STMnormalization effect due to Pavlovian con-ditioning is the somewhat faster learning ofa conditioned emotional response to a com-pound stimulus than to its component stim-uli (Kamin, 1969). Parametric studies ofcompound training trials using approxi-mately equally salient stimuli whose similar-ity is parametrically altered across experi-ments, followed by extinction of the com-pound, or of each component takenseparately across groups in each experiment,would provide useful theoretical informationabout the interaction between the degree ofSTM normalization and the rate of CERlearning.

Another piece of data reported in Kamin(1969) also suggests an STM normalizationeffect. His Group Y first learned a CER toa noise stimulus (N) on 16 trials, then re-ceived a nonreinforced compound light-noise(LN) stimulus for 8 trials, and finally received4 more nonreinforced N trials. His Group Zalso received 16 CER trials with stimulus N,

,.

I

but these trials were follows by 12 nonrein-forced trials with N. Three main effects werefound: The first LN trial in Group Y showeda larger suppression ratio than the first non-reinforced N trial in Group Z. The suppres-sion ratio increased on successive nonrein-forced trials of LN in Group Y and of N inGroup Z. On the first nonreinforced N trialin Group Y (its 25th trial), the suppressionratio suddenly dropped to the value that ithad on the first nonreinforced LN trial. Thissuppression ratio was, moreover, signifi-cantly lower than the suppression ratio onTrial 25 in Group Z.

Kamin was impressed by the rapidity withwhich the suppression ratio changed on thefirst nonreinforced LN trial and on the firstnonreinforced N trial for Group Y. He re-alized that the Y animals rapidly noticed Land that their processing of L somehow at-tenuated the suppression. In my theory thesurprising occurrence ofL abets its STM stor-age, weakens the STM storage of N via STMnormalization, and thereby reduces thenegative conditioned reinforcing signals fromL to the drive representations.

That an STM rather than an L TM effectis primary on the transitional trials is furthersuggested by what happens on Trial 25 inGroup Y. When N is then presented withoutL, its representation can acquire a largerSTM activity. This representation can thenread out-on that very trial-a larger nega-tive conditioned reinforcing signal to thedrive representations. The negative reinforc-ing L TM trace is there to be read out becausethe extinction of the N representation on LNtrials was slowed owing to its small STM ac-tivity.

46. Overshadowing During Key Pecking

Newman and Bardn (1965) reinforced pi-geons who pecked a vertical white line on agreen key (the S+) but not a green key alone(the S-). They tested cue discrimination bytilting the line at various orientations duringrecall trials. A generalization gradient ojpecking was found, indicating that the ver-tical line was discriminated. By contrast, n<Jgeneralization gradient was found if the S-on learning trials was a red key or if the S-was a vertical white line on a red key.


Newman and Benefeld (cited in Honig,1970) used a vertical white line on a greenkey as S+ arid a green key as S- but testedand found generalization of the line orien-tation on a black key. They also tested gen-eralization on a black key following trainingwithout a green S- and again found a gen-eralization gradient, by contrast with the casewhere testing used a green key. They inter-preted this effect as "cue utilization duringtesting rather than cue selection during learn-ing" (p. 202). This interpretation does notexplain how the orientation cue could belearned on training trials if it was not dis-criminated using a green background on testtrials 'yet could be discriminated using-a blackbackground on test trials if it was not learnedon training trials.

My explanation of these data begins bynoting that color cues are prepotent over ori-entation cues in the pigeon, other thingsbeing equal. Consequently, when a verticalwhite line on a green background is first pre-sented, the green representations will par-tially overshadow the orientation represen-tations. (I will talk about "green" and "ori-entation" representations as a shorthand formore sophisticated coding notions that wedo not need here.) Grossberg and Levine(1975) and Levine and Grossberg (1976) de-scribe some factors that control how prepo-tent representations can mask the STM ac,:tivities of other representations due to com-petitive feedback interactions.-When tbe line-on-green cues are first pre-sented, they enjoy an additional advantagein their STM storage. Their unexpectednessin the context of the experiment's situationalcues will strengthen the STM activities of theline-on-green cues as the STM activites of thesituational cue representations are re-bounded. These rebounds should elicit aP300 evoked potential.

After the line-on-green representations areinitially stored in STM, the green cues canincrease their relative STM advantage as theyacquire conditioned reinforcer and condi-tioned incentive motivation properties. Theydo this by means of the conditioned-rein-forcer-incentive-motivation loop, the poly-valent cell fiqng constraint, and the STMnormalization property in the manner de-scribed within Section 37.

The orientation representations can alsoacquire conditioned reinforcer and incentivemotivation properties just so long as theirSTM activities are not suppressed. Theirlearning rates will be slower than those of thegreen representations, because their samplingsignals in the conditionable pathways aresmaller due to their smaller STM activities.Hence, their conditioned pathways will re-main weak compared to those of the greenrepresentations. As conditioning continues,the orientation representations may be en-tirely suJjpressed if the conditioned advan-tage of the color cues becomes sufficientlygreat to drive orientational STM activities tozero by competitive feedback across the cuerepresentations.

The unexpected nonoccurrence of rewardin response to pecking the green key causesan antagonistic rebound that excites the off-cells of the previously most active STM rep-resentations. The active incentive motivationpathway~ thereupon sample a large off-re-sponse in the green representational dipoles(Figure 11). As this experimental contin-gency reoccurs on several trials, the net in-centive motivation feedback to the green di-poles becomes progressively smaller due todipole competition between the conditionedon-cell and off-cell pathways to these dipoles.This is just the extinction mechanism of Sec-tion 28 acting at the sensory representationsrather than at the drive representations.

Even zero net incentive feedback may notbe small enough to extinguish the green rep-resentation, however, because of the innateadvantage of color over orientation. Negativenet incentive feedback may be needed toovercome green's innate competitive advan-tage. Net negative feedback is needed if netpositive conditioned reinforcer-incentivefeedback to the orientation representation is.not sufficient to offset the innate competitiveadvantage of the color representation whenthe latter receives net zero conditioned feed-back.

This framework explains why the whitevertical line is discriminable on a black back-ground during test trials even if it is not dis-criminable on a green background during testtrials in an experiment -without a green S-on learning trials. Removing green on testtrials' eliminates competitive feedback from


drive representation. The green representa-tion is consequently shut off, the orientationrepresentation is disinhibited, and the cyclerepeats itself.

Any viable alternative to the present net-work description must also avoid this prob-lem of perseverating prepotent representa-tions. In particular, a more sophisticatedcoding analysis would replace "green"representations and "orientation" represen-tations with heterarchical network encodingswherein one representation's prepotence formasking other representations would dependon its heterarchical position with respect toall representations. In Grossberg (1978b, Sec-tions 25-47) I suggest some rules wherebyheterarchical masking can be tiesigned tochoose those chunks that provide the mostinformative prediction in a prescribed cuecontext during recognition and recall.

48. Two Distinct P300 Evoked Potentialsin Cortex and Hippocampus

The previous discussion of overshadowingduring key pecking suggests a striking psy-chophysiological prediction. I have arguedthat the unexpected nonoccurrence of rewardin response to pecking the green key cangradually extingui~h the net incentive moti-vation to the green representation. This oc-curs as the incentive motivation L TM tracessample the antagonistic rebound Within thegreen representation on successive nonrein-forced trials. The reset of the green represen-tation also has an immediate effect on eachtrial. Offset of this representation rapidlyshuts off conditioned reinforcer input to thepositive drive representation. If the green rep-resentation has been conditioned to the driverepresentation on sufficiently many preViousreinforced trials, the reduction in condi-tioned reinforcer input will be large enoughto overcome the STM hysteresis that defendsthe positive drive representation against re-set. Then a rebound Will occur within thedrive dipole itself, thereby activating itsnegative qrive representation. If a new cueis stored in STM at the time of this negativedrive rebound, it Will become a negative con-ditioned reinforcer by sampling the rebound.

Let us consider the minimal assumptionthat any massive antagonistic rebound in a

the color representations to the orientationrepresentations. The STM field is therebyrenormalized. In the renormalized field, evensmall conditioned-reinforcer-incentive-mo-tivation feedback signals can provide thewhite vertical line representation with a sig-nificant competitive advantage for STMstorage.

47. The Problem of Perseverating PrepotentCues

The above discussion shows how theconditioned -reinforcer -incentive -motiva-tion feedback loop enables representationsto overcome innate competitive STM dis-advantages. Some further remarks mightclarify why the incentive motivation pathwaymust send branches to both the on-cells andoff-cells of cortical dipoles, just as the con-ditioned reinforcer pathway sends branchesto both the on-cells and off-cells of the driverepresentation dipoles. The main benefit isthat some cues can lose net positive feedbackas other cues gain net positive feedback whileboth sets of cues are conditioned to the samedrive representation. This property avoidsthe following dilemma:

Suppose the rebound that conditions zeronet feedback to the green representation oc-curs among the drive representations ratherthan among the cue representations. Thenrebound activates a negative drive represen-tation, and the net conditioned reinforceroutput controlled by the green representationbecomes small, rather than the net incentivemotivational output driven by a large con-ditioned reinforcer output becoming small,as in Section 46. This mechanism is unstablefor the following reason: As soon as the ori-entation representation takes over in STM,its positive conditioned reinforcer signals ac-tivate the positive drive representation. Whenthis drive representation sends incentive mo-tivation feedback to the cortex, the green rep-resentation receives conditioned positivefeedback because the negative drive repre-sentation is momentarily inhibited. Then thegreen representation can quickly overshadowthe orientation representation because of itsinnate competitive advantage. As soon as thegreen representation is reinstated in STM, itsconditioned reinforcer signals cause readoutof net negative incentive from the competing

C-o~~BITle~~~t-i6 A..."D A TTE..."fI()N- -jf{1-

catecholamine dipole system is registered asa P300 evoked potential-keeping in mindthat rebounds in different brain regions mayoccur yet ultimately be associated with dis-tinct evoked potentials. Then the above dis-cussion predicts that the nonoccurrence ofexpected reward can trigger a cortical P300by mismatching a learned expectancy. Thereset of cortical STM can thereupon triggera hippocampal P300 by rapidly withdrawingconditioned reinforcer input. The size of thissecond P300 should, moreover, depend onthe strength of the conditioned reinforcer dueto the number of preceding conditioningtrials. If these P300 predictions hold up, theywill clarify that a P300 can be elicited bydifferent operations in different brain re-gions. They will also refine our understand-ing of the information processing substratesof overshadowing and discrimination learn-ing by distinguishing rebounds that motiva-tionally extinguish cues owing to their cog-nitive irrelevance from rebounds that directlyelicit new conditioned reinforcer learning.

49. Nonmonotonic Relation Between P300and CNV

The key pecking experiment also suggestsa psychophysiological test that would arguefor P300 as a measure of STM reset andagainst P300 as a measure ofLTM learning.I suggest that the unexpected occurrence ofa reward in response to the line-on-green cuewill elicit a P300. As this P300 shrinks onsuccessive rewarded trials, I suggest that theline-on-green cue will elicit a growing moti-vational CNV that reflects the progressiveconditioning of positive net incentive moti-vation (Cant & Bickford, 1967; Irwin, Re-bert, McAdam, & Knott, 1966). By contrast,I suggest that the unexpected nonoccurrenceof a reward in response to green alone willelicit a P300. As this P300 shrinks on suc-cessive un rewarded trials, the green cueshould elicit a shrinking motivational CNVas the net incentive motivation of the irrel-evant green cue is extinguished. In the formercase a conditioned response is learned,whereas in the latter case a conditioned re-sponse is extinguished. If these predictionsare verified, then we can conclude that P300size does not differentiate opposite outcomes

in LTM because a monotonic decrease inP300 can predict either CNV increase (learn-ing) or CNV decrease (extinction) within thesame e~periment.

50. Some Comparisons With the FormalModels

Pearce and Hall (1980) ascribe extinction!Q.- co.mpetition between CS-US and CS-US (US = no US) associations due to an

"inhibitory link between the US and USmemories" (p. 546), They suggest this con-cept to replace Rescorlaand Wagner's (1972)notion that extinction is due to weakeningof previously established associations. Myown concept of how a conditioned reinforc-ing cue's input to a gated dipole is extin-guished is superficially similar to Pearce andHall's (Grossberg, 1972a, 1972b). I also sug-gest, however, that the competitive extinc-tion process is mediated by the drive repre-sentations and is due to gated dipole re-bounds. Neither these concepts nor theirmechanistic substrates appear in the formalmodels.

Instead, the formal models restrict them-selves to links between CS and US memories,which in turn read out the CR. In my theory,readout of the CR does not require activationof a US memory, but only of the LTM-en-coded patterns that were sampled by the CSfrom STM when the US was active, as inSections 15 and 18. These LTM-encodedpatterns can be a fractional component orother transformation of the US, due to non-isotropy of CS and US sampling pathwaysacross'the network or due to STM 'transfor-mations of the US pattern before it i~ en-coded in L TM at the CS-activated synapticknobs. I do not see how direct links from CSto US can account for the sometimes signif-icant differences between UR and CR,whereas an STM-mediated theory can easilydo so (Seligman & Hager, 1972).

Pearce and Hall (1980) suggest "that aUS representation is activated only by theomission of an expected US" (p. 543) andsuggest a fQ!:!Dula for the intensity X of thereinforcer US, namely,

X = V2; -V2; -A. (14)

I agree that an off-cell rebound can be acti-

568- STEPHE1'~ 6ReSSBER6-

,vated by the nonoccurrence of an expectedevent, mediated by a mismatch-contingentarousal burst. However, this is not the onlyway to activate a US in my theory. Just assudden offset of shock can trigger relief(Denny, 1971), a rebound can also be causedby the mere offset of a reinforcer. Further-more, the Equation 14 for rebound size isinadequate for many reasons: It does not ex-plain the inverted U in reinforcement (Ber-lyne, 1969). It does not explain the analgesiceffect whereby cutting J units of shock in halfis less rewarding then shutting J/2 units ofshock off (Campbell & Kraeling, 1953). Itdoes not explain why the reinforcing effectof shock offset should depend on the priorduratien of shock (Boe, 1966; Borozci,Storms, & Broen, 1964; Church, Raymond,& Beauchamp, 1967; Keehn, 1963; Strouthes,1965). All of these properties obtain in gateddipoles(Grossberg, 1972b). Moreover, Equa-tion 14 lumps together LTM expectancy,STM matching, nonspecific arousal, andSTM rebound properties in a way that ob-scures their mechanistic substrates, notablytheir influence on STM and L TM patternsrather than parameters.

At bottom, the formal models are led tothese difficulties because they do not ade-quately distinguish the STM and L TMmechanisms that are used in the processingof expected and unexpected events. Conse-quently, the formal models cannot easilymake the distinction that a surprising CS canreset STM in a manner that favors its ownsubsequent STM storage, whereas a fully pre-dictable US can also be stored (or, as Pearceand Hall would say, "processed") in STM byresonating with an active feedback expec-tancy. The recent theorizing of Wagner (1978)on STM priming perhaps comes closest tomaking these distinctions within the streamof formal models.

\

51. Schedule Interactions and BehavioralContrast

Similar difficulties occur in recent modelsof instrumental conditioning. Instead ofoveremphasizing L TM properties at the ex-pense of STM properties, the Hinson andStaddon (1978) theory of schedule interac-

.

.

L

r

I

r

t

iJ'hysiological J'sychology, IYO4, J{j, 4'O-4'Y.

Brown, J. L. Afterimages. In C. H. Graham (Ed.), Visionand visual perception. New York: Wiley, 1965.

Butcher, L. L. (Ed.). Cholinergic-monoaminergic inter-actions in the brain. New York: Academic Press. 1978.

Campbell, B. A., & Kraeling, D. Response strength asa function of drive level and amount of drive reduc-tion. Journal of Experimental Psychology, 1953,45,97-101.

Cant, B. R., & Bickford, R. G. The effect of motivationon the contingent negative variation (CNV). Electro-

tions and behavioral contrast completely for-sakes L TM effects to focus on STM com-petitive properties. In Grossberg (1981b) Ishow that the same theoretical ideas that Isketched herein can overcome some diffi-culties that Hinson and Staddon face in ex-plaining their data, and I make some predic-tions to test these ideas.

Pavlovian and instrumental experimentsthat have heretofore been analyzed in a frag-mentary fashion by formal models, at thecost of implying internal paradoxes and re-stricting their predictive power, can be un-derstood in a unified fashion in terms of afew psychophysiological mechanisms whoseexistence can be more directly validated byinterdisciplinary experimental paradigms.

Reference Note

I. Rusinov, V. S. An electrophysiological analysis of theconnecting function in the cerebral cortex in the pres-ence of a dominant area. Paper presented at the XIXInternational Physiology Congress, Montreal, 1953.

ReferencesAnand, B. K., & Pillai, R. V. Activation of single neu-

rones in the hypothalamic feeding centres: Effect ofgastric distension. Journal of Physiology, 1967,19263-77.

Beach, F. A. Characteristics of masculine "sex drive.'In M. R. Jones (Ed.), Nebraska Symposium on Mo-tivation (Vol. 4). Lincoln: University of Nebrask~Press, 1956.

Berlyne, D. E. The reward-value of indifferent stimu.lation. In J. T. Tapp (Ed.), Reinforcement and be.havior. New York: Academic Press, 1969.

Boe, E. E. Effect of punishment duration and intensit)on the extinction of an instrumental respon~. Journaof Experimental Psychology, 1966, 72, 125-131.

Bolles, R. C. Theory of motivation. New York: Harpe& Row, 1967.

Borozci, G., Storms, L. H., & Broen, W. E. Respon~suppression and recovery of responding at differendeprivation levels as functions of intensity and du~.tion .of pu~~hm.en~. Jo~~! of. C?~p~CG!ive am

CONDmo~ ATTENTION 569

encephalography and Clinical Neurophysiology, 1967,23, 594.

Carpenter, G. A., & Grossberg, S. Adaptation and trans-mitter gating in vertebrate photoreceptors. Journal ofTheoretical Neurobiology, 1981,1,1-42.

Church, R. M.Response suppression. In B. A. Campbell& R. M. Church (Eds.), Punishment and aversive be-havior. New York: Appleton-Century-Crofts, 1969.

Church, R. M.,Raymond, G. A., & Beauchamp, R. D.Response suppression as a function of intensity andduration of punishment. Journal of Comparative andPhysiological Psychology, 1967, 63, 39-44.

Denny, M. R. Relaxation theory and experiments. InF. R. Brush (Ed.), Aversive conditioning and learning.New York: Academic Press, 1971.

Dexter, W. R., & Merrill, H. K. Role of contextual dis-crimination in fear conditioiring. Journal ofCompar-ative and Physiological Psychology, 1969, 69, 677-681.

Dickinson, A., Hall, G., & Mackintosh, N. J. Surpriseand the attenuation of blocking. Journal of Experi-mental Psychology: Animal Behavior Processes, 1976,2,213-222.

Dickinson, A., & Mackintosh, N. J. Reinforcer specific-ity in the enhancement of conditioiring by posttrialsurprise. Journal of Experimental Psychology: AnimalBehaviorProcesses, 1979,5,162-177.

Dunham, P.J. Puirishment: Method and theory. Psy-chological Review, 1971, 78, 58-70.

Dunham, P. J.,Mariner, A., & Adams, H. Enhancementof off-key pecking by on-key puirishment. Journal ofthe Experimental Analysis of Behavior, 1969,1,156-166.

Elias, S. A., & Grossberg, S. Pattern formation, contrastcontrol, and oscillations in the short term memory ofshunting on-center off-surround networks. BiologicalCybernetics, 1975,20, 69-98.

Epstein, A. N., Kissileff, H. R., & Stellar, E. (Eds.). Theneuropsychology of thirst: New findings and advancesin concepts. Washington, D.C.: V. H. Winston, 1973.

Estes, W. K. Outline of a theory of punishment. InB. A. Campbell & R. M. Church (Ed.), Punishmentand aversive behavior. New York: Appleton:Century-Crofts, 1969.

Estes, W. K., & Skinner, B. F. Some quantitative prop-erties of anxiety. Journal of Experimental Psychology,1941,29,390-400.

Freeman, W. J. Mass action in the nervous system. NewYork: Academic Press, 1975.

Freeman, W. J. EEG analysis gives model of neuronaltemplate-matching mechanism for sensory searchwith olfactory bulb. Biological Cybernetics, 1980,35,221-234.

Freeman, W. J. A neural mechairism for generalizationover equivalent stimuli in the olfactory system. In S.Grossberg (Ed.), Mathematical psychology and psy-chophysiology. Providence, R.I.: American Mathe-matical Society, 1981.

Frey, P. W., & Sears, R. J. Model of conditioning in-corporating the Rescorla-Wagner associative axiom,a dynamic attention rule, and a catastrophe rule. Psy-chologicalReview, 1978,85,321-340.

Friedhoff, A. J. (Ed.). Catecholamines and behavior: Vol.1. Basic neurobiology. New York: Plenum Press,

1975.(a)Friedhoff; A. J. (Ed.). Catecholamines and behavior: Vol.

2. Neuropsychopharmacology. New York: Plenum

Press, 1975.(b)Gabriel, M., Foster, K., Orona, E., Saltwick, S. E., &

Stanton, M. Neuronal activity of cingulate cortex,anteroventral thalamus and hippocampal formationin discriminative conditioning: Encoding and extrac-tion of the significance of conditional stimuli. Prog-ress in Psychobiology and Physiological Psychology,1980,9, 125-231.

Gibson, J. J. The ecological'approach to visual percep-tion. Boston: Houghton Mifflin, 1979.

Grossberg, S. The theory of embedding fields with ap-plications to psychology and neurophysiology. NewYork: Rockefeller Institute for Medical Research, .

1964.Grossberg, S. Nonlinear difference-differential equations

in prediction and learning theory. Proceedings of theNational Academy of Sciences, 1967,58, 1329-1334.

Grossberg, S. Some physiological and biochemical con-sequences of psychological postulates. Proceedings ofthe National Academy of Sciences, 1968, 60, 758-765.

Grossberg, S. Embedding fields: A theory of learningwith physiological implications. Journal of Mathe-matical Psychology, 1969,6, 209-239.(a)

Grossberg, S. On the serial learning of lists. Mathemat-ical Biosciences, 1969,4, 201-253.(b)

Grossberg, S. Some networks that can learn, remember,and reproduce any number of complicated space-time patterns, I. Journal of Mathematics and Me-chanics, 1969,19, 53-91.(c)

Grossberg, S; On the dynamics of operant conditioning.Journal of Theoretical Biology, 1971,33,225-255.

Grossberg, S. A neural theory of punishment and avoid-ance, I. Qualitative theory. Mathematical Biosciences,1972,15, 39-67.(a)

Grossberg, S. A neural theory of punishment and avoid-

ance, ll. Quantitative theory. Mathematical Biosci-

ences, 1972,15, 253-285.(b)Grossberg, S. Pattern learning by functional-differential

neural networks with arbitrary path weights. In K.Schmitt (Ed.), Delayandfunctional-dijferentiafequa-tions and their applications. New York: AcademicPress, 1972.(c)

Grossberg, S. Contour enhancement, short-term mem-ory, and constancies in reverberating neural networks.Studies in Applied Mathematics, 1973, 52, 217-257.

Grossberg, S. Classical and instrumental learning byneural networks. In R. Rosen & F. Snell (Eds.), Prog-ress in theoretical biology (Vol. 3). New York: Aca-demic Press, 1974.

Grossberg, S. A neural model of attention, reinforce-ment, and discrimination learning. International Re-view of Neurobiology, 1975,18,263-327.

Grossberg, S. Adaptive pattern classification and uni-versal recoding, I: Parallel development and codingof neural feature detectors. Biological Cybernetics,1976,23, 121-134.(a)

Grossberg, S. Adaptive pattern classification and uni-


versa! recoding, II: Feedback, expectation, olfaction,and illusions. Biological Cybernetics, 1976,23, 187-202.(b)

Grossberg, S. Behavioral contrast in short-tenn memory:Serial binary memory models or parallel continuousmemory models? Journal of Mathematical Psychol-ogy, 1978,17, 199-219.(a)

Grossberg, S. A theory of human memory: Self-orga-nization and perfonnance of sensory-motor codes,maps, and plans. In R. Rosen & F. Snell (Eds.), Prog-ress in theoretical biology (Vol. 5). New York: Aca-demic Press, 1978.(b)

Grossberg, S. How does a brain build a cognitive code?Psychological Review, 1980,87,1-51.

Grossberg, S. Adaptive resonance in development, per-ception, and cognition. InS. Grossberg (Ed.), Math-emaiical psychology and psychophysiology. Provi-dence, R.I.: American Mathematical Society, 1981.(a)

Grossberg, S. Psychophysiological substrates of scheduleinteractions and behavioral contrast. In S. Grossberg(Ed.), Mathematical psychology and psychophysiol-ogy. Providence, R.I.: American Mathematical Soci-ety, 1981.(b)

Grossberg, S. Some psychophysiological and phanna-cological correlates of a developmental, cognitive,andmotivational theory. In J. Cohen, R. Karrer, & P.Tueting (Eds.), Cognition and brain activity. NewYork: New York Academy of Sciences, 1982.(a)

Grossberg, S. Studies of mind and brain: Neural prin-ciples of learning, perception, development, cognition.and motor control. Boston: Reidel Press, 1982.(b)

Grossberg, S., & Levine, D. S. Some developmental andattentional biases in contrast enhancement and short-tenn memory of recurrent neural networks. Journalof Theoretical Biology, 1975,53,341-380.

Grossberg, S., & Pepe.. J. Spiking threshold and over-arousal effects in serial learning. Journal of StatisticalPhysics, 1971,3,95-125.

Groves, P. M., Young, S. J., & Wilson, C. J. Nigro-striatal relations and the mechanisms of action of am-phetamine. In L. L. Butcher (Ed.), Cholinergic-mono-aminergic interactions in the brain. New York: Aca-demic Press, 1978.

Hall, G., & Pearce,J. M. Latent inhibition ofaCS duringCS-US pairings. Journal of Experimental Psychology:Animal Behavior Processes, 1979,5,31-42.

Hammond, L. J. Retardation of fear acquisition by apreviously inhibitory CS. Journal of Comparative andPhysiological Psychology, 1968, 66, 7?6~758.

Hebb, D. O. The organization of behavior. New York:Wiley, 1949.

Hebb, D. O. Drives and theC.N.S. (conceptual nervoussystem). Psychological Review, 1955,62,243-254.

Held, R. Exposure-history as a factor in maintainingstability of perception and coordination. Journal ofNervous and Mental Diseases, 1961,132,26-32.

Held, R. Dissociation of visual functions by deprivationand rearrangement. Psychologische Forschung, 1967,31, 338-348.

Held, R., & Hein, A. Movement-produced stimulationin the development of visually guided behavior. Jour-nal of Comparati~e and Physiological Psychology,1963,56, 872-876.

II

Helmholtz, H. von. Handbuch der physiologischen optik(1st ed.). Leipzig, Gernlan Democl:atic Republic:Voss, 1866.

Helmholtz, H. von. Physiological optics (Vol. 2) (J. P.Southall, Ed.). New York: Dover, 1962. (Originallypublished, 1866.)

Hinson, J. M., & Staddon, J. E. R. Behavioralcompeti-tion: A mechanism for schedule interactions. Science,1978,202,432-434.

Honig, W. K. Attention and the modulation of stimuluscontrol. In D. I. Mostofsky (Ed.), Attention: Contem-porary theory and analysis. New York: Appleton.(;en-tury-Crofts, 1970.

Irwin, D. A., Rebert, C. S., McAdam, D. W., & Knott,J. R. Slow potential change (CNV) in the human EEGas a function of motivational variables. Electroen-cephalography and Clinical Neurophysiology, 1966,

21,412-413.Janowitz, H. D., Hanson, M. E., & Grossman, M. I.

Effect of intravenouSly administered glucose on foodintake in the dog. American Journal of Physiology,

1949,156,87-91.John,E. R. Neural processes during learning. In R. W.

Russell (Ed.), Frontiers in physiological psychology.New York: Academic Press, 1966.

John, E. R. Mechanisms of memory. New York: Aca-demic Press, 1967.

Kaczmarek, L. K., & Babloyantz, A. Spatiotemporalpatterns in epileptic seizures. Biological Cybernetics,1977,26,199-208.

Kamin, L. J. "Attention-like" processes in classical con-ditioning. In M. R. Jones (Ed.), Miami Symposiumon the Prediction of Behavior: Aversive stimulation.Miami: University of Miami Press, 1968.

Kamin, L. J. Predictability, surprise, attention, and con-ditioning. In B. A. Campbell & R. M. Church (Eds.),Punishment and aversive behavior. New York: Ap-pleton-Century-Crofts, 1969.

Keehn, J. D. Effect of shock duration on Sidman avoid-ance response rates. Psychological Reports, 1963,13,852.

Kremer, E. F. Effect of posttrial episodes on cqnditioningin compound conditioned stimuli. Journal of Exper-imental Psychology: Animal Behavior Processes, 1979,5, 130-141,

Lantz, A. E. Effects of number of trials, interstimulusinterval and dishabituation during CS habituation onsubsequent conditioning in a CER paradigm, AnimalLearning and Behavior, 1973,1,273-277.

Le Magnen, J. Regulation of food intake. In F. Reichs-man (Ed.), Hunger and satiety in health and disease.Basel, Switzerland: S. Karger, 1972.

Levine, D. S., & Grossberg, S. Visual illusions in neuralnetworks: Line neutralization, tilt aftereffect, and an-gle expansioq. Journal of Theoretical Biology, 1976,61,477-504.

Lubow, R. E., Rifkin, B., & Alek, M. The context effect:The relationship between stimulus preexposure andenvironmental preexposure determines subsequentlearning. Journal of Experimental Psychology: AnimalBehavior Processes, 1976,2,38-47.

Macchi, G., & Rinvik, E. Thalamo-telencephalic cir-

57]

~NDmONING

AND A mmJ_Q~-

cuits: A neuroanatomical survey. In A. Remond (Ed.),Handbook of ~lectroencephalography and clinical neu-rophysiology (Vol. 2, Pt. A). Amsterdam: Elsevier,1976.

Mackintosh, N. J. An analysis of overshadowing andblocking. Quarterly Journal of Experimental Psy-chology, 1971,23,118.,.125.

Mackintosh, N. J. A theory of attention: Variations inthe associability Qf stimuli with reinforcement.. Psy-chological Review, 1975, 82, 276-298.

Mackintosh, N. J. Overshadowing and stimulus inten-sity. Animal Learning and Behavior; 1976, 4, 186-192.

Mackintosh, N. J., Bygrave, D. J., & Picton, B. M. B.Locus of the effect of a surprising reinforcer in theattenuation of blocking. Quarterly Journal of Exper-imental Psychology, 1977, 29, 327~336.

Mackintosh, N. J., & Reese, B. One-trial overshadowing.Quarterly Journal of Experimental Psychology, 1979,31,519.,.526.

Maier, S. F., Seligman, M. E. P., & Solomon; R. L.Pavlovian fear conditioning and learned helplessnesseffects on escape and avoidance behavior of (a) theCS-US contingency and (b) the independence of theUS and voluntary responding. In B. A. Campbell &R. M. Church (Eds.), Punishment and aversive be-havior.:New York: Appleton-Century-Crofts, .1969.

Masterson, F. A. Is termination of a warning signal aneffective reward for the rat? Journal of Comparativeand Physiological Psychology, 1970, 72,4.71-475.

McAllister, W. R., & McAllister, D. E. Behavioral mea-surement of conditioned fear. In F. R. Brush (Ed.),Aversive conditioning and learning. New York: Aca-demic Press, 1971.

Miller, N. E. Learning resistance to pain and fear: Effectsof overlearning, exposure, and rewarded exposure incontext. Journal of Experimental Psychology, 1960,60,137-145.

Morrell, F. Electrophysiological contributions to theneural basis of learning. Physiological Review, 1961,41,443-494.

NMtlInen, R., Hukkanen, S., & Jarvilechto, T. Magni-tude of stimulus deviance and brain potentials. InH. H.' Kornhuber & L. Deecke (Eds.), Progress ilibrain research: Vol. 54. Motivation, motor and sensoryprocesses of the brain. New York: Elsevier, 1980.

Newman, F. L., & Baron, M. R. Stimulus generalizationalong the dimension of angularity: A comparison oftraining procedures. Journal of Comparative andPhysiological Psychology, 1965, 60, 59-63.

O'Keefe, J. 0., & Nadel, r:. The hippocampus as a cog-nitive map. Oxford, England: Oarendon Press, 1978.

Pearce, J. M., & Hall, G. A model for Pavlovian learning:Variations in the effectiveness of conditioned but notof unconditioned stimuli. Psychological Review, 1980,87, 532-552.

Rescorla, R. A. Establishment of a positive reinforcerthrough contrast with shock. Journal of Comparativeand Physiological Psychology, 1969,67,260-263:

Rescor1a, R. A., & 'LoLordo, V. M. Inhibition of avoid-ance behavior. Journal of Comparative and Physio-

logical Psychology, 1965,59,406-412.Rescorla, R. A., & Wagner, A. R. A theory of Pavlovian

"'

-5n -S+EP-HENGROSSBERG

~

fect. American Journal of Psychology, 1963, 76, 429-435.(a)

Wallach, H., & Karsh, E. B. Why the modification ofstereoscopic depth-perception is so rapid. AmericanJournal of Psychology, 1963, 76,413-420.(b)

Wallach, H., Moore, M. E., & Davidson, L. Modifica-tion of stereoscopic depth-perception. AmericanJour-nal of Psychology, 1963, 76, 191-204.

Weisman, R. G., & Litner, J. S. The course of Pavlovianextinction and inhibition of fear in rats. Journal ofComparative and Physiological Psychology, 1969,69,667-672.

Received Octo~r 29, 1981Revision receIved February 7, 1982 .

cessing mechanism for self-generated or retrieval-gen-erated depression in perfonnance. In T. J. Tighe &R. N. Leaton (Eds.), Habituation: Perspectives fromchild development. animal behavior. and neurophys-iology. Hillsdale, N.J.: Erlbaum, 1976.

Wagner, A. R. Expectancies and the priming of STM.In S. H. Hulse, H. Fowler, & W. K. Honig (Eds.),Cognitive processes in animal behavior. Hillsdale,N.J.: Erlbaum, 1978.

Wagner, A. R., & Rescorla, R. A.Inhibition in Pavlovianconditioning: Application of a theory. In R. A. Boakes& M. S. Halliday (Eds.), Inhibition and learning. NewYork: Academic Press, 1972.

Wagner,A.R.,RudY,J. W., & Whitlow,J. W. Rehearsalin animal conditioning. Journal of Experimental Psy-chology, 1973,97,407-426.

Wallach, H., & Karsh, E. B. The modification of ste-

reoscopic depth-perception and the kinetic depth-ef-

Processing of Expected and Unexpected Events During...

Documents

Transcript of Processing of Expected and Unexpected Events During...