Dopamine reveals neural circuitmechanisms of fly memoryScott Waddell

Department of Neurobiology, University of Massachusetts Medical School, 364 Plantation Street, Worcester, MA 01605, USA

Review

Glossary

Action selection: often referred to as choosing ‘what to do next’. The task of

selecting between a number of behavioral actions to best achieve a goal.

Appetitive memory: a memory whose expression involves approach behavior.

In the fly experiments discussed here, this involves training flies by pairing one

odorant with sucrose reward and another with nothing. When subsequently

given the choice between the two odorants in a T-maze, the flies preferentially

approach the previously sugar-paired odorant. Appetitive learning and

memory performance is facilitated by hunger and suppressed by satiety and

is therefore considered to be a goal-directed action.

Aversive memory: a memory whose expression involves avoidance behavior.

In the fly experiments discussed here this involves training flies by pairing one

odorant with electric shock and another with nothing. When subsequently

given the choice between the two odorants in a T-maze the flies preferentially

avoid the previously shock-paired odorant. A new automated version of the

assay also allows trial-and-error learning where single flies sample two odor

streams and learn to avoid the odor stream triggering punishment.

Phasic dopamine release: transient release of dopamine evoked by neural

activity in DA neurons.

Prediction error: the difference between the value of the actual reward (or

punishment) experienced and the expected reward (or punishment).

Reinforcement: depending on the context, reinforcement is either a rewarding

or aversive stimulus or the neural representation of the reinforcing stimulus.

Reinforcement learning: how artificial or neural systems learn by trial-and-

error to maximize rewards and minimize punishments.

Temporal difference (TD) algorithm: a computational solution to calculating

and updating a prediction error involving an ongoing comparison of the

present and predicted state/situation.

Tonic dopamine release: continuous and sustained ‘background’ dopamine

A goal of memory research is to understand how chan-ging the weight of specific synapses in neural circuits inthe brain leads to an appropriate learned behavioralresponse. Finding the relevant synapses should allowinvestigators to probe the underlying physiological andmolecular operations that encode memories and permittheir retrieval. In this review I discuss recent work inDrosophila that implicates specific subsets of dopamin-ergic (DA) neurons in aversive reinforcement and appe-titive motivation. The zonal architecture of these DAneurons is likely to reveal the functional organizationof aversive and appetitive memory in the mushroombodies. Combinations of fly DA neurons might codenegative and positive value, consistent with a motiva-tional systems role as proposed in mammals.

IntroductionThe neuromodulator dopamine has been ascribed manyimportant roles in the vertebrate and invertebrate nervoussystem. In mammals these functions include motor control,reinforcement, motivation, memory, arousal and attention[1–4]. A dysfunctional DA system has been linked to anumber of neurological and psychiatric disorders includingParkinson’s disease, schizophrenia, depression, drug addic-tion and attention deficit disorder [3–5]. The organization ofthemammalianDAsystem is extraordinarily complex, withneurons clustered into nine major nuclei and projectingbroadly throughout the brain [6]. Evidence suggests thatDAneurons from at least two of these clusters in the ventraltegmental area (VTA) and substantia nigra process infor-mation from rewarding stimuli and distribute this infor-mation to areas of the brain involved inmotivation and goal-directedbehavior. In themouse thereareapproximately20–

30 000 DA neurons that innervate the striatum, limbic andcortical areas, and eachbrain structure receives amixture ofprojections from both the VTA and substantia nigra. It istherefore apparent that thesemajor brain regions are inner-vated by a large and heterogeneous population of DAneurons. This complexity represents a considerable barrierto understanding how the dopamine system precisely influ-ences nervous system function.

Perhaps surprisingly, dopamine has also been impli-cated in the control of movement [7–9], reinforcement [10],motivation [11], memory [10,12,13], arousal [14–18] andstimulus saliency [19] in the fruit fly Drosophila melano-gaster. However, in the Drosophila brain these processesare orchestrated by a total of only 600 DA neurons whosecell bodies are organized into fifteen clusters [20,21]. Thisrelative simplicity provides an obvious advantage for

Corresponding author: Waddell, S. (scott.waddell@umassmed.edu).

0166-2236/$ – see front matter � 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.tins.2010.

understanding the functional organization of a completeDA system. In addition, sophisticated genetic manipula-tion in Drosophila allows one to interrogate the molecularand cellular properties of the DA system with spatial andtemporal precision. Taking advantage of this geneticarsenal (see Table S1 in the supplementary materialonline), investigators have recently described the anatomyof parts of the DA system in the fly at single-cell resolution[22,23] and have functionally dissected the role of sub-populations of DA neurons in locomotion [9,24], reinforce-ment [11,23–25] (Glossary) and motivation [11]; in somecases to the level of a single morphological type of DAneuron [9,11,25]. This analysis has indicated that distinctfly DA neurons may influence these neural processes byinnervating discrete anatomical zones.

The ability to identify and genetically label small groupsor even single DA neurons that execute particular tasks inthe fly would allow one to determine their underlying phys-iological operations. Precise and reproducible cell-specificexpression permits live-imaging of genetically encodedreporters of neural activity and aids placement of pipettesfor direct electrophysiological recording. Pioneering studiesof appetitive conditioning in monkeys, pairing auditory orvisual cues with fruit juice or a small piece of apple, haveestablished a popular model for midbrain DA neurons in

release.

07.001 Trends in Neurosciences, October 2010, Vol. 33, No. 10 457

Review Trends in Neurosciences Vol.33 No.10

reward learning [26]. In this reinforcement learning model,phasic firing of midbrain DA neurons drives changes insynaptic strength of receptive circuits by signaling devi-ations or ‘errors’ between predicted reward and actualreward, instead of simply coding the occurrence of a reward-ing stimulus. Themagnitude of DA neuron activity scales tothe unpredictability of the reward until the animal haslearned an accurate prediction. At that point, reinforcementrecedes to zero and the DA neurons instead fire in responseto the stimulus predicting reward, which is proposed toguide action selection. Importantly, if no reward is pre-sented following thepredictive stimulus,DAneuronactivityis suppressed below baseline at the time of the expectedreward.MammalianmidbrainDAneurons thereforeappearto provide a constant update of the environment and code apositive prediction error if actual reward is more thanpredicted, no error if prediction matches actual reward,and a negative error if actual reward is less than expected[26]. Although prediction error models have a strong quan-titative and theoretical framework in temporal difference(TD), or TD algorithms of machine learning [27,28], DAneurons are not universally accepted to function this way[29]. Finding that certain fly DA neurons operate like theirmammalian counterpartswould both validate the use of thefly as a model for DA neurons and permit genetic dissectionof the underlying neural algorithms. In this review I willillustrate how the fly DA system is being dissected byfocusing on recent work investigating a role for DA neuronsin reinforcement [22,24,25] and motivation [11] in the con-text of fly olfactory memory.

Teaching fruit fliesA number of assays have been developed to address howexperience is represented in the fly brain. Here I focus onthe study of variations of two classic associative olfactory

Figure 1. Model of the fly brain detailing the position and anatomy of the mushroom

neurons that synapse in the antennal lobes. Transformed information is then carried on

to poorly characterized neurons in the lateral horn. The approximately 2500 Kenyon ce

based on the bundling of their projections in the region of the MB called the lobes. Each

branch vertically to the a lobe and one horizontally to the b lobe. Similarly, each neuron

lobe. The g neurons send a single unbranched axon horizontally in the g lobe. All thre

removed in the left hand MB to reveal the position of the ab (green) and ab (red) lobe

collected by Shamik DasGupta.

458

learning paradigms [30,31]. Presenting flies with an odorpaired with electric shock produces a labile aversive odormemory [30], whereas multiple spaced training trials forman aversive long-term memory that lasts for days [32].Alternatively, conditioning hungry flies by pairing anodorant with sugar forms appetitive odor memory [31]and a single 2 minute training session can form long-termappetitive memory that lasts for days [33,34].

Memory in either paradigm is observed at given timesafter training as a selective avoidance of, or attraction to,the previously reinforced odorant, when given a choicebetween that odorant and another. Understanding thedifference between aversive and appetitive memory pro-cesses has been a driving force in the field for several yearswith the prevailing model being that, unlike in mammals,dopamine in Drosophila mediates aversive reinforcementwhereas reward is carried by octopamine [10,35]. A collec-tion of recent papers [11,22,24,36,37] have added to theneuroanatomical resolution of our understanding of aver-sive and appetitive memory processes and together theychallenge a strict dissociation between the monoaminesand reinforcement processing in the fly. Coincidentally,recent work has suggested that discrete populations ofmammalian DA neurons convey appetitive and aversivesignals [38]. In this review I discuss these findings toillustrate that we are fast approaching a detailed under-standing of the neural circuit mechanisms of fly learningand that these mechanisms could turn out to be morecomparable to mammals than was previously appreciated.

Odor memories in mushroom body neuronsMushroom body (MB) neurons (Kenyon cells) are some ofthe third order neurons of the insect olfactory system, anda large body of literature points to these neurons as thelikely location for olfactory memories [39] (Figure 1). The

bodies. Olfactory information enters the brain through axons of olfactory sensory

the axons of projection neurons (blue) to the MB neuron dendrites in the calyx and

lls in each MB are roughly subdivided into three morphologically distinct groups

Kenyon cell that contributes to the ab subdivision bifurcates and sends one axon

in the ab lobe bifurcates and sends one axon branch to the a lobe and one to the b

e lobes are shown in the right hand MB whereas the g lobe (magenta) has been

s. Scale bar, 100 mm. Image constructed by Wolf Huetteroth from confocal stacks

Review Trends in Neurosciences Vol.33 No.10

�2500 MB neurons can be functionally subdivided basedon morphology, molecular markers [40–43], and by theirroles in memory processing. Experiments blocking synap-tic output from different MB neuron populations suggestthat the ab neurons are required during learning, for earlymemory retrieval, and for memory consolidation [33,44,45]whereas the ab neurons are required for early retrieval[46,47] and are particularly crucial for latermemory retrie-val [33,46,48,49]. There is some indication that olfactorymemory processes might be distributed between ab

neurons and the unbranched g neurons [49–53] but exper-iments have revealed little or no consequence of blocking g

neuron output for short-term memory (STM) [46,52] orlong-term memory (LTM) [49]. Therefore their contri-bution to guiding behavior is presently enigmatic. A plaus-ible model, supported by the majority of data regarding therequirement of different subsets of MB neurons, is thatinitial memory is laid down in parallel in ab neurons andab neurons and it is refined to only ab neurons throughconsolidation involving ab and dorsal pairedmedial (DPM)neuron activity [33,36,39,44].

Electro- and opto-physiological recordings at the level ofKenyon cell bodies suggest that, as in the locust [54], odorsare sparsely coded as a few action potentials in a smallnumber of Kenyon cells [55] – amode that would endow theMB with the capacity to store a large collection of highlyspecific olfactory memories. The key to models involving‘parallel memory traces’ is that each odor is representedacross the ensemble and in each class of Kenyon cell.Unfortunately, direct evidence for such a model is cur-rently scant. Electrophysiological recordings from naıveflies found that ab neurons were more broadly tuned andresponded more vigorously to odors than ab neurons and g

neurons – with g neurons being the least responsive andmostly responding below threshold [55]. In addition, arecent detailed structural study of the MB [56] showedthat dendrites from ab neurons and ab neurons are pre-dominantly localized to a peripheral zone in the MB calyxwhereas g neuron dendrites mostly occupy the center.These data are consistent with prior studies [43,57] andwith the notion that some projection neurons couldsynapse with ab neurons and ab neurons in the samemicroglomeruli in the calyx [56]. This anatomywould allowsome olfactory information to be simultaneously deliveredto certain ab neurons and ab neurons.

It is possible that odor-evoked activity is more wide-spread across the MB neuron population and that evensubthreshold or biochemical signals can be coded intomemory as suggested by a study in the moth [58]. Inaddition, odor representations in the MB could be alteredby learning. It has been proposed that learning recruitsnew spiking neurons to the odor representation in the fruitfly antennal lobe [59] and it is therefore possible that g

neurons become more responsive after training. Othershave proposed that learning increases the reliability ofodor-evoked Kenyon cell firing and instead further spar-sens odor representations in the Kenyon cells of the locust[60]; this might also be expected to increase the level ofdiscrimination between odors. Assuming technical bar-riers can be overcome, it should be helpful to image activityin ensembles of MB neurons with more sensitive geneti-

cally-encoded reporters and at the spatial resolutionrequired to distinguish individual neurons so as to differ-entiate between these possibilities.

An important question is whether olfactory memoriesare represented at MB input and/or output synapses.Olfactory input is most evidently received from cholinergicprojection neurons in the MB calyx whereas output isbelieved to occur from the MB lobe region. However, it isclear that the MB lobes also receive input and thereforeboth pre- and postsynaptic plasticity could occur in the loberegion. Nevertheless, most current models of memory for-mation in flies favor presynaptic mechanisms of plasticityin MB neurons. To resolve the mechanisms and loci ofmemory we need to be able to localize memory-relevantplasticity to individual MB synapses rather than to entiregroups of neurons. Recent work suggests an additionalsophisticated anatomical and functional layer of subdivi-sion within the individual MB lobes [11,22,24,61] thatcould guide microscopic study to the relevant places inthe MB.

DA reinforcement pathways intersect the MBIt is reasonable to expect that the odor-activated MBneurons that also receive innervation from the relevantreinforcement systems harbor synapses where memoriesare written [35]. The general model is that neurotrans-mission fromDAneurons provides the reinforcing effects ofpunitive electric shock [10,35]. So where do theDAneuronsproject within the MB? Recent detailed clonal analysisdemonstrated a remarkable and unforeseen complexityand diversity of DA neurons that innervate the MB [22].Eight different types of individual DA neurons were ident-ified that project from three discrete clusters of cell bodiesto non-overlapping zones that decorate most of the MB.Furthermore, there is substantial heterogeneity within agiven cluster. For example, the protocerebral posteriorlateral (PPL) 1 cluster was found to contain five types ofDA neuron projecting to exclusive zones on the MB lobesincluding some that specifically innervate the a and a lobes(Figure 2). This type of circuit element is an expectation ofmodels predicting the formation of parallel early memorytraces. Functional imaging of the DA neurons innervatingthese different zones has revealed a variety of responseparameters [22], consistent with the idea that each zoneserves a different function.

Another recent study employing a new single-fly versionof the olfactory conditioning assay identified the DAneurons in PPL1 to be required for negative reinforcement[24]. Silencing most DA neurons in the fly impairs aversiveconditioning whereas blocking most DA neurons, but notthose in the PPL1 cluster, has no effect. More impressively,by expressing a light-addressable neuronal trigger in DAneurons it was shown that flies can be aversively con-ditioned by coupling odor exposure with remote activationof the DA neurons, but only if the PPL1 neurons areincluded. These experiments suggest that the PPL1 DAneurons carry reinforcement. As mentioned above and asdescribed in Figure 2, PPL1 includes DA neurons thatinnervate the peduncle, heel, junction, stalk, a tip and a

tip zones on the MB [22,24]. A different recent study usinga heat-activated neural trigger determined that specific

459

Figure 2. Zones of the mushroom body innervated by DA neurons. (a) Projection

view of a subset of DA neurons in the protocerebral posterior lateral (PPL) 1 cluster.

A brain from a transgenic fly expressing a red fluorescent protein in MB neurons

and a green fluorescent protein in MB-MP neurons that innervate the MB-heel and

peduncle region (shown as green in panel b) and also the neurons shown as

magenta in panel b that seem to project around the MB. Image adapted, with

permission, from Ref. [11]. Scale bar, 20 mm. (b) The PPL1 cluster. The Butcher’s

cut illustration summarizes data from Refs. [11,22,25]. The image shows one

neuron projecting to the tip of the a lobe (red), one to the tip of the a tip (orange),

one to the upper stalk of the vertical lobes (MB-V1 neuron, yellow), one to the

lower stalk and junction region (MB-MV1 neuron, blue) and one to the heel and

distal peduncle (MB-MP neuron, green). In addition, PPL1 includes a neuron that

innervates the anterior superior and inferior medial protocerebrum (magenta), that

runs along a similar path to the MB-MP neurons but does not appear to innervate

the MB, and another neuron that innervates the region behind the a lobe and

sends another branch into the central complex (grey cell body, projection not

shown for clarity). All neurons shown have a projection to a similar zone on the

contralateral MB (dotted lines with arrowhead). The cell body locations are not

stereotyped. Although the number of neurons in each class has not been

exhaustively determined, there appear to be at least two MB-MP neurons per

PPL1 cluster [11]. There are also likely to be multiple neurons in some of the other

classes because four of the 12 neurons in PPL1 remain to be accounted for. (c) The

PPL2ab and protocerebral anterior medial (PAM) clusters. The illustration

summarizes data described in Refs. [22,25]. At least two neurons from the

PPL2ab cluster innervate the ipsilateral MB calyx (brown and blue). Tyrosine

hydroxylase staining suggests that DA neurons project to discrete zones of the

horizontal b, b and g MB lobes (marked with black dotted lines for b and orange for

g) but the only individual PAM neurons described so far ramify on the tip of the b

lobe (MB-M3 neurons, cyan) and a similar zone on the contralateral MB (dotted line

with arrowhead) or further along the b lobe (MB-MVP1 neurons, purple).

Review Trends in Neurosciences Vol.33 No.10

460

activation of the MB-MP neurons (named after the regionsof theMB they innervate – themedial lobe and peduncle) inthe PPL1 cluster could substitute for aversive reinforce-ment [25] although acutely blocking these neurons did notimpair aversive memory formation [11]. It therefore seemslikely that aversivememories lie underneath one of theMBzones innervated by PPL1 neurons and that if the MB-MPneurons are involved in aversive reinforcement, they workin concert with other types of PPL1 neuron.

Finding that neurons projecting to the vertical a and a

lobes are likely to be those that carry aversive reinforce-ment is consistent with two previous live-imaging studiesthat identified a vertical branch-specific plasticity aftertraining. An early 30–60 minute increase in odor-evokedCa2+ and synaptic transmission was observed in the pro-jections of DPM neurons on the vertical lobes consistentwith the time at which their output is required to con-solidate memory [62]. Furthermore, 3–9 hours after LTMtraining odor-evoked Ca2+ increased in only the a projec-tions of ab neurons [48] (Figure 1). Taken together with thecrucial requirement for ab neurons in LTM [33,48,49], itseems reasonable to conclude that aversive LTM must berepresented in the synapses of the a projections. Given thatflies lacking the vertical MB a projections have no LTM,but retain STM [63], it is conceivable that labile memory isserved by the b projections of the same ab neurons, andperhaps the g neurons (Figure 1). Individual DA neuronsfrom the protocerebral anterior medial (PAM) cluster arelikely to innervate comparably complex zones within thehorizontal b, b and g lobes [22] (Figure 2C) and recent workusing a heat-activated neural trigger showed that acti-vation of the MB–M3 neurons from the PAM cluster cansubstitute for an aversive stimulus in conditioning [25].The MB–M3 neurons (three neurons named after theregion of the MB they innervate – the medial lobe) prim-arily innervate the tip of the b lobe and sparsely innervatea small zone near the tip of the b lobe. Importantly,blocking MB-M3 neuron output seems to specifically dis-rupt a component of labile memory consistent with MBsynapses in the b lobe coding labile aversive memory.

Compartmentalization of dopamine-dependentintracellular signalingDo intracellular cascades downstream of dopamine tell usmore about the organization of memory? The Drosophilagenome encodes four dopamine receptors: DopR (dDA1)[64], DopR2 (dopamine receptor in mushroom bodies,DAMB) [65], D2R/DD2R [66] and DopEcR [67]. ThedDA1 andDAMB receptors both stimulate cyclic adenosinemonophosphate (cAMP) synthesis and are known to beexpressed at high levels in the MB [64,65,68]. However,they appear to be distributed across the MB and are notobviously organized into the zones demarcated by theafferent DA neurons. It therefore seems likely that thesame type of dopamine receptor localized to different zonesin theMB neurons is fulfilling a variety of roles by respond-ing to distinct DA source neurons.

Two recent live-imaging studies [36,37] using geneti-cally encoded fluorescent reporters monitoring cAMP pro-duction or cAMP-dependent protein kinase (PKA) activitysuggest that although the dopamine receptors might be

Review Trends in Neurosciences Vol.33 No.10

evenly distributed, the signals they generate are notuniform. Surprisingly, bath application of dopamine toan isolated brain in vitro caused an increase in cAMPsynthesis across the MB lobes, whereas dopamine per-fused on the brain of a living fly caused a vertical a lobe-specific PKA activation, suggesting that DA receptors andthe cAMP and PKA dynamic are differentially coupledand/or regulated within different branches of the sameneurons [36,37]. Further experiments suggested thedunce gene encoded cAMP phosphodiesterase constrainsthe observable PKA dynamic to the vertical lobes. Strik-ingly, learning-defective dunce mutant flies exhibitedPKA activation across the entire MB when dopaminewas applied to the brain [37]. These experiments areconsistent with a model wherein cAMP levels and PKAactivity are transient in b projections of ab neurons due tothe rapid turnover of cAMP by DUNCE phosphodiester-ase, whereas in the a projections cAMP levels accumulateand activate PKA. Experiments combining acetylcholine(ACh) with dopamine application, designed to mimiccoincident MB neuron depolarization and reinforcementsignaling, led to a synergistic rise in cAMP and PKAactivity [36,37]. Furthermore, stimulating cAMP syn-thesis was sufficient to enhance ACh-evoked Ca2+ influx[36]. Prolonged cAMP signals have long been considered tobe a prerequisite for LTM formation [69] and therefore theimaging data revealing that a prolonged cAMP signal isfacilitated in the LTM-relevant a lobe are entirely con-sistent with this model.

In addition to spatially and temporally regulating cAMPturnover, cells are known to organizePKAsignaling. PKA istethered to specific subcellular locations, at certain timesand in close proximity to select substrates, by assemblingmultiprotein complexes via interactions between the PKAregulatory subunits and a large number of A-kinase anchor-ing proteins (AKAPs) [70]. AKAPs can also nucleate distinctcollections of adenylate cyclases, phosphodiesterases, Epacs(cAMP stimulated guaninine nucleotide exchange factors)and phosphatases, as well as other kinases such as proteinkinaseC. The spatial and temporal complexity of intracellu-larPKAsignaling is therefore likely tobeextremely complexand its dissection will be tricky. However, in addition to thelive-imaging studies mentioned above, two other reportshave indicated a role for PKA-anchoring in memory. Inter-fering with PKA binding to AKAPs by overexpressing ablocking peptide in MB neurons specifically disrupted anintermediate form of aversive memory [71] whereas fliesthat are mutant for the Yu-encoded AKAP have defectiveaversive long-term memory [72]. Determining the distri-bution of the different AKAPs in MB neurons and under-standing the consequences of their misregulation couldprove highly informative.

Itwill be also be important to establish the organizationand regulation of cAMP signaling that results from dopa-mine released fromafferentDAneurons onto a specificMBzone. For example, will dopamine release onto the tip ofthe a lobe lead to cAMP synthesis that is preciselyrestricted to that zone – and are downstream intracellularsignaling complexes uniquely organized within that zone?Furthermore, the very different anatomy of the individualDA neurons in PPL1 and functional imaging data [22]

suggest that these DA neurons receive discrete presyn-aptic input and could be differentially controlled. It willtherefore be crucial to investigate how DA neurons areregulated and whether particular groups of DA neuronsare simultaneously controlled and, if so, in what direction.In order for the DA neurons in the fly to code predictionerror, and switch from being driven more strongly byaversive stimuli than odorants in the naıve state to beingpreferentially driven by the conditioned odor in well-trained flies, they must be efferent to both types of infor-mation. They are likely to be pre- and postsynaptic toKenyon cells in theMB lobes because live-imaging showedthat they respond when flies are exposed to odorant orshock [22,73].

What about appetitive memory in the fly?We know less about the neural circuitry and reinforcementsignaling of appetitivememory formation following pairingof odors with sugar. Appetitive memory is similarlyimpaired in mutant flies that disrupt the cAMP pathwayand it is expected that the intersection between olfactorysignals and positive reinforcement also occurs in the MB[10], although there is also an apparent role in appetitivelearning for plasticity in projection neurons [74]. Further-more, the same dissection of MB subfields seems to applywith ab neurons being required for consolidation and ab

neurons for retrieval [33,44]. Current models propose thatpositive reinforcement is mediated by octopaminergic (OA)neurons [10,35,75], consistent with classic studies in thehoneybee [76–78]. We do not know where the relevant OAneurons project to in the fly brain; innervation of the MB isnotably scant, especially compared to that of DA neurons,and is most evident in the calyx and heel region [61,79,80].This positioning of input might be a crucial element ofspatial control, with innervation of the calyx perhapsmodulating the incoming projection neurons and MBneuron dendrites [79].

Recent data suggest that the spatial organization ofdownstream OA signaling is remarkably broad. The onlyOA receptor studied so far, OAMB (octopamine receptor inmushroom body), appears to be localized throughout theMB [42,81,82]. Furthermore, live-imaging experiments inwhich OA was bath-applied to a brain in vitro or perfusedon the exposed brain of a live fly suggest they are similarlycoupled and regulated – cAMP levels were elevated andPKA was activated in all MB regions [36,37]. In contrast tothe application of dopamine, no branch-specific PKA acti-vation was evident after OA application. Furthermore, atthe behavioral level, genetic manipulations expected todisrupt PKA localization and impair aversive memoryhad no effect on appetitive memory [71]. Although it seemslikely that appetitive memory will also be localized tospecific synapses, so far investigating OA signals has notprovided any resolution. However, it could be that we aresomewhat misled by focusing on OA alone in appetitivereinforcement. An important earlier study found a role forthe dDA1 receptor in appetitive and aversive learning [83](a result recently replicated in larvae [23]) challenging thestraightforward DA versus OA dissociation between aver-sive and appetitive memory and suggesting a more com-plex involvement of DA. It is therefore entirely possible

461

Figure 3. Hypothetical models for the involvement of DA neurons in aversive and

appetitive reinforcement. (a) DA neurons representing aversive reinforcement.

Studies from Refs [24,25] suggest that neurons in PPL1 convey negative

reinforcement whereas live-imaging data from Ref. [22] indicate that DA neurons

innervating the lower stalk, junction and a tip regions are most strongly activated

by shock. In this illustration a selection of DA neurons innervating these specific

MB zones and the MB-M3 neurons on the tip of the b lobe [25] are activated

(green). (b) Octopaminergic (OA) neuron innervation of the mushroom body (MB).

Single cell analysis has determined that OA-ventral unpaired medial (VUM) a2

neurons innervate the MB calyx whereas OA-ventral paired medial (VPM) 4

neurons innervate the MB heel [23]. Should the OA neurons that innervate the MB

be those that are important for appetitive reinforcement in Drosophila, they are

appropriately placed to influence the activity of the DA neurons in the calyx and in

the MB heel (the MB-MP neurons). (c) DA neurons that could be relieved by

appetitive reinforcement. OA neuron activation can substitute for sugar

presentation in appetitive conditioning in Drosophila larvae [75] and in the

honeybee [76]. In this model, OA neuron action would relieve the DA neurons in

the heel (MB-MP neurons) and those in the calyx that can convey negative signals

(now red rather than green in panel A). Hunger inhibits the MB-MP DA neurons to

promote appetitive memory retrieval [11]. Therefore, appetitive learning could

involve a similar MB-MP relief mechanism that relies on OA.

Review Trends in Neurosciences Vol.33 No.10

that DA is also involved in appetitive reinforcement(Figure 3).

Consistent with this, recent studies have implicated DAneurons in a different aspect of appetitive memory. Asubset of DA neurons in the PPL1 cluster were found tobe involved in a hierarchical inhibitory mechanism thatcontrols appetitive memory performance [11]. Appetitivememory performance is acutely regulated by the relative

462

satiety state of the fly and is most efficient in hungry flies[11,33]. The MB-MP neurons in the PPL1 cluster arerequired to suppress appetitive memory performance insatiated flies because blocking their output mimicked fooddeprivation and released behavioral expression of appeti-tive memory [11]. Conversely, stimulating the MB-MPneurons in hungry flies with a hot or cold-sensitive triggersuppressed memory performance. MB-MP neurons projectspecifically to the heel and the ab neuron strata at the baseof the peduncle [11,22,61], consistent with a role for theseneurons in modulating the efficiency of memory retrievalthrough these sites.

As mentioned previously, activation of MB-MP neurons(Figure 2A,B) can substitute for reinforcement in aversiveconditioning [25] but they are apparently not the PPL1neurons that are crucial for providing negative reinforce-ment value from electric shock. Blocking these neuronsduring acquisition revealed that they are dispensable foraversive (and appetitive) memory formation [11]. Takentogether with the extensive literature suggesting localiz-ation of aversive memory to the vertical a lobes [22,48,62],it is likely that the other PPL1 neurons that innervate theMB vertical lobes include those providing the negativereinforcing effects of electric shock [24]. It has been pro-posed thatMB-MPneurons represent an inhibitory gate onMB function that is released in hungry flies to promoteappetitive memory retrieval [11], and this model is com-patible with the interpretation that MB-MP neurons pro-vide reinforcement and motivational control. It is perhapsworth noting that some mammalian midbrain DA neuronsencode value-related reward signals [26] whereas othersare activated by both rewarding and aversive stimuli[38,84]. At least some of these DA neurons are inhibitedby aversive cues [85] through neurons in the lateral habe-nula that have been proposed to code negative predictionerror [86]. This organization suggests that the balancebetween the activity of the distinct midbrain DA neuronsmight code positive and negative value. Flies could havesimilar discrete populations of DA neurons that carryaversive value and others that carry appetitive value. Ifthis is the case, aversive signals could excite one group andinhibit the other, and vice versa for appetitive signals, andthe balance between the two systems would then be acrucial determinant of appetitive and aversive learningand memory-driven behavior (Figure 3). Such a balancingact might explain why dDA1 receptors are required foraversive and appetitive learning [83] because the fly isunable to encode the relative balance between aversive andappetitive signals. This antagonism between DA subsys-tems could in turn be regulated by hunger and satiety. Infact, it has been reported that sucrose produces mildaversive learning in fed flies and this switches to appetitivelearning as the fly becomes more hungry [87]. The flytherefore seems to attribute different values to sucroseduring learning [87], as well as to appetitive memoryretrieval [11], depending on relative satiety state. Furtherdetailed analysis of the MB-MP neurons and other indi-vidual types of DA neurons innervating the MB shouldresolve whether MB-MP neurons operate as part of aneural system that codes value and conveys motivationalsignals.

Box 1. Outstanding questions

� What is the high-resolution connectivity of DA neurons to MB

neurons? Do the DA processes synapse on MB neurons and/or

other MB input and output neurons? Do particular MB neurons

drive DA neurons? Do any of the DA neurons relay information

from one MB neuron to others?

� What are the inputs to the different DA neurons?

� Are groups of DA neurons coregulated? Does the activity of some

increase while that of others decreases?

� What are the firing characteristics of DA neurons? Are some of

them tonic and others phasic? If so, is there an interaction

between tonic and phasic dopamine release?

� Do fly DA neurons code negative or positive prediction error? If

not, what do the signals represent and where do the relevant

neurons project onto the MB?

� How does DA neuron activity alter the physiology of MB neurons?

� What DA receptors are involved and how are they coupled to

downstream cell signaling pathways?

� Do the anatomically distinct DA neurons that innervate the MB

perform different tasks? If so, does their anatomy also mark

relevant MB output zones?

Review Trends in Neurosciences Vol.33 No.10

DA zones and the control of other brain functionsTheMBhas been implicated in a large number of behaviorsthat do not involve olfactory stimuli, andmany of these alsoinvolve dopamine. These functions include sleep and arou-sal [17,88–90], visual context generalization [91], salience-based choice behavior [19] and behavioral flexibility [92]. Ittherefore seems clear that the MB can no longer simply beviewed as a repository for olfactory memories, and it mightbetter be described as a brain center controlling actionselection. The incredible diversity of DA neurons thatproject from a variety of places in the brain to the MBsuggests that each of these functions might be served by adifferent MB zone and be regulated by a unique subset ofDA neurons [22,61]. Mapping the input to each of these DAneurons and understanding when, where and how theirreleased DA influences MB physiology is now one of themajor challenges to be addressed (Box 1). We should soonbe able to determine whether individual DA neurons in thefly operate in the same way as larger groups of DA neuronsin mammals.

AcknowledgementsI thank Wolf Huetteroth and Shamik DasGupta for Figure 1 andBenjamin Leung, Jena Pitman, Wolf Huetteroth, Emmanuel Perisse,Geraldine Wright and the anonymous reviewers for comments on themanuscript. Work in the Waddell lab is supported by grants MH09883,MH081982 and GM085788 from the National Institutes of Health.

Appendix A. Supplementary dataSupplementary data associated with this article can befound, in the online version, at doi:10.1016/j.tins.2010.07.001.

References1 Joshua, M. et al. (2009) The dynamics of dopamine in control of motor

behavior. Curr. Opin. Neurobiol. 19, 615–6202 Dayan, P. and Balleine, B.W. (2002) Reward, motivation, and

reinforcement learning. Neuron 36, 285–2983 Wise, R.A. (2004) Dopamine, learning and motivation. Nat. Rev.

Neurosci. 5, 483–4944 Montague, P.R. et al. (2004) Computational roles for dopamine in

behavioural control. Nature 431, 760–767

5 Heinz, A. and Schlagenhauf, F. (2010) Dopaminergic dysfunction inschizophrenia: salience attribution revisited. Schizophr. Bull. 36,472–485

6 Bjorklund, A. and Dunnett, S.B. (2007) Dopamine neuron systems inthe brain: an update. Trends Neurosci. 30, 194–202

7 Pendleton, R.G. et al. (2002) Effects of tyrosine hydroxylasemutants onlocomotor activity in Drosophila: a study in functional genomics.Behav. Genet. 32, 89–94

8 Lima, S.Q. and Miesenbock, G. (2005) Remote control of behaviorthrough genetically targeted photostimulation of neurons. Cell 121,141–152

9 Kong, E.C. et al. (2010) A pair of dopamine neurons target the D1-likedopamine receptor DopR in the central complex to promote ethanol-stimulated locomotion in Drosophila. PLoS One 5, e9954

10 Schwaerzel, M. et al. (2003) Dopamine and octopamine differentiatebetween aversive and appetitive olfactory memories in Drosophila. J.Neurosci. 23, 10495–10502

11 Krashes, M.J. et al. (2009) A neural circuit mechanism integratingmotivational state with memory expression in Drosophila. Cell 139,416–427

12 Tempel, B.L. et al. (1984) Mutations in the dopa decarboxylase geneaffect learning in Drosophila. Proc. Natl. Acad. Sci. U. S. A. 81, 3577–

358113 Zhang, S. et al. (2008) Increased dopaminergic signaling impairs

aversive olfactory memory retention in Drosophila. Biochem.Biophys. Res. Commun. 370, 82–86

14 Kume, K. et al. (2005) Dopamine is a regulator of arousal in the fruit fly.J. Neurosci. 25, 7377–7384

15 Andretic, R. et al. (2005) Dopaminergic modulation of arousal inDrosophila. Curr. Biol. 15, 1165–1175

16 Foltenyi, K. et al. (2007) Neurohormonal and neuromodulatory controlof sleep in Drosophila. Cold Spring Harb. Symp. Quant. Biol. 72,565–571

17 Seugnet, L. et al. (2008) D1 receptor activation in the mushroom bodiesrescues sleep-loss-induced learning impairments in Drosophila. Curr.Biol. 18, 1110–1117

18 Lebestky, T. et al. (2009) Two different forms of arousal in Drosophilaare oppositely regulated by the dopamine D1 receptor ortholog DopRvia distinct neural circuits. Neuron 64, 522–536

19 Zhang, K. et al. (2007) Dopamine-mushroom body circuit regulatessaliency-based decision-making in Drosophila. Science 316, 1901–1904

20 Budnik, V. and White, K. (1987) Genetic dissection of dopamine andserotonin synthesis in the nervous system of Drosophila melanogaster.J. Neurogenet. 4, 309–314

21 Nassel, D.R. and Elekes, K. (1992) Aminergic neurons in the brain ofblowflies and Drosophila: dopamine- and tyrosine hydroxylase-immunoreactive neurons and their relationship with putativehistaminergic neurons. Cell Tissue Res. 267, 147–167

22 Mao, Z. and Davis, R.L. (2009) Eight different types of dopaminergicneurons innervate the Drosophila mushroom body neuropil:anatomical and physiological heterogeneity. Front. Neural Circuits3, 5

23 Selcho, M. et al. (2009) The role of dopamine in Drosophila larvalclassical olfactory conditioning. PLoS One 4, e5897

24 Claridge-Chang, A. et al. (2009) Writing memories with light-addressable reinforcement circuitry. Cell 139, 405–415

25 Aso, Y. et al. (2010) Specific dopaminergic neurons for the formation oflabile aversive memory. Curr. Biol. DOI: 10.1016/j.cub.2010.06.048

26 Schultz, W. et al. (1997) A neural substrate of prediction and reward.Science 275, 1593–1599

27 Sutton, R.S. and Barto, A.G. (1998) Reinforcement Learning: AnIntroduction, MIT Press

28 Maia, T.V. (2009) Reinforcement learning, conditioning, and the brain:Successes and challenges. Cogn. Affect. Behav. Neurosci. 9, 343–364

29 Redgrave, P. and Gurney, K. (2006) The short-latency dopamine signal:a role in discovering novel actions? Nat. Rev. Neurosci. 7, 967–975

30 Tully, T. and Quinn, W.G. (1985) Classical conditioning and retentionin normal and mutant Drosophila melanogaster. J. Comp. Physiol. A157, 263–277

31 Tempel, B.L. et al. (1983) Reward learning in normal and mutantDrosophila. Proc. Natl. Acad. Sci. U. S. A. 80, 1482–1486

32 Tully, T. et al. (1994) Genetic dissection of consolidated memory inDrosophila. Cell 79, 35–47

463

Review Trends in Neurosciences Vol.33 No.10

33 Krashes, M.J. and Waddell, S. (2008) Rapid consolidation to a radishand protein synthesis-dependent long-term memory after single-session appetitive olfactory conditioning in Drosophila. J. Neurosci.28, 3103–3113

34 Colomb, J. et al. (2009) Parametric and genetic analysis of Drosophilaappetitive long-term memory and sugar motivation. Genes BrainBehav. 8, 407–415

35 Heisenberg, M. (2003) Mushroom body memoir: from maps to models.Nat. Rev. Neurosci. 4, 266–275

36 Tomchik, S.M. and Davis, R.L. (2009) Dynamics of learning-relatedcAMP signaling and stimulus integration in the Drosophila olfactorypathway. Neuron 64, 510–521

37 Gervasi, N. et al. (2010) PKA dynamics in aDrosophila learning center:coincidence detection by rutabaga adenylyl cyclase and spatialregulation by dunce phosphodiesterase. Neuron 65, 516–529

38 Matsumoto, M. andHikosaka, O. (2009) Two types of dopamine neurondistinctly convey positive and negative motivational signals. Nature459, 837–841

39 Keene, A.C. and Waddell, S. (2007) Drosophila olfactory memory:single genes to complex neural circuits. Nat. Rev. Neurosci. 8, 341–354

40 Yang, M.Y. et al. (1995) Subdivision of the Drosophila mushroombodies by enhancer-trap expression patterns. Neuron 15, 45–54

41 Ito, K. et al. (1997) The Drosophila mushroom body is a quadruplestructure of clonal units each of which contains a virtually identical setof neurones and glial cells. Development 124, 761–771

42 Crittenden, J.R. et al. (1998) Tripartite mushroom body architecturerevealed by antigenic markers. Learn Mem. 5, 38–51

43 Strausfeld, N.J. et al. (2003) The mushroom bodies of Drosophilamelanogaster: an immunocytological and golgi study of Kenyon cellorganization in the calyces and lobes. Microsc. Res. Tech. 62, 151–169

44 Krashes, M.J. et al. (2007) Sequential use of mushroom body neuronsubsets duringDrosophila odormemory processing.Neuron 53, 103–115

45 Wang, Y. et al. (2008) Imaging of an early memory trace in theDrosophila mushroom body. J. Neurosci. 28, 4368–4376

46 McGuire, S.E. et al. (2001) The role of Drosophila mushroom bodysignaling in olfactory memory. Science 293, 1330–1333

47 Dubnau, J. et al. (2001) Disruption of neurotransmission inDrosophilamushroom body blocks retrieval but not acquisition of memory.Nature411, 476–480

48 Yu, D. et al. (2006) Drosophila alpha/beta mushroom body neuronsform a branch-specific, long-term cellular memory trace after spacedolfactory conditioning. Neuron 52, 845–855

49 Isabel, G. et al. (2004) Exclusive consolidated memory phases inDrosophila. Science 304, 1024–1027

50 Zars, T. et al. (2000) Localization of a short-termmemory inDrosophila.Science 288, 672–675

51 McGuire, S.E. et al. (2003) Spatiotemporal rescue of memorydysfunction in Drosophila. Science 302, 1765–1768

52 Akalal, D.B. et al. (2006) Roles for Drosophilamushroom body neuronsin olfactory learning and memory. Learn Mem. 13, 659–668

53 Blum, A.L. et al. (2009) Short- and long-term memory in Drosophilarequire cAMP signaling in distinct neuron types. Curr. Biol. 19, 1341–

135054 Perez-Orive, J. et al. (2002) Oscillations and sparsening of odor

representations in the mushroom body. Science 297, 359–36555 Turner, G.C. et al. (2008) Olfactory representations by Drosophila

mushroom body neurons. J. Neurophysiol. 99, 734–74656 Leiss, F. et al. (2009) Synaptic organization in the adult Drosophila

mushroom body calyx. J. Comp. Neurol. 517, 808–82457 Tanaka, N.K. et al. (2004) Integration of chemosensory pathways in the

Drosophila second-order olfactory centers. Curr. Biol. 14, 449–45758 Ito, I. et al. (2008) Sparse odor representation and olfactory learning.

Nat. Neurosci. 11, 1177–118459 Yu, D. et al. (2004) Altered representation of the spatial code for odors

after olfactory classical conditioning; memory trace formation bysynaptic recruitment. Neuron 42, 437–449

60 Finelli, L.A. et al. (2008) Synaptic learning rules and sparse coding in amodel sensory system. PLoS Comput. Biol. 4, e1000062

61 Tanaka, N.K. et al. (2008) Neuronal assemblies of the Drosophilamushroom body. J. Comp. Neurol. 508, 711–755

62 Yu, D. et al. (2005) Drosophila DPM neurons form a delayed andbranch-specific memory trace after olfactory classical conditioning.Cell 123, 945–957

464

63 Pascual, A. and Preat, T. (2001) Localization of long-term memorywithin the Drosophila mushroom body. Science 294, 1115–1117

64 Sugamori, K.S. et al. (1995) A primordial dopamine D1-like adenylylcyclase-linked receptor from Drosophila melanogaster displaying pooraffinity for benzazepines. FEBS Lett. 362, 131–138

65 Han, K.A. et al. (1996) DAMB, a novel dopamine receptor expressedspecifically in Drosophila mushroom bodies. Neuron 16, 1127–1135

66 Hearn, M.G. et al. (2002) A Drosophila dopamine 2-like receptor:Molecular characterization and identification of multiple alternativelyspliced variants. Proc. Natl. Acad. Sci. U. S. A. 99, 14554–14559

67 Srivastava, D.P. et al. (2005) Rapid, nongenomic responses toecdysteroids and catecholamines mediated by a novel Drosophila G-protein-coupled receptor. J. Neurosci. 25, 6145–6155

68 Kim, Y.C. et al. (2003) Expression of a D1 dopamine receptor dDA1/DmDOP1 in the central nervous system of Drosophila melanogaster.Gene Expr. Patterns 3, 237–245

69 Kandel, E.R. (2001) The molecular biology of memory storage: adialogue between genes and synapses. Science 294, 1030–1038

70 Scott, J.D. and Pawson, T. (2009) Cell signaling in space and time: whereproteins come together and when they’re apart. Science 326, 1220–1224

71 Schwaerzel, M. et al. (2007) Signaling at A-kinase anchoring proteinsorganizes anesthesia-sensitive memory in Drosophila. J. Neurosci. 27,1229–1233

72 Lu, Y. et al. (2007) The AKAP Yu is required for olfactory long-termmemory formation in Drosophila. Proc. Natl. Acad. Sci. U. S. A. 104,13792–13797

73 Riemensperger, T. et al. (2005) Punishment prediction bydopaminergic neurons in Drosophila. Curr. Biol. 15, 1953–1960

74 Thum, A.S. et al. (2007) Multiple memory traces for olfactory rewardlearning in Drosophila. J. Neurosci. 27, 11132–11138

75 Schroll, C. et al. (2006) Light-induced activation of distinct modulatoryneurons triggers appetitive or aversive learning in Drosophila larvae.Curr. Biol. 16, 1741–1747

76 Hammer, M. (1993) An identified neuron mediates the unconditionedstimulus in associative olfactory learning in honeybees. Nature 366,59–63

77 Hammer, M. and Menzel, R. (1998) Multiple sites of associative odorlearning as revealed by local brain microinjections of octopamine inhoneybees. Learn. Mem. 5, 146–156

78 Farooqui, T. et al. (2003) Modulation of early olfactory processing by anoctopaminergic reinforcement pathway in the honeybee. J. Neurosci.23, 5370–5380

79 Sinakevitch, I. and Strausfeld, N.J. (2006) Comparison of octopamine-like immunoreactivity in the brains of the fruit fly and blow fly. J.Comp. Neurol. 494, 460–475

80 Busch, S. et al. (2009) A map of octopaminergic neurons in theDrosophila brain. J. Comp. Neurol. 513, 643–667

81 Han, K.A. et al. (1998) A novel octopamine receptor with preferentialexpression inDrosophilamushroom bodies. J. Neurosci. 18, 3650–3658

82 Lee, H.G. et al. (2003) Octopamine receptor OAMB is required forovulation in Drosophila melanogaster. Dev. Biol. 264, 179–190

83 Kim, Y.C. et al. (2007) D1 dopamine receptor dDA1 is required in themushroom body neurons for aversive and appetitive learning inDrosophila. J. Neurosci. 27, 7640–7647

84 Schultz, W. (2010) Dopamine signals for reward value and risk: basicand recent data. Behav. Brain Funct. 6, 24

85 Ungless, M.A. et al. (2004) Uniform inhibition of dopamine neurons inthe ventral tegmental area by aversive stimuli. Science 303, 2040–2042

86 Matsumoto, M. and Hikosaka, O. (2007) Lateral habenula as a source ofnegative reward signals in dopamine neurons. Nature 447, 1111–1115

87 Liu, X. et al. (2009) The GABAA receptor RDL suppresses theconditioned stimulus pathway for olfactory learning. J. Neurosci. 29,1573–1579

88 Pitman, J.L. et al. (2006) A dynamic role for the mushroom bodies inpromoting sleep in Drosophila. Nature 441, 753–756

89 Joiner, W.J. et al. (2006) Sleep in Drosophila is regulated by adultmushroom bodies. Nature 441, 757–760

90 Andretic, R. et al. (2008) Drosophila D1 dopamine receptor mediatescaffeine-inducedarousal.Proc.Natl.Acad.Sci.U.S.A.105,20392–20397

91 Liu, L. et al. (1999) Context generalization in Drosophila visuallearning requires the mushroom bodies. Nature 400, 753–756

92 Brembs, B. (2009) Mushroom bodies regulate habit formation inDrosophila. Curr. Biol. 19, 1351–1355