Mapping Neural Interactivity onto Regional Activity: An Analysisof Semantic Processing and Response Mode Interactions

Janine M. Jennings,* Anthony R. McIntosh,* and Shitij Kapur*,†*Rotman Research Institute of Baycrest Centre, Toronto, Ontario, Canada, M6A 2E1; †PET Centre of the Clarke Institute of Psychiatry,

University of Toronto, Toronto, Ontario, Canada

Received July 7, 1997

Neuroimaging studies of cognition have typicallybeen designed to identify brain regions that are activeduring a cognitive process. However, identifying howbrain regions interact may be equally important. In arecent study we found that the pattern of activationassociated with a semantic task differed depending onhow subjects made a response, suggesting that therewas an interaction between the neural systems under-lying response mode and semantic processing (J. M.Jennings et al., 1997, NeuroImage 5, 229–239). Thisresult raises two important questions, which we exam-ined here: (1) How did the regions underlying semanticperformance influence one another, or interact, toproduce a different pattern of activation in each case?(2) What can be learned about the neurobiology ofsemantic processing when different regions are identi-fied as a function of response? We addressed thesequestions using structural equation modeling. Thistechnique produced functional network models repre-senting the effect of different regions on each otherduring the semantic task for each response. A commonnetwork of regions associated with semantic process-ing was observed and included the left inferior frontaland left superior temporal cortices, with other regionsbrought into that network depending on response(e.g., right middle frontal). Moreover, changes in theinfluences among these regions across response condi-tion predicted the pattern of activation found previ-ously. These results show how an arbitrary responsecan affect the neural pathways associated with a cogni-tive process, likely due to the parallel and reentrantorganization of the brain, and emphasize the impor-tance of examining functional connections when study-ing cognition. r 1998 Academic Press

INTRODUCTION

Brain imaging techniques provide an invaluable toolfor exploring the neural basis of cognition. For manystudies, the aim is to determine whether particularbrain regions are more or less active than other regions

through the subtraction of regional cerebral blood flow(rCBF) patterns between different tasks (e.g., Petersenet al., 1988). The results of this work have providedfundamental knowledge about the neural areas under-lying various cognitive activities (Cabeza and Nyberg,1997). However, recently, we found that neural pro-cesses may not always combine in the additive mannerthat is assumed when using the subtraction technique(Jennings et al., 1997), a concern which has been voicedin the neuroimaging literature with increasing fre-quency (Bookheimer et al., 1995; Friston et al., 1996;Sergent et al., 1992; for an interesting debate on thisissue see papers by Poeppel, 1996a,b; and Demonet etal., 1996).

In our study, PET scans were obtained while subjectscarried out a semantic task, in which they decidedwhether a word represented something ‘‘living,’’ and aletter task, in which they determined whether wordscontained the letter ‘‘a’’ (Jennings et al., 1997). The twotasks were designed to differ only in the access andutilization of word meaning, as discussed elsewhere(Kapur et al., 1994). Subjects performed each task threetimes, and each time answers were indicated by use of adifferent response mode (mouse clicking, spoken re-sponding, or silent thought). According to the subtrac-tion paradigm (e.g., Petersen et al., 1988), there shouldbe an additive relationship between response type andsemantic processing. That is, subtracting the lettertask from the semantic task for each response modeshould result in the same pattern of rCBF. In actuality,the pattern of activation following subtraction variedwith response. There was a different pattern of brainactivation in each case, and statistical testing revealeda significant interaction between processing task andresponse mode.

These results suggest that the neural basis of thesemantic task was not simply the addition of the neuralcomponents subserving the semantic process plus theneural components underlying a response, but arosefrom the interactive engagement of both sets of compo-nents. Such interactions do not seem surprising when

NEUROIMAGE 7, 244–254 (1998)ARTICLE NO. NI980325

2441053-8119/98 $25.00Copyright r 1998 by Academic PressAll rights of reproduction in any form reserved.

one considers the biological organization of the brain.As pointed out by Sergent (Sergent et al., 1992; Ser-gent, 1994), the additivity assumption requires informa-tion processing to occur in a serial, feed-forward man-ner without retroactivation or feedback. However, thereis evidence that higher cognitive functions involve therapid integration of information across several sensoryand behavioral domains through reciprocal and long-range feedback connections among widely distributedbrain regions (Edelman, 1987; Mesulam, 1990; Tononiet al., 1992, 1994). In addition, a particular site oftensubserves multiple cognitive operations depending onits functional connections with other areas (McIntosh etal., 1996, 1997). Given the interconnectedness of thebrain, determining the neurobiology of a cognitiveprocess may be better accomplished by examining howregions influence one another during task performancerather than regional localization.

One effective technique for exploring the relation-ships among regions is network analysis using struc-tural equation modeling (McIntosh and Gonzalez-Lima, 1994; McIntosh et al., 1994). A thoroughdiscussion of the practical and theoretical issues sur-rounding the use of this approach in neuroimaging hasbeen covered elsewhere, particularly in a review byMcIntosh and Gonzalez-Lima (1994). Their paper de-tails the mathematical foundation of neural structuralequation modeling and addresses the use of anatomicalconstraints for modeling using simulations and empiri-cal demonstrations. Briefly though, the method pro-duces brain models that represent the effect of differentregions on each other through their neuroanatomicalconnections. The influence of different regions is repre-sented by path coefficients, which indicate the strengthand type (positive or negative) of an effect and providean idea of how regions interact as a network to supportcognitive performance.

Network analysis is particularly advantageous formaking sense of the discrepant patterns of activationmentioned above. By determining the functional net-works associated with the semantic task for eachresponse, we can compare them to establish whetherthe different patterns of activation associated withsemantic processing can be attributed to differentialinfluences among the same brain regions as a functionof response. Moreover, we can learn more about theneurobiology of semantic processing. Trying to identifythe specific regions associated with semantic perfor-mance through activation analysis alone is problematicbecause different regions would be specified dependingon which response condition was considered. Instead,examining the influences among brain regions allowsus to determine whether they are any common connec-tions that are maintained regardless of response andcan thus be attributed to semantic processing.

In summary, the following study was carried out

using functional network analysis to determine: (1)whether differences in effective connectivity among thebrain regions associated with semantic processing ineach response condition (mouse clicking, spoken re-sponding, silent thought) can account for the statisticalinteractions that were found in our previous study(Jennings et al., 1997) and (2) whether there is anetwork of regions that appear to support semanticprocessing regardless of response mode.

MATERIAL AND METHODS

The methodology for data collection and rCBF analy-sis have been reported elsewhere (Jennings et al.,1997). However, these details are presented brieflybelow since the activation analyses provide the basisfor incorporating regions into the functional models.

PET and Behavioral Data

Data were obtained from 12 right-handed partici-pants (6 male/6 female) with a mean age of 27.3 years.Each participant underwent six PET scans. For three ofthe scans they made semantic judgments, decidingwhether each word presented on the computer screencould be considered living or nonliving; for the otherthree scans they carried out a letter processing task,checking each word presented on the computer screenfor the letter ‘‘a.’’ For each task, 48 concrete nouns(three to eight letters in length) were presented visu-ally for a duration of 2 s followed by a 500-ms interval.Participants responded ‘‘yes’’ or ‘‘no’’ during each taskby clicking the response buttons of a computer mouse(mouse-click condition), responding aloud (spoken re-sponse condition), or by thinking silently to themselves(silent thought condition). Each response condition waspaired with both processing tasks (i.e., six differentscan tasks). Task order was counterbalanced, such thateach participant received a different ordering with therestriction that tasks using the same response modenever occurred consecutively.

Participants were scanned using a GEMS-Scanditro-nix PC-2048 head scanner (15 slices, 6.5 mm apart,inplane FWHM 5 5–6 mm) with injections of 40 mCi of[15O]water for each scan. Integrated regional countswere used as an index of rCBF. Following the final scan,participants were given a recognition test to ensurethat they were performing both the semantic and theletter processing tasks consistently across each re-sponse mode. If so, there should be a levels of process-ing effect—recognition memory should be higher in thesemantic task than the letter task (Craik and Lockhart,1972) for each response condition, with comparablelevels of recognition regardless of response. Because wefound this pattern of memory performance, we canassume that any nonadditivity in the brain data did notarise from discrepancies in effort or attention during

245MAPPING NEURAL INTERACTIVITY

the tasks as a function of response. Moreover, allsubjects showed highly accurate performance duringthe semantic and letter tasks for the spoken response(0.98 and 0.99, respectively) and mouse click (0.94 and0.99, respectively) conditions, with no significant inter-action between processing task and response mode.

Activation Analyses

Images were spatially transformed, smoothed with a10-mm isotropic Gaussian filter, and corrected forindividual differences in whole brain CBF throughratio transformation using SPM95 (Friston et al., 1995).

Statistical testing for a processing task 3 responsemode interaction was performed with a new multivari-ate analysis method, partial least squares (McIntosh etal., 1996). Partial least squares (PLS) describes therelationship between a set of contrasts that code for theexperimental design and the resultant brain images,allowing the examination of changes in the pattern ofrCBF across an entire image for each processing task/response mode pairing. Because PLS considers all sixconditions and their respective images simultaneously,it is more appropriate when the experimental questionis aimed at the level of entire neural systems (McIntoshet al., 1996). Moreover, in a fully parameterized or-thonormal design matrix, in which all experimentaldegrees of freedom are represented, PLS provides anomnibus index of whether main effects and interactionsare significant (McIntosh et al., 1996). Because the PLSsolutions are invariant to orthonormal rotation of thedesign matrix (Bookstein, 1991). In contrast, the uni-variate complement of tests for main effects and interac-tions (the conventional ANOVA) is better used to assessinteractions when the hypothesis is focused on a particu-lar brain area (Friston et al., 1996).

Structural Equation Modeling

To determine the functional network underlying se-mantic processing for each response mode, we appliedstructural equation modeling to the data. We first chosea set of regions, which were identified in the PLSanalysis, for the reasons outlined below. Each regionwas represented by its peak voxel, and the globallyscaled activity counts of each peak were used to com-pute interregional correlations of activity among thebrain regions for each response mode, as done else-where (Horwitz, 1989; Horwitz et al., 1995; McIntosh etal., 1994; Nyberg et al., 1996). The anatomic connectiv-ity between these regions was then derived from theliterature on primate neuroanatomy, with a focus onthe connectivity of prefrontal and anterior temporalregions (Pandya and Yeterian, 1990; Petrides and Pan-dya, 1988). Recent neuroanatomical work has sug-gested that strong cytoarchitectural similarities existbetween the monkey and human prefrontal cortex,

which implies that the use of primate connectivity forfunctional modeling is a reasonable approximation forhumans (Petrides and Pandya, 1994).

The interregional correlations and anatomic modelwere combined to create structural equation modelsusing LISREL (Version 8, Scientific Software Inc.) todetermine the weights or path coefficients for eachanatomic connection. These values represent the influ-ence of each region on its efferent areas and reflect howmuch a unit change in activity in one region affectsactivity in the region to which it projects. Path coeffi-cients were derived through a process of iterative datafitting to the observed pattern of interregional covari-ances of activity. Since they were based on functionalactivity measured across participants, they reflect an‘‘average’’ functional influence within a given task andindicate the stability and direction of that influencewithin the sample.

Because the value of a path coefficient represents thedirect effect that one region has on another in themodel, coefficients can be regarded as indices of ‘‘effec-tive connectivity’’ (Aertsen et al., 1989). Effective connec-tivity contrasts with ‘‘functional connectivity’’ (Fristonet al., 1993), which describes interregional correlationsof activity without indicating how these correlationsare mediated. The interpretation of an effective connec-tion, as derived from a neural systems-level model(McIntosh and Gonzalez-Lima, 1994), can best be con-ceived of as the overall effect that several cell en-sembles in one area has on the ensembles in another.The sign and strength of the coefficients can be thoughtto reflect ensemble excitation (positive coefficient) orinhibition (negative coefficient). Ensemble excitation/inhibition are statements about the nature of theinfluences between brain areas, as presently defined bythe pattern of covariances in rCBF; they do not neces-sarily map onto the level of individual neurons orcolumns of neurons. This point follows from demonstra-tions showing that the emergent pattern of influenceson one level of organization (i.e., single cells) may notcorrespond directly to another level, such as large-scaleneural systems (Douglas et al., 1995; Somers et al.,1995). Further elaboration can be found elsewhere(Nyberg et al., 1996).

Task-related changes in functional influences or net-works were assessed statistically by comparing a model,where all path coefficient estimates were fixed to beequivalent between conditions, versus a model in whichthe estimates were unconstrained. These omnibus com-parisons between conditions were carried out using thestacked model or multiple group approach in LISREL(Joreskog and Sorbom, 1989; McIntosh and Gonzalez-Lima, 1992; McIntosh et al., 1994). Rather than estimat-ing a model for each condition separately, the modelswere combined in a single program run. This processinvolved statistically comparing the null functional

246 JENNINGS, MCINTOSH, AND KAPUR

model (equivalent path coefficients between conditions)with the alternative model (coefficients allowed todiffer) by subtracting the goodness-of-fit x2 value for thealternative model from that of the null model. If thealternative model produced a significantly lower x2

value, then the coefficients that were free to varybetween conditions could be considered statisticallydifferent across the three response modes, indicatingthat the functional networks associated with semanticprocessing vary with response. The significance of thisx2 test was assessed using the difference in degrees offreedom between the two models. It is important to notethat the overall models (i.e., all connections together)were statistically compared rather than individualconnections within a model because our interest lies inhow the functional network as a whole is organized andhow a given region in the model is influenced by allother regions acting together.

RESULTS AND DISCUSSION

Regional Activation Changes

Detailed results from the PLS analysis have beendescribed previously (Jennings et al., 1997), but will besummarized here to highlight the main findings. Theanalysis showed several commonalties in activationpatterns when semantic processing was compared toletter processing across the three response modes,primarily increased rCBF to left BA 45 (inferior frontalgyrus). However, this main effect was not pure—thedegree to which these common regions distinguishedsemantic and letter processing varied significantlywith response. Specifically, the magnitude of increasein rCBF associated with semantic processing was great-est for the mouse-click condition, somewhat less for thespoken response condition, and very small with silentthought (Jennings et al., 1997).

Further evidence for a significant interaction be-tween semantic processing and response was evident inthe large number of active regions that were discrepantacross the three conditions (Jennings et al., 1997). Forthe mouse-click condition, semantic processing wasassociated with increased rCBF to left BA 10 (superiorfrontal gyrus), right BA 9 and 10 (middle frontal gyrus),and left BA 22 (superior temporal gyrus) whereas thespoken response condition revealed increased rCBF toleft BA 6/8 (medial frontal gyrus), left fusiform gyrus,and BA 31 (posterior cingulate). The silent thoughtcondition was similar to the other two responses show-ing activation in right BA 9 and 10 (middle frontalgyrus) and left fusiform gyrus, but there was alsoadditional unique activation in right BA 9/46 (middlefrontal gyrus), the right lentiform nucleus, and rightBA 40 (inferior parietal lobule).

Structural Equation Models

From the set of activated regions identified in ourPLS analysis, we selected a subset of regions, whichincluded left BA 45, right BA 32, right cerebellum, leftBA 10, and bilateral BA 22. These regions were chosenbecause they have all been identified in previous neuro-imaging studies of semantic processing (for review seeCabeza and Nyberg, 1997) yet showed intriguing inter-actions as a function of response type. In addition, weincluded right BA 9 and BA 10, which are not typicallyfound during semantic processing but figured promi-nently in our results. These eight regions, the coordi-nates of their peak voxels in stereotaxic space (Talara-ich and Tournoux, 1988), and the response condition(s)for which they were significantly activated are listed inTable 1. The interregional correlations between theseregions and the anatomic connections among them(Fig. 1) were combined to create the structural equationmodels.

The resulting functional networks proved to be signifi-cantly different across the three response modes, x2

diff(48) 5 114.04, P , 0.001. However, examining thenetworks for each response suggested that they com-prised two distinct components. One, there were anumber of effective connections underlying the seman-tic task that were similar for each response mode (Fig.2), such as the influence between left BA 22 and left BA45. However, the strength of these connections variedwith response. The strongest connections were evidentin the mouse-click condition and weakest seen withsilent thought. Two, the remaining connections werehighly discrepant across the three conditions (Fig. 3).For example, the input connection from the cerebellumto right BA 9 varied markedly with response, beinghighly positive in the mouse-click condition, weaklypositive with spoken responding, and strongly negativewith silent thought. Both aspects of the models areexplored further below.

TABLE 1

Areas of Increased rCBF Associated withSemantic Processing

x y z Brodmann’s areas Mouse Spoken Thought

234 28 4 Left area 45 * * *6 22 36 Right area 32 * *

10 288 228 Right cerebellum * *40 48 12 Right area 10 * *42 30 28 Right area 9 * *58 236 16 Right area 22 * *

28 60 12 Left area 10 *262 236 8 Left area 22 *

Note. Asterisks indicate regions that were active for each responsemode.

247MAPPING NEURAL INTERACTIVITY

Common Network

The set of effective connections that were similaracross each response condition consisted of influencesbetween left BA 22 and left BA 45; left BA 22 and BA 32;left BA 45 and right BA 10; right BA 10 and left BA 10;and right BA 10 and BA 32 (Fig. 2). These connectionscould comprise a common functional network, which islargely responsible for semantic processing, since theywere comparable in effect (positive or negative) regard-less of response, although the magnitude of influencevaried.

To determine if this network of connections could beconsidered consistent or common, we tested whetherthe changes in effective strength were significantlydifferent by comparing a null model (all connectionsheld constant) with an alternative model, in which only

the ‘‘common’’ network connections were allowed tovary with condition. The results showed that there wasno significant difference between the null and alterna-tive model; the influences among these ‘‘common’’ re-gions did not differ significantly with response, x2

diff(12) 5 15.11, P . 0.05. As further confirmation, wetested whether the significance of our initial x2 teststemmed mainly from the connections that were highlydiscrepant across each response condition and were notpart of the common network. The same null model wascompared against a second alternative model, in whichonly the ‘‘discrepant’’ network was free to vary and thecommon network was fixed. This comparison proved tobe significant, x2 diff(36) 5 98.99, P , 0.001, suggestingthat the discrepant effective connections were thestrongest source of difference across the three responseconditions.

Although these post hoc tests help confirm the idea ofa common semantic network, those connections maynot be solely associated with semantic processing butmay be part of a more general processing networkwhich is also present in the letter task. That is, thecommon connections between left BA 22 and left BA 45or left BA 22 and BA 32, for example, may also be seenin a functional model representing the letter conditionacross all three response modes. To examine this possi-bility we used the same regions and anatomical net-work to determine a set of effective connections forletter detection. This analysis yielded a model that wassignificantly different across the three response condi-tions, x2 diff(48) 5 81.22, P , 0.005, and showed noevidence of consistently duplicating the common connec-tions we identified with the semantic task. In short, ourcommon network seems to be specific to semantic

FIG. 1. Graphic representation of the regions included in thefunctional network analysis and the neuroanatomical connectionsamong them.

FIG. 2. Graphic representation of the effective connections associated with the semantic task that were common across the three responsemodes. The direct effect is proportional to the arrow width for each path. Values for the width gradient are given in the legend at the bottom.Positive path coefficients are represented as solid black arrows, and negative path coefficients are represented as dashed arrows. Paths wherethe coefficient was at or near zero are not shown.

248 JENNINGS, MCINTOSH, AND KAPUR

processing in our experiment. It should be noted thatthis model of letter detection was not meant to offer anyinformation about that task, since the regions includedin the model were only used to provide a check on thesemantic condition.

Neural Influences Underlying Semantic/ResponseInteractions

As mentioned above, a number of connections in themodels differed significantly across the three responseconditions (Fig. 3), and it is these differential influencesthat seem to reflect the statistical interactions foundbetween semantic processing and response mode. Thesame regions appear to affect one another very differ-ently during the semantic task depending on howsubjects make a response. Moreover, these variationsare in keeping with the interaction effects seen in theoriginal PLS analysis. Before considering interpreta-tions of these network connections though, one needs toremember that the models are not complete representa-tions of the semantic task. Due to parametric con-straints, only a subset of the regions activated duringperformance can be included for modeling. Nonethe-less, even these somewhat limited models shed light onwhy certain regions were activated during the semantictask in only one or two response conditions.

Consider the afferent influences to right BA 10 (Fig.3). In the mouse-click condition there was a strongpositive common connection from left BA 45, a weakpositive input from the cerebellum, and a moderatelystrong influence from left BA 10 with a moderatenegative connection from right BA 9. The silent thoughtcondition showed a weaker positive connection fromleft BA 45 and a weak positive path from the cerebel-lum combined with a very strong positive influence

from right BA 9 and only a weak negative influencefrom left BA 10. In contrast, the spoken responsecondition showed weak positive connections from leftBA 45 and left BA 10 with only a moderate positiveinfluence from the cerebellum and two negative connec-tions (right BA 9 and BA 32). The absence of significantactivation seen in right BA 10 with spoken respondingrelative to the other two modes could be attributed tothose weaker positive connections and additional nega-tive pathway.

A similar type of explanation can be applied to theactivation found in BA 32 (Fig. 3). This region wassignificantly more active in the mouse-click and spokenresponse conditions than with silent thought, a resultwhich is consistent with the inputs identified in themodels. In the mouse-click condition, there was a verystrong positive connection from left BA 45 with amoderate positive common connection from left BA 22,regions which, in turn, were most strongly activatedwith that response type. There were also two negativepathways acting on BA 32, although their influence didnot cancel out the positive effects. The spoken responsecondition also had several positive influences acting onBA 32 from left BA 22, left BA 10 and right BA 22, withonly weak negative influences from left BA 45 and rightBA 10. In contrast, the silent thought condition showedonly two weak positive influences (left BA 22 and rightBA 9) with negative inputs from left BA 45 and right BA10. This pattern of weaker positive connectivity helpsexplain the lack of activation in BA 32 during the silentthought condition compared to the other two responses.Inspection of the other connections in the modelsprovide further insights about the pattern of activationfound with the original PLS analysis. The differentialpatterns of activation for all areas, except the cerebel-

FIG. 3. Graphic representation of the semantic networks for each response condition. The direct effect is proportional to the arrow widthfor each path. Values for the width gradient are given in the legend at the bottom. Positive path coefficients are represented as solid blackarrows, and negative path coefficients are represented as dashed arrows; paths where the coefficient was at or near zero are not shown. Thecommon network is represented by gray arrows.

249MAPPING NEURAL INTERACTIVITY

lum and right BA 22, which had too few inputs to drawmeaningful conclusions, can be related to their networkconnections.

Predicting Regional Activation Levelsfrom the Functional Models

The differences in regional activation between thethree response conditions, identified with PLS, seem tobe attributable to the discrepant influences seen acrossour models. If this idea is valid then the activity level ofany region within a condition should be predicted bythe activity levels of the other regions in the model. Wetested this notion by carrying out a multiple regressionanalysis within each response condition, in whichactivity in a given region was regressed on the activityof areas that projected to it (Fig. 1). For example, activity inleft BA 45 was regressed on the activity of left BA 22and BA 32. If there is a strong positive correlationbetween the predicted activity values from the regres-

sion and the actual activity counts, then the notion thatactivity levels of a region are determined by the influ-ences identified in our models would be confirmed.

This result proved to be the case. The regressionanalysis showed positive correlations between the pre-dicted and actual activity values for all regions in themodel for each response condition. For instance, ascatterplot of the observed values plotted against thepredicted values for right BA 10 (Fig. 4) depicts thestrong correlation between the predicted and actualactivity values for each response mode. The results ofthe regression analyses for all regions in the model withthe exception of the cerebellum and right BA 22, whichhad too few inputs for this analysis, can be seen inTable 2. Almost all regions show strong correlationsbetween the predicted and actual activity values withthe exception of left BA 10, left BA 45, and BA 32 in thesilent thought condition, where a number of the affer-ent influences are weak or even zero. In short, theobserved level of activation in a region seems to be well

FIG. 4. Scatterplots of observed activity values versus predicted activity values for right BA 10 following regression analysis for eachresponse condition.

250 JENNINGS, MCINTOSH, AND KAPUR

predicted from the afferent influences in the model,suggesting that differential activation of these regionsis caused by changes in the effective connectivityamong them.

While few of the correlations from the regressionanalysis would be significant by conventional test, thefact that these correlations are even moderately high issurprising when one considers that the models werenot meant to account for all influences on a region. Forinstance, a more veridical model would include effectsfrom additional areas and influences from diffuselyacting neurotransmitter systems (e.g., locus ceruleus,basal forebrain). However, such a complete depictionhas never been the goal of network analysis (McIntoshand Gonzalez-Lima, 1994). Rather, we have demon-strated, with structural equation modeling, that changesin effective connectivity correspond to the differentialpatterns of activation obtained from PLS. The regres-sion analysis extends this result by showing that theregions included in the models provide a good, albeitnot perfect, prediction of within-task activity patterns.These results are important because there is no a priorireason to expect a relationship between the within-tasksubject variance that is used for path analysis and thebetween-task variance used for PLS and univariatemethods. The fact this relationship exists validates theapproach of using within-task covariances to drawinferences about task-related changes in functionalnetworks.

GENERAL DISCUSSION

We previously demonstrated that the pattern ofrCBF associated with a semantic processing task var-ied as a function of how participants were asked tomake a response (Jennings et al., 1997). This interac-tion effect took two forms. One, the semantic taskactivated several brain regions consistently (e.g., leftBA 45) regardless of response mode. However, thedegree of activation in those areas varied with condi-tion, with the greatest activation seen in the mouse-click condition, somewhat less with spoken responding,

and least with silent thought. Two, other regions provedto be uniquely activated for a given response or acti-vated with only two response modes. For example,activation of left BA 10 was unique to the mouse-clickcondition, while the cerebellum was activated only withmouse-click and spoken responding.

These interactions can be startling if one holds theassumption that the neural processes which underliecognition and behavior are additive. Subtracting thepattern of rCBF during a letter checking task from thatobtained during a semantic task should produce thesame pattern of activation regardless of whether sub-jects respond to each task by clicking a mouse, speakingaloud, or thinking silently to themselves. Whatever thetask differences between the semantic and letter pro-cessing conditions may be, response mode or outputwould be assumed to be a constant across the two tasksthat should not have an interactive effect with processing.

To understand why such interactions may have oc-curred, we undertook a functional network analysis ofour semantic task for each response. The ensuingmodels showed how the patterns of rCBF revealed byour PLS analysis may have arisen from the operation ofa common network of regions with other areas re-cruited into that network depending on response (e.g.,right BA 9, cerebellum, right BA 22). These models helpexplain the original statistical interaction effects thatwere found. The activation of regions common to allresponse conditions, such as left BA 45, seem to havevaried with response because the common pathwaysamong those regions differed in strength across condi-tions (strongest connections in the mouse-click condi-tion and weakest seen with silent thought) and becausethe other influences among these regions were highlydisparate. The network analysis further revealed thatregional differences, unique to a single response modeor shared across only two conditions, also arose fromchanges in neural interactions as a function of re-sponse. For example, activation of left BA 10, whichwas only significant with the mouse-click response,seems to have been related to stronger positive affer-ents in that condition relative to the other two modes.

These findings are particularly important becausethey are unobtainable from examination of regionalactivation alone. For instance, some regions identifiedas part of the common network through network analy-sis (e.g., left BA 22) were not found to be significantlyactive for all three response conditions (see Table 1).Moreover, even the simplified network used here pro-duced network analysis and multiple regression resultswhich demonstrate that activity changes within a givenregion are determined by influences of other anatomi-cally related regions.

TABLE 2

Correlations (R) between Predicted and Actual ActivityCounts Following a Multiple Regression Analysis, in Whichthe Activity in a Given Region Was Regressed on the Activityof the Areas That Projected to It for Each Response Condition

L BA 22 L BA 45 BA 32 L BA 10 R BA 9 R BA 10

ResponseMouse 0.73 0.73 0.68 0.67 0.75 0.90Spoken 0.75 0.56 0.89 0.64 0.83 0.78Silent thought 0.49 0.44 0.32 0.13 0.77 0.74

251MAPPING NEURAL INTERACTIVITY

Implications for the Neurobiology of SemanticProcessing

What have we learned then about the neurobiology ofthe semantic task from path analysis? The presence ofleft BA 45 in the common network associated with thesemantic task replicates numerous studies that haveshown preferential activation of this region in semanticprocessing (for review see Cabeza and Nyberg, 1997;Kapur et al., 1994). The feedback loop between left BA45 and left BA 22 found for each response mode alsosuggests that this latter region is important for seman-tic processing, although the magnitude of its activationseems to be mediated by response—it was only signifi-cantly active in the mouse-click condition. This result isin keeping with other studies that have shown inconsis-tent activation of left BA 21/22 during a semantic taskas a function of response mode and presentation rate(Raichle, 1994; Wise et al., 1991).

An interaction between cognitive processing andresponse also seems critical when examining activationof BA 32 (anterior cingulate) during a semantic task.Identified in a number of semantic studies, it has beendescribed as part of an anterior attentional system thatis engaged in selection for response (Petersen et al.,1988; Petersen and Fiez, 1993). However, a more recentview suggests that the anterior cingulate is critical formodulating and funneling a cognitive/motor commandfrom cortical regions to the motor system (Paus et al.,1993). Our path analysis data suggest this latter viewis probable. Although the models allow feedback fromBA 32 to other regions, the strongest influences seem tobe inputs, suggesting that BA 32 may be more associ-ated with orchestrating a response after semanticprocessing is complete, a view that makes sense giventhe relatively diminished activation in that region withsilent thought.

One of the more surprising findings, perhaps, in-volves right BA 10. Most rCBF data show that leftfrontal activation is largely related to semantic re-trieval whereas right frontal activation is more tied toepisodic retrieval (Nyberg et al., 1996). However, ourfunctional network suggests that right BA 10 may playmore of a role in a semantic processing task than haspreviously been suspected. A positive input from left BA45 with a positive output back to left BA 10, common toall response modes, suggests that the right frontalregion does participate in the semantic task. Onereason this region may be frequently overlooked inother work lies with the weak positive and additionalnegative influence found in the spoken response condi-tion, which appears to reduce right frontal activationbelow a significant level. Given most semantic process-ing studies involve vocal responding, right frontalinvolvement may have been consistently suppressed inthese tasks and gone unidentified (see also Martin et

al., 1996, for right frontal activation during silentnaming).

Neural Context

In summary, network analysis demonstrated thatthere were a set of common influences that could berelated to the semantic processing task with certaininteractions between constituents of this network depen-dent on response mode. These results can be under-stood as a manifestation of a more general principlethought of as ‘‘neural context,’’ in which the role a givenregion plays in a particular process is affected by thecontext of its interactions with other regions, which, inturn, are influenced by task demands (McIntosh et al.,1997). The neural substrates of cognitive operationsmay thus be more sensitive to the specifics of taskdemands than previously thought, which could allow aricher repertoire of operations to be subserved by thesame brain regions.

The path analysis results do contain instances wherethe same brain region may be activated in two taskconditions for potentially different reasons. For ex-ample, if we consider right BA 10 we can see thatactivation in the mouse-click condition arose from thecommon positive input from left BA 45 and from apositive connection from left BA 10. In contrast, thesilent thought condition had a weak common connec-tion with left BA 45 but a strong positive influence fromright BA 9. Although, right BA 10 was part of thecommon network, it seems to also be activated foradditional and different purposes through a disparatepattern of connections. It should be pointed out thatneural context is not meant to imply that there are noconsistencies in brain activation patterns related tocognitive function. As mentioned, some of the compo-nents of the common network, such as left BA 45, havebeen identified in other imaging studies of semanticperformance (Cabeza and Nyberg, 1997).

Implications for Cognitive Neuroscience

The influence of task demands or response modalityon semantic processing is not only apparent at theneural level with rCBF analysis. Such effects can alsobe identified in behavior. One compelling examplecomes from case studies of patients suffering with opticaphasia. Individuals with this disorder cannot namevisually presented objects, but can indicate access tosemantic information through other response modali-ties, such as gesturing. Moreover, they can correctlyname items that they have explored in the tactiledomain, suggesting that an interaction between input,semantic processing and output modality must occur(Coslett and Saffran, 1992; Hillis and Caramazza,1995; Manning and Campbell, 1992). Although we

252 JENNINGS, MCINTOSH, AND KAPUR

found no differences in behavioral measures as a func-tion of response, such interactions can be seen withhealthy individuals (Brooks, 1968).

Studies of other cognitive domains have also foundthat task demands can have a strong impact on theneural regions and pathways associated with a particu-lar process. For example, Bookheimer et al. (1995) usedPET to show that oral reading activates a differentneural pathway from that of silent reading, againsuggesting that response modality or output can influ-ence the neurobiology of a cognitive operation. Addi-tional evidence has been obtained in the visual domainby Goodale and colleagues (Milner and Goodale, 1993).They propose that the inferior parietal lobule and areasin the premotor and prefrontal cortex are involved in aform of visual processing that is necessary for action,while an occipitotemporal processing stream is moreconcerned with the visual processes that support percep-tion. Similar to the reading example, the pathwaysassociated with visual processing appear to differ de-pending on the output requirements of a task. Althoughthe output or response modes required in our semantictask would appear to be an irrelevant task componentadded to a semantic function, the network analysisresults correspond to the data described above. Ourfindings suggest that even an arbitrary response canstrongly affect and alter the neural pathways associ-ated with a cognitive process.

ACKNOWLEDGMENTS

We thank the staff at the PET Centre and Natasha Rajah fortechnical assistance and Stefan Kohler, Michael Smith, and EndelTulving for comments on early versions of the manuscript. This workwas partly supported by a Natural Sciences and Engineering Re-search Council Grant to ARM.

REFERENCES

Aertsen, A. M. H. J., Gerstein, G. L., Habib, M. K., and Palm, G. 1989.Dynamics of neuronal firing correlation: Modulation of ‘‘effectiveconnectivity.’’ J. Neurophysiol. 61:900–917.

Bookheimer, S. Y., Zeffiro, T. A., Blaxton, T., Gaillard, W., andTheodore, W. 1995. Regional cerebral blood flow during objectnaming and word reading. Hum. Brain Map. 3:93–106.

Bookstein, F. L. 1991. Morphometric Tools for Landmark Data:Geometry and Biology. Cambridge Univ. Press, Cambridge, UK.

Brooks, L. 1968. Spatial and verbal components of the act of recall.Can. J. Psychol. 22:349–367.

Cabeza, R., and Nyberg, L. 1997. Imaging cognition: An empiricalreview of PET studies with normal subjects. J. Cog. Neurosci.9:1–26.

Coslett, H. B., and Saffran, E. M. 1992. Optic aphasia and the righthemisphere: A replication and extenstion. Brain Lang. 43:148–161.

Craik, F. I. M., and Lockhart, R. S. 1972. Levels of processing: Aframework for memory research. J. Verb. Learn. Verb. Behav.11:671–684.

Demonet, J. F., Fiez, J. A., Paulesu, E., Petersen, S. E., and Zatorre,R. J. 1996. Brain Lang. 55:352–379.

Douglas, R. J., Koch, C., Mahowald, M., Martin, K. A. C., and Suarez,H. H. 1995. Recurrent excitation in neocortical circuits. Science269:981–985.

Edelman, G. M. 1987. Neural Darwinism. Basic Books, New York.Friston, K. J., Frith, C. D., Liddle, P. F., and Frackowiak, R. S. J.

1993. Functional connectivity: The principal-component analysisof large (PET) data sets. J. Cereb. Blood Flow Metab. 13:5–14.

Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J-P., Frith, C. D.,and Frackowiak, R. S. J. 1995. Statistical parametric maps infunctional imaging: A general linear approach. Hum. Brain Map.2:189–210.

Friston, K. J., Price, C. J., Flectcher, P., Moore, C., Frackowiak,R. S. J., and Dolan, R. J. 1996. The trouble with cognitivesubtraction. NeuroImage 4:97–104.

Hillis, A. E., and Caramazza, A. 1995. Cognitive and neural mecha-nisms underlying visual and semantic processing: Implicationsfrom optic aphasia. J. Cog. Neurosci. 7(4):457–478.

Horwitz, B. 1989. Functional neural systems analyzed by use ofinterregional correlations of glucose metabolism. In VisuomotorCoordination: Amphibians, Comparisons, Models and Robots (J-P.Ewert, and M. A. Arbib, Eds.), pp. 873–892. Plenum, New York.

Horwitz, B., McIntosh, A. R., Haxby, J. V., Furey, M., Salerno, J. A.,Schapiro, M. B., Rapoport, S. I., and Grady, C. L. 1995. Alteredbrain functional interactions during visual processing in Alzheimertype dementia. NeuroReport 6:2287–2292.

Jennings, J. M., McIntosh, A. R., Kapur, S., Tulving, E., and Houle, S.1997. Cognitive subtractions may not add up: The interactionsbetween semantic processing and response mode. NeuroImage5:229–239.

Joreskog, K. G., and Sorbom, D. 1989. LISREL 7 User’s ReferenceGuide. Scientific Software, Mooresville, IN.

Kapur, S., Rose, R., Liddle, P. F., Zipursky, R. B., Brown, G. M., Stuss,D., Houle, S., and Tulving, E. 1994. The role of left prefrontal cortexin verbal processing: Semantic processing or willed action? NeuroRe-port 5:2193–2196.

Manning, L., and Campbell, R. 1992. Optic aphasia with sparedaction naming: A description and possible loci of impairment.Neuropsychologia 30:587–592.

Martin, A., Wiggs, C. L., Ungerledier, L. G., and Haxby, J. V. 1996.Neural correlates of category specific knowledge. Nature 379:649–652.

McIntosh, A. R., Bookstein, F. L., Haxby, J. V., and Grady, C. L. 1996.Spatial pattern analysis of functional brain images using partialleast squares. NeuroImage 3:143–157.

McIntosh, A. R., and Gonzalez-Lima, F. 1992. Structural modeling offunctional visual pathways mapped with 2-deoxyglucose: Effects ofpatterned light and footshock. Brain Res. 578:75–86.

McIntosh, A. R., and Gonzalez-Lima, F. 1994. Structural equationmodeling and its application to network analysis in functionalbrain imaging. Hum Brain Map. 2:2–22.

McIntosh, A. R., Grady, C. L., Ungerleider, L. G., Haxby, J. V.,Rapoport, S. I., and Horwitz, B. 1994. Network analysis of corticalvisual pathways mapped with PET. J. Neurosci. 14:655–666.

McIntosh, A. R., Nyberg, L., Bookstein, F. L., and Tulving, E. 1997.Differential functional connectivity of prefrontal and medial tempo-ral cortices during episodic memory retrieval. Hum Brain Map.5:323–327.

Mesulam, M. M. 1990. Large-scale neurocognitive networks anddistributed processing for attention, language and memory. Ann.Neurol. 28:597–613.

Milner, A. D., and Goodale, M. A. 1993. Visual pathways to perception

253MAPPING NEURAL INTERACTIVITY

and action. In Progress in Brain Research (T. P. Hicks, S. Molotchni-koff, and T. Ono, Eds.), Vol. 95, pp. 317–337. Elsevier, Amsterdam.

Nyberg, L., Cabeza, R., and Tulving, E. 1996. PET studies of encodingand retrieval: The HERA Model. Psych. Bull. Rev. 3:135–148.

Nyberg, L., McIntosh, A. R., Cabeza, R., Nilsson, L-G., Houle, S.,Habib, R., and Tulving, E. 1996. Network analysis of PET rCBFdata: Ensemble inhibition during episodic memory retrieval. J.Neurosci. 16:3753–3759.

Pandya, D. N., and Yeterin, E. H. 1990. Prefrontal cortex in relationto other areas in rhesus monkey: Architecture and connections.Prog. Brain Res. 85:63–94.

Paus, T., Petrides, M., Evans, A. C., and Meyer, E. 1993. Role of thehuman anterior cingulate cortex in control of oculomotor manualand speech responses: A positron emission tomography study. J.Neurophysiol. 70:453–469.

Petersen, S. E., Fox, P. T., Posner, M. I., Mintun, M., and Raichle, M.E. 1988. Positron emission tomographic studies of the corticalanatomy of single-word processing. Nature 331:585–589.

Petersen, S. E., and Fiez, J. A. 1993. The processing of single wordsstudied with positron emission tomography. Annu. Rev. Neurosci.16:509–530.

Petrides, M., and Pandya, D. N. 1988. Association fibers to the frontalcortex from the superior temporal region in the rhesus monkey. J.Comp. Neurol. 273:52–66.

Petrides, M., and Pandya, D. N. 1994. Comparative architectonicanalysis of the human brain and the macaque frontal cortex. InHandbook of Neuropsychology (F. Boller and J. Grafman, Eds.), pp.17–58. Amsterdam, Elsevier.

Poeppel, D. 1996a. A critical review of PET studies of phonologicalprocessing. Brain Lang. 55:317–351.

Poeppel, D. 1996b. Some remaining questions about studying phono-logical processing with PET: Response to Demonet, Fiez, Paulesu,Petersen and Zatorre (1996). Brain Lang. 55:380–385.

Raichle, M. E. 1994. Images of the mind: Studies with modernimaging techniques. Annu. Rev. Psychol. 45:333–356.

Sergent, J., Zuck, E., Levesque, M., and MacDonald, B. 1992.Positron emission tomography study of letter and object process-ing: Empirical findings and methodological considerations. Cereb.Cortex 2:68–80.

Sergent, J. 1994. Brain-imaging studies of cognitive functions. TrendsNeurosci. 17:221–227.

Somers, D. C., Nelson, S. B., and Sur, M. 1995. An emergent model oforientation selectivity in cat visual cortical simple cells. J. Neuro-sci. 15:5448–5465.

Talairach, J., and Tournoux, P. 1988. Co-planar Stereotaxic Atlas ofthe Human Brain. Thieme, New York.

Tononi, G., Sporns, O., and Edelman, G. M. 1992. Reentry and theproblem of integrating multiple cortical areas: Simulation ofdynamic integration in the visual system. Cereb. Cortex 2:310–335.

Tononi, G., Sporns, O., and Edelman, G. M. 1994. A measure for braincomplexity: Relating functional segregation and integration in thenervous system. Proc. Natl. Acad. Sci. USA 91:5033–5037.

Wise, R. J., Chollet, F., Hadar, U., Friston, K., Hoffner, E., andFrackowiak, R. 1991. Distribution of cortical neural networksinvolved in word comprehension and word retrieval. Brain 114:1803–1817.

254 JENNINGS, MCINTOSH, AND KAPUR