Jesse J. Prinz (2000), A Neurofunctional Theory of Visual Consciousness

7/30/2019 Jesse J. Prinz (2000), A Neurofunctional Theory of Visual Consciousness

1/17

Consciousness and Cognition 9, 243259 (2000)

doi:10.1006/ccog.2000.0442, available online at http://www.idealibrary.com on

A Neurofunctional Theory of Visual Consciousness

Jesse Prinz

Philosophy-Neuroscience-Psychology Program, Department of Philosophy, Washington University,

Campus Box 1073, One Brookings Drive, St. Louis, Missouri 63130-4899

E-mail: [email protected]

This paper develops an empirically motivated theory of visual consciousness. It be-gins by outlining neuropsychological support for Jackendoffs (1987) hypothesis that vi-sual consciousness involves mental representations at an intermediate level of processing.

It then supplements that hypothesis with the further requirement that attention, whichcan come under the direction of high level representations, is also necessary for conscious-ness. The resulting theory is shown to have a number of philosophical consequences. Ifcorrect, higher-order thought accounts, the multiple drafts account, and the widely heldbelief that sensation precedes perception will all be found wanting. The theory will alsobe used to illustrate and defend a methodology that fills the gulf between functionalistswho ignore the brain and neural reductionists who repudiate functionalism. 2000 AcademicPress

Over the past few years, the number of philosophical theories of consciousnesshas proliferated. This proliferation has been fueled by well-publicized efforts to bringconsciousness under empirical scrutiny. Central to these efforts have been attemptsto locate the neural substrates of conscious experience. Philosophers have found someinspiration in this empirical work. We have supplemented our thought experimentswith descriptions of actual brain disorders that are being investigated in hospitalsand laboratories around the world. Our theories are now informed by the case studieswhose exotic reality we are forced to accommodate. Nevertheless, much philosophi-

cal output is still produced from the armchair. Empirical findings are conve-niently deployed to dress up conceptually motivated accounts. Few philosophers haveappropriated detailed findings from cognitive neuropsychology, let alone builttheories around them. I would like to reverse the order of business. I suggest thatwe begin by seeing what kind of story is dictated by the lessons of the lab and thensee whether our philosophical theories and distinctions hold up under the micro-scope. Toward that end, I will begin by sketching a neurally informed theory ofconscious experience that accords with empirical results. I will then use this theory,

and the methodology behind it, as a litmus to briefly assess some of the proposalsthat have been suggested by other philosophers. The discussion proceeds in threeparts. Section 1 describes the neural bases of stages in visual processing, Section2 develops a theory of visual consciousness, and Section 3 explores philosophicalramifications.

This article is part of a special issue of this journal on Metacognition and Consciousness, with Thomas

O. Nelson and Georges Rey as Guest Editors.

243

1053-8100/00 $35.00Copyright 2000 by Academic Press

All rights of reproduction in any form reserved.


2/17

244 JESSE PRINZ

1. LOCATING VISUAL CONSCIOUSNESS

1.1. Marrs Levels

If one begins with empirical findings, it is sensible to start with a restricted goal.

Not every aspect of consciousness has been empirically explored in equal detail.The most carefully studied phenomenon has been visual consciousness, the kind ofexperience that accompanies visual perception. Therefore, visual consciousness is anatural starting place. One can hope that a theory of visual consciousness will general-izethat other modalities, at least, will give rise to conscious experience in muchthe same way. For the purposes of this discussion, I will remain largely neutral aboutthat question (though see Section 2.5).

An account of visual consciousness must ultimately cohere with an account of

visual perception. Computational cognitive psychologists often characterize visualperception as a multistage process. Three major stages are generally identified, anddubbed low-, intermediate-, and high-level vision. Each stage encodes dif-ferent information about a stimulus and then passes that information onto the nextstage for further processing. An influential characterization of these levels is foundin Marrs (1982) celebrated theory. Though wrong in detail, I will assume that Marrspartition is essentially correct. The information encoded in low-level vision is a reti-notopically organized collection of blobs and edges. Marr calls representations at thislevel primal sketches. On his account, intermediate-level vision encodes informationabout surfaces, depth and shape from the viewers perspective. Here, representationsare called 21/2D sketches. High-level vision yields viewpoint invariant structural de-scriptions of objects, which can be matched with stored representations to achieveobject recognition. These are Marrs 3D models.

Ray Jackendoff (1987) has argued that Marrs intermediate-level, the 21/2D sketch,is the locale of visual consciousness or phenomenal awareness (I will use such termsinterchangeably). This hypothesis enjoys empirical support. Some of that supportcomes from neuroscience. The brain region that corresponds best with this level ofvisual processing is also the best candidate for a locale of visual consciousness. Orso I shall presently argue. It will turn out, however, that this story needs to be slightlyembellished, for mere activity at the intermediate level does not seem to be sufficientfor visual experience.

1.2. Visual Levels in the Brain

Cognitive neuropsychologists have corroborated the claim that visual information

is processed in successive stages. The functional characteristics of these stages andtheir patterns of interaction have not coincided perfectly with Marrs account, butthe mapping is close enough to be of interest. Although no consensus exists, thereare some widely accepted opinions about which regions of the brain are most closelycorrelated with Marrs levels. Low-level vision is generally associated with primaryvisual cortex or V1. Here information is encoded in retinotopically arranged cellsthat respond selectively to wavelength, movement, and edges at various orientations.At this stage, too little processing has occurred to support recognition tasks.

Intermediate-level vision is best correlated with extrastriate cortical regions. Here,


3/17

THEORY OF VISUAL CONSCIOUSNESS 245

FIG. 1. Levels of visual processing and some possible neural correlates.

information is parceled out from the lower level into specialized processors, whichyield information about shadow, color, shape, and motion (Zeki, 1992). Like Marrsintermediate level, extrastriate cortex typically encodes shape information usingviewer-centered coordinates.

The most natural candidate for the neural substrate of Marrs object-centered highlevel of representation is inferotemporal cortex (IT). Cells in this region are moreindifferent to the size, orientation, and position of objects in the visual field. Lesionstudies and neuroimaging confirm that this is also the primary locale of object recog-nition.

There are important differences between the neural systems involved in visualperception and those identified by Marr. For example, stereoscopic depth is computedin V1, but not in Marrs low level (Poggio & Poggio, 1984); Marr does not describethe division of labor in the extrastriate regions that are correlated with his intermediatelevel (Zeki, 1992), and the high-level processors in IT might not decompose objectsinto volumetric primitives as Marr supposed (Tanaka, 1992). Despite these differ-

ences, neuroscience divides visual processing into several stages and these can beconstructively, if loosely, compared to Marrs. The key points of this comparisonare summarized in Fig. 1.

1.3. The Locale of Consciousness

Following Jackendoff, it is natural to ask whether any one of these processinglevels could qualify as the locale of visual consciousness. As I will use the term, thelocale of visual consciousness is a level (or area) that is active during conscious

experience and encodes information that is available in conscious experience.At first blush, the low level of visual processing seems to be the most obvious

candidate for the locale of visual consciousness. The destruction of V1, the mainneural correlate of low-level vision, apparently results in the loss of visual experience.Still, there is mounting evidence that V1 cannot be the locale of consciousness (seeCrick & Koch, 1995, and Koch & Braun, 1996, for reviews). For one thing, visualhallucination can occur for a period after V1 has been destroyed (Seguin, 1886).Similarly, some subjects who experience blindsight after V1 damage continue to have

phenomenal experiences in the blind fields under certain conditions (Weiskrantz,


4/17

246 JESSE PRINZ

FIG. 2. Illusory contours in a Kanizsa triangle.

1997). As Crick and Koch emphasize, V1 also seems to lack information that is

available to consciousness. First, our experience of colors can remain constant acrossdramatic changes in wavelengths (Land, 1964). Zeki (1983) has shown that suchcolor constancy is not registered V1. Second, V1 does not seem responsive to illusorycontours across gaps in a visual array (von der Heydt, Peterhans, & Baumgartner,1984). If V1 were the locale of consciousness, we would not experience the lines ina Kanizsa triangle (Fig. 2). The fact that consciousness is lost when V1 is destroyedis, therefore, better interpreted as evidence that V1 is a primary source of inputs toanother region in which consciousness can rightfully be said to reside.

It would also be implausible to locate consciousness in high-level visual proces-sors. If Marr is right, high-level vision represents objects using object-centered struc-tural descriptions. These abstract away from aspects of vantage point, including orien-tation, size, and position. In contrast, our visual experience of objects always presentsthem from a particular vantage point (Jackendoff, 1987). Moreover, there is neurolog-ical evidence that we can have visual phenomenology even when we lose access tothe high-level representations in IT. This is presumably what happens in cases ofassociative agnosia in which subjects can accurately draw visual stimuli without be-ing able to identify them (Farah, 1990).

Heres where the intermediate level comes in. Jackendoff locates consciousnessin the intermediate level on computational and psychological grounds. As Crick andKoch (1995) have emphasized, this proposal enjoys further neuropsychological sup-port. First, there is evidence from brain lesions that extrastriate cortex is necessaryfor visual consciousness. For example, a form of color blindnesss, or achromatopsia,results from V4 damage,1 and motion blindness, or akinetopsia, results from MTdamage (Zeki, 1992). Damage in extrastriate areas also seems responsible for apper-ceptive agnosias in which awareness of form is seriously impaired (Farah, 1990).Studies of healthy brains have supported this case by showing that V4 encodes con-stant color, V2 encodes illusory contours, and MT encodes illusory motion with atime course that matches our phenomenal experience (Zeki, 1983; von der Heydt etal., 1984; Tootell, Reppas, Dale, Look, Sereno, Brady, & Rosen, 1995).

As a working hypothesis, it seems reasonable to conclude that visual consciousnessis located in the intermediate-level visual processing systems in extrastriate cortex.

1 David Hilbert tells me that many no longer consider V4 a color area. Some nearby region might be

the real locus of color processing and color blindness.


5/17


FIG. 3.Two fragmented images.

But this hypothesis needs a bit of refinement. For one thing, it would be hasty toconclude that mere activity in the intermediate level produces visual phenomenology.In cases of masked priming, rapidly presented visual stimuli can apparently be recog-nized without entering into awareness. Recognition generally requires processing inall visual areas, including those at the intermediate level. Therefore, mere activity isnot enough. Under what conditions, one must ask, do intermediate-level activitiesenter awareness? I will postpone my answer to this question until Section 2.1, becauseI must first describe a range of phenomena that can steer us in the right direction.

1.4. The Contribution of Higher Levels

I suggested above that activity in V1 might contribute to visual consciousness byproviding inputs to extrastriate regions. I believe that high-level visual areas makean even more substantive contribution. Though not necessary for phenomenology,high-level vision is where objects are interpreted, and interpretation affects phenome-nology. There is considerable evidence for this claim.

First, consider our experience of fragmented images, like the famous Dalmatianphotograph of R. C. James (reproduced in Fig. 3 with another fragmented image).Such images are very difficult to interpret initially but impossible not to interpretonce we have figured out what they represent (Rock, 1984). The Dalmatian imageis so familiar to most of us that we immediately organize it into the correct bound-aries, but first-time viewers are stumped. Such images cannot be processed bottom-up; they seem to require assistance from higher levels of processing. That high-levelinfluence delivers a meaningful segmentation which transforms our phenomenal ex-

perience.The phenomenal impact of interpretation is also evidenced by our experience of

multistable figures, which are experienced differently depending on how they areinterpreted. Chambers and Reisberg (1992) showed that the mental image we formof a duck-rabbit (Fig. 4) is experienced differently depending on how it is interpreted.They asked subjects to study a duck-rabbit figure until they were able to form vividmental images of it. Then they showed the same subjects three duck-rabbit figuresand asked them to guess which one they had been trained on. One was the original,

the second had been slightly altered in the region corresponding to the ducks face,


6/17

248 JESSE PRINZ

FIG. 4. A duck-rabbit.

and the third had been slightly altered in the region corresponding to the rabbits

face. They found that subjects who had interpreted the original figure as a duck wereable to rule out the image with the altered duck face, but were at chance at decidingbetween the two other figures. Those who interpreted the original figure as a rabbitexhibited the opposite pattern. While interpreting a duck-rabbit as a duck, subjectsneglect details on rabbit face, and while interpreting it as a rabbit they neglect theduck face. When asked to reconstrue the images in memory, subjects bring the facecorresponding to the new interpretation into focus and lose focus of the face corre-sponding to the original interpretation. This shifting is presumably mediated by high-level vision.

Further evidence for a contribution of high-level areas comes from studying binoc-ular rivalry. When two distinct stimuli are present to each eye, our conscious perceptalternates between them. Using single cell recordings, Scheinberg and Logothetis(1997) found that 90% of tested cells in IT correspond to the experienced perceptas opposed to 2025% in extrastriate regions. A natural interpretation is that thehigh-level activity dictates which intermediate-level activities become conscious. Itfixes on an interpretation and thereby renders the corresponding subset of intermedi-ate neurons conscious.

All these considerations suggest that high-level visual systems make a contributionto visual awareness. If awareness is located in the intermediate level, as argued above,this contribution must be explained in terms of an effect that the high level has onintermediate processing. I will now sketch a proposal along those lines.

2. THE AIR THEORY OF CONSCIOUSNESS

2.1. Attention and Awareness

How does high-level vision affect the intermediate level? My description of theChambers and Reisberg research suggests an answer. When we fix on an interpreta-tion of the duck-rabbit figure, we seem to focus on those potions of the figure thatare most relevant to the interpretation. This is most naturally described as an atten-tional effect. We preferentially attend to one side of the figure. In cases of binocularrivalry, attention might be responsible for marking the subset of intermediate-levelneurons corresponding to the high-level interpretation. Attention might also be usedto group together the components of a fragmented picture in ways that respect our

interpretation. It is well established that attention increases activity in intermediate-


7/17


FIG. 5. The circuit that gives rise to consciousness on the AIR theory.

level (and lower-) visual areas (Corbetta, Miezen, Dobmeyer, Shulman, & Petersen,1991). The phenomena under consideration suggest that such attentional effects can

be guided by high-level visual areas.Putting all this together suggests the following picture. When we see a visual stimu-

lus, it is propagated unconsciously through the levels of our visual system. Whensignals arrive at the high level, interpretation is attempted. If the high level arrivesat an interpretation, it sends an efferent signal back into the intermediate level withthe aid of attention. Aspects of the intermediate-level representation that are mostrelevant to interpretation are neurally marked in some way, while others are eitherunmarked or suppressed. When no interpretation is achieved (as with fragmented

images or cases of agnosia), attentional mechanisms might be deployed somewhatdifferently. They might search or scan the intermediate level, attempting tofind groupings that will lead to an interpretation. Both the interpretation-driven en-hancement process and the interpretation-seeking search process might bring the at-tended portions of the intermediate level into awareness. This proposal can be sum-marized by saying that visual awareness derives from Attended Intermediate-levelRepresentations (AIRs). The circuit I have described is depicted in Fig. 5.

The AIR theory offers an explanation of how high-level vision can influence phe-

nomenology even though phenomenology is located at the intermediate level. It alsocontributes to an explanation of the question raised at the end of Section 1.3 regardingthe lack of awareness in masked priming. Any interpreted visual stimulus presumablypasses through both intermediate- and high-level visual areas. If mere activity in theintermediate level was sufficient for consciousness, the lack of awareness in maskedpriming would be inexplicable. On the present story, mere activity is not enough.Awareness only occurs after intermediate-level representations are enhanced by at-tention. This cannot occur in cases of masked priming, because, by the time an inter-

pretation is achieved at the high level, the intermediate-level representation is nolonger there to be enhanced. It has been masked by a subsequent stimulus.

2.2. Unilateral Neglect

The AIR theory also enjoys support from studies of unilateral neglect. Neglectgenerally follows lesions to right parietal cortex and results in a lack of awarenessof objects and object parts on the contralesional side. Asked to describe or draw avisual stimulus, neglect subjects will omit features on the left. Neglect is generally

described as an attention disorder. It is thought that attention mechanisms located in


8/17

250 JESSE PRINZ

FIG. 6. Damage resulting in unilateral neglect.

or mediated by regions of parietal cortex are damaged resulting in impaired phenome-nal access. The situation is diagrammed in Fig. 6.

Neglect is illuminating in several ways. First, if the standard interpretations are

correct, it demonstrates that damage to attentional systems can lead to deficits inawareness. In making attention requisite to visual consciousness, the AIR theory pre-dicts this result. Neglect has also provided further evidence that interpretation canoccur without awareness. In a fascinating series of studies, it has been discovered thatsome neglect subjects tacitly process information in their neglected fields (Marshall &Halligan, 1988; Bisiach & Rusconi, 1990). In one case, neglect subjects see twopictures of a vase that are exactly alike except that one has flowers extending outon the left-hand side. Because the flowers fall in the neglected region, neglect subjects

say the two vases look alike and believe that both are empty. They nevertheless claimthat they prefer the vase with the flowers, unable to provide the correct explanationfor this preference. The AIR story explains this by saying such subjects have intactobject recognition systems (as in Fig. 6). Their phenomenal deficit derives from anattention deficit. Without the phenomenological experience of seeing flowers, sub-jects form the false belief that none have been detected.

A third phenomenon exhibited in neglect can also be explained on the AIR theory.It has been well established that some cases of neglect are object-based. Subjects

neglect the left sides of perceived objects rather than the left side of their visual field.To show this, Tipper and Behrmann (1996) showed neglect subjects a barbell-shapedframe, formed by two circles and a connecting line. When targets are flashed on theleft circle of the barbell frame, neglect subjects fail to detect it, but they do detecttargets flashed on the right. The surprising result is that this pattern of response re-verses after the subjects see the barbell rotated 180. After witnessing this rotation,neglect subjects fail to detect targets flashed on the right-hand side, the side that hadformerly been on the left. They seem to assign an intrinsic axis to the object in the

initial trial, and they subsequently neglect the left side of that axis no matter whereit appears relative to retinal coordinates. This suggests that the kind of attention thatis disrupted in neglect depends on object representations. This coheres with the AIRtheory, according to which high-level object representations are formed prior to back-projecting the attention that gives rise to awareness. Attention is directed by priorinterpretation.

2.3. Awareness Without Attention?

The AIR theory states that visual awareness arises when the high levels of pro-cessing direct attention to intermediate-level representations. This implies that aware-


9/17


ness cannot occur without attention. Some might regard this as a fatal implicationof the theory. There are some empirical results that suggest we can have awarenesswithout attention. This had led some authors to believe that attention is not necessaryfor phenomenal experience. Fortunately, I think these results can be readily inter-

preted in ways that are compatible with the AIR theory. In fact, they provide someevidence in its favor.I will briefly consider two studies that claim to show awareness without attention.

First, Braun and Julesz (1998) tested the effects of visual pop-out during attention-taxing discrimination tasks. They found that subjects could detect the color and orien-tation of peripheral stimuli even when they were presented during shape identificationtasks that exhaust attentional resources. From this, they conclude that awareness ofcolor and orientation can occur without being attended to. There are three reasons

to think that this result does not contradict the AIR theory. First, the AIR theorysays that consciousness occurs after detection, when attention is projected back intointermediate areas. The fact that there is a high-cost attention task occurring duringdetection of peripheral stimuli in the test conditions does not rule out the possibilitythat a subsequent attentional backprojection occurs and gives rise to awareness ofthose stimuli. Second, there is considerable evidence that there are several differentkinds of attention mechanisms in the brain (a point that I will revisit shortly). It ispossible that the high-cost attention task exhausts just one kind of attention, leavingothers untaxed and, hence, available to give rise to awareness of the periphery.

Finally, Braun and Julesz interpret their own results in a way that supports the AIRtheory. They suggest that visual pop-out follows a Global Competition process inwhich the visual system selects the most salient attributes and suppresses less salientattributes. They then suggest that this Global Competition might occur in inferotem-poral cortex (Braun & Julesz, 1998, p. 20). This implies that processing in inferotem-poral cortex, the locus of high-level vision, precedes visual awareness. That is pre-cisely what the AIR theory claims. The only difference is that Braun and Julesz do notcall the selection and suppression process an instance of attention. Though reluctant toengage in a verbal dispute, I am inclined to think that this is a paradigmatic case ofwhat is generally described as an attentional process. If so, the Global Competitionmodel coincides with the AIR theory.

The second study that alleges to show awareness without attention uses a similarparadigm (Rock, Linnett, Grant, & Mack, 1992). Here, subjects are shown a seriesof cross-hair images in which they are asked to determine which of the two inter-secting lines is longer. On one of these trials, an unexpected target stimulus is flashedin one corner of the image. The subjects are then asked whether they saw the target,

and, if so, they are asked to describe it. Many of the subjects report seeing the stimu-lus, and they accurately identify its color and location, though not its shape. Theauthors conclude that an unexpected stimulus can enter awareness without attention.

This conclusion is premature. As with the other experiment, it is perfectly possiblethat awareness of the peripheral stimulus comes after an attentional process involvingGlobal Competition has taken place. The fact that subjects do not expect the stimulusto appear does not show that attention is not applied after the fact. It supports thepossibility that awareness-conferring attention occurs after preprocessing. It must

also be noted that 25% of the subjects tested using the paradigm did not perceivethe surprise stimulus. There is a possibility that the other 75% were allocating


10/17

252 JESSE PRINZ

some attention to the periphery. Using a very similar paradigm, More and Egeth(1997) found that only 010% of their subjects consciously recognized peripheralstimuli.

So striking is this pattern that Mack and Rock (1998) have now written a book

revising the claim in their original article and arguing that conscious perception can-not occur without attention. They have also corroborated a second tenet of the AIRtheory, viz consciousness-conferring attention is allocated under the control of high-level representations. To demonstrate this, they achieved a visual version of the cock-tail party effect by quickly displaying words while subjects performed attention-con-suming line comparison tasks. Subjects would see the word if it was their own name,but fail to see it otherwise. As with other cases we have seen, attention, and henceconsciousness, seems to depend on prior interpretation.

2.4. Anatomical Bases

It is natural to ask what parts of the brain underlie the attentional processes thatbring intermediate-level representations into awareness. This question, it turns out,is quite difficult to answer. Attention, like many folk-psychological concepts, is amongrel. When we analyze it, we find that several different things fall under thislabel. For example, there is a kind of attention associated with vigilance as when wekeep our focus on the same monotonous task, and another kind of attention associatedwith selection as when we pick out an object from a sea of distracters. These concep-tually distinct forms of attention apparently have distinct neural correlates (seePosner & Raichle, 1994, for a review). Investigating the neural correlates of attentionalso leads to distinctions that are more finely grained than our various concepts ofattention. For example, the notion of an object-based attention mechanism was intro-duced earlier in discussing neglect. This can be distinguished from a kind of spatialattention that focuses on the relative locations of entire objects rather than their parts.There is evidence that these are located in anatomically distinct regions of parietalcortex (Humphreys & Ridoch, 1994). Given this motley assortment of attentionmechanisms, and others that I have not mentioned, one can rightfully wonder whichcontribute to awareness.

I will not provide a definite answer to this question. I think it is an important topicfor future research. I have implicated object-based attention, a scanning mechanism,and a Global Competition mechanism, all of which may or may not be independent.It is unlikely that only one kind of attention contributes to awareness. Brain lesionsin different areas (e.g., posterior parietal, inferior parietal, and frontal cortex) can all

give rise to different forms of neglect. Instead, I think there are a family of mecha-nisms that are united by their common ability to exploit high-level information inmaking intermediate-level states conscious. An adequate taxonomy of these, and anunderstanding of how they operate, still elude us.

I have also said nothing about how neurons at the intermediate level becomemarked by these attentional processes. What makes the marked neurons differentfrom other intermediate-level activities that do not enter awareness? How are theymarked? This is also an important empirical question demanding further research.

There are several distinct, but compatible, possibilities. One possibility is that the


11/17


neurons that enter consciousness have a different firing pattern. A second possibilityis that those neurons enter into a reentrant feedback relation with neurons in attentionmechanisms. A third possibility is that they are endowed with a special kind of con-nectivity. In particular, they might become accessible to other regions of the brain,

which gives them a privileged role in information processing. The last suggestionraises a possibility that has been of considerable interest to those studying the neuralcorrelates of consciousness. Perhaps intermediate-level representations enter aware-ness when they broadcast to areas of prefrontal cortex that have been associated withworking memory (Wilson, Scalaidhe, & Goldman-Rakic, 1993). A number of authorshave argued that such connections to prefrontal cortex are essential to consciousexperience. First, Nakamura and Mishkin (1986) have done primate ablation studiessuggesting that removal of prefrontal cortex results in complete blindness. Second,

Crick and Koch (1995) argue that conscious states must be connected to prefrontalareas, because prefrontal areas mediate rational action and verbal reporting, and con-scious states contribute to both. Finally, Sharaie, Weiskrantz, Barbur, Simmons, Wil-liams, and Brammer (1997) did fMRI studies that contrasted activity in the brain ofa blindsighted subject during those conditions in which he does and does not experi-ence phenomenal awareness in his blind field. When he reported phenomenal expe-rience, there was a significant increase in visual areas of prefrontal cortex, notablyarea 46.

Despite this evidence, I do not think the case for prefrontal involvement is terriblyconvincing. The last two arguments run the risk of mistaking cause for effect. Even ifprefrontal activity always occurs when visual states become conscious (an unprovenhypothesis), it does not show that projections to prefrontal activity are essential toconsciousness. Conscious states might be accessible to prefrontal areas without beingconscious in virtue of such accessibility (see Block, 1998 for another, related lineof objection). The ablation work is more suggestive, but there is now some evidencethat it is misleading. First, when more localized lesions are created in prefrontal area46, which Sharaie et al. (1997) appeal to, monkeys only seem to lose their abilityto preserve spatial memories during delayed matching tasks. Their actual visionseems to be unimpaired. More important, Gilman, Saunders, Rickrode, and Mishkin(1998) have recently reevaluated the Nakamura and Mishkin results by performinga similar experiment. After removing the frontal lobe portion in a Nakamura andMishkin-style ablation, they found that monkeys continue to display visual discrimi-nation and visually-guided action. The earlier results might have merely given theappearance of blindness by disrupting connections between the visual system andsystems that mediate motor responses to visual stimuli. The new results suggest that

monkeys might retain visual experience without interaction between visual areas andthe relevant parts of the frontal lobe. If attention confers awareness by connectingintermediate-level representations to other brain systems, we might want to look else-where.

The complete articulation of the AIR hypothesis will depend on the developmentof an adequate theory of attention. For now, the appeal to attention can only functionas a placeholder. Evidence suggests that attention confers awareness; now we mustfigure out exactly what the relevant form of attention is and how it contributes. These

promissory notes do not trivialize the AIR hypothesis, because the invocation of


12/17

254 JESSE PRINZ

attention is already a substantive, testable, and controversial claim. One can, however,refine that claim by exploring the nature and neural basis of attention.

2.5. Other Modalities

The AIR theory is intended as a theory of visual awareness. It has a structure,however, that might extend to other sensory modalities. Like vision, other modalitiesinvolve different levels of processing and are affected by attention. It is possible thatawareness is associated with attended intermediate-level representations throughoutthe brain. Jackendoff (1997) takes his hypothesis about the privileged status of theintermediate level to extend beyond vision. I would like to leave this an open questionfor further research. It is worth noting, however, a couple of suggestive phenomena

involving auditory language processing.The AIR theory says that visual recognition is attempted before the onset of visualawareness. This processing order is also evidenced in audition. When we listen tolanguage, our comprehension of words can affect our experience of them. A notableexample is the phenomenon of phoneme restoration. Listeners will often report hear-ing a phoneme that has been omitted from a spoken sentence. The phoneme theyhear will be one that renders the word in which it occurs meaningful relative to itssentential context. For example, Warren (1970) asked subjects to listen to the follow-ing two sentences with an omitted phoneme.

It was found that the _eel was on the axle.It was found that the _eel was on the orange.

When they listened to the first sentence, they heard the incomplete word as wheel,and when they listened to the second, they heard the incomplete word as peel.Thus, the experience of these words clearly depended on the way latter parts of thesentence were interpreted. Assuming that sentence interpretation occurs at an uncon-scious level of semantic processing, these examples demonstrate that our phonologi-cal experience of sentences occurs after they have been interpreted. This result ac-cords well with the AIR theory if we follow Jackendoff in thinking of thephonological level as the intermediate stage of language processing. The intermediatelevel is the locale of awareness, but it only achieves that status after a higher levelof processing.

A second auditory phenomenon is the famous cocktail party effect. When we arefocused on a nearby conversation, we lose conscious awareness of other conversa-tions going on in the background. Those conversations pop back into awareness,however, if one of their participants mentions our name or uses some other salient

word. The same phenomenon can be reproduced using dichotic listening paradigms,in which a different recording is played in each ear. Subjects will lose their awarenessof one recording until a salient word occurs. Once again, it is natural to suppose thatinterpretation precedes awareness. When our unconscious semantic system recog-nizes that an unattended word might be significant to us, it brings that word intophenomenal awareness. This has been classically interpreted as an attention phenom-enon. Interpretation gives rise to phenomenology by directing attention to relevantstimuli. Such examples suggest that the AIR theory might be applicable in the audi-

tory domain.


13/17


3. PHILOSOPHICAL RAMIFICATIONS

3.1. Against HOT AIR

Philosophers have proposed a number of theories of consciousness. Many of these

have been informed by empirical, or at least introspective, evidence. At the sametime, few have been erected around our knowledge of the brain. The AIR theorydiffers in that respect. It is a neurally informed theory of consciousness. If this theoryis correct, it has a number of important philosophical consequences. For one thing,it is incompatible with a number of existing philosophical hypotheses. I cannot dojustice to all of these hypotheses here, but I want to raise the specter of possibleempirical disconfirmation.

One hypothesis that seems to be threatened by the AIR theory is the Higher-Order

Thought (HOT) theory of consciousness eloquently advanced by Rosenthal (1986).According to that theory, states become conscious in virtue of being the intentionaltargets of higher-order thoughts. For example, I would become visually consciousof seeing a dog in virtue of having a thought that I would express by saying, Imseeing a dog.

The HOT theory is similar to the AIR theory in that both think states can becomeconscious in virtue of activities at higher levels of processing. They also agree thatinterpretation precedes conscious experience. Despite these similarities, the AIR the-ory differs from the HOT theory in important respects. The high-level states thatdirect attention into intermediate levels are not HOTs. First, they are not higher-order states. A higher-order state is one that represents another state. High-levelperceptual states fail to qualify because they do not represent intermediate-levelstates. Instead, they represent the distal objects that they are deployed to recognize.Moreover, the high-level states invoked by the AIR theory are not thoughts. Thoughtsare presumably amodal propositional attitudes. They are the kinds of things that enterinto reasoning and dispose us to make verbal reports. In contrast, the relevant high-level states on the AIR theory are perceptual representations. Finally, the AIR theorysays that intermediate-level states can become conscious even when we lack higher-level states to interpret them. This is what happens when interpretation fails and weresort to attentional scanning. If AIR is right, HOT is not.

3.2. The Stalinesque Theater

The next account I will mention is Dennetts (1991). Dennett criticizes other ac-counts for becoming ensnared in debates about whether consciousness arises from

processes that are Stalinesque or Orwellian. Stalinesque accounts have it that informa-tion is edited or censored before becoming conscious. Orwellian accounts have itthat we experience an unedited stream of information, which is subsequently revisedand fed back into consciousness erasing memories of what came before. On his ownmultiple-drafts theory neither story is right. Both mistakenly believe there is somecenter of consciousness. So, they think there is a real question as to whether informa-tion gets into that center prior to or after processing. In actual fact, he claims, thereis no center. There are multiple narratives going on simultaneously.

The evidence marshaled in favor of the AIR theory tells against Dennetts account.


14/17

256 JESSE PRINZ

On the AIR theory, there is something that qualifies as a (somewhat distributed)center of consciousness, viz. intermediate-level processors. These are comprised ofdifferent component subprocessors, but they qualify as a center in Dennetts sense,because they are anatomically and functionally separable from other parts of the

brain. In healthy brains they harbor a single, coherent narrative separated from com-peting drafts with the help of attention. I have also suggested that conscious pro-cessing is Stalinesque. Attention gives rise to awareness after information has beencensored or modified by an interpretational level of processing. The turnstile thatushers information into consciousness is more complex than intuitions might dictate,because it involves high-level feedback. But, there seems to be a fact of the matterabout when and under what conditions information enters awareness. If the AIRTheory is right, Dennetts claims cannot be sustained.

3.3. Sensing Perception

This response to Dennett also exposes a problem for classic sense-data theories.Sense-data theorists traditionally assume that phenomenology precedes interpreta-tion. When we see an object, we first become phenomenally aware of sense-datum.This is the level of sensation. Perception occurs when the sense-datum is interpretedas representing something. In one respect, the AIR theory corroborates this kind ofaccount. High-level vision delivers interpretations of intermediate-level representa-

tions, and intermediate-level representations are those of which we are phenomenallyaware. On the other hand, the AIR theory puts interpretation before phenomenology.We become visually conscious of a stimulus after it has been identified, if identifica-tion takes place. We dont perceive our conscious sensations, as the sense-data theo-rists would have it. Instead, we sense our perceptions. In this respect, the sense-datatheorists had it backward.

3.4. Gray Matters

Philosophers have offered various solutions to the mindbody problem. Some saythat phenomenal mental states are not identical to any states describable by existingphysical science. Others say there is a type identity between phenomenal states andcertain kind of brain states. Still others say that phenomenal states can be identifiedwith functional roles, which are delineated in turn by the platitudes of folk psychologyor by the discoveries of empirical psychology. The AIR theory falls into this lastcategory, what Block (1978) calls Psychofunctionalism. It uses empirical science tocorrelate phenomenal states with a particular kind of functional role. That functional

role can be stated in more or less computational terms by appealing to levels ofinformation processing. But this functional organization is mirrored by the organiza-tion of the nervous system; functional components are anatomically distinct. Thefunctional role of consciousness can be described using a high-level neural descrip-tion. This can be regarded as a kind of Neurofunctionalism.

Neurofunctionalism allows for a fairly straightforward correlation between mindand brain. It suggests that an account of the mind can be naturally accommodatedwithin physical science. It also shows that the brain matters when studying the mind.

Looking at the brain is one of the best ways to understand how cognition is function-


15/17


ally organized. This insight is in stark contrast to early functionalists who appealedto computational roles in order to escape brain-based theories of experience (e.g.,Putnam, 1960). Functionalists have been skeptical of brain science ever since, andthose who take the brain seriously have been skeptical about functionalism (e.g.,

Block, 1978; Searle, 1980; Churchland, 1986). I think both sides are mistaken. Brain-friendly philosophers sometimes talk as if it is sufficient to correlate conscious stateswith brain states without talking about function. To say that phenomenology occurswhen such and such part of the brain lights up is not satisfying from an explanatoryperspective. To understand how the mental could be physical, we must recognizethat mental states play certain functional roles and carry certain information. We canlearn about these roles and how they are implemented by looking at the brain. Cor-relating mental states with neural states becomes illuminating when we can see a

mutually enlightening correspondence between the functional organization of each.Systems-level neuroscience typically follows this stricture. It concerns itself withfunctional organization and develops theories of cognition by observing how the brainparcels out information-processing tasks. It is time to decimate the barriers betweenfunctionalists and fans of the brain. Even if it is wrong in its details, the AIR theorycan be interpreted as a plea for this more egalitarian methodology.

4. CONCLUSION

I have developed a theory of visual consciousness by appealing to lessons frompsychology and cognitive neuroscience. The theory states that consciousness occurswhen intermediate-level visual processors are enhanced or scanned by attentionmechanisms, which can operate under the direction of high-level visual processors.This theory is not necessitated by the evidence I have presented, but it offers oneviable explanation. It also coincides, in part, with other empirically motivated theoriesof consciousness that appeal to intermediate-level representations (Jackendoff, 1987;Koch & Braun, 1996) or afford a central role to attention (Posner & Rothbart, 1992;

Mack & Rock, 1998). If the AIR story is right, a number of philosophical hypotheseswill be disconfirmed. That possibility suggests a more general methodological moral.We should try to build our theories of consciousness around empirical evidence.Included in this effort will be a thorough examination of the neural correlates ofconscious experience. Many philosophers find the search for neural correlates to belittle more than a parlor sport for scientists who fail to understand the relationshipbetween mind and brain. This attitude is undermined by the discovery that a neurallyinformed theory can have significant implications for claims that have been defended

within philosophy.

ACKNOWLEDGMENTS

Versions of this paper have been presented at the 1998 Society for Philosophy and Psychology Confer-ence, in Minneapolis and the 1998 Southern Society for Philosophy and Psychology Conference in NewOrleans. Some of this material has also been presented at Washington University and at the City Univer-sity of New York, Graduate Center. I thank my audiences, and express special gratitude to David Rosen-thal, Georges Rey, Bill Bechtel, Murat Aydede, David Hilbert, Dan Gilman, Albert Yonas, and Michael

Tye for some useful discussions.


16/17

258 JESSE PRINZ

REFERENCES

Bisiach, E., & Rusconi, M. L. (1990). Break-down of perceptual awareness in unilateral neglect. Cortex,26, 643649.

Block, N. (1978). Troubles with functionalism, Minnesota Studies in the Philosophy of Science. 9, 261

325.Block, N. (1998). How not to find the neural correlate of consciousness. In S. Hameroff, A. W. Kasz-

niak, & A. C. Scott (Eds.), Toward a science of consciousness II. Cambridge, MA: MIT Press.

Braun, J., & Julesz, B. (1998). Withdrawing attention at little or no cost: Detection and discriminationtasks. Perception & Psychophysics, 60, 123.

Chambers, D., & Reisberg, D. (1992). What an image depicts depends on what an image means. CognitivePsychology, 24, 145174.

Churchland, P. S. (1986). Neurophilosophy: Toward a unified science of the mind-brain. Cambridge,MA: MIT Press.

Corbetta, M., Miezen, F. M., Dobmeyer, S., Shulman, D. L., & Petersen, S. E. (1991). Selective anddivided attention during visual discriminations of shape, color, and speed: Functional anatomy bypositron emission tomography. Journal of Neuroscience, 11, 23832402.

Crick, F., & Koch, C. (1995). Are we aware of activity in primary visual cortex? Nature, 375, 121123.

Dennett, D. C. (1991). Consciousness explained. New York: Little, Brown.

Farah, M. J. (1990). Visual agnosia: Disorders of object recognition and what they tell us about normalvision. Cambridge, MA: MIT Press.

Gilman, D. J., Saunders, R. C., Rickrode D., & Mishkin, M. (1998). Effects of frontal cortex lesions on

visual awareness in the rhesus monkey. Manuscript under review.

Humphreys, G. W., & Riddoch, M. J. (1994). Attention to within-object and between-object spatialrepresentations: Multiple sites for visual selection. Cognitive Neuropsychology, 11, 207241.

Jackendoff, R. (1987). Consciousness and the computational mind. Cambridge, MA: MIT Press.

Koch, C., & Braun, J. (1996). Towards a neuronal correlate of visual awareness. Current Opinion inNeurobiology, 6, 158164.

Land, E. H. (1964). The retinex. Scientific American, 52, 247264.

Mack, A., & Rock, I. (1998). Inattentional blindness. Cambridge, MA: MIT Press.

Marr, D. (1982). Vision: A computational investigation into the human representation and processingof visual information. New York: W. H. Freeman.

Marshall, J. C., & Halligan, P. W. (1988). Blindsight and insight in visio-spatial neglect. Nature, 336,766767.

Moore, C. M., & Egeth, H. (1997). Perception without attention: Evidence of grouping under conditionsof inattention. Journal of Experimental Psychology: Human Perception and Performance, 23, 339352.

Nakamura, R. K., & Mishkin, M. (1986). Chronic blindness following lesions of nonvisual cortex inthe monkey. Experimental Brain Research, 63, 173184.

Poggio, G. F., & Poggio, T. (1984). The analysis of stereopsis. Annual Review of Neuroscience, I, 379412.

Posner, M. I., & Raichle, M. E. (1994). Images of mind. New York: W. H. Freeman.

Posner, M. I., & Rothbart, M. K. (1992). Attentional mechanisms and conscious experience. In A. Mil-ner & M. Rugg (Eds.), The neuropsychology of consciousness. New York: Academic Press.

Putnam, H. (1960). Minds and machines. In S. Hook (Ed.), Dimensions of mind. New York: New YorkUniversity Press.

Rock, I. (1984). Perception. New York: Scientific American Books.

Rock, I., Linnett, C. M., Grant, P., & Mack, A. (1992). Perception without attention: Results of a new

method. Cognitive Psychology, 24, 502534.


17/17


Rosenthal, D. M. (1986). Two concepts of consciousness. Philosophical Studies, 49, 329359.

Scheinberg, D. L., & Logothetis, N. K. (1997). The role of temporal cortical areas in perceptual organiza-tion. Proceedings of the National Academy of Science, 94, 34083413.

Searle, J. R. (1980). Minds, brains and programs. Behavioral and Brain Sciences, 3, 417457.

Seguin, E. G. (1886). A contribution to the pathology of hemianopsis of central origin (cortex-hemianop-

sia). Journal of Nervous and Mental Diseases, 13, 138.

Sharaie, A., Weiskrantz, L., Barbur, J. L., Simmons, A., Williams, S. C. R., & Brammer, M. J. (1997).Pattern of neuronal activity associated with conscious and unconscious processing of visual signals.Proceedings of the National Academy of Science, 94, 94069411.

Tanaka, K. (1992). Inferotemporal cortex and higher visual functions. Current Opinion in Neurobiology,2, 502505.

Tipper, S. P., & Behrmann, M. (1996). Object-centered not scene-based visual neglect.Journal of Experi-mental Psychology: Human Perception and Performance, 22, 12611278.

Tootell, R. B. H., Reppas, J. B., Dale, A. M., Look, R. B., Sereno, M. I., Brady, T. J., & Rosen, B. R.

(1995). Functional MRI evidence for a visual motion after effect in human cortical area MT/V5.Nature, 375, 139141.

von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron re-sponses. Science, 224, 12601262.

Warren, R. M. (1970). Perceptual restoration of missing speech sounds. Science, 167, 392393.

Weiskrantz, L. (1997). Consciousness lost and found. Oxford: Oxford Univ. Press.

Wilson, F. A. W., O Scalaidhe, S. P., & Goldman-Rakic, P. S. (1993). Dissociation of object and spatialprocessing domains in primate prefrontal cortex. Science, 260, 19551958.

Zeki, S. (1983). Colour coding in the cerebral cortex: The reaction of cells in monkey visual cortex towavelengths and colour. Neuroscience, 9, 741756.

Zeki, S. (1992). The visual image in mind and brain. Scientific American, 267, 6976.

Received March 2, 2000

Jesse J. Prinz (2000), A Neurofunctional Theory of Visual Consciousness

Documents

Transcript of Jesse J. Prinz (2000), A Neurofunctional Theory of Visual Consciousness