Confirmation Bias: A Ubiquitous Phenomenon in Many Guises

Review of General Psychology1998, Vol. 2, No. 2, 175-220

Copyright 1998 by the Educational Publishing Foundation1089-2680«8/$3.00

Confirmation Bias: A Ubiquitous Phenomenon in Many Guises

Raymond S. NickersonTufts University

Confirmation bias, as the term is typically used in the psychological literature, connotesthe seeking or interpreting of evidence in ways that are partial to existing beliefs,expectations, or a hypothesis in hand. The author reviews evidence of such a bias in avariety of guises and gives examples of its operation in several practical contexts.Possible explanations are considered, and the question of its utility or disutility isdiscussed.

When men wish to construct or support a theory, howthey torture facts into their service! (Mackay, 1852/1932, p. 552)

Confirmation bias is perhaps the best known and mostwidely accepted notion of inferential error to come outof the literature on human reasoning. (Evans, 1989, p. 41)

If one were to attempt to identify a singleproblematic aspect of human reasoning thatdeserves attention above all others, the confirma-tion bias would have to be among the candidatesfor consideration. Many have written about thisbias, and it appears to be sufficiently strong andpervasive that one is led to wonder whether thebias, by itself, might account for a significantfraction of the disputes, altercations, and misun-derstandings that occur among individuals,groups, and nations.

Confirmation bias has been used in thepsychological literature to refer to a variety ofphenomena. Here I take the term to represent ageneric concept that subsumes several morespecific ideas that connote the inappropriatebolstering of hypotheses or beliefs whose truthis in question.

Deliberate Versus SpontaneousCase Building

There is an obvious difference betweenimpartially evaluating evidence in order to cometo an unbiased conclusion and building a case tojustify a conclusion already drawn. In the firstinstance one seeks evidence on all sides of a

Correspondence concerning this article should be ad-dressed to Raymond S. Nickerson, Department of Psychol-ogy, Paige Hall, Tufts University, Medford, Massachusetts02155. Electronic mail may be sent to [email protected].

question, evaluates it as objectively as one can,and draws the conclusion that the evidence, inthe aggregate, seems to dictate. In the second,one selectively gathers, or gives undue weightto, evidence that supports one's position whileneglecting to gather, or discounting, evidencethat would tell against it.

There is a perhaps less obvious, but alsoimportant, difference between building a caseconsciously and deliberately and engaging incase-building without being aware of doing so.The first type of case-building is illustrated bywhat attorneys and debaters do. An attorney'sjob is to make a case for one or the other side ofa legal dispute. The prosecutor tries to marshalevidence to support the contention that a crimehas been committed; the defense attorney triesto present evidence that will support thepresumption that the defendant is innocent.Neither is committed to an unbiased weighing ofall the evidence at hand, but each is motivated toconfirm a particular position. Debaters alsowould be expected to give primary attention toarguments that support the positions they aredefending; they might present counterargu-ments, but would do so only for the purpose ofpointing out their weaknesses.

As the term is used in this article and, Ibelieve, generally by psychologists, confirma-tion bias connotes a less explicit, less con-sciously one-sided case-building process. Itrefers usually to unwitting selectivity in theacquisition and use of evidence. The linebetween deliberate selectivity in the use ofevidence and unwitting molding of facts to fithypotheses or beliefs is a difficult one to draw inpractice, but the distinction is meaningfulconceptually, and confirmation bias has more todo with the latter than with the former. The

175

176 RAYMOND S. NICKERSON

assumption that people can and do engage incase-building unwittingly, without intending totreat evidence in a biased way or even beingaware of doing so, is fundamental to theconcept.

The question of what constitutes confirmationof a hypothesis has been a controversial matteramong philosophers and logicians for a longtime (Salmon, 1973). The controversy is exem-plified by Hempel's (1945) famous argumentthat the observation of a white shoe isconfirmatory for the hypothesis "All ravens areblack," which can equally well be expressed incontrapositive form as "All nonblack things arenonravens." Goodman's (1966) claim thatevidence that something is green is equally goodevidence that it is "grue"—grue being definedas green before a specified future date and bluethereafter—also provides an example. A largeliterature has grown up around these and similarpuzzles and paradoxes. Here this controversy islargely ignored. It is sufficiently clear for thepurposes of this discussion that, as used ineveryday language, confirmation connotes evi-dence that is perceived to support—to increasethe credibility of—a hypothesis.

I also make a distinction between what mightbe called motivated and unmotivated forms ofconfirmation bias. People may treat evidence ina biased way when they are motivated by thedesire to defend beliefs that they wish tomaintain. (As already noted, this is not tosuggest intentional mistreatment of evidence;one may be selective in seeking or interpretingevidence that pertains to a belief without beingdeliberately so, or even necessarily being awareof the selectivity.) But people also may proceedin a biased fashion even in the testing ofhypotheses or claims in which they have nomaterial stake or obvious personal interest. Theformer case is easier to understand in common-sense terms than the latter because one canappreciate the tendency to treat evidenceselectively when a valued belief is at risk. But itis less apparent why people should be partial intheir uses of evidence when they are indifferentto the answer to a question in hand. An adequateaccount of the confirmation bias must encom-pass both cases because the existence of each iswell documented.

There are, of course, instances of one wishingto disconfirm a particular hypothesis. If, forexample, one believes a hypothesis to be untrue,

one may seek evidence of that fact or give undueweight to such evidence. But in such cases, thehypothesis in question is someone else's belief.For the individual who seeks to disconfirm sucha hypothesis, a confirmation bias would be abias to confirm the individual's own belief,namely that the hypothesis in question is false.

A Long-Recognized Phenomenon

Motivated confirmation bias has long beenbelieved by philosophers to be an importantdeterminant of thought and behavior. FrancisBacon (1620/1939) had this to say about it, forexample:

The human understanding when it has once adopted anopinion (either as being the received opinion or asbeing agreeable to itself) draws all things else tosupport and agree with it. And though there be a greaternumber and weight of instances to be found on theother side, yet these it either neglects and despises, orelse by some distinction sets aside and rejects; in orderthat by this great and pernicious predetermination theauthority of its former conclusions may remaininviolate.. . . And such is the way of all superstitions,whether in astrology, dreams, omens, divine judg-ments, or the like; wherein men, having a delight insuch vanities, mark the events where they are fulfilled,but where they fail, although this happened muchoftener, neglect and pass them by. (p. 36)

Bacon noted that philosophy and the sciencesdo not escape this tendency.

The idea that people are prone to treatevidence in biased ways if the issue in questionmatters to them is an old one among psycholo-gists also:

If we have nothing personally at stake in a disputebetween people who are strangers to us, we areremarkably intelligent about weighing the evidence andin reaching a rational conclusion. We can be convincedin favor of either of the fighting parties on the basis ofgood evidence. But let the fight be our own, or let ourown friends, relatives, fraternity brothers, be parties tothe fight, and we lose our ability to see any other side ofthe issue than our own. . . . The more urgent theimpulse, or the closer it comes to the maintenance ofour own selves, the more difficult it becomes to berational and intelligent. (Thurstone, 1924, p. 101)

The data that I consider in what follows donot challenge either the notion that peoplegenerally like to avoid personally disquietinginformation or the belief that the strength of abias in the interpretation of evidence increaseswith the degree to which the evidence relatesdirectly to a dispute in which one has a personal

CONFIRMATION BIAS 177

stake. They are difficult to reconcile, however,with the view that evidence is treated in a totallyunbiased way if only one has no personalinterest in that to which it pertains.

The following discussion of this widelyrecognized bias is organized in four majorsections. In the first, I review experimentalevidence of the operation of a confirmation bias.In the second, I provide examples of the bias atwork in practical situations. The third sectionnotes possible theoretical explanations of thebias that various researchers have proposed. Thefourth addresses the question of the effects ofthe confirmation bias and whether it serves anyuseful purposes.

Experimental Studies

A great deal of empirical evidence supportsthe idea that the confirmation bias is extensiveand strong and that it appears in many guises.The evidence also supports the view that onceone has taken a position on an issue, one'sprimary purpose becomes that of defending orjustifying that position. This is to say thatregardless of whether one's treatment of evi-dence was evenhanded before the stand wastaken, it can become highly biased afterward.

Hypothesis-Determined InformationSeeking and Interpretation

People tend to seek information that theyconsider supportive of favored hypotheses orexisting beliefs and to interpret information inways that are partial to those hypotheses orbeliefs. Conversely, they tend not to seek andperhaps even to avoid information that would beconsidered counterindicative with respect tothose hypotheses or beliefs and supportive ofalternative possibilities (Koriat, Lichtenstein, &Fischhoff, 1980).

Beyond seeking information that is support-ive of an existing hypothesis or belief, it appearsthat people often tend to seek only, or primarily,information that will support that hypothesis orbelief in a particular way. This qualification isnecessary because it is generally found thatpeople seek a specific type of information thatthey would expect to find, assuming thehypothesis is true. Also, they sometimes appearto give weight to information that is consistentwith a hypothesis but not diagnostic with respect

to it. These generalizations are illustrated byseveral of the following experimental findings.

Restriction of attention to a favored hypoth-esis. If one entertains only a single possibleexplanation of some event or phenomenon, oneprecludes the possibility of interpreting data assupportive of any alternative explanation. Evenif one recognizes the possibility of otherhypotheses or beliefs, perhaps being aware thatother people hold them, but is strongly commit-ted to a particular position, one may fail toconsider the relevance of information to thealternative positions and apply it (favorably)only to one's own hypothesis or belief.

Restricting attention to a single hypothesisand failing to give appropriate consideration toalternative hypotheses is, in the Bayesianframework, tantamount to failing to take likeli-hood ratios into account. The likelihood ratio isthe ratio of two conditional probabilities,p(D\Hx)lp(p\Hj), and represents the probabilityof a particular observation (or datum) if onehypothesis is true relative to the probability ofthat same observation if the other hypothesis istrue. Typically there are several plausiblehypotheses to account for a specific observation,so a given hypothesis would have severallikelihood ratios. The likelihood ratio for ahypothesis and its complement, p{D\H)lp{D\~H), is of special interest, however,because an observation gives one little evidenceabout the probability of the truth of a hypothesisunless the probability of that observation, giventhat the hypothesis is true, is either substantiallylarger or substantially smaller than the probabil-ity of that observation, given that the hypothesisis false.

The notion of diagnosticity reflects theimportance of considering the probability of anobservation conditional on hypotheses otherthan the favored one. An observation is said tobe diagnostic with respect to a particularhypothesis to the extent that it is consistent withthat hypothesis and not consistent, or not asconsistent, with competing hypotheses and inparticular with the complementary hypothesis.One would consider an observation diagnosticwith respect to a hypothesis and its complementto the degree that the likelihood ratio, p{D\H)lp(D\~H), differed from 1. An observation thatis consistent with more than one hypothesis issaid not to be diagnostic with respect to thosehypotheses; when one considers the probability


of an observation conditional on only a singlehypothesis, one has no way of determiningwhether the observation is diagnostic.

Evidence suggests that people often do theequivalent of considering only p(D\H) andfailing to take into account the ratio of this andp(D\~H), despite the fact that considering onlyone of these probabilities does not provide alegitimate basis for assessing the credibility of//(Beyth-Marom & Fischhoff, 1983; Doherty &Mynatt, 1986; Doherty, Mynatt, Tweney, &Schiavo, 1979; Griffin & Tversky, 1992; Kern &Doherty, 1982; Troutman & Shanteau, 1977).This tendency to focus exclusively on the casein which the hypothesis is assumed to be true isoften referred to as a tendency toward pseudodi-agnosticity (Doherty & Mynatt, 1986; Dohertyet al., 1979; Fischhoff & Beyth-Marom, 1983;Kern & Doherty, 1982). Fischhoff and Beyth-Marom (1983) have argued that much of whathas been interpreted as a confirmation bias canbe attributed to such a focus and the consequen-tial failure to consider likelihood ratios.

Preferential treatment of evidence supportingexisting beliefs. Closely related to the restric-tion of attention to a favored hypothesis is thetendency to give greater weight to informationthat is supportive of existing beliefs or opinionsthan to information that runs counter to them.This does not necessarily mean completelyignoring the counterindicative information butmeans being less receptive to it than tosupportive information—more likely, for ex-ample, to seek to discredit it or to explain itaway.

Preferential treatment of evidence supportingexisting beliefs or opinions is seen in thetendency of people to recall or produce reasonssupporting the side they favor—my-sidebias—on a controversial issue and not to recallor produce reasons supporting the other side(Baron, 1991, 1995; Perkins, Allen, & Hafner,1983; Perkins, Farady, & Bushey, 1991). Itcould be either that how well people remember areason depends on whether it supports theirposition, or that people hold a position becausethey can think of more reasons to support it.Participants in the study by Perkins, Farady, andBushey were capable of generating reasons forholding a view counter to their own whenexplicitly asked to do so; this finding ledTishman, Jay, and Perkins (1993) to interpret the

failure to do so spontaneously as a motivationalproblem as distinct from a cognitive limitation.

Baron (1995) found that, when asked to judgethe quality of arguments, many people werelikely to rate one-sided arguments higher thantwo-sided arguments, suggesting that the bias isat least partially due to common beliefs aboutwhat makes an argument strong. In keeping withthis result, participants in a mock jury trial whotended to use evidence selectively to build oneview of what happened expressed greaterconfidence in their decisions than did those whospontaneously tried to weigh both sides of thecase (D. Kuhn, Weinstock, & Flaton, 1994).

When children and young adults were givenevidence that was inconsistent with a theorythey favored, they often "either failed toacknowledge discrepant evidence or attended toit in a selective, distorting manner. Identicalevidence was interpreted one way in relation to afavored theory and another way in relation to atheory that was not favored" (D. Kuhn, 1989, p.677). Some of Kuhn's participants were unableto indicate what evidence would be inconsistentwith their theories; some were able to generatealternative theories when asked, but they did notdo so spontaneously. When they were asked torecall their theories and the related evidence thathad been presented, participants were likely torecall the evidence as being more consistentwith the theories than it actually was. Thegreater perceived consistency was achievedsometimes by inaccurate recall of theory andsometimes by inaccurate recall of evidence.

Looking only or primarily for positive cases.What is considerably more surprising than thefact that people seek and interpret information inways that increase their confidence in favoredhypotheses and established beliefs is the factthat they appear to seek confirmatory informa-tion even for hypotheses in whose truth valuethey have no vested interest. In their pioneeringconcept-discovery experiments, Bruner, Good-now, and Austin (1956) found that participantsoften tested a hypothesized concept by choosingonly examples that would be classified asinstances of the sought-for concept if thehypothesis were correct. This strategy precludesdiscovery, in some cases, that an incorrecthypothesis is incorrect. For example, supposethe concept to be discovered is small circle andone's hypothesis is small red circle. If one teststhe hypothesis by selecting only things that are


small, red, and circular, one will never discoverthat the class denned by the concept includesalso small circular things that are yellow or blue.Several investigators (Levine, 1970; Millward& Spoehr, 1973; Taplin, 1975; Tweney et al.,1980; Wason & Johnson-Laird, 1972) subse-quently observed the same behavior of partici-pants testing only cases that are members of thehypothesized category.

Some studies demonstrating selective testingbehavior of this sort involved a task invented byWason (1960) in which people were asked tofind the rule that was used to generate specifiedtriplets of numbers. The experimenter presenteda triplet, and the participant hypothesized therule that produced it. The participant then testedthe hypothesis by suggesting additional tripletsand being told, in each case, whether it wasconsistent with the rule to be discovered. Peopletypically tested hypothesized rules by producingonly triplets that were consistent with them.Because in most cases they did not generate anytest items that were inconsistent with thehypothesized rule, they precluded themselvesfrom discovering that it was incorrect if thetriplets it prescribed constituted a subset of thoseprescribed by the actual rule. Given the triplet2-4-6, for example, people were likely to comeup with the hypothesis successive even numbersand then proceed to test this hypothesis bygenerating additional sets of successive evennumbers. If the 2-4-6 set had actually beenproduced by the rule numbers increasing by 2,numbers increasing in size, or any three positivenumbers, the strategy of using only sets ofsuccessive even numbers would not reveal theincorrectness of the hypothesis because everytest item would get a positive response.

The use only of test cases that will yield apositive response if a hypothesis under consider-ation is correct not only precludes discoveringthe incorrectness of certain types of hypotheseswith a correct hypothesis, this strategy wouldnot yield as strongly confirmatory evidence,logically, as would that of deliberately selectingtests that would show the hypothesis to bewrong, if it is wrong, and failing in the attempt.To the extent that the strategy of looking onlyfor positive cases is motivated by a wish to findconfirmatory evidence, it is misguided. Theresults this endeavor will yield will, at best, beconsistent with the hypothesis, but the confirma-tory evidence they provide will not be as

compelling as would the failure of a rigorousattempt at disconfirmation.

This point is worth emphasizing because thepsychological literature contains many refer-ences to the confirmatory feedback a participantgets when testing a hypothesis with a positivecase. These references do not generally distin-guish between confirmatory in a logical senseand confirmatory in a psychological sense. Theresults obtained by Wason (1960) and otherssuggest that feedback that is typically inter-preted by participants to be strongly confirma-tory often is not logically confirmatory, or atleast not strongly so. The "confirmation" theparticipant receives in this situation is, to somedegree, illusory. This same observation appliesto other studies mentioned in the remainder ofthis article.

In an early commentary on the triplet-ruletask, Wetherick (1962) argued that the experi-mental situation did not reveal the participants'intentions in designating any particular triplet asa test case. He noted that any test triplet couldeither conform or not conform to the rule, asdefined by the experimenter, and it could alsoeither conform or not conform to the hypothesisbeing considered by the participant. Any giventest case could relate to the rule and hypothesisin combination in any of four ways: conform-conform, conform-not conform, not conform-conform, and not conform-not conform. Weth-erick argued that one could not determinewhether an individual was intentionally attempt-ing to eliminate a candidate hypothesis unlessone could distinguish between test cases thatwere selected because they conformed to ahypothesis under consideration and those thatwere selected because they did not.

Suppose a participant selects the triplet 3-5-7and is told that it is consistent with the rule (therule being any three numbers in ascendingorder). The participant might have chosen thistriplet because it conforms to the hypothesisbeing considered, say numbers increasing bytwo, and might have taken the positive responseas evidence that the hypothesis is correct. On theother hand, the participant could have selectedthis triplet in order to eliminate one or morepossible hypotheses (e.g., even numbers ascend-ing; a number, twice the number, three times thenumber). In this case, the experimenter'spositive response would constitute the discon-


firming evidence (with respect to these hypoth-eses) the participant sought.

Wetherick (1962) also pointed out that a testtriplet may logically rule out possible hypoth-eses without people being aware of the factbecause they never considered those hypoth-eses. A positive answer to the triplet 3-5-7logically eliminates even numbers ascendingand a number, twice the number, three times thenumber, among other possibilities, regardless ofwhether the participant thought of them. But ofcourse, only if the triplet was selected with theintention of ruling out those options should itsselection be taken as an instance of a falsifica-tion strategy. Wetherick's point was that withoutknowing what people have in mind in makingthe selections they do, one cannot tell whetherthey are attempting to eliminate candidates fromfurther consideration or not.

Wason (1962, 1968/1977) responded to thisobjection with further analyses of the data fromthe original experiment and data from additionalexperiments showing that although some partici-pants gave evidence of understanding theconcept of falsification, many did not. Wasonsummarized the findings from these experi-ments this way: "there would appear to becompelling evidence to indicate that evenintelligent individuals adhere to their ownhypotheses with remarkable tenacity when theycan produce confirming evidence for them"(1968/1977, p. 313).

In other experiments in which participantshave been asked to determine which of severalhypotheses is the correct one to explain somesituation or event, they have tended to askquestions for which the correct answer would beyes if the hypothesis under consideration weretrue (Mynatt, Doherty, & Tweney, 1977; Shak-lee & Fischhoff, 1982). These experiments areamong many that have been taken to reveal notonly a disinclination to test a hypothesis byselecting tests that would show it to be false if itis false, but also a preference for questions thatwill yield a positive answer if the hypothesis istrue.

Others have noted the tendency to askquestions for which the answer is yes if thehypothesis being tested is correct in the contextof experiments on personality perception (Hod-gins & Zuckerman, 1993; Schwartz, 1982;Strohmer & Newman, 1983; Trope & Bassock,1982, 1983; Trope, Bassock, & Alon, 1984;

Zuckerman, Knee, Hodgins, & Miyake, 1995).Fischhoff and Beyth-Marom (1983) also notedthe possibility that participants in such experi-ments tend to assume that the hypothesis theyare asked to test is true and select questions thatwould be the least awkward to answer if that isthe case. For instance, participants askedassumed extroverts (or introverts) questions thatextroverts (or introverts) would find particularlyeasy to answer.

Overweighting positive confirmatory in-stances. Studies of social judgment provideevidence that people tend to overweight positiveconfirmatory evidence or underweight negativediscomfirmatory evidence. Pyszczynski andGreenberg (1987) interpreted such evidence assupportive of the view that people generallyrequire less hypothesis-consistent evidence toaccept a hypothesis than hypothesis-inconsistentinformation to reject a hypothesis. Theseinvestigators argued, however, that this asymme-try is modulated by such factors as the degree ofconfidence one has in the hypothesis to beginwith and the importance attached to drawing acorrect conclusion. Although they saw the needfor accuracy as one important determinant ofhypothesis-evaluating behavior, they suggestedthat other motivational factors, such as needs forself-esteem, control, and cognitive consistency,also play significant roles.

People can exploit others' tendency to over-weight (psychologically) confirming evidenceand underweight disconfirming evidence formany purposes. When the mind reader, forexample, describes one's character in more-or-less universally valid terms, individuals whowant to believe that their minds are being readwill have little difficulty finding substantiatingevidence in what the mind reader says if theyfocus on what fits and discount what does notand if they fail to consider the possibility thatequally accurate descriptions can be produced iftheir minds are not being read (Fischhoff &Beyth-Marom, 1983; Forer, 1949; Hyman,1977). People who wish to believe in astrologyor the predictive power of psychics will have noproblem finding some predictions that haveturned out to be true, and this may suffice tostrengthen their belief if they fail to considereither predictions that proved not to be accurateor the possibility that people without the abilityto see the future could make predictions withequally high (or low) hit rates. A confirmation


bias can work here in two ways: (a) people mayattend selectively to what is said that turns out tobe true, ignoring or discounting what turns outto be false, and (b) they may consider onlyp(D\H), the probability that what was saidwould be said if the seer could really see, andfail to consider p(D\~H), the probability thatwhat was said would be said if the seer had nospecial psychic powers. The tendency of gam-blers to explain away their losses, thus permit-ting themselves to believe that their chances ofwinning are higher than they really are, alsoillustrates the overweighting of supportiveevidence and the underweighting of opposingevidence (Gilovich, 1983).

Seeing what one is looking for. Peoplesometimes see in data the patterns for whichthey are looking, regardless of whether thepatterns are really there. An early study byKelley (1950) of the effect of expectations onsocial perception found that students' percep-tions of social qualities (e.g., relative sociability,friendliness) of a guest lecturer were influencedby what they had been led to expect from a priordescription of the individual. Forer (1949)demonstrated the ease with which people couldbe convinced that a personality sketch was anaccurate depiction of themselves and theirdisinclination to consider how adequately thesketch might describe others as well.

Several studies by Snyder and his colleaguesinvolving the judgment of personality traits lendcredence to the idea that the degree to whichwhat people see or remember corresponds towhat they are looking for exceeds the correspon-dence as objectively assessed (Snyder, 1981,1984; Snyder & Campbell, 1980; Snyder &Gangestad, 1981; Snyder & Swann, 1978a,1978b). In a study representative of this body ofwork, participants would be asked to assess thepersonality of a person they are about to meet.Some would be given a sketch of an extrovert(sociable, talkative, outgoing) and asked todetermine whether the person is that type.Others would be asked to determine whether theperson is an introvert (shy, timid, quiet).Participants tend to ask questions that, if givenpositive answers, would be seen as stronglyconfirmatory and that, if given negative an-swers, would be weakly disconfirmatory of thepersonality type for which they are primed tolook (Snyder, 1981; Snyder & Swann, 1978a).Some of the questions that people ask in these

situations are likely to evoke similar answersfrom extroverts and introverts (Swann, Giuli-ano, & Wegner, 1982)—answers not verydiagnostic with respect to personality type.When this is the case, askers find it easy to seethe answers they get as supportive of thehypothesis with which they are working, inde-pendently of what that hypothesis is.

The interpretation of the results of thesestudies is somewhat complicated by the findingof a tendency for people to respond to questionsin a way that, in effect, acquiesces to whateverhypothesis the interrogator is entertaining (Len-ski & Leggett, 1960; Ray, 1983; Schuman &Presser, 1981; Snyder, 1981; Zuckerman et al.,1995). For example, if the interrogator hypoth-esizes that the interviewee is an extrovert andasks questions that an extrovert would beexpected to answer in the affirmative, theinterviewee is more likely than not to answer inthe affirmative independently of his or herpersonality type. Snyder, Tanke, and Berscheid(1977) have reported a related phenomenon.They found that male participants acted differ-ently in a phone conversation toward femaleparticipants who had been described as attrac-tive than toward those who had been describedas unnattractive, and that their behavior evokedmore desirable responses from the "attractive"than from the "unattractive" partners.

Such results suggest that responders mayinadvertently provide evidence for a workinghypothesis by, in effect, accepting the assump-tion on which the questions or behavior arebased and behaving in a way that is consistentwith that assumption. Thus the observer'sexpectations become self-fulfilling propheciesin the sense suggested by Merton (1948, 1957);the expectations are "confirmed" because thebehavior of the observed person has beenshaped by them, to some degree. Studies ineducation have explored the effect of teachers'expectations on students' performance and haveobtained similar results (Dusek, 1975; Meichen-baum, Bowers, & Ross, 1969; Rosenthal, 1974;Rosenthal & Jacobson, 1968; Wilkins, 1977;Zanna, Sheras, Cooper, & Shaw, 1975).

Darley and Fazio (1980) noted the importanceof distinguishing between the case in which thebehavior of a target person has changed inresponse to the perceiver's expectation-guidedactions and that in which behavior has notchanged but is interpreted to have done so in a


way that is consistent with the perceiver'sexpectations. Only the former is considered anexample of a self-fulfilling prophecy. In thatcase, the change in behavior should be apparentto an observer whose observations are notsubject to distortion by the expectations inquestion. When people's actions are interpretedin ways consistent with observers' expectationsin the absence of any interaction betweenobserver and observed, the self-fulfilling-prophecy effect cannot be a factor.

Darley and Gross (1983) provided evidenceof seeing what one is looking for under the latterconditions noted above. These authors had twogroups of people view the same videotape of achild taking an academic test. One of the groupshad been led to believe that the child'ssocioeconomic background was high and theother had been led to believe that it was low. Theformer group rated the academic abilities, asindicated by what they could see of herperformance on the test, as above grade level,whereas the latter group rated the same perfor-mance as below grade level. Darley and Grosssaw this result as indicating that the participantsin their study formed a hypothesis about thechild's abilities on the basis of assumptionsabout the relationship between socioeconomicstatus and academic ability and then interpretedwhat they saw in the videotape so as to make itconsistent with that hypothesis.

Numerous studies have reported evidence ofparticipants seeing or remembering behaviorthat they expect. Sometimes the effects occurunder conditions in which observers interactwith the people observed and sometimes underconditions in which they do not. "Confirmed"expectations may be based on ethnic (Duncan,1976), clinical (Langer & Abelson, 1974;Rosenhan, 1973; Swann et al., 1982), educa-tional (Foster, Schmidt, & Sabatino, 1976;Rosenthal & Jacobson, 1968), socioeconomic(Rist, 1970), and lifestyle (Snyder & Uranowitz,1978) stereotypes, among other factors. Andthey can involve induced expectations regardingoneself as well as others (Mischel, Ebbensen, &Zeiss, 1973).

Pennebaker and Skelton (1978) have pointedout how a confirmation bias can reinforce theworst fears of a hypochondriac. The body moreor less continuously provides one with anassortment of signals that, if attended to, could

be interpreted as symptoms of illness of one sortor another. Normally people ignore thesesignals. However, if one suspects that one is ill,one is likely to begin to attend to these signalsand to notice those that are consistent with theassumed illness. Ironically, the acquisition offactual knowledge about diseases and theirsymptoms may exacerbate the problem. Uponlearning of a specific illness and the symptomsthat signal its existence, one may look for thosesymptoms in one's own body, thereby increas-ing the chances of detecting them even if theyare not out of the normal range (Woods,Matterson, & Silverman, 1966).

Similar observations apply to paranoia and avariety of neurotic or psychotic states. If onebelieves strongly that one is a target of otherpeople's ill will or aggression, one is likely to beable to fit many otherwise unaccounted-forincidents into this view. As Beck (1976) hassuggested, the tendency of people sufferingfrom mental depression to focus selectively oninformation that gives further reason to bedepressed and ignore information of a morepositive nature could help perpetuate the de-pressed state.

The results of experiments using such abstracttasks as estimating the proportions of beads ofdifferent colors in a bag after observing thecolor(s) of one or a few beads shows that datacan be interpreted as favorable to a workinghypothesis even when the data convey nodiagnostic information. The drawing of beads ofa given color may increase one's confidence in ahypothesis about the color distribution in thebag even when the probability of drawing a beadof that color is the same under the workinghypothesis and its complement (Pitz, 1969;Troutman & Shanteau, 1977). After one hasformed a preference for one brand of acommercial product over another, receivinginformation about an additional feature that iscommon to the two brands may strengthen one'spreexisting preference (Chernev, 1997). Simi-larly, observers of a sports event may describe itvery differently depending on which team theyfavor (Hastorf & Cantril, 1954).

It is true in science as it is elsewhere (Mitroff,1974) that what one sees—actually or metaphori-cally—depends, to no small extent, on what onelooks for and what one expects. Anatomist A.Kolliker criticized Charles Darwin for having


advanced the cause of teleological thinking, andbotanist Asa Gray praised Darwin for the samereason. Meanwhile, biologist T. H. Huxley andnaturalist Ernst Haeckel praised Darwin forthoroughly discrediting thinking of this kind(Gigerenzer et al., 1989).

Several investigators have stressed the impor-tance of people's expectations as sources of biasin their judgments of covariation (Alloy &Abramson, 1980; Alloy & Tabachnik, 1984;Camerer, 1988; Crocker, 1981; Golding &Rorer, 1972; Hamilton, 1979; D. Kuhn, 1989;Nisbett & Ross, 1980). The belief that twovariables are related appears to increase thechances that one will find evidence consistentwith the relationship and decrease the chancesof obtaining evidence that tends to disconfirm it.Judgments of covariation tend to be moreaccurate when people lack strong preconcep-tions of the relationship between the variables ofinterest or when the relationship is consistentwith their preconceptions than when they havepreconceptions that run counter to the relation-ship that exists.

The perception of a correlation where noneexists is sometimes referred to as an illusorycorrelation. The term also has been appliedwhen the variables in question are correlated butthe correlation is perceived to be higher than itactually is Chapman and Chapman (1967a,1967b, 1969; see also Chapman, 1967) did earlywork on the phenomenon. These investigatorshad participants view drawings of humanfigures, each of which occurred with twostatements about the characteristics of theperson who drew it. Statements were paired withdrawings in such a way as to ensure norelationship between drawings and personalitytypes, nevertheless participants saw the relation-ships they expected to see.

Nisbett and Ross (1980) summarized theChapmans' work on illusory correlation bysaying that "reported covariation was shown toreflect true covariation far less than it reflectedtheories or preconceptions of the nature of theassociations that 'ought' to exist" (p. 97).Goldberg (1968) said that this research "illus-trates the ease with which one can 'learn'relationships which do not exist" (p. 493). Forpresent purposes, it provides another illustrationof how the confirmation bias can work: thepresumption of a relationship predisposes one to

find evidence of that relationship, even whenthere is none to be found or, if there is evidenceto be found, to overweight it and arrive at aconclusion that goes beyond what the evidencejustifies.

A form of stereotyping involves believing thatspecific behaviors are more common amongpeople who are members of particular groupsthan among those who are not. There is aperceived correlation between group member-ship and behavior. Such perceived correlationscan be real or illusory. One possible explanationof the perception of illusory correlations is thatunusual behavior by people in distinctive groupsis more salient and easily remembered thansimilar behavior by people who are not mem-bers of those groups (Feldman, Camburn, &Gatti, 1986; Hamilton, Dugan, & Trolier, 1985).Another possibility is that, once a person isconvinced that members of a specific groupbehave in certain ways, he or she is more likelyto seek and find evidence to support the beliefthan evidence to oppose it, somewhat indepen-dently of the facts.

Some investigators have argued that peopletypically overestimate the degree to whichbehavior in different situations can be predictedfrom trait variables or the degree to which anindividual's typical social behavior can bepredicted from a knowledge of that person'sbehavior on a given occasion (Kunda & Nisbett,1986). An illusion of consistency, according tothis view, leads people to misjudge the extent towhich a friendly individual's behavior is consis-tently friendly or a hostile individual's behavioris consistently hostile (Jennings, Amabile, &Ross, 1982; Mischel, 1968; Mischel & Peake,1982). It is easy to see how a pervasiveconfirmation bias could be implicated in thisillusion: once one has categorized an individualas friendly or hostile, one will be more attunedto evidence that supports that categorizationthan to evidence that undermines it.

A more subtle problem relating to categoriza-tion involves the phenomenon of reification.Taxonomies that are invented as conceptualconveniences often come to be seen as represent-ing the way the world is really structured. Giventhe existence of a taxonomy, no matter howarbitrary, there is a tendency to view the world interms of the categories it provides. One tends to


fit what one sees into the taxonomic bins athand. In accordance with the confirmation bias,people are more likely to look for, and find,confirmation of the adequacy of a taxonomythan to seek and discover evidence of itslimitations.

Formal Reasoning and the Selection Task

I have already considered a task invented byWason and much used in rule-discovery experi-ments that revealed a tendency for people to testa hypothesized rule primarily by consideringinstances that are consistent with it. Wason(1966, 1968) invented another task that also hasbeen widely used to study formal reasoning. In awell-known version of this task, participants seean array of cards and are told that each card has aletter on one side and a number on the other.Each of the cards they see shows either a vowel,a consonant, an even number, or an odd number,and participants are asked to indicate whichcards one would have to turn over in order todetermine the truth or falsity of the followingstatement: If a card has a vowel on one side thenit has an even number on the other side.

Suppose the array that people performing thistask see is as follows:

Given this set of cards, experimenters havegenerally considered selection of those showingA and 7 to be correct, because finding an oddnumber on the other side of the A or finding avowel behind the 7 would reveal the statementto be false. The cards showing B and 4 havebeen considered incorrect selections, becausewhatever is on their other sides is consistentwith the statement. In short, one can determinethe claim to be false by finding either the cardshowing the A or the card showing the 7 to beinconsistent with it, or one can determine theclaim to be true by finding both of these cards tobe consistent with it. Wason found that peopleperforming this task are most likely to selectonly the card showing a vowel or the cardshowing a vowel and the one showing an evennumber; people seldom select either the cardshowing a consonant or the one showing an oddnumber. Numerous investigators have obtainedessentially the same result. Experiments withthis task and variants of it have been

reviewed many times (Cosmides, 1989; Evans,1982; Evans, Newstead, & Byrne, 1993; Twe-ney & Doherty, 1983; Wason & Johnson-Laird,1972).

The logic of Wason's selection task is that ofthe conditional: if P then Q. In the case of theabove example, P is there is a vowel on one side,and Q is there is an even number on the other.Selecting the card showing the 7 is analogous tochecking to see if the not-Q case is accompaniedby not-P, as it must be if the conditional is true.The basic finding of experiments with the tasklends support to the hypothesis—which issupported also by other research (Evans, New-stead, & Byrne, 1993; Hollis, 1970)—thatpeople find the modus tollens argument (not-Q,therefore not-P) to be less natural than themodus ponens form (P, therefore Q). And it isconsistent with the idea that, given the objectiveof assessing the credibility of a conditionalassertion, people are more likely to look for thepresence of the consequent given the presenceof the antecedent than for the absence of theantecedent given the absence of the consequent.

Several experiments have shown that perfor-mance of the selection task tends to beconsiderably better when the problem is couchedin familiar situational terms rather than ab-stractly (Johnson-Laird, Legrenzi, & Legrenzi,1972; Wason & Shapiro, 1971), although it is byno means always perfect in the former case(Einhorn & Hogarth, 1978). People also gener-ally do better when the task is couched in termsthat require deontic reasoning (deciding whethera rule of behavior—e.g., permission, obligation,promise—has been violated) rather than indica-tive reasoning (determining whether a hypoth-esis is true or false; Cheng & Holyoak, 1985;Cosmides, 1989; Cosmides & Tooby, 1992;Gigerenzer & Hug, 1992; Griggs & Cox, 1993;Kroger, Cheng, & Holyoak, 1993; Manktelow &Over, 1990, 1991; Markovits & Savary, 1992;Valentine, 1985; Yachanin & Tweney, 1982).

In relating confirmation bias to Wason'sselection task, it is important to make threedistinctions. The first is the distinction betweenthe objective of specifying which of four cardsin view must be turned over in order todetermine the truth or falsity of the assertionwith respect to those four cards and theobjective of saying which of the four types ofcards represented should be turned over in orderto determine the plausibility of the assertion


more generally. I believe that in most of theearlier experiments, the first of these tasks wasthe intended one, although instructions have notalways made it explicit that this was the case.The distinction is critical because what oneshould do when given the first objective is clearand not controversial. However the answer towhat one should do when given the secondobjective is considerably more complicated anddebatable.

When the task is understood in the first way,the only correct answer is the card showing thevowel and the one showing the odd number.However, when one believes one is being askedhow one should go about obtaining evidenceregarding the truth or falsity of an assertion ofthe form if P then Q where P and Q can beunderstood to be proxies for larger sets, it is nolonger so clear that the cards showing P and ~Qare the best choices. Depending on the specificsof the properties represented by P and Q andwhat is known or assumed about their preva-lence in the population of interest, P may be theonly selection that is likely to be very informa-tive, and selection of ~Q may be essentiallypointless (Kirby, 1994b; Nickerson, 1996; Oaks-ford & Chater, 1994; Over & Evans, 1994).

Although experimenters have often, perhapsmore often than not, intended that the selectiontask be interpreted in the first of the two waysdescribed, the second interpretation seems to memore representative of the kind of conditionalreasoning that people are required to do ineveryday life; it is hard to think of everydayexamples of needing to determine the truth orfalsity of a claim about four entities like thecards in the selection task, all of which areimmediately available for inspection. Perhaps atendency to carry over to the task, uncritically,an effective approach in the world of everydayhypothesis testing that is not always effective inthe contrived world of the laboratory contributesto performance on the selection task.

The second and third distinctions pertain toinstances when the task is understood to involveonly the four cards in view. They are thedistinctions that were made in the context of thediscussion of how confirmation bias relates to"find-the-rule" tasks and other tasks involvingconditional syllogisms, namely the distinctionbetween evidence that is confirmatory by virtueof the presence of two entities rather than bytheir joint absence and the distinction between

logical and psychological confirmation. Asapplied to the selection task, confirmation biashas typically connoted a tendency to look for thejoint occurrence of the items specified in thetask description. In the original version of thetask, this means looking for instances of thejoint occurrence of a vowel and an even number,and thus the turning over of the cards showing Aand 4, inasmuch as these are the only cards ofthe four shown that could have both of thenamed properties.

Selection of evidence that can only beconfirmatory of the hypothesis under consider-ation—or avoidance of evidence that couldpossibly falsify the hypothesis—is one interpre-tation of the confirmation bias that is notconsistent with the selection of A and 4 inWason's task. A is a potentially falsifying card;if the rule that cards that have a vowel on oneside have an even number on the other is false,turning over the card showing A could reveal thefact. So in selecting A one is not avoiding thepossibility of falsification; one is performing anappropriate test, even in the strictest Popperiansense. The only way to avoid the possibility ofdisconfirming the hypothesized rule in Wason'stask is to select either or both of the cardsshowing B and 4, inasmuch as nothing that is onthe other sides of these cards could beinconsistent with it.

The fact that, when given the selection task,people are at least as likely to select A as they areto select 4 weighs against the idea that theirbehavior can be attributed to a logically baseddesire to seek only confirming evidence and toavoid potentially disconfirming evidence. I saylogically based because it may be that peopleselect A because they are seeking confirmingevidence, even though their selection alsoprovides the opportunity to acquire falsifyingevidence. From a strictly logical point of view,although selecting A is part of the correctsolution, selecting only A, or A and 4, not onlyfails to ensure the discovery that the hypoth-esized rule is false if it is false, it also fails toestablish its truth if it is true. The only way todemonstrate its truth is to rule out the possibilitythat it is false, and this means checking all (both)cases (A and 7) that could possibly show it to befalse if it is.

So again, behavior that has been interpretedas evidence of a confirmation bias is notstrongly confirmatory in a logical sense, al-


though it may well be seen (erroneously) as soby those who display it. Inasmuch as anobservation can be confirming in a psychologi-cal sense (i.e., interpreted as confirming evi-dence) independently of whether it is logicallyconfirmatory, perhaps the confirmation biasshould be thought of as a tendency to seekevidence that increases one's confidence in ahypothesis regardless of whether it should. Thedistinction between logical confirmation andpsychological confirmation deserves greateremphasis than it has received.

Although researchers have interpreted theconsiderable collection of results that have beenobtained from experiments with variations ofthe selection task as generally reflective of aconfirmation bias, alternative hypotheses havealso been proposed. L. J. Cohen (1981) arguedthat it is not necessary to suppose that people doanything more in testing a conditional rule thancheck whether the consequent holds true when-ever the antecedent does. The fact that peoplesometimes select the card representing Q (4 inthe case of the present example) as well as theone representing P (A) is because they falselyperceive ifP then Q to be equivalent to ifQ thenP (conversion error); and the fact that they donot select the card representing not-Q (7 in thepresent example) is attributed to a failure toapply the law of the contrapositive.

Several investigators have argued that peopleinterpret the conditional relationship ifP then Qas equivalent to the biconditional relationship ifand only if P then Q, or that they fail todistinguish between ifP then Q and ifQ then P,especially as the if-then construction is used ineveryday language (Chapman & Chapman,1959; L. J. Cohen, 1981; Henle, 1962; Legrenzi,1970; Politzer, 1986; Revlis, 1975a, 1975b).Others have assumed the operation of amatching bias whereby people tend to focus on(and select) cards explicitly named in thestatement of the rule to be tested (Evans, 1972;Evans & Lynch, 1973; Hoch & Tschirgi, 1983).Evans (1989) has suggested that people maketheir selections, without thinking much aboutthem, on the basis of a "preconscious heuristicjudgment of relevance, probably linguisticallydetermined" (p. 108).

Recent theorizing about the selection task hasgiven rise to several new explanations of howpeople interpret it and deal with it, especially aspresented in various concrete frames of refer-

ence. An idea common to several of theseexplanations is that people's performance of thetask is indicative of behavior that has proved tobe adaptively effective in analogous real-worldsituations. Cosmides (1989), for example, hasargued that the typical results are consistent withthe idea that people are predisposed to look forcheating on a social contract and are particularlyskilled at detecting it. Consequently, they dowell on selection tasks in which checking thenot-Q condition is analogous to looking for aninstance of violating a tacit social agreement,like failing to meet an obligation. Gigerenzerand Hug (1992) have defended a similar point ofview.

Liberman and Klar (1996) proposed analternative explanation of the results of researchon cheating detection. They argued that acheating-detection perspective is neither neces-sary nor sufficient to yield the P and not-Qselection; they contend that the logically correctchoice will be likely if the conditional ifP thenQ is understood to represent a deterministic (asopposed to a probabilistic) unidirectional rela-tionship between P and Q, if what constitutes aviolation of the rule is clear, and if it isunderstood that the task is to look for such aviolation.

Other investigators have begun to view theselection task as more a problem of dataselection and decision making than of logicalinference (Kirby, 1994a, 1994b; Manktelow &Over, 1991, 1992). Oaksford and Chater (1994),for example, have proposed a model of the taskaccording to which people should make selec-tions so as to maximize their gain in informationregarding the tenability of the hypothesis thatthe conditional ifP then Q is true relative to thetenability that it is false. I have done an analysisthat leads to a similar conclusion, at least whenthe task is perceived as that of deciding whetherthe conditional is true in a general sense, asopposed to being true of four specific cards(Nickerson, 1996).

Wason's selection task has proved to be oneof the most fertile paradigms in experimentalpsychology. It would be surprising if all theresults obtained with the many variations of thetask could be accounted for by a single simplehypothesis. I believe that, taken together, theresults of experimentation support the hypoth-esis that one of the several factors determiningperformance on this task is a confirmation bias


that operates quite pervasively. This is not tosuggest that a confirmation bias explains all thefindings with this task, or even that it is the mostimportant factor in all cases, but only that itexists and often plays a substantive role.

The Primacy Effect and Belief Persistence

When a person must draw a conclusion on thebasis of information acquired and integratedover time, the information acquired early in theprocess is likely to carry more weight than thatacquired later (Lingle & Ostrom, 1981; Sher-man, Zehner, Johnson, & Hirt, 1983). This iscalled the primacy effect. People often form anopinion early in the process and then evaluatesubsequently acquired information in a way thatis partial to that opinion (N. H. Anderson &Jacobson, 1965; Jones & Goethals, 1972;Nisbett & Ross, 1980; Webster, 1964). FrancisBacon (1620/1939) observed this tendencycenturies ago: "the first conclusion colors andbrings into conformity with itself all that comeafter" (p. 36).

Peterson and DuCharme (1967) had peoplesample a sequence of colored chips and estimatethe probability that the sequence came from anurn with a specified distribution of colors ratherthan from a second urn with a differentdistribution. The sampling was arranged so thatthe first 30 trials favored one urn, the second 30favored the other, and after 60 trials the evidencewas equally strong for each possibility. Partici-pants tended to favor the urn indicated by thefirst 30 draws, which is to say that the evidencein the first 30 draws was not countermanded bythe evidence in the second 30 even thoughstatistically it should have been.

Bruner and Potter (1964) showed people thesame picture on a series of slides. The first slidewas defocused so the picture was not recogniz-able. On successive slides the focus was madeincreasingly clear. After each presentation theparticipant was asked to identify what wasshown. A hypothesis formed on the basis of adefocused picture persisted even after thepicture was in sufficiently good focus thatparticipants who had not looked at the poorlyfocused picture were able to identify it correctly.Jones, Rock, Shaver, Goethals, and Ward (1968)had participants form opinions about people'sproblem-solving abilities by watching them inaction. The problem solvers were judged to be

more competent if they solved many problemsearly in a problem series and few late than ifthey did the reverse.

The primacy effect is closely related to (andcan perhaps be seen as a manifestation of) beliefpersistence. Once a belief or opinion has beenformed, it can be very resistive to change, evenin the face of fairly compelling evidence that itis wrong (Freedman, 1964; Hay den & Mischel,1976; Luchins, 1942,1957; Rhine & Severance,1970; Ross, 1977; Ross & Lepper, 1980; Ross,Lepper, & Hubbard, 1975). Moreover it can biasthe evaluation and interpretation of evidencethat is subsequently acquired. People are morelikely to question information that conflicts withpreexisting beliefs than information that isconsistent with them and are more likely to seeambiguous information to be confirming ofpreexisting beliefs than disconfirming of them(Ross & Anderson, 1982). And they can be quitefacile at explaining away events that areinconsistent with their established beliefs (Hen-rion & Fischhoff, 1986).

I have already discussed studies by Pitz(1969) and Troutman and Shanteau (1977)showing that people sometimes take norma-tively uninformative data as supportive of afavored hypothesis. One could interpret resultsfrom these studies as evidence of a belief-preserving bias working in situations in whichthe "information" in hand is not really informa-tive. This is in keeping with the finding that twopeople with initially conflicting views canexamine the same evidence and both findreasons for increasing the strength of theirexisting opinions (Lord, Ross, & Lepper, 1979).The demonstration by Pitz, Downing, andReinhold (1967) that people sometimes interpretevidence that should count against a hypothesisas counting in favor of it may be seen as anextreme example of the confirmation biasserving the interest of belief preservation.

Ross and his colleagues have also shownexperimentally that people find it extremelyeasy to form beliefs about or generate explana-tions of individuals' behavior and to perseverein those beliefs or explanations even afterlearning that the data on which the beliefs orexplanations were originally based were ficti-tious (Ross et al., 1975; Ross, Lepper, Strack, &Steinmetz, 1977). Ross et al. (1975), forexample, had people attempt to distinguishbetween authentic and unauthentic suicide


notes. As the participants made their choices,they were given feedback according to apredetermined schedule that was independent ofthe choices they made, but that insured that thefeedback some participants received indicatedthat they performed far above average on thetask while that which others received indicatedthat they performed far below average.

Following completion of Ross et al.'s (1975)task, researchers informed participants of thearbitrary nature of the feedback and told themthat their rate of "success" or "failure" waspredetermined and independent of their choices.When the participants were later asked to ratetheir ability to make such judgments, those whohad received much positive feedback on theexperimental task rated themselves higher thandid those who had received negative feedback,despite being told that they had been givenarbitrary information. A follow-up experimentfound similar perseverance for people whoobserved others performing this task (but did notperform it themselves) and also observed thedebriefing session.

Nisbett and Ross (1980) pointed out how aconfirmation bias could contribute to the perse-verance of unfounded beliefs of the kindinvolved in experiments like this. Receivingfeedback that supports the assumption that oneis particularly good or particularly poor at a taskmay prompt one to search for additionalinformation to confirm that assumption. To theextent that such a search is successful, the beliefthat persists may rest not exclusively on thefraudulent feedback but also on other evidencethat one has been able to find (selectively) insupport of it.

It is natural to associate the confirmation biaswith the perseverance of false beliefs, but in factthe operation of the bias may be independent ofthe truth or falsity of the belief involved. Notonly can it contribute to the perseverance ofunfounded beliefs, but it can help make beliefsfor which there is legitimate evidence strongerthan the evidence warrants. Probably few beliefsof the type that matter to people are totallyunfounded in the sense that there is nolegitimate evidence that can be marshalled forthem. On the other hand, the data regardingconfirmation bias, in the aggregate, suggest thatmany beliefs may be held with a strength ordegree of certainty that exceeds what theevidence justifies.

Own-Judgment Evaluation

Many researchers have done experiments inwhich people have been asked to express theirdegree of confidence in judgments that theyhave made. When participants have expressedconfidence as probability estimates or as ratingsthat, with some plausible assumptions, can betransformed into probability estimates, it hasbeen possible to compare confidence withperformance on the primary task. Thus research-ers can determine for each confidence judgmentthe percentage of the correct items on theprimary task to which that judgment wasassigned. Plots of actual percent correct againstpercent correct "predicted" by the confidencejudgments are often referred to as calibrationcurves; perfect calibration is represented by theunit line, which indicates that for a givenconfidence level, X, the proportion of all thejudgments with that level that were correct was X.

In general, people tend to express a higherdegree of confidence than is justified by theaccuracy of their performance on the primarytask, which is to say that calibration studies havetypically shown overconfidence to be morecommon than underconfidence (Einhorn &Hogarth, 1978; Fischhoff, 1982; Lichtenstein &Fischhoff, 1977; Lichtenstein, Fischhoff, &Phillips, 1977; Pitz, 1974; Slovic, Fischhoff, &Lichtenstein, 1977). Kahneman and Tversky(1973) refer to the confidence that people feelfor highly fallible performance as the illusion ofvalidity. Being forced to evaluate one's views,especially when that includes providing reasonsagainst one's position, has reduced overconfi-dence in some instances (Fischhoff, 1977; Hoch,1984, 1985; Koriat, Lichtenstein, & Fischhoff,1980; Tetlock & Kim, 1987). But generallyoverconfidence has only been reduced, noteliminated, and providing reasons against one'sposition is not something that most people dospontaneously.

One explanation of overconfidence starts withthe assumption that people tend to be goodjudges of their knowledge as it relates tosituations they are likely to encounter ineveryday life. This explanation of overconfi-dence also notes that a minimum requirementfor observing good calibration in experimentalsituations is that the questions people are toanswer and that one to be used to judge theprobability of the correctness of their answers


must be selected in such a way as to ensure thatvalid cues in the natural environment remainvalid in the experimental situation. Juslin (1993)has argued that certain strategies commonlyused to select items in experiments more or lessensure that valid knowledge from the partici-pants' natural environment will be less valid inthe experimental situation. More specifically,the argument is that such selection strategiestypically result in sets of items for which cuesleading to wrong answers are overrepresentedrelative to their commonness in the naturalenvironment. Experiments showing good calibra-tion for items selected at random from a setassumed to be representative of the naturalenvironment (Gigerenzer, Hoffrage, & Kleinbolt-ing, 1991; Juslin, 1993, 1994) support thisexplanation.

I believe this argument to be a forceful one.The evidence is fairly strong that some of theoverconfidence reported, especially in experi-ments that require people to answer general-knowledge questions in a forced-choice format,is an artifact of item-selection procedures thatdo not ensure that cues have the same validity inexperimental situations that they have in typicalreal-life situations. Not all studies of confidenceor calibration have used general-knowledgequestions and the forced-choice format foranswers, however, and overconfidence has alsobeen observed in other contexts. Researchershave shown that assessments of peoples' abilityto recognize faces, for example, correlate poorlywith performance on a face-recognition task(Baddeley & Woodhead, 1983) and observedthat retrospective judgments of comprehensionof expository texts are higher than justified(Glenberg, Wilkinson, & Epstein, 1982).

Experts are not immune from the illusion ofvalidity. Attorneys tend, in the aggregate, toexpress confidence of obtaining outcomes betterthan those they actually obtain (Loftus &Wagenaar, 1988; Wagenaar & Keren, 1986).Other professionals who have been found to beoverconfident when making judgments in theirown areas of expertise include physicians(Lusted, 1977), psychologists (Oskamp, 1965),and engineers (Kidd, 1970). Experts appear todo better when there is a reliable basis forstatistical prediction, such as when predictingbridge hands (Keren, 1987) or betting odds forhorse racing (Hausch, Ziemba, & Rubenstein,1981). On the other hand, despite the fact that

clinical diagnoses based on case statistics tendto be more accurate than those based on clinicaljudgments, clinicians typically have greaterconfidence in their own judgments than in thosederived statistically from incidence data (Arkes,Dawes, & Christensen, 1986; Goldberg, 1968;Meehl, 1954, 1960; Sawyer, 1966). In contrast,professional weather forecasters tend to berelatively well calibrated, at least with respect totheir weather predictions (Murphy & Winkler,1974, 1977; Winkler & Murphy, 1968); this hasbeen attributed to the fact that they receiveconstant and immediate feedback regarding theaccuracy of their predictions, which is the kindof information that makes learning feasible.

Griffin and Tversky (1992) have made adistinction between strength (extremeness) ofevidence and weight (predictive validity) ofevidence. They hypothesized that people tend tofocus primarily on strength and make some(typically insufficient) adjustments in responseto weight. This hypothesis leads to the expecta-tion that overconfidence will be the ruleespecially when strength is high and weight low,whereas underconfidence becomes more likelywhen the opposite is the case. This account isconsistent with the finding that overconfidenceis the general rule because of the assumedgreater importance attached to evidentiarystrength. The account is even more predictive ofgeneral overconfidence if one assumes thatstrong evidence is more common than weightyevidence, as Griffin and Tversky use these terms.

Another account of why people tend to beoverconfident of their own knowledge is thatwhen one has produced a tentative answer to aquestion, one's natural inclination is to searchmemory for evidence to support that answer andto fail to consider possible alternative answers(Graesser & Hemphill, 1991; Griffin, Dunning,& Ross, 1990; Hoch, 1985; Koriat et al., 1980;Shaklee & Fischhoff, 1982). This is similar tothe explanation already mentioned of whypeople sometimes persevere with a belief evenafter learning that the information on which thebelief was based was fictitious: after havingformed the belief they sought and foundindependent data to corroborate it.

The Confirmation Biasin Real-World Contexts

As the foregoing review shows, the confirma-tion bias has been observed in a variety of guises


in many experimental situations. What makes itespecially important to understand is that it canhave significant consequences in many nonlabo-ratory contexts. The point is illustrated in whatfollows by a few examples.

Number Mysticism

Pythagoras discovered, by experimentation,how the pitch of a sound emitted by a vibratingstring depends on the length of the string andwas able to state this dependency in terms ofsimple numerical ratios: (a) two strings of thesame material under the same tension differ inpitch by an octave when one of the strings istwice as long as the other, and (b) two strings thelengths of which are in the ratio 2 to 3 willproduce a note and its fifth, and so on.

Observation was not the new thing thatPythagoras did in his study of harmonicrelationships; people had been observing theheavens and recording what they saw for a longtime. What he did that was new was manipulatewhat he was observing and take notice of theeffects of those manipulations. He has beencalled the first experimentalist. Ironically, in-stead of establishing experimentation as anespecially fruitful way to investigate the proper-ties of the physical world, Pythagoras's discov-ery helped to usher in what Bell called "thegolden age of number mysticism" and to delaythe acceptance of experimentation as the pri-mary method of science for 2000 years (Bell,1946/1991). Pythagoras's intellectual heirs wereso convinced of the validity of his pronounce-ment, "everything is number," that many of themost able thinkers over the next 2 milleniadevoted much of their cognitive energy to thepursuit of numerology and the confirmation ofits basic assumptions. It took a Galileo to kindlean interest in experimentation that would notagain sputter and die.

Work done as recently as the mid-19thcentury involving the great pyramid of Egypt,which has extraordinary fascination for modernobservers of artifacts of ancient cultures, illus-trates the relevance of the confirmation bias toPythagorean numerological pursuits. Much ofthis fascination is due to certain mathematicalrelationships discussed first by John Taylor(1859) and shortly later by Charles Smyth(1864/1890). Taylor noted, among other facts,that the ratio of twice the pyramid's base to its

height was roughly the same as the ratio of thediameter of a circle to its circumference, whichis to say, IT. Smyth was inspired by Taylor'sobservations and set himself the task ofdiscovering other mathematical relationships ofinterest that the monument might hide.

Smyth discovered that the ratio of thepyramid's base to the width of a casing stonewas 365, the number of days in a year, and thatthe pyramid's height multiplied by 109 wasapproximately equal to the distance from theearth to the sun. By comparing pyramid lengthmeasurements in various ways, he was able tofind numbers that correspond to many quantita-tive properties of the world that were presum-ably unknown when the pyramid was built.These include the earth's mean density, theperiod of precession of the earth's axis, and themean temperature of the earth's surface. VonDaniken (1969) used the existence of suchrelationships as the basis for arguing that theearth had been visited by intelligent extraterres-trials in the past.

Gardner (1957) referred to Smyth's book as aclassic of its kind illustrating beautifully "theease with which an intelligent man, passionatelyconvinced of a theory, can manipulate hissubject matter in such a way as to make itconform to precisely held opinions" (p. 176). Hepointed out that a complicated structure like thepyramid provides one with a great assortment ofpossible length measurements, and that anyonewith the patience to juggle them is quite sure tofind a variety of numbers that will coincide withsome dates or figures that are of interest forhistorical or scientific reasons. One simplymakes an enormous number of observations,tries every manipulation on measurements andmeasurement relationships that one can imag-ine, and then selects from the results those fewthat coincide with numbers of interest in othercontexts. He wrote, "Since you are bound by norules, it would be odd indeed if this search forPyramid 'Truths' failed to meet with consider-able success" (p. 177). The search for pyramidtruths is a striking illustration of how a bias toconfirm is expressed by selectivity in the searchfor and interpretation of information.

Witch Hunting

From a modern vantage point, the convictionsand executions of tens of thousands of individu-


als for practicing witchcraft during the 15th,16th, and 17th centuries in Western Europe andto a lesser extent in 18th-century New England,is a particularly horrific case of the confirmationbias functioning in an extreme way at thesocietal level. From the perspective of many,perhaps most, of the people of the time, belief inwitchcraft was perfectly natural and sorcery waswidely viewed as the reason for ills and troublesthat could not otherwise be explained. Execution-ers meted out punishment for practising witch-craft—generally the stake or the scaffold—withappalling regularity. Mackay (1852/1932) putthe number executed in England alone, duringthe first 80 years of the 17th century, at 40,000.

People believed so strongly in witchcraft thatsome courts had special rules of evidence toapply only in cases involving it. Mackay(1852/1932) quoted Bodinus, a 17th-centuryFrench authority, as follows:

The trial of this offense must not be conducted likeother crimes. Whoever adheres to the ordinary courseof justice perverts the spirit of the law, both divine andhuman. He who is accused of sorcery should never beacquitted, unless the malice of the prosecutor be clearerthan the sun; for it is so difficult to bring full proof ofthis secret crime, that out of a million witches not onewould be convicted if the usual course were followed!(p. 528)

Torture was an accepted and widely practicedmeans of exacting confessions from accusedpersons. Failure to denounce a witch was insome places a punishable offense.

It is hard for those who live in a society thatrecognizes the principle that a person accused ofa crime is to be presumed innocent until provenguilty beyond a reasonable doubt to imaginewhat it must have been like to live at a timewhen—at least with respect to the accusation ofwitchcraft—just the opposite principle held. Onthe other hand, it is perhaps too easy to assumethat nothing like the witchcraft mania couldoccur in our enlightened age. A moment'sreflection on the instances of genocide orattempted genocide that have occurred in recenttimes should make people wary of any suchassumption. These too can be seen as instancesof what can happen when special rules ofevidence are used to protect and support beliefsthat individuals, groups or nations want, forwhatever reasons, very much to hold. Awarenessof a prevalent bias toward confirmation evenunder relatively ideal conditions of evidence

evaluation should make people doubly cautiousabout the potential for disaster when circum-stances encourage the bias to be carried toexcess.

Policy Rationalization

Tuchman (1984) described a form of confirma-tion bias at work in the process of justifyingpolicies to which a government has committeditself: "Once a policy has been adopted andimplemented, all subsequent activity becomesan effort to justify it" (p. 245). In the context of adiscussion of the policy that drew the UnitedStates into war in Vietnam and kept the U.S.military engaged for 16 years despite countlessevidences that it was a lost cause from thebeginning, Tuchman argued that once a policyhas been adopted and implemented by agovernment, all subsequent activity of thatgovernment becomes focused on justification ofthat policy:

Wooden headedness, the source of self deception is afactor that plays a remarkably large role in government.It consists in assessing a situation in terms ofpreconceived fixed notions while ignoring or rejectingany contrary signs. It is acting according to wish whilenot allowing oneself to be deflected by the facts. It isepitomized in a historian's statement about Philip II ofSpain, the surpassing wooden head of all sovereigns:"no experience of the failure of his policy could shakehis belief in its essential excellence." (p. 7)

Folly, she argued, is a form of self-deceptioncharacterized by "insistence on a rooted notionregardless of contrary evidence" (p. 209).

At the beginning of this article, I gaveexamples of bias in the use of information thatwould not be considered illustrative of theconfirmation bias, as that term is used here.These examples involved intentional selectivityin the use of information for the consciouspurpose of supporting a position. Anotherexample is that of politicians taking note of factsthat are consistent with positions they havetaken while intentionally ignoring those that areinconsistent with them. This is not to suggest,however, that all selective use of information inthe political arena is knowing and deliberate andthat confirmation bias, in the sense of unwittingselectivity, is not seen here. To the contrary, Isuspect that this type of bias is especiallyprevalent in situations that are inherentlycomplex and ambiguous, which many political


situations are. In situations characterized byinteractions among numerous variables and inwhich the cause-effect relationships are ob-scure, data tend to be open to many interpreta-tions. When that is the case, the confirmationbias can have a great effect, and people shouldnot be surprised to see knowledgeable, well-intentioned people draw support for diametri-cally opposed views from the same set of data.

Medicine

The importance of testing theories, hypoth-eses, speculations, or conjectures about theworld and the way it works, by understandingtheir implications vis-a-vis observable phenom-ena and then making the observations necessaryto check out those implications, was a commontheme in the writings of the individuals whogave science its direction in the 16th, 17th, and18th centuries. This spirit of empirical criticismhad not dominated thinking in the precedingcenturies but was a new attitude. Theretoforeobservation, and especially controlled experi-mentation, had played second fiddle to logic andtradition.

Consider, for example, the stagnated status ofmedical knowledge: "For 1500 years the mainsource of European physicians' knowledgeabout the human body was not the body itself. . . [but] the works of an ancient Greekphysician [Galen]. 'Knowledge' was the barrierto knowledge. The classic source became arevered obstacle." Boorstin (1985, p. 344), whowrote these words, noted the irony of the factthat Galen himself was an experimentalist andurged others who wished to understand anatomyor medicine to become hands-on investigatorsand not to rely only on reading what others hadsaid. But Galen's readers found it easier to relyon the knowledge he passed on to them than tofollow his advice regarding how to extend it.

Although some physicians did "experiment"with various approaches to the treatment ofdifferent diseases and ailments, as Thomas(1979) pointed out, until quite recently, thisexperimentation left something to be desired asa scientific enterprise:

Virtually anything that could be thought up for thetreatment of disease was tried out at one time oranother, and, once tried, lasted decades or evencenturies before being given up. It was, in retrospect,the most frivolous and irresponsible kind of human

experimentation, based on nothing but trial and error,and usually resulting in precisely that sequence.Bleeding, purging, cupping, the administration ofinfusions of every known plant, solutions of everyknown metal, every conceivable diet including totalfasting, most of these based on the weirdest imaginingsabout the cause of disease, concocted out of nothing butthin air—this was the heritage of medicine up until alittle over a century ago. It is astounding that theprofession survived so long, and got away with somuch with so little outcry. Almost everyone seems tohave been taken in. (p. 159)

How is it that ineffective measures could becontinued for decades or centuries without theirineffectiveness being discovered? Sometimespeople got better when they were treated;sometimes they did not. And sometimes they gotbetter when they were not treated at all. Itappears that people's beliefs about the efficacyof specific treatments were influenced morestrongly by those instances in which treatmentwas followed by recovery than in either those inwhich it was not or those in which it occurredspontaneously. A prevailing tendency to focusexclusively or primarily on positive cases—cases in which treatment was followed byrecovery—would go a long way toward account-ing for the fact that the discoveries that certaindiseases have a natural history and people oftenrecover from them with or without treatmentwas not made until the 19th century.

Fortunately such a tendency is no longercharacteristic of medical science as a whole; butone would be hard pressed to argue that it nolonger influences the beliefs of many peopleabout the efficacy of various treatments ofmedical problems. Every practitioner of a formof pseudomedicine can point to a cadre ofpatients who will testify, in all sincerity, tohaving benefited from the treatment. Moregenerally, people engage in specific behaviors(take a pill, rest, exercise, think positively) forthe express purpose of bringing about a specifichealth-related result. If the desired result occurs,the natural tendency seems to be to attribute it towhat was done for the purpose of causing it;considering seriously the possibility that theresult might have been obtained in the absenceof the associated "cause" appears not to comenaturally to us but to have to be learned.

Medical diagnosis, as practiced today, hasbeen the subject of some research. In looking forcauses of illness, diagnosticians tend to generateone or a small set of hypotheses very early in the


diagnostic session (Elstein, Shulman, & Sprafka,1978). The guidance that a hypothesis in handrepresents for further information gathering canfunction as a constraint, decreasing the likeli-hood that one will consider an alternativehypothesis if the one in hand is not correct.Failure to generate a correct hypothesis has beena common cause of faulty diagnosis in somestudies (Barrows, Feightner, Neufeld, & Nor-man, 1978; Barrows, Norman, Neufeld, &Feightner, 1977). A hypothesis in hand also canbias the interpretation of subsequently acquireddata, either because one selectively looks fordata that are supportive of the hypothesis andneglects to look for disconfirmatory data (Bar-rows et al., 1977) or because one interprets datato be confirmatory that really are not (Elstein etal., 1978).

Studies of physicians' estimates of the prob-ability of specific diagnoses have often shownestimates to be too high (Christensen-Szalanski& Bushyhead, 1981/1988; DeSmet, Fryback, &Thornbury, 1979). Christensen-Szalanski andBushyhead (1981/1988) had physicians judge,on the basis of a medical history and physicalexamination, the probability that patients at awalk-in clinic had pneumonia. Only about 20percent of the patients who were judged to havepneumonia with a probability between .8 and .9actually had pneumonia, as determined by chestX rays evaluated by radiologists. These investi-gators also got physicians to rate the variouspossible outcomes from such a diagnosis andfound no difference between the values assignedto the two possible correct diagnoses and nodifference between the values assigned to thepossible incorrect ones. This suggests that theoverly high estimates of probability of diseasewere not simply the result of a strong preferenceof false positive over false negative results.

Results from other studies suggest thatphysicians do not do very well at revisingexisting probability estimates to take intoaccount the results of diagnostic tests (Berwick,Fineberg, & Weinstein, 1981; Cassells, Schoen-berger, & Grayboys, 1978). A diagnostic findingthat fails to confirm a favored hypothesis may bediscounted on the grounds that, inasmuch asthere is only a probabilistic relationship betweensymptoms and diseases, a perfect match is not tobe expected (Elstein & Bordage, 1979).

Judicial Reasoning

In the context of judicial proceedings, anattempt is made to decouple the process ofacquiring information from that of drawingconclusions from that information. Jurors areadmonished to maintain open minds during thepart of a trial when evidence is being presented(before the jury-deliberation phase); they are notsupposed to form opinions regarding what theverdict should be until all the evidence has beenpresented and they have been instructed by thejudge as to their decision task. During thejury-deliberation phase, the jury's task is toreview and consider, collectively, the evidencethat has been presented and to arrive, throughdiscussion and debate, at a consensus on theverdict. They are to be careful to omit fromconsideration any evidence that came to lightduring the trial that was deemed inadmissibleand ordered stricken from the record.

The admonition to maintain an open mindduring the evidence-presentation phase of a trialseems designed to counter the tendency to forman opinion early in an evidence-evaluationprocess and then to evaluate further evidencewith a bias that favors that initial opinion.Whether jurors are able to follow this admoni-tion and to delay forming an opinion as to thetruth or falsity of the allegations against theaccused until the jury-deliberation phase is amatter of some doubt. It is at least a plausiblepossibility that individual jurors (and judges)develop their personal mental models of "whatreally happened" as a case develops andcontinuously refine and elaborate those modelsas evidence continues to be presented (Holstein,1985). To the extent that this is the way jurorsactually think, a model, as it exists at any pointin time, may strongly influence how newinformation is interpreted. If one has come tobelieve that a defendant is guilty (or innocent),further evidence that is open to various interpre-tations may be seen as supportive of that belief.Opinions formed about a defendant on the basisof superficial cues (such as demeanor whilegiving testimony) can bias the interpretation ofinherently ambiguous evidence (Hendry &Shaffer, 1989).

Results of mock-trial experiments indicatethat, although there are considerable individualdifferences among mock jurors with respect to


how they approach their task (D. Kuhn et al.,1994), jurors often come to favor a particularverdict early in the trial process (Devine &Ostrom, 1985). The final verdicts that juriesreturn are usually the same as the tentative onesthey initially form (Kalven & Zeisel, 1966;Lawson, 1968). The results of some mock trialssuggest that deliberation following the presenta-tion of evidence tends to have the effect ofmaking a jury's average predeliberation opinionmore extreme in the same direction (Myers &Lamm, 1976).

The tendency to stick with initial tentativeverdicts could exist because in most cases theinitial conclusions stand up to further objectiveevaluation; there is also the possibility, however,that the tentative verdict influences jurors'subsequent thinking and biases them to look for,or give undo weight to, evidence that supports it.This possibility gains credence from the findingby Pennington and Hastie (1993) that partici-pants in mock-jury trials were more likely toremember statements consistent with theirchosen verdict as having been presented as trialevidence than statements that were inconsistentwith this verdict. This was true both ofstatements that had been presented (hits) and ofthose that had not (false positives).

Science

Polya (1954a) has argued that a tendency toresist the confirmation bias is one of the ways inwhich scientific thinking differs from everydaythinking:

The mental procedures of the trained naturalist are notessentially different from those of the common man,but they are more thorough. Both the common man andthe scientist are led to conjectures by a few observa-tions and they are both paying attention to later caseswhich could be in agreement or not with the conjecture.A case in agreement makes the conjecture more likely,a conflicting case disproves it, and here the differencebegins: Ordinary people are usually more apt to lookfor the first kind of cases, but the scientist looks for thesecond kind. (p. 40)

If seeking data that would disconfirm ahypothesis that one holds is the general ruleamong scientists, the history of science gives usmany exceptions to the rule (Mahoney, 1976,1977; Mitroff, 1974). Michael Faraday waslikely to seek confirming evidence for ahypothesis and ignore such disconfirming evi-dence as he obtained until the phenomenon

under study was reasonably well understood, atwhich time he would begin to pay moreattention to disconfirming evidence and activelyseek to account for it (Tweney & Doherty,1983). Louis Pasteur refused to accept orpublish results of his experiments that seemed totell against his position that life did not generatespontaneously, being sufficiently convinced ofhis hypothesis to consider any experiment thatproduced counterindicative evidence to benecessarily flawed (Farley & Geison, 1974).When Robert Millikan published the experimen-tal work on determining the electric charge of asingle electron, for which he won the Nobelprize in physics, he reported only slightly morethan half (58) of his (107) observations, omittingfrom publication those that did not fit hishypothesis (Henrion & Fischhoff, 1986).

It is not so much the critical attitude thatindividual scientists have taken with respect totheir own ideas that has given science thesuccess it has enjoyed as a method for makingnew discoveries, but more the fact that indi-vidual scientists have been highly motivated todemonstrate that hypotheses that are held bysome other scientist(s) are false. The insistenceof science, as an institution, on the objectivetestability of hypotheses by publicly scrutablemethods has ensured its relative independencefrom the biases of its practitioners.

Conservatism among scientists. The factthat scientific discoveries have often metresistance from economic, technological, reli-gious, and ideological elements outside sciencehas been highly publicized. That such discover-ies have sometimes met even greater resistancefrom scientists, and especially from those whosetheoretical positions were challenged or invali-dated by those discoveries, is no less a fact ifless well known (Barber, 1961; Mahoney, 1976,1977). Galileo, for example, would not acceptKepler's hypothesis that the moon is responsiblefor the tidal motions of the earth's oceans.Newton refused to believe that the earth couldbe much older than 6,000 years on the strengthof the reasoning that led Archbishop Usher toplace the date of creation at 4,004 BC. Huygensand Leibniz rejected Newton's concept ofuniversal gravity because they could not acceptthe idea of a force extending throughout spacenot reducible to matter and motion.

Humphrey Davy dismissed John Dalton'sideas about the atomic structure of matter as


more ingenious than important. William Thom-son (Lord) Kelvin, who died in 1907, someyears after the revolutionary work of JosephThomson on the electron, never accepted theidea that the atom was decomposable intosimpler components. Lev Landau was willing in1932 to dismiss the laws of quantum mechanicson the grounds that they led to such a ridiculousprediction as the contraction of large burned outstars to essentially a point. Arthur Eddingtonrejected Subrahmanyan Chandrasekhar's predic-tion in the early 1930s that cold stars with amass of more than about one half that of the sunwould collapse to a point. Chandrasekhar wassufficiently discouraged by Eddington's reactionto leave off for the better part of his professionalcareer the line of thinking that eventually led tothe now widely accepted theory of black holes.

Theory persistence. The history of sciencecontains many examples of individual scientiststenaciously holding on to favored theories longafter the evidence against them had becomesufficiently strong to persuade others withoutthe same vested interests to discard them. I. B.Cohen (1985) has documented many of thesestruggles in his account of the role of revolutionin science. All of them can be seen as examplesof the confirmation bias manifesting itself as anunwillingness to give the deserved weight toevidence that tells against a favored view.

Roszak (1986) described the tenacity withwhich adherents to the cosmology of Ptolemyclung to their view in the face of mountingevidence of its untenability in this way:

The Ptolemaic cosmology that prevailed in ancienttimes and during the Middle Ages had been compro-mised by countless contradictory observations overmany generations. Still, it was an internally coherent,intellectually pleasing idea; therefore, keen mindsstood by the familiar old system. Where there seemedto be any conflict, they simply adjusted and elaboratedthe idea, or restructured the observations in order tomake them fit. If observations could not be made to fit,they might be allowed to stand along the culturalsidelines as curiosities, exceptions, freaks of nature. Itwas not until a highly imaginative constellation of ideasabout celestial and terrestrial dynamics, replete withnew concepts of gravitation, inertia, momentum, andmatter, was created that the old system was retired,(p. 91)

Roszak pointed out also that scientists through-out the 18th and 19th centuries retained otherinherited ideas in the fields of chemistry,

geology, and biology in adjusted forms, despiteincreasing evidences of their inadequacies.

Science has held, often for very long times,some beliefs and theories that could have beeninvalidated if a serious effort had been made toshow them to be false. The belief that heavierbodies fall faster than lighter ones, for example,prevailed from the time of Aristotle until that ofGalileo. The experiments that Galileo per-formed to demonstrate that this belief was falsecould have been done at any time during that2000-year period. This is a particularly interest-ing example of a persisting false belief becauseit might have been questioned strictly on thebasis of reasoning, apart from any observations.Galileo posed a question that could have beenasked by Aristotle or by anybody who believedthat heavier bodies fall faster than lighter ones:If a 10 pound weight falls faster than a 1 poundweight, what will happen when the two are tiedtogether? Will the 11 pound combination fallfaster than the 10 pound weight, or will it fallmore slowly because the 1 pound weight willhold back the 10 pound one?

Hawking (1988) argued that the fact that theuniverse is expanding could have been predictedfrom Newton's theory of gravity at any timeafter the late 17th century, and noted thatNewton himself realized that the idea ofuniversal gravity begged an explanation as towhy the stars did not draw together. Theassumption that the universe was static was astrong and persistent one, however. Newtondealt with the problem by assuming the universewas infinite and consequently had no center ontowhich it could collapse. Others introduced anotion that at very great distances gravitybecame repulsive instead of attractive. Einstein,in order to make his general theory of relativitycompatible with the idea of a static universe,incorporated a "cosmological constant" in thetheory which, in effect, nullified the otherwiseexpected contraction. Einstein later was embar-rassed by this invention and saw it as his greatestmistake.

None of this is to suggest that scientistsaccept evidences of the inadequacy of anestablished theory with equanimity. Rather, Inote that the typical reaction to the receipt ofsuch evidences is not immediately to discard thetheory to which the inadequacies relate but tofind a way to defend it. The bias is definitely inthe direction of giving the existing theory the


benefit of the doubt, so long as there is room fordoubt and, in some cases, even when there isnot. The usual strategy for dealing withanomalous data is first to challenge the datathemselves. If they prove to be reliable, the nextstep is to complicate the existing theory justenough to accommodate the anomalous resultto, as T. S. Kuhn (1970) put it, "devisenumerous articulations and ad hoc modifica-tions of [the] theory in order to eliminate anyapparent conflict" (p. 78). If that proves to betoo difficult, one may decide simply to live withthe anomaly, at least for a while. When a theoryis confronted with too many anomalies to beaccommodated in this way—or when, as aconsequence of a series of modifications thetheory becomes too convoluted to manage andan alternative theory becomes available—thereis the basis of a paradigm shift and arevolutionary reorientation of thinking.

Overconfidence. Overconfidence in experi-mental results has manifested itself in thereporting of a higher-than-warranted degree ofcertainty or precision in variable measurements.Scientific investigators often have underesti-mated the uncertainty of their measurements andthus reported errors of estimate that have notstood the test of time. Fundamental constantsthat have been reported with uncertainty esti-mates that later proved too small include thevelocity of light, the gravitational constant, andthe magnetic moment of the proton (Henrion &Fischhoff, 1986).

The 1919 British expedition to West Africa totake advantage of a solar eclipse in order to testEinstein's prediction that the path of light wouldbe bent by a gravitational field represents anespecially noteworthy case of the reporting of ahigher-than-warranted degree of precision inmeasurement. Einstein had made the predictionin the 1915 paper on the general theory ofrelativity. Scientists later discovered that theerror of measurement was as great as the effectthat was being measured so that, as Hawking(1988) put it, "The British team's measurementhad been sheer luck, or a case of knowing theresult they wanted to get, not an uncommonoccurrence in science" (p. 32).

The predictions have subsequently beenverified with observations not subject to thesame measurement problems, but as first madeand reported, they suggest the operation aconfirmation bias of considerable strength. In a

detailed account of the event, Collins and Pinch(1993) noted that Eddington's data were noisy,that he had to decide which photographs tocount and which to ignore, and that he usedEinstein's theory to make these decisions. Asthey put it:

Eddington could only claim to have confirmed Einsteinbecause he used Einstein's derivations in deciding whathis observations really were, while Einstein's deriva-tions only became accepted because Eddington'sobservation seemed to confirm them. Observation andprediction were linked in a circle of mutual confirma-tion rather than being independent of each other as wewould expect according to the conventional idea of anexperimental test. (p. 45)

Collins and Pinch's account of the reportingof the results of the 1919 expedition and of thesubsequent widespread adoption of relativity asthe new standard paradigm of physics representsscientific advance as somewhat less inexorablydetermined by the cold objective assessment oftheory in the light of observational fact than it issometimes assumed to be.

Henrion and Fischhoff (1986) suggested thatthe overconfidence associated with the estimatesthey considered could have resulted fromscientists overlooking, for one reason or an-other, specific sources of uncertainty in theirmeasurements. This possibility is consistentwith the results of laboratory studies of judg-ment showing that people typically find it easierto think of reasons that support a conclusionthey have drawn than to think of reasons thatcontradict it and that people generally havedifficulty in thinking of reasons why their bestguess might be wrong (Koriat et al., 1980).

By way of rounding out this discussion ofconfirmation bias in science, it is worth notingthat prevailing attitudes and opinions canchange rapidly within scientific communities, asthey can in other communities. Today's revolu-tionary idea is tomorrow's orthodoxy. Ideasconsidered daring, if not bizarre or downrightridiculous when first put forward, can becomeaccepted doctrine or sometimes obvious truthsthat no reasonable person would contest inrelatively short periods of time. According toLakatos (1976)

Newton's mechanics and theory of gravitation was putforward as a daring guess which was ridiculed andcalled "occult" by Leibniz and suspected even byNewton himself. But a few decades later—in absenceof refutations—his axioms came to be taken asindubitably true. Suspicions were forgotten, critics


branded "eccentric" if not "obscurantist"; some of hismost doubtful assumptions came to be regarded as sotrivial that textbooks never even stated them. (p. 49,Footnote 1)

One can see a confirmation bias both in thedifficulty with which new ideas break throughopposing established points of view and in theuncritical allegiance they are often given oncethey have become part of the established viewthemselves.

Explanations of the Confirmation Bias

How is one to account for the confirmationbias and its prevalence in so many guises? Is it amatter of protecting one's ego, a simplereluctance to consider the possibility that abelief one holds or a hypothesis that one isentertaining is wrong? Is it a consequence ofspecific cognitive limitations? Does it reflect alack of understanding of logic? Does it persistbecause it has some functional value? That is,does it provide some benefits that are asimportant as, or in some situations moreimportant than, an attempt to determine the truthin an unbiased way would be?

The Desire to Believe

Philosophers and psychologists alike haveobserved that people find it easier to believepropositions they would like to be true thanpropositions they would prefer to be false. Thistendency has been seen as one manifestation ofwhat has been dubbed the Pollyanna principle(Matlin & Stang, 1978), according to whichpeople are likely to give preferential treatmentto pleasant thoughts and memories over unpleas-ant ones.

Finding a positive correlation between theprobability that one will believe a proposition tobe true and the probability that one will considerit to be desirable (Lefford, 1946; McGuire,1960; Weinstein, 1980, 1989) does not, in itself,establish a causal link between desirability andperceived truth. The correlation could reflect arelationship between truth and desirability in thereal world, whereby what is likely to be true islikely also to be desirable, and the same isconversely true. On the other hand, the evidenceis strong that the correlation is the result, at leastto some degree, of beliefs being influenced bypreferences. The continuing susceptibility of

people to too-good-to-be-true promises of quickwealth is but one illustration of the fact thatpeople sometimes demand very little in the wayof compelling evidence to drive them to aconclusion that they would like to accept.

Although beliefs can be influenced by prefer-ences, there is a limit to how much influencepeople's preferences can have. It is not the case,for most of us at least, that we are free to believeanything we want; what we believe must appearto us believable. We can be selective withrespect to the evidence we seek, and we can tiltthe scales when we weigh what we find, but wecannot completely ignore counterindicative evi-dence of which we are aware. Kunda (1990) hasmade this argument persuasively. The very factthat we sometimes seek to ignore or discountevidence that counts against what we would liketo believe bears witness to the importance weattach to holding beliefs that are justified.

More generally, one could view, somewhatironically perhaps, the tendency to treat dataselectively and partially as a testament to thehigh value people attach to consistency. Ifconsistency between beliefs and evidence wereof no importance, people would have no reasonto guard beliefs against data that are inconsistentwith them. Consistency is usually taken to be animportant requirement of rationality, possiblythe most important such requirement. Paradoxi-cally, it seems that the desire to be consistent canbe so strong as to make it difficult for one toevaluate new evidence pertaining to a statedposition in an objective way.

The quote from Mackay (1852/1932) that isused as an epigraph at the beginning of thisarticle stresses the importance of motivation inefforts to confirm favored views. Some investi-gators have argued, however, that the basicproblem is not motivational but reflects limita-tions of a cognitive nature. For example, Nisbettand Ross (1980) held that, on the whole,investigators have been too quick to attribute thebehavior of participants in experimental situa-tions to motivational biases when there wereequally plausible alternative interpretations ofthe findings:

One wonders how strongly the theory of self-servingbias must have been held to prompt such uncriticalacceptance of empirical evidence. . . . We doubt thatcareful investigation will reveal ego-enhancing orego-defensive biases in attribution to be as pervasive or


potent as many lay people and most motivationaltheorists presume them to be. (p. 233)

This argument is especially interesting in thepresent context, because it invokes a form ofconfirmation bias to account for the tendency ofsome investigators to attribute certain behaviorsto motivational causes and to ignore what, inNisbett and Ross's view, are equally likelyalternative explanations.

The role of motivation in reasoning has beena subject of debate for some time. Kunda (1990)noted that many of the phenomena that oncewere attributed to motivational variables havebeen reinterpreted more recently in cognitiveterms; according to this interpretation, conclu-sions that appear to be drawn only becausepeople want to draw them may be drawnbecause they are more consistent with priorbeliefs and expectancies. She noted too thatsome theorists have come to believe thatmotivational effects are mediated by cognitiveprocesses. According to this view, "[p]eoplerely on cognitive processes and representationsto arrive at their desired conclusions, butmotivation plays a role in determining which ofthese will be used on a given occasion" (Kunda,1990, p. 480).

Kunda defended this view, arguing that theevidence to date is consistent with the assump-tion that motivation affects reasoning, but itdoes so through cognitive strategies for access-ing, constructing, and evaluating beliefs:

Although cognitive processes cannot fully account forthe existence of self-serving biases, it appears that theyplay a major role in producing these biases in that theyprovide the mechanisms through which motivationaffects reasoning. Indeed, it is possible that motivationmerely provides an initial trigger for the operation ofcognitive processes that lead to the desired conclusions,(p. 493)

The primary cognitive operation hypoth-esized to mediate motivational effects is thebiased searching of memory. Evidence ofvarious types converges, she argued, on theconclusion that "goals enhance the accessibilityof those knowledge structures—memories, be-liefs, and rules—that are consistent with desiredconclusions" (p. 494); "Motivation will causebias, but cognitive factors such as the availablebeliefs and rules will determine the magnitudeof the bias" (p. 495).

Several of the accounts of confirmation biasthat follow stress the role of cognitive limita-

tions as causal factors. It is possible, however,and probable, in my view, that both motivationaland cognitive factors are involved and that eachtype can mediate effects of the other.

Information-Processing Basesfor Confirmation Bias

The confirmation bias is sometimes attributedin part to the tendency of people to gatherinformation about only one hypothesis at a timeand even with respect to that hypothesis toconsider only the possibility that the hypothesisis true (or only the possibility that it is false) butnot to consider both possibilities simultaneously(Tweney, 1984; Tweney & Doherty, 1983).Doherty and Mynatt (1986) argued, for ex-ample, that people are fundamentally limited tothink of only one thing at a time, and oncehaving focused on a particular hypothesis, theycontinue to do so. This, they suggested, explainswhy people often select nondiagnostic overdiagnostic information in Bayesian decisionsituations. Suppose that one must attempt todecide which of two diseases, A or B, a patientwith Symptoms X and Y has. One is informed ofthe relative frequency of Symptom X amongpeople who have Disease A and is then given thechoice of obtaining either of the following itemsof information: the relative frequency of peoplewith A who have Symptom Y or the relativefrequency of people with B who have symptomX. Most people who have been given choices ofthis sort opt for the first; they continue to focuson the hypothesis that the patient has A, eventhough learning the relative frequency of Ygiven A does not inform the diagnosis, whereaslearning the relative frequency of X given Bdoes.

Assuming a restricted focus on a singlehypothesis, it is easy to see how that hypothesismight become strengthened even if it is false. Anincorrect hypothesis can be sufficiently close tobeing correct that it receives a considerableamount of positive reinforcement, which may betaken as further evidence of the correctness ofthe hypothesis in hand and inhibit continuedsearch for an alternative. In many contextsintermittent reinforcement suffices to sustain thebehavior that yields it.

People also can increase the likelihood ofgetting information that is consistent withexisting beliefs and decrease the likelihood of


getting information that is inconsistent withthem by being selective with respect to wherethey get information (Frey, 1986). The idea thatpeople tend to expose themselves more toinformation sources that share their beliefs thanto those that do not has had considerablecredibility among social psychologists (Fes-tinger, 1957; Klapper, 1960). Sears and Freed-man (1967) have challenged the conclusivenessof much of the evidence that has been evoked insupport of this idea. They noted that, when givena choice of information that is supportive of aview one holds and information that is support-ive of an opposing view, people sometimesselect the former and sometimes the latter, andsometimes they show no preference. Behaviorin these situations seems to depend on a numberof factors in addition to the polarity of theinformation with respect to one's existing views,such as people's level of education or socialstatus and the perceived usefulness of theinformation that is offered.

Sears and Freedman (1967) stopped short,however, of concluding that people are totallyunbiased in this respect. They noted thepossibility that "dramatic selectivity in prefer-ences may not appear at any given moment intime, but, over a long period, people mayorganize their surroundings in a way thatensures de facto selectivity" (p. 213). Peopletend to associate, on a long-term basis, withpeople who think more or less as they do onmatters important to them; they read authorswith whom they tend to agree, listen to newscommentators who interpret current events in away that they like, and so on. The extent towhich people choose their associates because oftheir beliefs versus forming their beliefs becauseof their associates is an open question. But itseems safe to assume that it goes a bit in bothdirections. Finding lots of support for one'sbeliefs and opinions would be a naturalconsequence of principally associating withpeople with whom one has much in common.

Gilovich (1991) made the important point thatfor many beliefs or expectations confirmatoryevents are likely to be more salient thannonconfirmatory ones. If a fortune teller predictsseveral events in one's life sometime during theindefinite future, for example, the occurrence ofany predicted event is more likely to remind oneof the original prediction of that event than is itsnonoccurrence. Events predicted by a fortune

teller illustrate one-sided events. Unlike two-sided events, which have the characteristic thattheir nonoccurrence is as noticeable as theiroccurrence, one-sided events are likely to benoticed while their nonoccurrence is not. (Anexample of a two-sided event would be the tossof a coin following the prediction of a head. In thiscase the nonoccurrence of the predicted out-come would be as noticeable as its occurrence.)

Sometimes decision policies that rule out theoccurrence of certain types of events precludethe acquisition of information that is counter-indicative with respect to a hypothesis. Con-sider, for example, the hypothesis that onlystudents who meet certain admission require-ments are likely to be successful as collegestudents. If colleges admit only students whomeet those requirements, a critical subset of thedata that are necessary to falsify the hypothesis(the incidence of students who do not meet theadmission requirements but are neverthelesssuccessful college students) will not exist(Einhorn & Hogarth, 1978). One could arguethat this preclusion may be justified if thehypothesis is correct, or nearly so, and if thenegative consequences of one type of error(admitting many students who will fail) aremuch greater than those of the other type(failing to admit a few students who wouldsucceed). But the argument is circular because itassumes the validity of the hypothesis inquestion.

Some beliefs are such that obtaining evidencethat they are false is inherently impossible. If Ibelieve, for example, that most crimes arediscovered sooner or later, my belief may bereinforced every time a crime is reported by alaw enforcement agency. But by definition,undiscovered crimes are not discovered, so thereis no way of knowing how many or them thereare. For the same reason, if I believe that mostcrimes go undiscovered, there is no way todemonstrate that this belief is wrong. No matterhow many crimes are discovered, the number ofundiscovered crimes is indeterminable becausebeing undiscovered means being uncounted.

Some have also given information-processingaccounts of why people, in effect, consider onlythe probability of an event, assuming the truth ofa hypothesis of interest, and fail to consider theprobability of the same event, assuming thefalsity of that hypothesis. Evans (1989) argued


that one need not assume the operation of amotivational bias—a strong wish to con-firm—in order to account for this failure. Itcould signify, according to Evans, a lack ofunderstanding of the fact that, without aknowledge of p(D\~H), p(D\H) gives one nouseful information about p(H\D). That is,p(D\H), by itself, is not diagnostic with respectto the truth or falsity of H. Possibly peopleconfuse p(D\H) with p(H\D) and take itsabsolute value as an indication of the strength ofthe evidence in favor of H. Bayes's rule ofinverse probability, a formula for getting fromp(D\H) to p(H\D) presumably was motivated,in part, to resolve this confusion.

Another explanation of why people fail toconsider alternatives to a hypothesis in hand isthat they simply do not think to do so. Plausiblealternatives do not come to mind. This is seen bysome investigators to be, at least in part, a matterof inadequate effort, a failure to do a sufficientlyextensive search for possibilities (Baron, 1985,1994; Kanouse, 1972).

Positive-Test Strategy or Positivity Bias

Arguing that failure to distinguish amongdifferent senses of confirmation in the literaturehas contributed to misinterpretations of bothempirical findings and theoretical prescriptions,Klayman and Ha (1987) have suggested thatmany phenomena of human hypothesis testingcan be accounted for by the assumption of ageneral positive-test strategy. Application of thisstrategy involves testing a hypothesis either byconsidering conditions under which the hypoth-esized event is expected to occur (to see if itdoes occur) or by examining known instances ofits occurrence (to see if the hypothesizedconditions prevailed). The phenomenon bearssome resemblance to the finding that, in theabsence of compelling evidence one way or theother, people are more inclined to assume that astatement is true than to assume that it is false(Clark & Chase, 1972; Gilbert, 1991; Trabasso,Rollins, & Shaughnessy, 1971; Wallsten &Gonzalez- Vallejo, 1994).

Baron, Beattie, and Hershey (1988), whoreferred to the positive-test strategy as thecongruence heuristic, have shown that thetendency to use it can be reduced if people areasked to consider alternatives, but that they tendnot to consider them spontaneously. The fact

that people show a preference for questions thatwould yield a positive answer if the hypothesisis correct over questions that would yield anegative answer if the hypothesis is correctdemonstrates that the positive-test strategy is nota simple consequence of the confirmation bias(Baron et al., 1988; Devine, Hirt, & Gehrke,1990; Skov & Sherman, 1986).

One could view the results that Wason (1960)originally obtained with the number-triplet task,which constituted the point of departure formuch of the subsequent work on confirmationbias, as a manifestation of the positive-teststrategy, according to which one tests cases onethinks likely to have the hypothesized property(Klayman & Ha, 1987). As already noted, thisstrategy precludes discovering the incorrectnessof a hypothesized rule when instances thatsatisfy the hypothesized rule constitute a subsetof those that satisfy the correct one, but it is aneffective strategy when the instances that satisfythe correct rule constitute a subset of those thatsatisfy the hypothesized one.

Suppose, for example, that the hypothesizedrule is successive even numbers and the correctone is increasing numbers. All triplets thatsatisfy the hypothesized rule also satisfy thecorrect one, so using only such triplets as testcases will not reveal the incorrectness of thehypothesized rule. But if the hypothesized ruleis increasing numbers and the correct rule issuccessive even numbers, the positive-test strat-egy—which, in this case means trying varioustriplets of increasing numbers—is likely toprovide the feedback necessary to discover thatthe hypothesized rule is wrong. In both of theexamples considered, the strategy is to selectinstances for testing that satisfy the hypoth-esized rule; it is effective in revealing the rule tobe wrong in the second case, not because thetester intentionally selects instances that willshow the rule to be wrong if it is wrong, butbecause the relationship between hypothesizedand correct rules is such that the tester is likelyto discover the hypothesized rule to be wrong byselecting test cases intended to show that it isright.

Klayman and Ha (1987) analyzed variouspossible relationships between the sets oftriplets delimited by hypothesized and correctrules in addition to the two cases in which oneset is a proper subset of the other—overlappingsets, disjoint sets, identical sets—and showed


that the likely effectiveness of positive-teststrategy depends on which relationship pertains.They also analyzed corresponding cases inwhich set membership is probabilistic and drewa similar conclusion. With respect to disconfir-mation, Klayman and Ha argued the importanceof distinguishing between two strategies, oneinvolving examination of instances that areexpected not to have the target property, and theother involving examination of instances thatone expects to falsify, rather than verify, thehypothesis. Klayman and Ha contended thatfailure to make this distinction clearly in the pasthas been responsible for some confusion anddebate.

Several investigators have argued that apositive-test strategy should not necessarily beconsidered a biased information-gathering tech-nique because questions prompted by thisstrategy generally do not preclude negativeanswers and, therefore, falsification of thehypothesis being tested (Bassock & Trope,1984; Hodgins & Zuckerman, 1993; Skov &Sherman, 1986; Trope & Mackie, 1987). It isimportant to distinguish, however, betweenobtaining information from a test because thetest was intended to yield the informationobtained and obtaining information adventi-tiously from a test that was intended to yieldsomething other than what it did. Consider againthe number-triplet task. One might select, say,6-7-8 for either of the following reasons,among others: (a) because one believes the ruleto be increasing numbers and wants a triplet thatfits the rule, or (b) because one believes the ruleto be successive even numbers and wants toincrease one's confidence in this belief byselecting a triplet that fails to satisfy the rule.The chooser expects the experimenter's re-sponse to the selected triplet to be positive in thefirst case and negative in the second. In eithercase, the experimenter's response could beopposite from what the chooser expects andshow the hypothesized rule to be wrong; theresponse is adventitious because the test yieldedinformation that the chooser was not seekingand did not expect to obtain.

The fact that people appear to be more likelythan not to test a hypothesized rule by selectingcases that they believe will pass muster (that fitthe hypothesized rule) and lend credence to thehypothesis by doing so justifies describingtypical performance on find-the-rule tasks like

the triplet task as illustrative of a confirmationbias. People appear to be much less likely toattempt to get evidence for a hypothesized ruleby choosing a case that they believe does not fitit (and expecting an informative-negative re-sponse) or to select cases with the intendedpurpose of ruling out one or more currentlyplausible hypotheses from further consideration.When a test that was expected to yield anoutcome that is positive with respect to ahypothesized rule does not do so, it seemsappropriate to say that the positive-test strategyhas proved to be adventitiously informative.Clearly, it is possible to select an item that isconsistent with a hypothesized rule for thepurpose of revealing an alternative rule to bewrong, but there is little evidence that this isoften done. It seems that people generally selecttest items that are consistent with the rule theybelieve to be correct and seldom select itemswith falsification in mind.

Klayman and Ha (1987) argued that thepositive-test strategy is sometimes appropriateand sometimes not, depending on situationalvariables such as the base rates of the phenom-ena of interest, and that it is effective undercommonly occurring conditions. They arguedtoo, however, that people tend to rely on itoverly much, treating it as a default strategy tobe used when testing must be done underless-than-ideal conditions, and that it accountsfor many of the phenomena that are generallyinterpreted as evidence of a pervasive confirma-tion bias.

Evans (1989) has proposed an explanation ofthe confirmation bias that is similar in somerespects to Klayman and Ha's (1987) accountand also discounts the possibility that peopleintentionally seek to confirm rather than falsifytheir hypotheses. Cognitive failure, and notmotivation, he argued, is the basis of thephenomenon:

Subjects confirm, not because they want to, but becausethey cannot think of the way to falsify. The cognitivefailure is caused by a form of selective processingwhich is very fundamental indeed in cognition—a biasto think about positive rather than negative informa-tion, (p. 42)

With respect to Wason's (1960) results withthe number-triplet task, Evans suggested thatrather than attempting to confirm the hypothesesthey were entertaining, participants may simplyhave been unable to think of testing them in a


negative manner; from a logical point of view,they should have used potentially disconfirmingtest cases, but they failed to think to do so.

Evans (1989) cited the results of severalefforts by experimenters to modify hypothesis-testing behavior on derivatives of Wason'snumber-set task by instructing participantsabout the importance of seeking negative ordisconfirming information. He noted that al-though behavioral changes were sometimesinduced, more often performance was notimproved. This too was taken as evidence thatresults that have been attributed to confirmationbias do not stem from a desire to confirm, butrather from the difficulty people have in thinkingin explicitly disconfirmatory terms.

Evans used the same argument to account forthe results typically obtained with Wason's(1966, 1968) selection task that have generallybeen interpreted as evidence of the operation ofa confirmation bias. Participants select nameditems in this task, according to this view,because these are the only ones that come tomind when they are thinking about what to do.The bias that is operating is not that of wantingto confirm a tentative hypothesis but that ofbeing strongly inclined to think only of informa-tion that is explicitly provided in the problemstatement. In short, Evans (1989) distinguishedbetween confirmatory behavior, which he ac-knowledges, and confirmatory intentions, whichhe denies. The "demonstrable deficiencies in theway in which people go about testing andeliminating hypotheses," he contended, "are afunction of selective processing induced by awidespread cognitive difficulty in thinkingabout any information which is essentiallynegative in its conception" (p. 63). Thetendency to focus on positive information andfail to consider negative information is regardednot as a conscious cognitive strategy but as theresult of preattentive processes. Positive is notsynonymous with confirmatory, but as moststudies have been designed, looking for positivecases is tantamount to looking for confirmation.

The idea that people have a tendency to focusmore on positive than on negative information,like the idea of a confirmation bias, is an oldone. An observation by Francis Bacon (1620/1939) can again illustrate the point: "It is thepeculiar and perpetual error of the humanunderstanding to be more moved and excited byaffirmatives than negatives; whereas it ought

properly to hold itself indifferently disposedtowards both alike" (p. 36). Evans's (1989)argument that such a bias can account, at least inpart, for some of the phenomena that areattributed to a confirmation bias is an intuitivelyplausible one. A study in which Perkins et al.,(1983) classified errors of reasoning made ininformal arguments constructed by over 300people of varying age and educational levelsupports the argument. Many of the errorsPerkins et al. identified involved participants'failure to consider lines of reasoning that couldbe used to defeat or challenge their conclusions.

I will be surprised if a positivity bias turns outto be adequate to account for all of the ways inwhich what has here been called a confirmationbias manifests itself, but there are manyevidences that people find it easier to deal withpositive information than with negative: it iseasier to decide the truth or falsity of positivethan of negative sentences (Wason, 1959,1961);the assertion that something is absent takeslonger to comprehend than the assertion thatsomething is present (Clark, 1974); and infer-ences from negative premises require more timeto make or evaluate and are more likely to beerroneous or evaluated incorrectly than are thosethat are based on positive premises (Fodor,Fodor, & Garrett, 1975). How far the idea can bepushed is a question for research. I suspect thatfailure to try to construct counterarguments or tofind counterevidence is a major and relativelypervasive weakness of human reasoning.

In any case, the positivity bias itself requiresan explanation. Does such a bias have somefunctional value? Is it typically more importantfor people to be attuned to occurrences than tononoccurrences of possible events? Is languageprocessed more effectively by one who ispredisposed to hear positive rather than negativeassertions? Is it generally more important to beable to make valid inferences from positive thanfrom negative premises? It would not besurprising to discover that a positivity bias isadvantageous in certain ways, in which caseBacon's dictum that we should be indifferentlydisposed toward positives and negatives wouldbe wrong.

There is also the possibility that the positivitybias is, at least in part, motivationally based.Perhaps it is the case that the processing ofnegative information generally takes more effortthan does the processing of positive information


and often we are simply not willing to make theeffort that adequate processing of the negativeinformation requires. As to why the processingof negative information should require moreeffort than the processing of positive informa-tion, perhaps it is because positive informationmore often than not is provided by the situation,whereas negative information must be activelysought, from memory or some other source. It isgenerally clear from the statement of a hypoth-esis, for example, what a positive instance of thehypothesized event would be, whereas thatwhich constitutes a negative instance mayrequire some thought.

Conditional Reference Frames

Several investigators have shown that whenpeople are asked to explain or imagine why ahypothesis might be true or why a possible eventmight occur, they tend to become more con-vinced that the hypothesis is true or that theevent will occur, especially if they have notgiven much thought to the hypothesis or eventbefore being asked to do so (Campbell & Fairey,1985; Hirt & Sherman, 1985; Sherman et al.,1983). In some cases, people who were asked toexplain why a particular event had occurred andthen were informed that the event did not occurafter they did so, still considered the event more"likely" than did others who were not asked toexplain why it might occur (Ross et al., 1977).

Koehler (1991), who has reviewed much ofthe work on how explanations influence beliefs,has suggested that producing an explanation isnot the critical factor and that simply coming upwith a focal hypothesis is enough to increaseone's confidence in it. Anything, he suggested,that induces one to accept the truth of ahypothesis temporarily will increase one'sconfidence that it is, in fact, true. Callingattention to a specified hypothesis results in theestablishment of a focal hypothesis, and this inturn induces the adoption of a conditionalreference frame, in which the focal hypothesis isassumed to be true.

Adoption of a conditional reference frameinfluences subsequent hypothesis-evaluation pro-cesses in three ways, Koehler (1991) suggested:it affects the way the problem is perceived, howrelevant evidence is interpreted, and the direc-tion and duration of information search. Accord-ing to this author,

Once a conditional reference frame has been inducedby an explanation task, a certain inertia sets in, whichmakes it more difficult to consider alternative hypoth-eses impartially. In other words, the initial impressionseems to persist despite the person's efforts to ignore itwhile trying to give fair consideration to an alternativeview. (p. 503)

Hoch's (1984) findings regarding people whowere asked to generate reasons for expecting aspecified event and reasons against expecting itsupport Koehler's conclusion: in Hoch's studythose who generated the pro reasons first and thecon reasons later considered the event moreprobable than did those who generated the proand con reasons in the opposite order. Koehlerrelated the phenomenon to that of mental set orfixedness that is sometimes described in discus-sions of problem solving. He argued thatadopting a conditional reference frame todetermine confidence in a hypothesis is prob-ably a good general method but that, like otherheuristic methods, it also can yield overconfi-dence in some instances.

In a recent study of gambling behavior,Gibson, Sanbonmatsu, and Posavac (1997)found that participants who were asked toestimate the probability that a particular teamwould win the NBA basketball championshipmade higher estimates of that team's probabilityof winning than did control participants whowere not asked to focus on a single team; thefocused participants also were more willing tobet on the focal team. This suggests thatfocusing on one among several possible eventoutcomes, even as a consequence of beingarbitrarily forced to do so, can have the effect ofincreasing the subjective likelihood of thatoutcome.

Pragmatism and Error Avoidance

Much of the discussion of confirmation bias ispredicated on the assumption that, in thesituations in which it has generally beenobserved, people have been interested in deter-mining the truth or falsity of some hypoth-esises) under consideration. But determining itstruth or falsity is not the only, or necessarilyeven the primary, objective one might have withrespect to a hypothesis. Another possibility isthat of guarding against the making of certaintypes of mistakes.

In many real-life situations involving theevaluation of a meaningful hypothesis, or the


making of a choice with an outcome that reallymatters to the one making it, some ways ofbeing wrong are likely to be more regretablethan others. Investigators have noted that thisfact makes certain types of biases functional inspecific situations (Cosmides, 1989; Friedrich,1993; Hogarth, 1981; Schwartz, 1982). When,for example, the undesirable consequences ofjudging a true hypothesis to be false are greaterthan those of judging a false hypothesis to betrue, a bias toward confirmation is dictated bysome normative models of reasoning and bycommon sense.

Friedrich (1993) argued that "our inferenceprocesses are first and foremost pragmatic,survival mechanisms and only secondarily truthdetection strategies" (p. 298). In this view,peoples' inferential strategies are well suited tothe identification of potential rewards and theavoidance of costly errors, but not to theobjective of hypothesis testing in accordancewith the logic of science. Inference strategiesthat are often considered to be seriously flawednot only may have desired effects in real-worldcontexts, but may also be seen as correct whenjudged in terms of an appropriate standard.

To illustrate the point, Friedrich (1993) usedthe example of an employer who wants to testthe hunch that extroverts make the best sales-people. If the employer checked the salesperformance only of extroverts, found it to bevery good and, on this basis, decided to hire onlyextroverts for sales positions, one would say thatshe had not made an adequate test of her hunchbecause she did not rule out the possibility thatintroverts might do well at sales also. But if hermain objective was to avoid hiring people whowill turn out to be poor at sales, satisfyingherself that extroverts make good sales peoplesuffices; the fact that she has not discovered thatintroverts can be good at sales too could meanthat she will miss some opportunities by nothiring them, but it does not invalidate thedecision to hire extroverts if the objective is toensure that poor performers do not get hired.

Schwartz (1982) also argued that whenresponses have consequences that really matter,people are more likely to be concerned aboutproducing desirable outcomes than about deter-mining the truth or falsity of hypotheses.Contingent reinforcement may create functionalbehavioral units that people tend to repeatbecause the behaviors have worked in the past,

and a bias toward confirmation is one of thestereotyped forms of behavior in which theoperation of these units manifests itself. Inperforming a rule-discovery task, one may beattempting to maximize the probability ofgetting a positive response, which is tantamountto seeking a rule that is sufficient but notnecessary to do so. Given that one has identifieda condition that is sufficient for producingdesired outcomes, there may be no compellingreason to attempt to determine whether thatcondition is also necessary. Schwartz pointedout too that, especially in social situations suchas some of those studied by Snyder andcolleagues attempting to evaluate hypotheses byfalsification would require manipulating peopleand could involve considerable social cost.

Baron (1994) noted that truth seeking orhypothesis testing often may be combined withone or more other goals, and that one's behaviorthen also must be interpreted in the light of theother goal(s). If, for example, one is curious asto why a cake turned out well despite the factthat certain ingredients were substituted forthose called for by the recipe, one may bemotivated to explore the effects of the substitu-tions in such a way that the next experimentalcake is likely to turn out well too.

When using a truth-seeking strategy wouldrequire taking a perceived risk, survival is likelyto take precedence over truth finding, and it ishard to argue that rationality would dictateotherwise. It would seem odd to considerirrational the refusal, say, to eat mushrooms thatone suspected of being poison because thedecision is calculated to preserve one's well-being rather than to shed light on the question ofwhether the suspicion is indeed true. In general,the objective of avoiding disastrous errors maybe more conducive to survival than is that oftruth determination.

The desire to avoid a specific type of errormay coincidentally dictate the same behavior aswould the intention to determine the truth orfalsity of a hypothesis. When this is the case, thebehavior itself does not reveal whether theindividual's intention is to avoid the error or totest the hypothesis. Friedrich (1993) suggestedthat some behavior that has been taken asevidence of people's preference for normativediagnostic tests of hypotheses and their interestin accuracy could have been motivated insteadby the desire to avoid specific types of errors. In


other words, even when behavior is consistentwith the assumption of truth seeking, it some-times may be equally well interpreted, accord-ing to this view, in terms of error-minimizingstrategies.

The assumption that decisions made orconclusions drawn in many real-life situationsare motivated more by a desire to accomplishspecific practical goals or to avoid certain typesof errors than by the objective of determiningthe truth or falsity of hypotheses is a plausibleone. Pragmatic considerations of this sort couldoften lead one to accept a hypothesis as true—tobehave as though it were true—on less thancompelling evidence that it is so, thus constitut-ing a confirmation bias of sorts.

Educational Effects

At all levels of education, stress is placed onthe importance of being able to justify what onebelieves. I do not mean to question theappropriateness of this stress, but I do want tonote the possibility that, depending on how it isconveyed, it can strengthen a tendency to seekconfirming evidence selectively or establishsuch a tendency if it does not already exist. Ifone is constantly urged to present reasons foropinions that one holds and is not encouragedalso to articulate reasons that could be givenagainst them, one is being trained to exercise aconfirmation bias.

Narveson (1980) noted that when studentswrite compositions, they typically evaluate theirclaims by considering supporting evidence only.He argued that standard methods for teachingcomposition foster this. The extent to which theeducational process makes explicit the distinc-tion between case-building and evidence-weighing deserves more attention. If the distinc-tion is not made and what is actually case-building passes for the impartial use of evidence,this could go some way toward accounting forthe pervasiveness and strength of the confirma-tion bias among educated adults.

Ideally, one would like students, and peoplein general, to evaluate evidence objectively andimpartially in the formation and evaluation ofhypotheses. If, however, there are fairly perva-sive tendencies to seek or give undue weight toevidence that is confirmatory with respect tohypotheses that people already hold and to avoidor discount evidence that is disconfirmatory

with respect to them, there is a need to beespecially sensitive to the educational practicesthat could serve to strengthen an already strongbias.

Utility of Confirmation Bias

Most commentators, by far, have seen theconfirmation bias as a human failing, a tendencythat is at once pervasive and irrational. It is notdifficult to make a case for this position. Thebias can contribute to delusions of many sorts, tothe development and survival of superstitions,and to a variety of undesirable states of mind,including paranoia and depression. It can beexploited to great advantage by seers, soothsay-ers, fortune tellers, and indeed anyone with aninclination to press unsubstantiated claims. Onecan also imagine it playing a significant role inthe perpetuation of animosities and strifebetween people with conflicting views of theworld.

Even if one accepts the idea that theconfirmation bias is rooted more in cognitivelimitations than in motivation, can anyone doubtthat whenever one finds oneself engaged in averbal dispute it becomes very strong indeed? Inthe heat of an argument people are seldommotivated to consider objectively whateverevidence can be brought to bear on the issueunder contention. One's aim is to win and theway to do that is to make the strongest possiblecase for one's own position while countering,discounting, or simply ignoring any evidencethat might be brought against it. And what is trueof one disputant is generally true of the other,which is why so few disputes are clearly won orlost. The more likely outcome is the claim ofvictory by each party and an accusation ofrecalcitrance on the part of one's opponent.

But whenever an apparently dysfunctionaltrait or behavior pattern is discovered to bepervasive, the question arises as to how, if it isreally dysfunctional, did it get to be sowidespread. By definition, dysfunctional tenden-cies should be more prone to extinction thanfunctional ones. But aspects of reasoning thatare viewed as flawed from one perspectivesometimes can be considered appropriate, per-haps because they are adaptively useful incertain real-world situations from another per-spective (Arkes, 1991; Funder, 1987; Green-wald, 1980). Does the confirmation bias have


some adaptive value? Does it serve some usefulpurpose(s)?

Utility in Science

According to the principle of falsifiability(Popper, 1959), an explanation (theory, model,hypothesis) cannot qualify as scientific unless itis falsifiable in principle. This is to say theremust be a way to show the explanation to befalse if in fact it is false. Popper focused onfalsifiability as the distinguishing characteristicof the scientific method because of a convictionthat certain theories of the day (in particular,Marx's theory of history, Freud's theory ofpsychoanalysis, and Adler's individual psychol-ogy) "appeared to be able to explain practicallyeverything that happened within the fields towhich they referred" (Popper, 1962/1981, p.94). Thus subscribers to any one of thesetheories were likely to see confirming evidenceanywhere they looked. According to Popper(1959), the most characteristic element in thissituation seemed to be the incessant stream ofconfirmations, of observations which 'verified'the theories in question; and this point wasconstantly emphasized by their adherents" (p.94). There was, in Popper's view, no conceiv-able evidence that could be brought to bear onany of these theories that would be viewed by anadherent as grounds for judging it to be false.Einstein's theory of relativity impressed Popperas being qualitatively different in the importantrespect that it made risky predictions, which isto say predictions that were incompatible withspecific possible results of observation. Thistheory was, in principle, refutable by empiricalmeans and therefore qualified as scientificaccording to Popper's criterion.

Although Popper articulated the principle offalsifiability more completely than anyonebefore him, the idea has many antecedents inphilosophy and science. It is foreshadowed, forexample, in the Socratic method of refutation(elenchos), according to which what is to betaken as truth is whatever survives relentlessefforts at refutation (Maclntyre, 1988). Lakatos(1976, 1978) and Polya (1954a, 1954b) havediscussed the importance of this attitude inreasoning in mathematics—especially in proof-making—at length. The principle of falsifiabil-ity was also anticipated by T. H. Huxley(1894/1908), who spoke of "a beautiful hypoth-

esis killed by an ugly fact," and by DavidHartley (1748/1981), who proposed a rule offalse. According to this rule, the acceptability ofany supposition or hypothesis should be itsability to provide the basis for the deduction ofobservable phenomena: "He that forms hypoth-eses from the first, and tries them by the facts,soon rejects the most unlikely ones; and beingfreed from these, is better qualified for theexamination of those that are probable" (p. 90).

Hypotheses are strengthened more whenhighly competent scientists make concertedefforts to disprove them and fail than whenefforts at disproof are made by less competentinvestigators or in are made half-hearted ways.As Polya (1954a) put it, "the more danger, themore honor." What better support could Ein-stein's corpuscular theory of light have receivedthan Millikan's failure to show it to be wrong,despite 10 years of experimentation aimed atdoing so? (It is interesting to note thatthroughout this period of experimentation,Millikan continued to insist on the untenabilityof the theory despite his inability to show byexperiment its predictions to be in error.)

Despite the acceptance of the falsifiabilityprinciple by the scientific community as awhole, one would look long and hard to find anexample of a well-established theory that wasdiscarded when the first bit of disconfirmingevidence came to light. Typically an establishedtheory has been discarded only after a bettertheory has been offered to replace it. Perhapsthis should not be surprising. What astronomer,Waismann (1952) asked, would abandon Ke-pler's laws on the strength of a single observa-tion? Scientists have not discarded the idea thatlight travels at a constant speed of about 300,000kilometers per second and that nothing cantravel faster simply because of the discovery ofradio sources that seem to be emanating from asingle quasar and moving away from each otherat more than nine times that speed (Gardner,1976). It appears that, the principle of falsifiabil-ity notwithstanding, "Science proceeds onpreponderance of evidence, not on finality"(Drake, 1980, p. 55).

Application of the falsifiability principle tothe work of individual scientists seems toindicate that when one comes up with a newhypothesis, one should immediately try tofalsify it. Common sense suggests this too; if thehypothesis is false, the sooner one finds that out,


the less time one will waste entertaining it. Infact, as has already been noted, there is littleevidence that scientists work this way. To thecontrary, they often look much harder forevidence that is supportive of a hypothesis thanfor evidence that would show it to be false.

Kepler's laborious effort to find a connectionbetween the perfect polyhedra and the planetaryorbits is a striking example of a search by ascientist for evidence to confirm a favoredhypothesis. Here is his account of the connec-tion he finally worked out and his elation uponfinding it, as quoted in Boorstin (1985):

The earth's orbit is the measure of all things;circumscribe around it a dodecahedron, and the circlecontaining this will be Mars; circumscribe around Marsa tetrahedron, and the circle containing this will beJupiter; circumscribe around Jupiter a cube, and thecircle containing this will be Saturn. Now inscribewithin the earth an icosahedron, and the circlecontained in it will be Mercury. You now have thereason for the number of planets.... This was theoccasion and success of my labors. And how intensewas my pleasure from this discovery can never beexpressed in words. I no longer regretted the timewasted. Day and night I was consumed by thecomputing, to see whether this idea would agree withthe Copernican orbits, or if my joy would be carriedaway by the wind. Within a few days everythingworked, and I watched as one body after another fitprecisely into its place among the planets, (p. 310)

People are inclined to make light of thisparticular accomplishment of Kepler's today,but it was a remarkable intellectual feat. Theenergy with which he pursued what he saw as anintriguing clue to how the world works isinspiring. Bell (1946/1991) argued that it wasKepler's "Pythagorean faith in a numericalharmony of the universe" that sustained him "inhis darkest hours of poverty, domestic tragedy,persecution, and twenty-two years of discourage-ment as he calculated, calculated, calculated todiscover the laws of planetary orbits" (p. 181).The same commitment that kept Kepler inpursuit of confirmation of his polyhedral modelof planetary orbits yielded the three exquisitlybeautiful—and correct—laws of planetary mo-tion for which he is honored today.

My point is not to defend the confirmationbias as an effective guide to truth or even as aheuristically practical principle of logical think-ing. It is simply to note that the quest to findsupport for a particular idea is a commonphenomenon in science (I. B. Cohen, 1985;Holton, 1973), that such a quest has often

provided the motivation to keep scientistsworking on demanding intellectual problemsagainst considerable odds, and that the resultingwork has sometimes yielded lasting, if unex-pected, results.

Students of the scientific process have notedthe conservatism of science as an institution(I. B. Cohen, 1985; T. S. Kuhn, 1970), andillustrations of it were given in an earlier part ofthis article. This conservatism can be seen as aninstitutional confirmation bias of sorts. Shouldwe view such a bias as beneficial overall ordetrimental to the enterprise? An extensivediscussion of this question is beyond the scopeof this article. However, it can be argued that adegree of conservativism plays a stabilizing rolein science and guards the field against uncriticalacceptance of so-called discoveries that fail tostand the test of time.

Price (1963) referred to conservatism in thebody of science as "a natural counterpart to theopen-minded creativity that floods it with toomany new ideas" (p. 64). Justification for acertain degree of conservatism is found in theembarrassment that the scientific communityhas occasionally experienced as a consequenceof not being sufficiently sceptical of newdiscoveries. The discovery of magnetic mono-poles, which was widely publicized before aclose examination of the evidence forced moreguarded interpretations, and that of polywater,which motivated hundreds of research projectsover decade following its discovery in the1960s, are examples.

The scientific community's peremtory rejec-tion of Wegener's (1915/1966) theory of conti-nental drift when it was first put forth is oftenheld out as an especially egregious example ofexcessive—and self-serving—conservatism onthe part of scientists. The view has also beenexpressed, however, that the geologists whodismissed Wegener's theory, whatever theirmotivations, acted in a way that was conduciveto scientific success. Solomon (1992), who tookthis position, acknowledged that the behaviorwas biased and motivated by the desire toprotect existing beliefs, but she argued that"bias and belief perseverance made possible thedistribution of effort, and this in turn led to theadvancement of the debate over [continental]drift" (p. 443).

Solomon's (1992) review of this chapter inthe history of geological research makes it clear


that the situation was not quite as simple assome accounts that focus on the closed-mindedness of the geologists at the time wouldlead one to believe. The idea of continental driftwas not entirely new with Wegener, forexample, although Wegener was the first topropose a well-developed theory. A seriouslimitation of the theory was its failure to identifya force of sufficient magnitude to account for thehypothesized movement of continents. (Thenotion of plate tectonics and evidence regardingsea-floor spreading came much later; LeGrand,1988.)

Solomon (1992) argued that it was, in part,because Wegener was not trained as a geologistand therefore not steeped in the "stabilist"theories of the time, that his thinking, relativelyunconstrained by prior beliefs about the stabilityof continents, could easily embrace a possibilitythat was so contrary to the prevailing view. Shepointed out too that when geologists began toaccept the notion of drift, as evidence favoring itaccumulated, it was those with low publicationrates who were the most likely to do so: "theirbeliefs were less entrenched (cognitively speak-ing) than those who had reasoned more andproduced more, so belief perseverance was lessof an impediment to acceptance of drift" (p. 449).

Although Solomon (1992) argued that biasand belief perseverance were responsible formuch of the distribution of research effort thatled finally to the general acceptance of thetheory of continental drift, and to that of platetectonics to which it led in turn, she does notclaim that a productive distribution could nothave been effected without the operation ofthese factors. The question of whether thesefactors facilitated or impeded progress ingeology remains an unanswered one; it is notinconceivable that progress could have beenfaster if the distribution of effort were deter-mined on some other basis.

In any case, it can be argued that a certaindegree of conservativism serves a useful stabiliz-ing role in science and is consistent with, if notdictated by, the importance science attaches totestability and empirical validation. Moreover, ifZiman (1978) is right, the vast majority of thenew hypotheses put forward by scientists proveto be wrong:

Even in physics, there is no infallible procedure forgenerating reliable knowledge. The calm order andperfection of well-established theories, accredited by

innumerable items of evidence from a thousanddifferent hands, eyes and brains, is not characteristic ofthe front-line of research, where controversy, conjec-ture, contradiction and confusion are rife. The physicsof undergraduate test-books is 90% true; the contents ofthe primary research journals of physics is 90% false,(p. 40)

"According to temperament," Ziman noted,"one may be impressed by the coherence ofwell-established theories, or horrified by thecontradictions of knowledge in the making" (p.100).

Fischhoff and Beyth-Marom (1983) made arelated point when commenting on Mahoney's(1977) finding that scientists tended to be lesscritical of a fictitious study that reported resultssupportive of the dominant hypothesis in theirfield than of one that reported results that wereinconsistent with it. They noted that a reluctanceby scientists to relinquish pet beliefs is only oneinterpretation that could be put on the finding.Another possibility is that what appears to bebiased behavior reflects "a belief that investiga-tors who report disconfirming results tend to useinferior research methods (e.g., small samplesleading to more spurious results), to commitcommon mistakes in experimental design, or,simply, to be charlatans" (Fischoff and Beyth-Marom, 1983, p. 251). To the extent that such abelief is accurate, a bias against the readyacceptance of results that are disconfirming ofprevailing hypotheses can be seen as a safeguardagainst precipitous changes of view that mayprove to be unjustified. It does not follow, ofcourse, that this is the only reason for aconfirmation bias in science or the only effect.

Is Belief Perseverance Always Bad?

It is easy to see both how the confirmationbias helps preserve existing beliefs, whether trueor false, and how the perseverance of unjustifiedbeliefs can cause serious problems. Is thereanything favorable to be said about a bias thattends to perpetuate beliefs independently oftheir factuality? Perhaps, at least from thenarrow perspective of an individual's mentalhealth. It may help, for example, to protect one'sego by making one's favored beliefs lessvulnerable than they otherwise would be.Indeed, it seems likely that a major reason whythe confirmation bias is so ubiquitous and soenduring is its effectiveness in preserving


preferred beliefs and opinions (Greenwald,1980).

But even among people who might see somebenefit to the individual in a confirmation bias,probably few would contest the claim that whenthe tendency to persevere in a belief is so strongthat one refuses to consider evidence that doesnot support that belief, it is irrational and offendsour sense of intellectual honesty. That is not tosay that dogmatic confidence in one's ownbeliefs and intolerance of opposing views cannever work to one's advantage. Boorstin (1958)argued, for example, that it was precisely thesequalities that permitted the 17th-century NewEngland Puritans to establish a society with theingredients necessary for survival and prosper-ity. He wrote,

Had they spent as much of their energy in debating witheach other as did their English contemporaries, theymight have lacked the single-mindedness needed toovercome the dark, unpredictable perils of a wilder-ness. They might have merited praise as precursors ofmodern liberalism, but they might never have helpedfound a nation, (p. 9)

Contrary to the popular stereotype of thePuritans, they were not preoccupied withreligious dogma but rather with more practicalmatters because, as Boorstin noted, they had nodoubts and allowed no dissent. They worriedabout such problems as how to select leadersand representatives, the way to establish theproper limits of political power, and how toconstruct a feasible federal organization.

The question of the conditions under whichone should retain, reject, or modify an existingbelief is a controversial one (Cherniak, 1986;Harman, 1986; Lycan, 1988). The controversy isnot likely to be settled soon. Whatever theanswer to the question is, the confirmation biasmust be recognized as a major force that worksagainst easy and frequent opinion change.Probably very few people would be willing togive up long-held and valued beliefs on the firstbit of contrary evidence found. It is natural to bebiased in favor of one's established beliefs.Whether it is rational is a complicated issue thatcan too easily be treated simplistically; however,the view that a person should be sufficientlyobjective and open minded to be willing to tossout any belief upon the first bit of evidence thatit is false seems to me wrong for several reasons.

Many, perhaps most, of the beliefs that matterto individuals tend not to be the type that can be

falsified, in the Popperian sense, by a singlecounterindicative bit of data. They tend rather tobe beliefs for which both supportive andcounterindicative evidence can be found, andthe decision as to whether to hold them isappropriately made on the basis of the relativeweights or merits of the pro and con arguments.

Second, it is possible to hold a belief for goodand valid reasons without being able to produceall of those reasons on demand. Some beliefs areshaped over many years, and the fact that onecannot articulate every reason one has or hasever had for a particular one of them does notmean that it is unfounded. Also, as Nisbett andRoss (1980) pointed out, there are practical timeconstraints that often limit the amount ofprocessing of new information one can do. Inview of these limitations, the tendency topersevere may be a stabilizing hedge againstoverly frequent changes of view that wouldresult if one were obliged to hold only beliefsthat one could justify explicitly at a moment'snotice. This argument is not unlike the oneadvanced by Blackstone (1769/1962) in defenseof not lightly scuttling legal traditions.

Third, for assertions of the type that representbasic beliefs, there often are two ways to bewrong: to believe false ones, or to disbelievetrue ones. For many beliefs that people hold,these two possibilities are not equally accept-able, which is to say that an individual mightconsider it more important to avoid one type oferror than the other. This is, of course, theargument behind Pascal's famous wager.

To argue that it is not necessarily irrational torefuse to abandon a long-held belief uponencountering some evidence that appears contra-dictory is not to deny that there is such a thing asholding on to cherished beliefs too tenaciouslyand refusing to give a fair consideration tocounterindicative evidence. The line betweenunderstandable conservativism with respect tochanging established beliefs and obstinate closed-mindedness is not an easy one to draw. Butclearly, people sometimes persevere beyondreason. Findings such as those of Pitz (1969),Pitz et al. (1967), Lord et al. (1979), andespecially Ross et al. (1975), who showed thatpeople sometimes persevere in beliefs evenwhen the evidence on which the beliefs wereinitially formed has been demonstrated to themto be fraudulent, have provided strong evidenceof that fact.


Confirmation Bias CompoundingInadequate Search

The idea that inadequate effort is a basiccause of faulty reasoning is common amongpsychologists. Kanouse (1972) suggested thatpeople may be satisfied to have an explanationfor an event that is sufficient and not feel theneed to seek the best of all possibilities. Nisbettand Ross (1980) supported the same idea andsuggested that it is especially the case whenattempting to come up with a causal explanation:

The lay scientist seems to search only until a plausibleantecedent is discovered that can be linked to theoutcome through some theory in the repertoire. Giventhe richness and diversity of that repertoire, such asearch generally will be concluded quickly and easily.A kind of vicious cycle results. The subjective ease ofexplanation encourages confidence, and confidencemakes the lay scientist stop searching as soon as aplausible explanation is adduced, so that the complexi-ties of the task, and the possibilities for "alternativeexplanations" no less plausible than the first, are neverallowed to shake the lay scientist's confidence, (p. 119,120)

Perkins and his colleagues expressed essen-tially the same idea with their characterizationof people as make-sense epistimologists (Per-kins et al., 1983, 1991). The idea here is thatpeople think about a situation only to the extentnecessary to make sense—perhaps superficialsense—of it:

When sense is achieved, there is no need to continue.Indeed, because further examination of an issue mightproduce contrary evidence and diminish or cloud thesense of one's first pass, there is probably reinforce-ment for early closure to reduce the possibility ofcognitive dissonance. Such a makes-sense approach isquick, easy, and, for many purposes, perfectly ad-equate. (Perkins et al., 1991, p. 99)

Baron (1985, 1994) and Pyszczynski andGreenberg (1987) also emphasized insufficientsearch as the primary reason for the prematuredrawing of conclusions.

All of these characterizations of the tendencyof people to do a less-than-thorough searchthrough the possibilities before drawing conclu-sions or settling on causal explanations areconsistent with Simon's (1957,1983/1990) viewof humans as satisficers, as opposed to optimiz-ers or maximizers. The question of interest inthe present context is that of how the criterionfor being satisfied should be set. Given thatsearch is seldom exhaustive and assuming that,

in many cases, it cannot be, how much should beenough? How should one decide when to stop?

Without at least tentative answers to thesetypes of questions, it is difficult to say whetherany particular stopping rule should be consid-ered rational. Despite this vagueness withrespect to criteria, I believe that the prevailingopinion among investigators of reasoning is thatpeople often stop—come to conclusions, adoptexplanations—before they should, which is notto deny that they may terminate a search morequickly when time is at a premium and searchlonger when accuracy is critical (Kruglanski,1980; Kruglanski & Ajzen, 1983; Kruglanski &Freund, 1983). The search seems to be not onlyless than extensive but, in many cases, minimal,stopping at the first plausible endpoint.

If this view is correct, the operation of aconfirmation bias will compound the problem.Having once arrived at a conclusion, belief, orpoint of view, however prematurely, one maythereafter seek evidence to support that positionand interpret newly acquired information in away that is partial to it, thereby strengthening it.Instead of making an effort to test an initialhypothesis against whatever counterindicativeevidence might be marshalled against it, onemay selectively focus on what can be said in itsfavor. As the evidence favoring the hypothesismounts, as it is bound to do if one givescredence to evidence that is favorable andignores or discounts that that is not, one willbecome increasingly convinced of the correct-ness of the belief one originally formed.

Concluding Comments

We are sometimes admonished to be tolerantof the beliefs or opinions of others and critical ofour own. Laplace (1814/1956), for example,gave this eloquent advice:

What indulgence should we not have . . . for opinionsdifferent from ours, when this difference often dependsonly upon the various points of view where circum-stances have placed us! Let us enlighten those whomwe judge insufficiently instructed; but first let usexamine critically our own opinions and weigh withimpartiality their respective probabilities, (p. 1328)

But can we assess the merits of our ownopinions impartially? Is it possible to put abelief that one holds in the balance with anopposing belief that one does not hold and givethem a fair weighing? I doubt that it is. But that


is not to say that we cannot hope to learn to dobetter than we typically do in this regard.

In the aggregate, the evidence seems to mefairly compelling that people do not naturallyadopt a falsifying strategy of hypothesis testing.Our natural tendency seems to be to look forevidence that is directly supportive of hypoth-eses we favor and even, in some instances, ofthose we are entertaining but about which areindifferent. We may look for evidence that isembarrassing to hypotheses we disbelieve orespecially dislike, but this can be seen aslooking for evidence that is supportive of thecomplementary hypotheses. The point is that weseldom seem to seek evidence naturally thatwould show a hypothesis to be wrong and to doso because we understand this to be an effectiveway to show it to be right if it really is right.

The question of the extent to which theconfirmation bias can be modified by trainingdeserves more research than it has received.Inasmuch as a critical step in dealing with anytype of bias is recognizing its existence, perhapssimply being aware of the confirmation bias—ofits pervasiveness and of the many guises inwhich it appears—might help one both to be alittle cautious about making up one's mindquickly on important issues and to be somewhatmore open to opinions that differ from one'sown than one might otherwise be.

Understanding that people have a tendency tooverestimate the probable accuracy of theirjudgments and that this tendency is due, at leastin part, to a failure to consider reasons why thesejudgments might be inaccurate provides arationale for attempting to think of reasons forand (especially) against a judgment that is to bemade. Evidence that the appropriateness ofpeople's confidence in their judgments can beimproved as a consequence of such efforts isencouraging (Arkes, Faust, Guilmette, & Hart,1988; Hoch, 1984,1985; Koriatetal., 1980).

The knowledge that people typically consideronly one hypothesis at a time and often make theassumption at the outset that that hypothesis istrue leads to the conjecture that reasoning mightbe improved by training people to think ofalternative hypotheses early in the hypothesis-evaluation process. They could be encouragedto attempt to identify reasons why the comple-ment of the hypothesis in hand might be true.Again, the evidence provides reason for opti-mism that the approach can work (C. A.

Anderson, 1982; C. A. Anderson & Sechler,1986; Lord, Lepper, & Preston, 1984).

To the extent that what appear to be biases aresometimes the results of efforts to avoid certaintypes of decision errors (Friedrich, 1993),making these other types of possible errors moresalient may have a debiasing effect. On the otherhand, if the errors that one is trying to avoid are,in fact, more costly than bias errors, suchdebiasing might not be desirable in all instances.Here the distinction between the objective ofdetermining the truth or falsity of a hypothesisand that of avoiding an undesirable error (at theexpense of accepting the possibility of commit-ting a less undesirable one) is an important oneto keep in mind.

Finally, I have argued that the confirmationbias is pervasive and strong and have reviewedevidence that I believe supports this claim. Thepossibility will surely occur to the thoughtfulreader that what I have done is itself anillustration of the confirmation bias at work. Ican hardly rule the possibility out; to do sowould be to deny the validity of what I amclaiming to be a general rule.

References

Alloy, L. B., & Abramson, L. Y. (1980). The cognitivecomponent of human helplessness and depression:A critical analysis. In J. Garber & M. E. P.Seligman (Eds.), Human helplessness: Theory andapplications. New York: Academic Press.

Alloy, L. B., & Tabachnik, N. (1984). Assessment ofcovariation by humans and animals: The jointinfluence of prior expectations and current situ-ational information. Psychological Review, 91,112-148.

Anderson, C. A. (1982). Inoculation and counterexpla-nation: Debiasing techniques in the perseveranceof social theories. Social Cognition, 1, 126-139.

Anderson, C. A., & Sechler, E. S. (1986). Effects ofexplanation and counterexplanation on the develop-ment and use of social theories. Journal ofPersonality and Social Psychology, 50, 24-34.

Anderson, N. H., & Jacobson, A. (1965). Effect ofstimulus inconsistency and discounting instruc-tions in personality impression formation. Journalof Personality and Social Psychology, 2, 531-539.

Arkes, H. R. (1991). Costs and benefits of judgmenterrors: Implications for debiasing. PsychologicalBulletin, 110, 486-498.

Arkes, H. R., Dawes, R. M., & Christensen, C.(1986). Factors influencing the use of a decisionrule in a probabilistic task. Organizational Behav-ior and Human Decision Processes, 37, 93-110.


Arkes, H. R., Faust, D., Guilmette, T. J., & Hart, K.(1988). Elimination of the hindsight bias. Journalof Applied Psychology, 73, 305-307.

Bacon, F. (1939). Novum organum. In Burtt, E. A.(Ed.), The English philosophers from Bacon to Mill(pp. 24-123). New York: Random House. (Originalwork published in 1620)

Baddeley, A. D., & Woodhead, M. (1983). Improvingface recognition ability. In S. M. A. Lloyd-Bostock& B. R. Clifford (Eds.), Evaluating witnessevidence. Chichester, England: Wiley.

Barber, B. (1961). Resistence by scientists toscientific discovery. Science, 134, 596-602.

Baron, J. (1985). Rationality and intelligence. NewYork: Cambridge University Press.

Baron, J. (1991). Beliefs about thinking. In J. F. Voss,D. N. Perkins, & J. W. Segal (Eds.), Informalreasoning and education (pp. 169-186). Hillsdale,NJ: Erlbaum.

Baron, J. (1994). Thinking and deciding (2nd ed.).New York: Cambridge University Press.

Baron, J. (1995). Myside bias in thinking aboutabortion. Thinking and reasoning, 7, 221-235.

Baron, J., Beattie, J., & Hershey, J. C. (1988).Heuristics and biases in diagnostic reasoning II:Congruence, information, and certainty. Organiza-tional Behavior and Human Decision Processes,42, 88-110.

Barrows, H. S., Feightner, J. W., Neufeld, V. R., &Norman, G. R. (1978). Analysis of the clinicalmethods of medical students and physicians.Hamilton, Ontario, Canada: McMaster UniversitySchool of Medicine.

Barrows, H. S., Norman, G. R., Neufeld, V. R., &Feightner, J. W. (1977). Studies of the clinicalreasoning process of medical students and physi-cians. Proceedings of the sixteenth annual confer-ence on research in medical education. Washing-ton, DC: Association of American Medical Colleges.

Bassock, M., & Trope, Y. (1984). People's strategiesfor testing hypotheses about another's personality:Confirmatory or diagnostic? Social Cognition, 2,199-216.

Beattie, J., & Baron, J. (1988). Confirmation andmatching biases in hypothesis testing. QuarterlyJournal of Experimental Psychology, 40A, 269-298.

Beck, A. T. (1976). Cognitive therapy and theemotional disorders. New York: InternationalUniversities Press.

Bell, E. T. (1991). The magic of numbers. New York:Dover. (Original work published 1946)

Berwick, D. M., Fineberg, H. V., & Weinstein, M. C.(1981). When doctors meet numbers. AmericanJournal of Medicine, 71, 991.

Beyth-Marom, R., & Fischhoff, B. (1983). Diagnostic-ity and pseudodiagnosticity. Journal of Personalityand Social Research, 45, 1185-1197.

Blackstone, W. (1962). Commentaries on the laws of

England of public wrongs. Boston: Beacon.(Original work published 1769)

Boorstin, D. J. (1958). The Americans: The colonialexperience. New York: Vintage Books.

Boorstin, D. J. (1985). The discoverers: A history ofman's search to know his world and himself. NewYork: Vintage Books.

Bruner, J. S., Goodnow, J. J., & Austin, G. A. (1956).A study of thinking. New York: Wiley.

Bruner, J. S., & Potter, M. C. (1964). Interference invisual recognition. Science, 144, 424—425.

Camerer, C. (1988). Illusory correlations in percep-tions and predictions of organizational traits.Journal of Behavioral Decision Making, 1, 11-94.

Campbell, J. D., & Fairey, P. J. (1985). Effects ofself-esteem, hypothetical explanations, and verbal-ization of expectancies on future performance.Journal of Personality and Social Psychology, 48,1097-1111.

Cassells, W., Schoenberger, A., & Grayboys, T. B.(1978). Interpretation by physicians of clinicallaboratory results. New England Journal of Medi-cine, 299, 999.

Chapman, L. J. (1967). Illusory correlation inobservational report. Journal of Verbal Learningand Verbal Behavior, 6, 151-155.

Chapman, L. J., & Chapman, J. P. (1959). Atmo-sphere effect reexamined. Journal of ExperimentalPsychology, 58, 220-226.

Chapman, L. J., & Chapman, J. P. (1967a). Genesis ofpopular but erroneous psychodiagnostic observa-tions. Journal of Abnormal Psychology, 72, 193-204.

Chapman, L. J., & Chapman, J. P. (1967b). Illusorycorrelation as an obstacle to the use of validpsychodiagnostic signs. Journal of AbnormalPsychology, 74, 271-280.

Chapman, L. J. & Chapman, J. P. (1969). Illusorycorrelation as an obstacle to the use of validpsychodiagnostic signs. Journal of AbnormalPsychology, 74, 271-280.

Cheng, P. W., & Holyoak, K. J. (1985). Pragmaticreasoning schemas. Cognitive Psychology, 17,391-416.

Chernev, A. (1997). The effect of common features onbrand choice: Moderating the effect of attributeimportance. Journal of Consumer Research, 23,304-311.

Cherniak, C. (1986). Minimal rationality. Cambridge,MA: MIT Press.

Christensen-Szalanski, J. J., & Bushyhead, J. B.(1988). Physicians' use of probabilistic informationin a real clinical setting. In J. Dowie & A. Elstein(Eds.), Professional judgment. A reader in clinicaldecision making (pp. 360-373). Cambridge, En-gland: Cambridge University Press. (Original workpublished 1981)

Clark, H. H. (1974). Semantics and comprehension.In T. A. Sebeok (Ed.), Current trends in linguistics.


Volume 12: Linguistics and adjacent arts andsciences. The Hague, Netherlands: Mouton.

Clark, H. H., & Chase, W. B. (1972). On the processof comparing sentences against pictures. CognitivePsychology, 3, 472-517.

Cohen, I. B. (1985). Revolution in science. Cam-bridge, MA: Harvard University Press.

Cohen, L. J. (1981). Can human irrationality beexperimentally demonstrated? Behavioral and BrainSciences, 4, 317-331.

Collins, S., & Pinch, J. (1993). The Golem: Whateveryone should know about science. New York:Cambridge University Press.

Cosmides, L. (1989). The logic of social exchange:Has natural selection shaped how humans reason?Studies with the Wason selection task. Cognition,31, 187-276.

Cosmides, L., & Tooby, J. (1992). Cognitive adapta-tions for social exchange. In J. Barkow, L.Cosmides, & J. Tooby (Eds.), The adapted mind:Evolutionary psychology and the generation ofculture (pp. 162-228). New York: Oxford Univer-sity Press.

Crocker, J. (1981). Judgment of covariation by socialperceivers. Psychological Bulletin, 90, 272-292.

Darley, J. M., & Fazio, R. H. (1980). Expectancyconfirmation processes arising in the social interac-tion sequence. American Psychologist, 35, 867-881.

Darley, J. M., & Gross, P. H. (1983). A hypothesis-confirming bias in labelling effects. Journal ofPersonality and Social Psychology, 44, 20-33.

DeSmet, A. A., Fryback, D. G., & Thornbury, J. R.(1979). A second look at the utility of radiographicskull examinations for trauma. American Journalof Roentgenology, 132, 95.

Devine, P. G., Hirt, E. R., & Gehrke, E. M. (1990).Diagnostic and confirmation strategies in traithypothesis testing. Journal of Personality andSocial Psychology, 58, 952-963.

Devine, P. G., & Ostrom, T. M. (1985). Cognitivemediation of inconsistency discounting. Journal ofPersonality and Social Psychology, 49, 5-21.

Doherty, M. E., & Mynatt, C. R. (1986). The magicalnumber one. In D. Moates & R. Butrick (Eds.),Inference Ohio University Interdisciplinary Confer-ence 86 (Proceedings of the InterdisciplinaryConference on Inference; pp. 221-230). Athens:Ohio University.

Doherty, M. E., Mynatt, C. R., Tweney, R. D., &Schiavo, M. D. (1979). Pseudodiagnosticity. ActaPsychologica, 43, 111-121.

Drake, S. (1980). Galileo. New York: OxfordUniversity Press.

Duncan, B. L. (1976). Differential social perceptionand attribution of intergroup violence: Testing thelower limits of stereotyping of Blacks. Journal ofPersonality and Social Psychology, 34, 590-598.

Dusek, J. B. (1975). Do teachers bias children's

learning? Review of Educational Research, 45,661-684.

Einhorn, H. J., & Hogarth, R. M. (1978). Confidencein judgment: Persistence of the illusion of validity.Psychological Review, 85, 395—416.

Elstein, A. S., & Bordage, G. (1979). Psychology ofclinical reasoning. In G. Stone, F. Cohen, & N.Adler (Eds.) Health psychology: A handbook. SanFrancisco: Jossey-Bass.

Elstein, A. S., Shulman, L. S., & Sprafka, S. A.(1978). Medical problem solving: An analysis ofclinical reasoning. Cambridge, MA: HarvardUniversity Press.

Evans, J. St. B. T. (1972). On the problems ofinterpreting reasoning data: Logical and psychologi-cal approaches. Cognition, 1, 373-384.

Evans, J. St. B. T. (1982). The psychology ofdeductive reasoning. London: Routledge & KeganPaul.

Evans, J. St. B. T. (1989). Bias in human reasoning:Causes and consequences. Hillsdale, NJ: Erlbaum.

Evans, J. St. B. T., & Lynch, J. S. (1973). Matchingbias in the selection task. British Journal ofPsychology, 64, 391-397.

Evans, J. St., B. P., Newstead, E., & Byrne, R. M. J.(1993). Human reasoning: The psychology ofdeduction. Hove, England: Erlbaum.

Farley, J., & Geison, G. L. (1974). Science politicsand spontaneous generation in nineteenth-centuryFrance: The Pasteur-Pouchet debate. Bulletin forthe History of Medicine, 48, 161-198.

Feldman, J. M., Camburn, A., & Gatti, G. M. (1986).Shared distinctiveness as a source of illusory correla-tion in performance appraisal. Organizational Behav-ior and Human Decision Process, 37, 34-59.

Festinger, L. (1957). A theory of cognitive disso-nance. Stanford, CA: Stanford University Press.

Fischhoff, B. (1977). Perceived informativeness offacts. Journal of Experimental Psychology: HumanPerception and Performance, 3, 349-358.

Fischhoff, B. (1982). Debiasing. In D. Kahneman, P.Slovic, & A. Tversky (Eds.), Judgment underuncertainty: Heuristics and biases (pp. 422—444).Cambridge, England: Cambridge University Press.

Fischhoff, B., & Beyth-Marom, R. (1983). Hypoth-esis evaluation from a Bayesian perspective.Psychological Review, 90, 239-260.

Fodor, J. D., Fodor, J. A., & Garrett, M. F. (1975). Thepsychological unreality of semantic representa-tions. Linguistic Inquiry, 4, 515-531.

Forer, B. (1949). The fallacy of personal validation: Aclassroom demonstration of gullibility. Journal ofAbnormal and Social Psychology, 44, 118-123.

Foster, G., Schmidt, C , & Sabatino, D. (1976).Teacher expectancies and the label "learningdisabilities." Journal of Learning Disabilities, 9,111-114.

Freedman, J. L. (1964). Involvement, discrepancy,and change. Journal of Abnormal and SocialPsychology, 69, 290-295.


Frey, D. (1986). Recent research on selectiveexposure to information. In L. Berkowitz (Ed.),Advances in experimental social psychology (Vol.19, pp. 41-80). New York: Academic Press.

Friedrich, J. (1993). Primary error detection andminimization (PEDMIN) strategies in social cogni-tion: A reinterpretation of confirmation bias phe-nomena. Psychological Review, 100, 298-319.

Funder, D. C. (1987). Errors and mistakes: Evaluatingthe accuracy of social judgment. PsychologicalBulletin, 101, 79-90.

Gardner, M. (1957). Fads and fallacies in the name ofscience. New York: Dover.

Gardner, M. (1976). The relativity explosion. NewYork: Vintage Books.

Gibson, B., Sanbonmatsu, D. M., & Posavac, S. S.(1997). The effects of selective hypothesis testingon gambling. Journal of Experimental Psychology:Applied, 3, 126-142.

Gigerenzer, G., Hoffrage, U., & Kleinbolting, H.(1991). Probabilistic mental models: A Brun-swikian theory of confidence. Psychological Re-view, 98, 506-528.

Gigerenzer, G., & Hug, K. (1992). Domain-specificreasoning: Social contracts, cheating, and perspec-tive change. Cognition, 43, 127-171.

Gigerenzer, G., Swijtink, Z., Porter, T., Daston, L.,Beatty, J., & Kriiger, L. (1989). The empire ofchance: How probability changed science and every-day life. New York: Cambridge University Press.

Gilbert, D. T. (1991). How mental systems believe.American Psychologist, 46, 107-119.

Gilovich, T. (1983). Biased evaluation and persis-tence in gambling. Journal of Personality andSocial Psychology, 44, 1110-1126.

Gilovich, T. (1991). How we know what isn't so: Thefallibility of human reason in everyday life. NewYork: Free Press.

Glenberg, A. M., Wilkinson, A. C , & Epstein, W.(1982). The illusion of knowing: Failure in theself-assessment of comprehension. Memory andCognition, 10, 597-602.

Goldberg, L. R. (1968). Simple models or simpleprocesses? Some research in clinical judgment.American Psychologist, 23, 483—496.

Golding, S. L., & Rorer, L. G. (1972). Illusorycorrelation and subjective judgement. Journal ofAbnormal Psychology, 80, 249-260.

Goodman, N. (1966). The structure of appearance(2nd ed.). Indianapolis, IN: Bobbs-Merrill.

Graesser, A. C , & Hemphill, D. (1991). Questionanswering in the context of scientific mechanisms.Journal of Memory and Language, 30, 186-209.

Greenwald, A. G. (1980). The totalitarian ego:Fabrication and revision of personal history.American Psychologist, 35, 603-618.

Griffin, D. W., Dunning, D., & Ross, L. (1990). Therole of construal processes in overconfident predic-

tions about the self and others. Journal ofPersonality and Social Psychology, 59, 1128-1139.

Griffin, D. W., & Tversky, A. (1992). The weighing ofevidence and the determinants of confidence.Cognitive Psychology, 24, 411-435.

Griggs, R. A., & Cox, J. R. (1993). Permissionschemas and the selection task. In J. St. B. T. Evans(Ed.), The cognitive psychology of reasoning (pp.637-651). Hillsdale, NJ: Erlbaum.

Hamilton, D. L. (1979). A cognitive attributionalanalysis of stereotyping. In L. Berkowitz (Ed.),Advances in Experimental Social Psychology, (Vol.12). New York: Academic Press.

Hamilton, D. L., Dugan, P. M., & Trolier, T. K.(1985). The formation of stereotypic beliefs:Further evidence for distinctiveness-based illusorycorrelations. Journal of Personality and SocialPsychology, 48, 5-17.

Harman, G. (1986). Change in view: Principles ofreasoning. Cambridge, MA: MIT Press.

Hartley, D. (1981). Hypotheses and the "rule of thefalse." In R. D. Tweney, M. E. Doherty, & C. R.Mynatt (Eds.), Scientific thinking (pp. 89-91). NewYork: Columbia University Press. (Original workpublished 1748)

Hasdorf, A. H., & Cantril, H. (1954). They saw agame. Journal of Abnormal and Social Psychology,49, 129-134.

Hausch, D. B., Ziemba, W. T., & Rubenstein, M.(1981). Efficiency of the market for racetrackbetting. Management Science, 27, 1435-1452.

Hawking, S. W. (1988). A brief history of time: Fromthe big bang to black holes. New York: BantamBooks.

Hayden, T., & Mischel, W. (1976). Maintaining traitconsistency in the resolution of behavioral inconsis-tency: The wolf in sheep's clothing? Journal ofPersonality, 44, 109-132.

Hempel, C. (1945). Studies in the logic of confirma-tion. Mind, 54(213), 1-26.

Hendry, S. H., & Shaffer, D. R. (1989). On testifyingin one's own behalf: Interactive effects of eviden-tial strength and defendant's testimonial demeanoron jurors' decisions. Journal of Applied Psychol-ogy, 74, 539-545.

Henle, M. (1962). On the relation between logic andthinking. Psychological Review, 69, 366-378.

Henrion, M., & Fischhoff, B. (1986). Assessinguncertainty in physical constants. American Jour-nal of Physics, 54, 791-798.

Hirt, E. R., & Sherman, S. J. (1985). The role of priorknowledge in explaining hypothetical events.Journal of Experimental Social Psychology, 21,519-543.

Hoch, S. J. (1984). Availability and inference inpredictive judgment. Journal of ExperimentalPsychology: Learning, Memory, and Cognition,10, 649-662.


Hoch, S. J. (1985). Counterfactual reasoning andaccuracy in predicting personal events. Journal ofExperimental Psychology: Learning, Memory, andCognition, 11, 719-731.

Hoch, S. J., & Tschirgi, J. E. (1983). Cue redundancyand extra logical inferences in a deductivereasoning task. Memory and Cognition, 11, 200-209.

Hodgins, H. S., & Zuckerman, M. (1993). Beyondselecting information: Baises in spontaneous ques-tions and resultant conclusions. Journal of Experi-mental Social Psychology, 29, 387^07.

Hogarth, R. M. (1981). Beyond discrete biases:Functional and dysfunctional aspects of judgmen-tal heuristics. Psychological Bulletin, 90, 197-217.

Hollis, M. (1970). Reason and ritual. In B. R. Wilson(Ed.), Rationality (pp. 221-239). Oxford, England:Blackwell.

Holstein, J. A. (1985). Jurors' interpretations and jurydecision making. Law and Human Behavior, 9,83-99.

Holton, G. (1973). Thematic origins of scientificthought. Cambridge, MA: Harvard UniversityPress.

Huxley, T. H. (1908). Discourses, biological andgeological. In Collected essays. (Vol. 8). London:MacMillan and Company. (Original work pub-lished 1894)

Hyman, R. (1977). Cold reading. The SkepticalInquirer, 1, 18-37.

Jennings, D., Amabile, T. M., & Ross, L. (1982).Informal covariation assessment: Data-based ver-sus theory-based judgments. In A. Tversky, D.Kahneman, & P. Slovic (Eds.), Judgment underuncertainty: Heuristics and biases. New York:Cambridge University Press.

Johnson-Laird, P. N., Legrenzi, P., & Legrenzi, M. S.(1972). Reasoning and a sense of reality. BritishJournal of Psychology, 63, 395-400.

Jones, E. E., & Goethals, G. (1972). Order effects inimpression formation: Attribution context and thenature of the entity. In E. E. Jones and others(Eds.), Attribution: Perceiving the causes ofbehavior. Morristown, NJ: General Learning Press.

Jones, E. E., Rock, L., Shaver, K. G., Goethals, G. R.,& Ward, L. M. (1968). Pattern of performance andability attribution: An unexpected primacy effect.Journal of Personality and Social Psychology, 10,317-340.

Juslin, P. (1993). An explanation of the hard-easyeffect in studies of realism of confidence in one'sgeneral knowledge. European Journal of CognitivePsychology, 5, 55-71.

Juslin, P. (1994). The overconfidence phenomenon asa consequence of informal experimenter-guidedselection of almanac items. Organizational Behav-ior and Human Decision Processes, 57, 226-246.

Kahneman, D., & Tversky, A. (1973). On the

psychology of prediction. Psychological Review,80, 237-251.

Kalven, H., & Zeisel, H. (1966). The American jury.Boston: Little, Brown.

Kanouse, D. E. (1972). Language, labeling, andattribution. In E. E. Jones, and others (Eds.),Attribution: Perceiving the causes of behavior.Morristown, NJ: General Learning Press.

Kelley, H. H. (1950). The warm-cold variable in firstimpressions of persons. Journal of Personality, 18,431^39.

Keren, G. B. (1987). Facing uncertainty in the gameof bridge: A calibration study. OrganizationalBehavior and Human Decision Processes, 39,98-114.

Kern, L., & Doherty, M. E. (1982). "Pseudodiagnos-ticity" in an idealized medical problem-solvingenvironment. Journal of Medical Education, 57,100-104.

Kidd, J. B. (1970). The utilization of subjectiveprobabilities in production planning. Ada Psycho-logica, 34, 338-347.

Kirby, K. N. (1994a). False alarm: A reply to Overand Evans. Cognition, 52, 245-250.

Kirby, K. N. (1994b). Probabilities and utilities offictional outcomes in Wason's four-card selectiontask. Cognition, 51, 1-28.

Klapper, J. T. (1960). The effects of mass communica-tions. Glencoe, IL: Free Press.

Klayman, J., & Ha, Y-W. (1987). Confirmation,disconfirmation, and information in hypothesistesting. Psychological Review, 94, 211-228.

Koehler, D. J. (1991). Explanation, imagination, andconfidence in judgment. Psychological Bulletin,110,499-519.

Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980).Reasons for confidence. Journal of ExperimentalPsychology: Human Learning and Memory, 6,107-118.

Kroger, J. K., Cheng, P. W., & Holyoak, K. J. (1993).Evoking the permission schema: The impact ofexplicit negation and a violation-checking context.In J. St. B. T. Evans (Ed.), The cognitivepsychology of reasoning (pp. 615-635). Hillsdale,NJ: Erlbaum.

Kruglanski, A. W. (1980). Lay epistemology processand contents. Psychological Review, 87, 70-87.

Kruglanski, A. W., & Ajzen, I. (1983). Bias and errorin human judgment. European Journal of SocialPsychology, 13, 1-44.

Kruglanski, A. W., & Freund, T. (1983). The freezingand unfreezing of lay-inferences: Effects onimpressional primacy, ethnic stereotyping, andnumerical anchoring. Journal of ExperimentalSocial Psychology, 19, 448^68.

Kuhn, D. (1989). Children and adults as intuitivescientists. Psychological Review, 96, 674-689.

Kuhn, D., Weinstock, M., & Flaton, R. (1994). How


well do jurors reason? Competence dimensions ofindividual variations in a juror reasoning task.Psychological Science, 5, 289-296.

Kuhn, T. S. (1970). The structure of scientificrevolutions (2nd ed). Chicago: University ofChicago Press.

Kunda, Z. (1990). The case for motivated reasoning.Psychological Bulletin, 108, 480-498.

Kunda, Z., & Nisbett, R. E. (1986). The psychomet-rics of everyday life. Cognitive Psychology, 18,195-224.

Lakatos, I. (1976). Proofs and refutations: The logicof mathematical discovery (J. Worrall & E. Zahar,Eds.). New York: Cambridge University Press.

Lakatos, I. (1978). Falsification and the methodologyof scientific research programmes. In I. LakatosThe methodology of scientific research pro-grammes (J. Worrall & G. Currie, Eds.; pp. 8-101).New York: Cambridge University Press.

Langer, E. J., & Abelson, R. P. (1974). A patient byany other name... : Clinical group differences inlabeling bias. Journal of Consulting and ClinicalPsychology, 42, 4-9.

Laplace, P. S. (1956). Concerning probability. In J. R.Newman (Ed.), The world of mathematics (Vol. 2;pp. 1325-1333). New York: Simon and Schuster.(Original work published 1814)

Lawson, R. (1968). Order of presentation as a factorin jury persuasion. Kentucky Law Journal, 56,523-555.

Lefford, A. (1946). The influence of emotionalsubject matter on logical reasoning. Journal ofGeneral Psychology, 34, 127-151.

LeGrand, H. E. (1988). Drifting continents andshifting theories: The modern revolution in geologyand scientific change. Cambridge, England: Cam-bridge University Press.

Legrenzi, P. (1970). Relations between language andreasoning about deductive rules. In G. B. Floresd'Arcais & W. J. M. Levelt (Eds.), Advances inpsychologistics (pp. 322-333). Amsterdam: NorthHolland.

Lenski, G. E., & Leggett, J. C. (1960). Caste, class,and deference in the research interview. AmericanJournal of Sociology, 65, 463^4-67.

Levine, M. (1970). Human discrimination learning.The subset-sampling assumption. PsychologicalBulletin, 74, 397-404.

Liberman, N., & Klar, Y. (1996). Hypothesis testingin Wason's selection task: Social exchange cheat-ing detection or task understanding. Cognition, 58,127-156.

Lichtenstein, S., & Fischhoff, B. (1977). Do thosewho know more also know more about how muchthey know? Organizational Behavior and HumanPerformance, 20, 159-183.

Lichtenstein, S., Fischhoff, B., & Phillips, L. D.(1977). Calibration of probabilities: The state of

the art. In H. Jungermann & G. de Zeeuw (Eds.),Decision making and change in human affairs (pp.275-324). Dordrecht, Netherlands: Reidel.

Lingle, J. H., & Ostrom, T. M. (1981). Principles ofmemory and cognition in attitude formation. InR. E. Petty, T. M. Ostrom, & T. C. Brock (Eds.),Cognitive responses in persuasive communica-tions: A text in attitude change (pp. 399-420).Hillsdale, NJ: Erlbaum.

Loftus, E. F, & Wagenaar, W. A. (1988). Lawyers'predictions of success. Jurimetrics Journal, 28,437^53.

Lord, C. G., Lepper, M. R., & Preston, E. (1984).Considering the opposite: A corrective strategy forsocial judgment. Journal of Personality and SocialPsychology, 47, 1231-1243.

Lord, C , Ross, L., & Lepper, M. R. (1979). Biasedassimilation and attitude polarization: The effectsof prior theories on subsequently consideredevidence. Journal of Personality and SocialPsychology, 37, 2098-2109.

Luchins, A. S. (1942). Mechanization in problemsolving: The effect of Einstellung. PsychologicalMonographs, 54, 1-95.

Luchins, A. S. (1957). Experimental attempts tominimize the impact of first impressions. In C. I.Hovland (Ed.), The order of presentation inpersuasion. New Haven, CT: Yale UniversityPress.

Lusted, L. B. (1977). A study of the efficacy ofdiagnostic reaiologic procedures: Final report ondiagnostic efficacy. Chicago: Efficacy Study Com-mittee of the American College of Radiology.

Lycan, W. C. (1988). Judgment and justification. NewYork: Cambridge University Press.

Maclntyre, A. (1988). Whose justice? Which rational-ity? Notre Dame, IN: University of Notre Dame Press.

Mackay, C. (1932). Extraordinary popular delusionsand the madness of crowds (2nd ed.). Boston: Page.(Original second edition published 1852)

Mahoney, M. J. (1976). Scientist as subject: Thepsychological imperative. Cambridge, MA: Ball-inger.

Mahoney, M. J. (1977). Publication prejudices.Cognitive Therapy and Research, 1, 161-175.

Manktelow, K. I., & Over, D. E. (1990). Deonticthought and the selection task. In K. J. Gilhooly,M. T. G. Keane, R. H. Logic, & G. Erdos. (Eds.),Lines of thinking (Vol. 1). London: Wiley.

Manktelow, K. I., & Over, D. E. (1991). Social rolesand utilities in reasoning with deontic conditionals.Cognition, 39, 85-105.

Manktelow, K. I., & Over, D. E. (1992). Utility anddeontic reasoning: Some comments on Johnson-Laird and Byrne. Cognition, 43, 183-188.

Markovits, H., & Savary, F. (1992). Pragmaticschemas and the selection task. Quarterly Journalof Experimental Psychology, 45A, 133-148.


Matlin, M. W., & Stang, D. J. (1978). The Pollyannaprinciple: Selectivity in language, memory andthought. Cambridge, MA: Shenkman.

McGuire, W. J. (1960). A syllogistic analysis ofcognitive relationships. In M. J. Rosenberg, C. I.Hovland, W. J. McGuire, R. P. Abelson, & J. W.Brehm (Eds.), Attitude organization and change(pp. 65-110). New Haven, CT: Yale UniversityPress.

Meehl, P. (1954). Clinical versus statistical predic-tion: A theoretical analysis and a review of theevidence. Minneapolis: University of MinnesotaPress.

Meehl, P. (1960). The cognitive activity of theclinician. American Psychologist, 15, 19-27.

Meichenbaum, D. H., Bowers, K. S., & Ross, R. R.(1969). A behavioral analysis of teacher expec-tancy effect. Journal of Personality and SocialPsychology, 13, 306-316.

Merton, R. K. (1948). The self-fulfilling prophecy.Antioch Review, 8, 193-210.

Merton, R. K. (1957). Priorities in scientific discov-ery: A chapter in the sociology of science.American Sociological Review, 22, 635-659.

Millward, R. B., & Spoehr, K. T. (1973). The directmeasurement of hypothesis-testing strategies. Cog-nitive Psychology, 4, 1-38.

Mischel, W. (1968). Personality and assessment.New York: Wiley.

Mischel, W., Ebbesen, E., & Zeiss, A. (1973).Selective attention to the self: Situational anddispositional determinants. Journal of Personalityand Social Psychology, 27, 129-142.

Mischel, W., & Peake, P. K. (1982). Beyond deja vuin the search for cross-situational consistency.Psychological Review, 89, 730-755.

Mitroff, I. (1974). The subjective side of science.Amsterdam: Elsevier.

Murphy, A. H., & Winkler, R. L. (1974). Subjectiveprobability forecasting experiments in meteorol-ogy: Some preliminary results. Bulletin of theAmerican Meteorological Society, 55, 1206-1216.

Murphy, A. H., & Winkler, R. L. (1977). Can weatherforecasters formulate reliable probability forecastsof precipitation and temperature? National WeatherDigest, 2, 2-9.

Myers, D. G., & Lamm, H. (1976). The grouppolarization phenomenon. Psychological Bulletin,83, 602-627.

Mynatt, C. R., Doherty, M. E., & Tweney, R. D.(1977). Confirmation bias in a simulated researchenvironment: An experimental study of scientificinferences. Quarterly Journal of ExperimentalPsychology, 29, 85-95.

Narveson, R. D. (1980). Development and learning:Complementary or conflicting aims in humanitieseducation? In R. G. Fuller, R. F. Bergstrom, E. T.Carpenter, H. J. Corzine, J. A. McShance, D. W.

Miller, D. S. Moshman, R. D. Narveson, J. L. Petr,M. C. Thornton, & V. G. Williams (Eds.),Piagetian programs in higher education (pp.79-88). Lincoln, NE: ADAPT Program.

Nickerson, R. S. (1996). Hempel's paradox andWason's selection task: Logical and psychologicalpuzzles of confirmation. Thinking and reasoning,2, 1-31.

Nisbett, R. E., & Ross, L. (1980). Human inference:Strategies and shortcomings of social judgement.Englewood Cliffs, NJ: Prentice-Hall.

Oaksford, M., & Chater, N. (1994). A rationalanalysis of the selection task as optimal dataselection. Psychological Review, 101, 608-631.

Oskamp, S. (1965). Overconfidence in case studyjudgments. Journal of Consulting Psychology, 29,261-265.

Over, D. E., & Evans, J. St. B. T. (1994). Hits andmisses: Kirby on the selection task. Cognition, 52,235-243.

Pennebaker, J. W., & Skelton, J. A. (1978). Psychologi-cal parameters of physical symptoms. Personalityand Social Psychology Bulletin, 4, 524-530.

Pennington, N., & Hastie, R. (1993). The story modelfor juror decision making. In R. Hastie (Ed.), Insidethe juror: The psychology of juror decision making(pp. 192-221). New York: Cambridge UniversityPress.

Perkins, D. N., Allen, R., & Hafner, J. (1983).Difficulties in everyday reasoning. In W. Maxwell(Ed.), Thinking: The frontier expands. Hillsdale,NJ: Erlbaum.

Perkins, D. N., Farady, M., & Bushey, B. (1991).Everyday reasoning and the roots of intelligence.In J. F. Voss, D. N. Perkins, & J. W. Segal (Eds.),Informal reasoning and education (pp. 83-106).Hillsdale, NJ: Erlbaum.

Peterson, C. R., & DuCharme, W. M. (1967). Aprimacy effect in subjective probability revision.Journal of Experimental Psychology, 73, 61-65.

Pitz, G. F. (1969). An inertia effect (resistance tochange) in the revision of opinion. CanadianJournal of Psychology, 23, 24-33.

Pitz, G. F. (1974). Subjective probability distributionsfor imperfectly know quantities. In L. W. Gregg(Ed.), Knowledge and cognition. New York:Lawrence Erlbaum Associates.

Pitz, G. F, Downing, L., & Reinhold, H. (1967).Sequential effects in the revision of subjectiveprobabilities. Canadian Journal of Psychology, 21,381-393.

Politzer, G. (1986). Laws of language use and formallogic. Journal of Psycholinguistic Research, 15,47-92.

Poly a, G. (1954a). Mathematics and plausiblereasoning. Volume 1: Induction and analogy inmathematics. Princeton, NJ: Princeton UniversityPress.


Polya, G. (1954b). Mathematics and plausiblereasoning. Volume 2: Patterns of plausible infer-ence. Princeton, NJ: Princeton University Press.

Popper, K. (1959). The logic of scientific discovery.New York: Basic Books.

Popper, K. (1981). Science, pseudo-science, andfalsifiability. In R. D. Tweney, M. E. Doherty, &C. R. Mynatt (Eds.), Scientific thinking (pp.92-99). New York: Columbia University Press.(Original work published in 1962)

Price, D. J. de S. (1963). Little science, big science.New York: Columbia University Press.

Pyszczynski, T., & Greenberg, J. (1987). Toward anintegration of cognitive and motivational perspec-tives on social inference: A biased hypothesis-testing model. Advances in experimental socialpsychology (pp. 297-340). New York: AcademicPress.

Ray, J. J. (1983). Reviving the problem of acquies-cence response bias. Journal of Social Psychology,121, 81-96.

Revlis, R. (1975a). Syllogistic reasoning: Logicaldecisions from a complex data base. In R. J.Falmagne (Ed.), Reasoning: Representation andprocess. New York: Wiley.

Revlis, R. (1975b). Two models of syllogisticinference: Feature selection and conversion. Jour-nal of Verbal Learning and Verbal Behavior, 14,180-195.

Rhine, R. J., & Severance, L. J. (1970). Ego-involvement, discrepancy, source credibility, andattitude change. Journal of Personality and SocialPsychology, 16, 175-190.

Rist, R. C. (1970). Student social class and teacherexpectations: The self-fulfilling prophecy in ghettoeducation. Harvard Educational Review, 40, 411-451.

Rosenhan, D. L. (1973). On being sane in insaneplaces. Science, 179, 250-258.

Rosenthal, R. (1974). On the social psychology ofself-fulfilling prophecy: Further evidence for Pyg-malion effects and their mediating mechanisms.New York: MSS Modular Publications, Module 53.

Rosenthal, R., & Jacobson, L. (1968) Pygmalion inthe classroom. New York: Hold, Rinehart andWinston.

Ross, L. (1977). The intuitive psychologist and hisshortcomings. In L. Berkowitz (Ed.), Advances inexperimental social psychology, 10, New York:Academic Press.

Ross, L., & Anderson, C. (1982). Shortcomings in theattribution process: On the origins and mainte-nance of erroneous social assessments. In A.Tversky, D. Kahneman, & P. Slovic (Eds.),Judgement under uncertainty: Heursitics andbiases. Cambridge, England: Cambridge Univer-sity Press.

Ross, L., & Lepper, M. R. (1980). The perseverance

of beliefs: Empirical and normative considerations.In R. Shweder & D. Fiske (Eds.), New directionsfor methodology of social and behavioral science:Fallibel judgement in behavioral research (Vol. 4,pp. 17-36). San Francisco: Jossey-Bass.

Ross, L., Lepper, M. R., & Hubbard, M. (1975).Perserverance in self perception and social percep-tion: Biased attributional processes in the debrief-ing paradigm. Journal of Personality and SocialPsychology, 32, 880-892.

Ross, L., Lepper, M. R., Strack, F, & Steinmetz, J. L.(1977). Social explanation and social expection:The effects of real and hypothetical explanationsupon subjective likelihood. Journal of Personalityand Social Psychology, 35, 817-829.

Roszak, T. (1986). The cult of information: Thefolklore of computers and the true art of thinking.New York: Pantheon Books.

Salmon, W. C. (1973). Confirmation. ScientificAmerican, 228(5), 75-83.

Sawyer, J. (1966). Measurement and prediction,clinical and statistical. Psychological Bulletin, 66,178-200.

Schuman, H., & Presser, S. (1981). Questions andanswers in attitude surveys. New York: AcademicPress.

Schwartz, B. (1982). Reinforcement-induced behav-ioral stereotypy: How not to teach people todiscover rules. Journal of Experimental Psychol-ogy: General, 111, 23-59.

Sears, D. O., & Freedman, J. L. (1967). Selectiveexposure to information: A critical review. PublicOpinion Quarterly, 31, 194-213.

Shaklee, H., & Fischhoff, B. (1982). Strategies ofinformation search in causal analysis. Memory andCognition, 10, 520-530.

Sherman, S. J., Zehner, K. S., Johnson, J., & Hirt,E. R. (1983). Social explanation: The role oftiming, set, and recall on subjective likelihoodestimates. Journal of Personality and SocialPsychology, 44, 1127-1143.

Simon, H. A. (1957). Models of man: Social andrational. New York: Wiley.

Simon, H. A. (1990). Alternative visions of rational-ity. In P. K. Moser (Ed.), Rationality in action:Contemporary approaches (pp. 189-204). NewYork: Cambridge University Press. (Original workpublished 1983)

Skov, R. B., & Sherman, S. J. (1986). Information-gathering processes: Diagnosticity, hypothesis con-firmatory strategies and perceived hypothesisconfirmation. Journal of Experimental SocialPsychology, 22, 93-121.

Slovic, P., Fischhoff, B., & Lichtenstein, S. (1977).Behavioral decision theory. Annual Review ofPsychology, 28, 1-39.

Smyth, C. P. (1890). Our inheritance in the great


pyramid (5th ed.). New York: Randolf. (Originalwork published 1864)

Snyder, M. (1981). Seek and ye shall find: Testinghypotheses about other people. In E. T. Higgins,C. P. Heiman, & M. P. Zanna (Eds.), Socialcognition: The Ontario symposium on personalityand social psychology (pp. 277-303). Hillsdale,NJ: Erlbaum.

Snyder, M. (1984). When belief creates reality. In L.Berkowitz (Ed.), Advances in experimental socialpsychology (Vol. 18). New York: Academic Press.

Snyder, M., & Campbell, B. H. (1980). Testinghypotheses about other people: The role of thehypothesis. Personality and Social PsychologyBulletin, 6, 421^26.

Snyder, M , & Gangestad, S. (1981). Hypotheses-testing processes. New directions in attributionresearch (Vol. 3). Hillsdale, NJ: Erlbaum.

Snyder, M., & Swann, W. B. (1978a). Behavioralconfirmation in social interaction: From socialperception to social reality. Journal of Experimen-tal Social Psychology, 14, 148-162.

Snyder, M., & Swann, W. B. (1978b). Hypothesis-testing processes in social interaction. Journal ofPersonality and Social Psychology, 36, 1202-1212.

Snyder, M , Tanke, E. D., & Berscheid, E. (1977).Social perception and interpersonal behavior: Onthe self-fulfilling nature of social stereotypes.Journal of Personality and Social Psychology, 35,656-666.

Snyder, M., & Uranowitz, S. W. (1978). Reconstruct-ing the past: Some cognitive consequences ofperson perception. Journal of Personality andSocial Psychology, 36, 941-950.

Solomon, M. (1992). Scientific rationality and humanreasoning. Philosophy of Science, 59, 439—455.

Strohmer, D. C , & Newman, L. J. (1983). Counselorhypothesis-testing strategies. Journal of Counsel-ing Psychology, 30, 557-565.

Swann, W. B., Jr., Giuliano, T., & Wegner, D. M.(1982). Where leading questions can lead: Thepower of conjecture in social interaction. Journalof Personality and Social Psychology, 42, 1025-1035.

Taplin, J. E. (1975). Evaluation of hypotheses inconcept identification. Memory and Cognition, 3,85-96.

Taylor, J. (1859). The great pyramid: Why was itbuilt? And who built it? London: Longman, Green,Longman & Roberts.

Tetlock, P. E., & Kim, J. I. (1987). Accountability andjudgment processes in a personality predictiontask. Journal of Personality and Social Psychol-ogy, 52, 700-709.

Thomas, L. (1979). The Medusa and the snail: Morenotes of a biology watcher. New York: VikingPress.

Thurstone, L. L. (1924). The nature of intelligence.London: Routledge & Kegan Paul.

Tishman, S., Jay, E., & Perkins, D. N. (1993).Teaching thinking dispositions: From transmissionto enculturation. Theory into practice, 32, 147-153.

Trabasso, T., Rollins, H., & Shaughnessey, E. (1971).Storage and verification stages in processingconcepts. Cognitive Psychology, 2, 239-289.

Trope, Y, & Bassock, M. (1982). Confirmatory anddiagnosing strategies in social information gather-ing. Journal of Personality and Social Psychology,43, 22-34.

Trope, Y, & Bassock, M. (1983). Information-gathering strategies in hypothesis-testing. Journalof Experimental Social Psychology, 19, 560-576.

Trope, Y, Bassock, M., & Alon, E. (1984). Thequestions lay interviewers ask. Journal of Person-ality, 52, 90-106.

Trope, Y, & Mackie, D. M. (1987). Sensitivity toalternatives in social hypothesis-testing. Journal ofExperimental Social Psychology, 23, 445-459.

Troutman, C. M., & Shanteau, J. (1977). Inferencesbased on nondiagnostic information. Organiza-tional Behavior and Human Performance, 19,43-55.

Tuchman, B. W. (1984). The march of folly: FromTroy to Vietnam. New York: Ballantine Books.

Tweney, R. D. (1984). Cognitive psychology and thehistory of science: A new look at Michael Faraday.In H. Rappart, W. van Hoorn, & S. Bern. (Eds.),Studies in the history of psychology and the socialsciences (pp. 235-246). Hague, The Netherlands:Mouton.

Tweney, R. D., & Doherty, M. E. (1983). Rationalityand the psychology of inference. Synthese, 57,139-161.

Tweney, R. D., Doherty, M. E., Worner, W. J., Pliske,D. B., Mynatt, C. R., Gross, K. A., & Arkkelin,D. L. (1980). Strategies of rule discovery in aninference task. Quarterly Journal of ExperimentalPsychology, 32, 109-123.

Valentine, E. R. (1985). The effect of instructions onperformance in the Wason selection task. CurrentPsychological Research and Reviews, 4, 214-223.

Von Daniken, E. (1969). Chariots of the gods? NewYork: Putnam.

Wagenaar, W. A., & Keren, G. B. (1986). Does theexpert know? The reliability of predictions andconfidence ratings of experts. In E. Hollnagel, G.Maneine, & D. Woods (Eds), Intelligent decisionsupport in process environments (pp. 87-107).Berlin: Springer.

Waismann, F. (1952). Verifiability. In A. G. N. Flew(Ed.), Logic and language. Oxford: Basil Blackwell.

Wallsten, T. S., & Gonzalez-Vallejo, C. (1994).Statement verification: A stochastic model of


judgment and response. Psychological Review,101, 490-504.

Wason, P. C. (1959). The processing of positive andnegative information. Quarterly Journal of Experi-mental Psychology, 11, 92-107.

Wason, P. C. (1960). On the failure to eliminatehypotheses in a conceptual task. Quarterly Journalof Experimental Psychology, 12, 129-140.

Wason, P. C. (1961). Response to affirmative andnegative binary statements. British Journal ofPsychology, 52, 133-142.

Wason, P. C. (1962). Reply to Wetherick. QuarterlyJournal of Experimental Psychology, 14, 250.

Wason, P. C. (1966). Reasoning. In B. M. Foss (Ed.),New horizons in psychology I, Harmondsworth,Middlesex, England: Penguin.

Wason, P. C. (1968). Reasoning about a rule.Quarterly Journal of Experimental Psychology, 20,273-281.

Wason, P. C. (1977). "On the failure to eliminatehypotheses...."—A second look. In P. N. Johnson-Laird & P. C. Wason (Eds.), Thinking: Readings incognitive science (pp. 307-314). Cambridge, En-gland: Cambridge University Press. (Original workpublished 1968)

Wason, P. C , & Johnson-Laird, P. N. (1972).Psychology of reasoning: Structure and content.Cambridge, MA: Harvard University Press.

Wason, P. C , & Shapiro, D. (1971). Natural andcontrived experience in a reasoning problem.Quarterly Journal of Experimental Psychology, 23,63-71.

Webster, E. C. (1964). Decision making in theemployment interview. Montreal, Canada: Indus-trial Relations Center, McGill University.

Wegener, A. (1966). The origin of continents andoceans (4th ed.; J. Biram, Trans.) London: Dover.(Original work published in 1915)

Weinstein, N. D. (1980). Unrealistic optimism about

future life events. Journal of Personality andSocial Psychology, 39, 806-820.

Weinstein, N. D. (1989). Optimistic biases aboutpersonal risks. Science, 246, 1232-1233.

Wetherick, N. E. (1962). Eliminative and enumerativebehaviour in a conceptual task. Quarterly Journalof Experimental Psychology, 14, 246-249.

Wilkins, W. (1977). Self-fulling prophecy: Is there aphenomenon to explain? Psychological Bulletin,84, 55-56.

Winkler, R. L., & Murphy, A. H. (1968). "Good"probability assessors. Journal of Applied Meteorol-ogy, 1, 751-758.

Woods, S., Matterson, J., & Silverman, J. (1966).Medical students' disease: Hypocondriasis inmedical education. Journal of Medical Education,41, 785-790.

Yachanin, S. A., & Tweney, R. D. (1982). The effectof thematic content on cognitive strategies in thefour-card selection task. Bulletin of the Psycho-nomic Society, 19, 87-90.

Zanna, M. P., Sheras, P., Cooper, J., & Shaw, C.(1975). Pygmalion and Galatea: The interactiveeffect of teacher and student expectancies. Journalof Experimental Social Psychology, 11, 279-287.

Ziman, J. (1978). Reliable knowledge. Cambridge,England: Cambridge University Press.

Zuckerman, M., Knee, R., Hodgins, H. S., & Miyake,K. (1995). Hypothesis confirmation: The jointeffect of positive test stratecy and acquiescenceresponse set. Journal of Personality and SocialPsychology, 68, 52-60.

Received August 1, 1997Revision received December 16, 1997

Accepted December 18, 1997

Confirmation Bias: A Ubiquitous Phenomenon in Many Guises

Documents

Transcript of Confirmation Bias: A Ubiquitous Phenomenon in Many Guises