Method and Factual Agreement in Science

Method and Factual Agreement in ScienceAuthor(s): Andrew McLaughlinSource: PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association,Vol. 1970 (1970), pp. 459-469Published by: SpringerStable URL: http://www.jstor.org/stable/495779 .

Accessed: 14/06/2014 07:44

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Springer is collaborating with JSTOR to digitize, preserve and extend access to PSA: Proceedings of theBiennial Meeting of the Philosophy of Science Association.

http://www.jstor.org

This content downloaded from 185.2.32.58 on Sat, 14 Jun 2014 07:44:34 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=springer

http://www.jstor.org/stable/495779?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


ANDREW MCLAUGHLIN

METHOD AND FACTUAL AGREEMENT

IN SCIENCE

One important view of the nature of facts in science sees the validity of factual claims as resting upon intersubjective agreement through observation, yet the conditions underlying such intersubjectivity are rarely explored.1 In this paper I should like to inquire into the role of shared method in arriving at intersubjective agreement concerning fact. One useful way of exploring this issue is to examine the methodological ele- ment in factual disputes in science, since such controversy may indicate important aspects of factual agreement.2

There are times in science when an appeal to factual evidence is success- ful in the choice between hypotheses. In fact it is clear that, at least on some occasions, there is agreement among scientists as to "what the facts are." Moreover, such factual agreement can be found between scientists who maintain quite different theories. This seems to imply that those who claim that facts in science are always 'dependent upon' theory in any important sense are mistaken - but the issue is more complicated than that. What factual agreement between proponents of differing theories does show is that there are things in scientific practice which I shall call theory neutralfacts. This concept minimally describes what happens in science when theoretical disputes are resolved by appeals to evidence. If proponents of differing theories have resolved their dispute through appeals to facts, then they must have been able to point to certain evidence which each will accept as the relevant court of appeal. Such a procedure does involve intersubjective agreement as to the facts and is descriptive of one important facet of scientific practice. This evidence accepted by proponents of two (or more) differing theories is what I shall call 'theory neutral facts'.

But the fact that some theoretical disputes in science are resolved by appeals to evidence does not, by itself, imply that those evidential claims occupy some privileged position sufficient to justify an important epistemological distinction between fact and theory. Rather, there is still room for the claim that facts in science depend upon theory. All that the

Boston Studies in the Philosophy of Science, Viii. All rights reserved.



460 ANDREW MCLAUGHLIN

existence of theory neutral facts implies is that there are some 'facts' which are taken as neutral between some theories, not that they are neutral between all theories. One important question, then, in discussing the epistemological relationship between fact and theory is whether there are some facts which are neutral between all theories. Such facts, if there are any, might be called theory free facts. The question of the presence or absence of such theory free facts is one of considerable interest in current disputes about the relation between theory and fact. This question, however, is not a simple one, since some care must be taken with the meanings given to terms such as 'theory' and 'theory dependence'. As both Achinstein3 and Spector4 have shown, there is no single distinction to be drawn between theory and observation. Rather there are several distinctions which may be drawn and these various distinctions do not seem to be equivalent. Consequently, the claim that there are (or are not) theory free facts in science must be accompanied with a specification of the meaning of 'theory'. For some possible meanings of this term there surely are theory free facts. Thus, if we take 'theory' to mean 'a particular low level scientific theory' - such as Darwinian evolutionary theory - then there surely are theory free facts. That is, there are relevant facts which, in no important sense, 'depend upon' Darwinian theory. Since in a paper of this length it is quite impossible to investigate the whole range of meanings which might be attached to 'theory', I would like to distinguish one important sense of this term and show that when 'theory' is understood in this way there are no theory free facts in science.

It is possible to distinguish between substantive scientific theories about "the way the world is" and methodological theories which are about the proper methods to be used in gaining scientific knowledge about the world. I do not mean to imply that these two types of theories are, in the final analysis, wholly independent, for what we take to be adequate methods probably depends upon what we believe to be the nature of the world. But in the early stages of analysis these two types of theories can be distinguished and discussed independently. There are, in all scientific inquiries, implicit or explicit beliefs concerning the proper procedures for obtaining knowledge about the world. Every method for gaining knowledge is a rule governed activity (at least implicitly), and the rules specify what are reliable ways of ascertaining fact. Churchman has put the general point this way:



METHOD AND FACTUAL AGREEMENT IN SCIENCE 461

Why are certain beliefs and estimates much mere reliable than others? The only conceivable answer is because experience has shown them to be. But then the answer, if sensible, implies that the reliability of a certain kind of information is based on a theory, namely that the information has had and will have a certain degree of reliability over time and a class of 'normal' observers.5

What I want to suggest is that agreement over theory, as that term applies to methods of inquiry, is a condition of agreement over facts. If that is true, then in this sense of 'theory', there are no theory free facts in science. There may, of course, be other senses of 'theory', some of which do allow theory free facts and some of which do not.

One way in which the dependence of factual agreement upon theoretical agreement (in the sense specified above) can be illuminated is to look at a factual dispute in science which involves disagreement over method. It is possible in science to deny any evidential claim the status of theory neutrality by criticizing the method by which it was obtained. This move is always possible because all scientific evidence is the product of an inquiry which uses one or another method. Moreover, any concrete application of a method involves utilizing specific procedures which may also be critized. We can talk very generally about the scientific method, less generally about the experimental method and even less generally about rather specific modes of inquiry, i.e. the non-directive interview. Con- troversy over the methods used in a particular inquiry can be simply about specific experimental designs or they can be about the appropriateness of the experimental method. For the purposes of the argument in this paper it is not necessary to carefully examine the various levels of generality which might be meant by the term 'method', although it is important to note the existence of different possible levels of methodological dispute and that the use of 'method' in this paper is rather wider than usual.

The claim that proposed evidence can be contested by criticizing the method which led to it is not entirely familiar, so it may be of value to illustrate the type of dispute to which I want to call attention. Since the experimental method is regarded by some as the paradigm of the scientific method, it is of value to look at an example which utilizes rigorous experimental procedures. Verbal learning theory in psychology uses such a method and an interesting controversy over the basic nature of learning has developed in this area. The dispute concerns whether the learning of particular items occurs gradually or all at once. The general belief among learning theorists has been that learning is incremental. This means that




even if a subject does not respond correctly to an item in a particular reinforced trial, the subject has nevertheless learned an association to some extent. Thus, it is claimed, the probability of the subject correctly responding to the stimulus in the future has increased. According to the incrementalist position, the repetition of an item does increase learning, and sufficient repetition may lead to correct response. The other side of this dispute maintains that learning is an event that either happens - in which case the subject responds correctly - or else the association is not learned. Thus, if learning is not incremental, but rather is all or none, then the repetition of an item does not promote learning, except insofar as repetition provides more instances when the item might be learned. On the all or none view, either the association is correctly learned or it is not - there is no middle ground. Controversy over this issue can be traced back in one form or another at least to the 1930's, but for our purposes we may date the issue in its contemporary form from an important study by Irwin Rock in 1957.6 Rock presented evidence that when the learning task is to link up two items, such as nonsense syllables or letters and numbers, it does not make any difference how often a pair of items are presented as to the probability that the subject will learn their association. Essentially, what Rock did was to replace items missed on each trial with new items not previously presented. He found that this dropping out of items and their replacement did not affect the rate the paired items were learned. There is not time, nor is it necessary for my point, to go into the details of Rock's work and subsequent evidence concerning one trial learning. Rather, what is important is to examine one way of attempting to deny the status of theory neutrality to Rock's evidence through methodological criticism.

Since Rock's claim, if granted, involves considerable change in the theoretical beliefs of most learning theorists, resistance was to be ex- pected. In this light it is interesting to note one theorist's initial resistance and subsequent acceptance of all or none learning. Estes recognized that the traditional design of learning experiments would obscure all or none learning, if that is how learning occurs.7 The traditional experimental design in learning theory averages the performances of individual subjects and constructs learning curves on the basis of aggregate performance. Such a practice is reasonable for the primary interest is in learning, not the idiosyncratic behavior of individual subjects. But such a procedure




would mask all or none learning, if that is how learning occurs. This is so because it is the individual subject and not the aggregate of those subjects which learns either incrementally or all or none. Consequently, Estes developed some new experimental designs which allowed him to examine how individual subjects performed. Since Estes had set out to falsify Rock's position he was surprised to find that his results indicated that learning was all or none.8 One of the conclusions which he draws from his experience is of particular interest: "The findings we have considered in this paper suggest that in point of fact some of the most firmly entrenched concepts and principles of learning theory may be in a sense artifacts of a conventionalized methodology."9 Thus, Estes concludes that much of what has been taken as theory neutral fact is actually methodologically contaminated, so that it is not neutral evidence between incremental and all or none theories of learning. The evidence which has been appealed to in the past is an 'artifact' of conventional experimental designs. The method (meaning here the traditional design of learning experiments) has become the subject of controversy, removing the evidence obtained by that method from theory neutral status.

But just as the traditional design of the learning experiment can be ques- tioned, so can the designs used by Rock and Estes.'0 Thus, Leo Postman claims that the evidence for all or none theories is contaminated by the methods used and that the incrementalist position can still be main- tained." According to Postman, Rock's evidence can be discounted because of two factors. In the first place, some of his results were due to 'artifacts inherent' in his procedure, for he used a 'slow' rate of presenting items to be learned.'2 This slow rate of presentation allowed subjects to rehearse what they were learning, and Postman cites evidence to show that faster rates of presenting items prevent such a rehearsal strategy.

Secondly, there was "a source of bias which is inherent in ... [Rock's] experimental design," for the method of replacing missed items leads to the selection of more easily learned items.'3 Further, according to Postman, "important differences in materials and procedures" may ex- plain the differences between Estes and Postman's findings, making it "'clear that the empirical inadequacy of the incremental model is still far from being a generally established fact."''4 What is important to note in Postman's reaction to evidence taken by some as telling against the incremental position is that he wishes to deny the status of theory neutral-




ity to that evidence by criticizing the procedures used in obtaining it. Another position taken in this controversy is of interest here. Instead of

taking Rock's evidence based upon 'slow' rates of presenting items as an artifact, one might take this evidence to be of crucial importance. This is Miller's reaction.15 He notes that one theory of learning utilizing computer simulation techniques is based upon a conceptualization of the learning process closely related to the all or none position. If one adopts this approach, then the full importance of Rock's experiments may not have been fully appreciated. According to Miller: The Feigenbaum and Simon theory assumes that subjects will rehearse a few items and let the others go unnoticed on each trial. Postman's experimental demonstration that if you interfere with a rehearsal strategy, you destroy the Rock effect is strong evidence that many people do follow an EPAM I strategy of memorization. Postman refers to this rehearsal strategy as an artifact, but if Feigenbaum and Simon are on the right track it may well be the most important fact of all.16 A few points must be made about this too brief presentation of a complicated dispute. In the first place, I do not mean to suggest that the claims and counter-claims about results being artifacts or resulting from biases inherent in experimental designs is simply an arbitrary procedure. Several sorts of reasons are offered by each writer to substantiate his particular claims about what is fact and what is artifact. Yet it should be apparent that the claim that certain evidence can be discounted because of the method by which it was obtained is not a simple factual claim. Rather, it involves invoking a methodological theory about the proper methods for gaining knowledge. If calling the considerations invoked in the controversy discussed above 'theoretical' seems inflated, it is well to remember that the dispute is relatively 'low level,' concerning particular experimental designs and carried out within the context of general acceptance of the experimental method. More general controversies, such as the question of the general appropriateness of the experimental method for the study of man, do have a distinct 'theoretical' aroma to them.

It is, moreover, worth noting that what is claimed to be a methodological artifact is typically related to the substantive theoretical beliefs of the inquirer. The relationship between what is claimed to be artifact and substantive theories of the learning process is clearest in the attempt to assess the importance of 'slow' and 'fast' rates of item presentation and rehearsal strategies of the subjects. Such a relationship between claims of evidence being artifactual and substantive theories does not appear to be necessary,




however, as Estes' experience led him to significantly revise his theoretical position. Finally, it should be noted that both the all or none and the incrementalist theories became increasingly complicated in the course of this controversy.17

By far the most important result, for the purposes of this paper, is to note that the claim that certain results are artifacts of the method by which they were obtained, and therefore not suitable candidates for theory neutral facts, is one which is always possible. This is so because all scientific evidence is obtained by some method and any method can become the subject of controversy. Such controversies involve theories about the proper methods of gaining knowledge about 'the way the world is.' Facts in science depend both upon the nature of the world and the methods used to investigate the world. If this is true, then methodological agreement is a condition of factual agreement and there are no theory free facts in science. Although such methodological agreement is one of the bases of intersubjectivity in science, there are probably quite a few other conditions of such intersubjectivity.

It could be asked at this point whether agreement concerning methods is either a necessary or a sufficient condition for intersubjectivity about facts. It is fairly clear that such agreement is not a sufficient condition for factual agreement, since agreement over method does not require agreement about fact. But it is also not at all clear that such agreement is a necessary condition, since disagreement about method does not require disagreement about fact. Two inquirers could agree about some particular factual claim, yet disagree about the proper method for ascertaining that factual claim. Hence the sense in which shared theories about methods is a condition for agreement over factual claims is that such theoretical agreement makes intersubjective agreement concerning fact more likely.

It is true that some agreements of this type could be only apparent, and the verbal agreement could hide some deeper disagreement. Thus, if the claim is that an object has a certain mass, then if different theories about the methods of ascertaining mass are held, it is possible that the term 'mass' has importantly different meanings. The verbal agreement as to the mass of an object, in such a case, could obscure the fact that the two inquirers might mean rather different things in their verbally similar statements. While such agreement may be only apparent in some cases, this does not mean that such agreement must always be only apparent.




Rather, it seems that agreement over factual claims arrived at through differing methods constitutes stronger evidence for that claim than agreement involving only one method. Such would be the case, of course, only if there is reason to have confidence in both the methods used. One of the results of recognizing the influence of method upon factual agreement is the prescription that we ought to seek a plurality of methods, since factual agreement which results from the application of various methods seems to be more thoroughly grounded than agreement arising from the use of only one method.18

It is worth emphasizing that the range of possible methodological dispute extends well beyond the sort of disagreement concerning particular experimental designs which have been considered above. It is simply not the case that there is only one 'scientific method' which is used in all scientific inquiries. This is surely true, at least, if one seeks a charac- terization of methods which are reasonably descriptive of the way inquiries are actually conducted in science. Perhaps this is clearest in the social sciences where there are highly visible methodological disagreements over the best means to gain knowledge. Some inquirers think that the experimental method is best, while others adopt survey research techniques, or use mathematical models, or computer simulation; still others insist that more 'subjective' methods such as participant observation are the proper ways to study man. Such methodological disagreements stem partly from divergent beliefs about the types of knowledge which ought to be sought,19 but such disagreements also involve differing theories about the proper methods of gaining reliable knowledge. It is perhaps more difficult to see what methodological controversy would look like in the natural sciences. Clearly there are examples of low level methodological disputes in the natural sciences (e.g. over proper measurement techniques) but, it could be asked, what would be the force of questioning the general experimental method in the natural sciences. This objection has some force, but two points make it less compelling. It is not clear that the variety of concrete methods utilized in the natural sciences are usefully as all being examples of one single method - namely the experimental method. For example, are inquiries stemming from mathematical physics best understood as 'experimental', or rather might there be many types of experiments which exemplify different methods of inquiry. It might turn out that if we pay closer attention to what natural scientists actually do,




there may be such diversity of procedures that the use of a single term to describe these various methods is misleading. But even if there is a certain general uniformity of procedure which can be successfully discerned in the natural sciences, it doesn't follow that such uniformity is desirable. Rather such uniformity may mask the influence that the standard method has upon the substantive results of inquiry. And it is simply not true that the experimental method is the only possible method for the natural sciences. To make only one suggestion as to alternatives, it is possible that the method of simulation could be used in the natural sciences and the use of such an alternative might provide a useful check on the influence of the experimental method on the substantive results achieved.

At this point a distinction which has been tacitly presupposed in my argument should be made explicit. There is, I think, an important difference between what I have called theory neutral facts and what might be called data. What functions as evidence in science are theory neutral facts, not data. Data are what are proposed as neutral fact but not all are accepted as such. The existence in scientific inquiry of the difference between data and fact can be most easily illustrated by noting the difference between denying data the status of fact and denying that data are truly data at all. We have already examined the process of denying data the status of fact in examining the dispute in learning theory. Contrast that type of dispute with a much rarer sort of dispute which involves accusing a scientist of deliberately misreporting the results of his inquiry. Such an accusation is much more serious than the claim that some evidence is the artifact of the method employed and is treated quite differently by the scientific community. Distinguishing between data and fact captures this difference nicely, for it makes room for the important difference between impugning the honesty of an inquirer and merely denying his data the status of theory neutrality.

A final point I want to make involves indicating a fruitful area for further inquiry. Noting the methodological aspect of intersubjectivity in science suggests that such intersubjectivity is neither a brute fact nor does it just fortuitously happen. Rather it is an achievement. There is reason to believe that such intersubjectivity as can be attained has a basis in shared theories about methods, but the piocess of achieving intersubjectivity is much more complex than that. After a study of the problems involved in




reaching intersubjective agreement in a particular interdisciplinary research project, Stewart E. Perry concludes that: "The process of making public what is our private experience is the social-cultural-psychological act in science. The necessity to share and to reach consensus implies an engagement in [social] interaction."20 Further investigation into the conditions underlying intersubjective agreement should involve recognizing the psychological and social dimensions of such agreement. Inquiry in these directions may prove quite fruitful in illuminating the nature of fact in science.

Lehman College, City University of New York

NOTES

1 The following passage brings out the idea of intersubjective agreement as the basis of fact in science, and it also shows how intersubjectivity can be taken as 'given': "The condition thus imposed upon the observational vocabulary of science is of a pragmatic character: it demands that each term included in that vocabulary be of such a kind that under suitable conditions, different observers can, by means of direct observation, arrive at a high degree of agreement on whether the term applies to a given situation.... That human beings are capable of developing observational vocabu- laries that satisfy the given requirement is a fortunate circumstance: without it, science as an intersubjective enterprise would be impossible." (C. Hempel, Aspects of Scientific Explanation, Free Press, N.Y., 1965, p. 127.) 2 A similar strategy for exploring a different condition of intersubjectivity was adopted by Hanson. See his Patterns of Discovery, Cambridge University Press, 1958, p. 18. 3 P. Achinstein, 'The Problem of Theoretical Terms', American Philosophical Quarterly 2, 1965. 4 M. Spector, 'Theory and Observation, I and II', British Journal for the Philosophy of Science 17, No. 1 and 2. 5 C. West Churchman, Prediction and Optimal Decision, Prentice-Hall, Englewood Cliffs, N.J., 1961, p. 80, italics in original. 6 I. Rock, 'The Role of Repetition in Associative Learning', American Journal of Psychology 70, 1957; see also I. Rock and W. Heimer, 'Further Evidence of One Trial Associative Learning', American Journal of Psychology 72, 1959. 7 W. K. Estes, 'Learning Theory and the New "Mental Chemistry"', The Psychological Review 67, 1960. 8 Ibid., p. 215. 9 Ibid., p. 221. 10 Estes seems to neglect this possibility and thinks he is at least on the way to experimental designs which are beyond dispute and which will lead directly to the truth about learning theory: "While I would not for a moment deprecate the role of imagination in science, I suspect that it will begin to serve us effectively in learning theory only as




we begin to accumulate reliable determinations of the effects of single variables upon single learning trials in individual organisms. If by continual simplification of our experimental analyses and refinement of our mensurational procedures we can achieve these determinations, we may find that the long sought laws of association may be not merely 'instigated', or even 'suggested', but literally dictated in form by empirical data." (Ibid., p. 222.) 11 L. Postman, 'One Trial Learning', in Verbal Behavior (ed. by C. N. Cofer and B. Musgrave), McGraw Hill, N.Y., 1963. 12 Ibid., p. 302. 13 Ibid., p. 303. 14 Ibid., p. 319. 15 G. A. Miller, 'Comments of Professor Postman's Paper', on Cofer and Musgrave, op. cit. 16 Ibid., p. 326. 17 For example, see R. C. Atkinson and R. C. Calfe, 'Mathematical Learning Theory', in Scientific Psychology (ed. by B. B. Wolman), Basic Books, N.Y., 1965. 18 J take this to be an application of Feyerabend's prescription concerning theoretical pluralism. See, for example, his 'How to Be a Good Empiricist - A Plea for Tolerance in Matters Epistemological', in Philosophy of Science; The Delaware Seminar, Vol. 2, (ed. by B. Baumrin), Wiley, N.Y., 1963. 19 See my paper 'Science, Reason and Value', Theory and Decision, forthcoming. 20 S. E. Perry, The Human Nature of Science, Free Press, N.Y., 1966, p. 236.



Method and Factual Agreement in Science

Documents

Transcript of Method and Factual Agreement in Science