On Probability as a Basis for Action On Probability as a Basis for Action

7/28/2019 On Probability as a Basis for Action On Probability as a Basis for Action

1/8

145

On Probability As a Basis For Action

by

W. EDWARDS DEMING

Reprinted fromThe American Statistician, Vol. 29, No.4. 1975. pp . 146-152


2/8

On Probability As a Basis For Action'"

AbstradThe aim of the author is improvement of statisticalpractice. The author distinguishes bptwepn enumprativpstudies and analytic studies. An enumerative study hasfor its aim an estimate of the numbpr of units of aframe that bplong to a specificd class. An analytic studyhas for its aim a basis for action on the causc-systpm or

the process, in order to improve product of the future.A fair price to pay for an invpntory is an examplp of anpnumerative study. Tests of varieties of wheat, insecticides, drugs, manufacturing processes, are examplpsof analytic studies: the choice of variety or treatmentwill affect the future out-turn of whpat, future patients,future product. Techniques and methods of inferencethat arc applicable to enumerative studies lead tofaulty design and faulty inference for analytic problpms.

I t is possible, in an enumerative problem, to n'duceerrors of sampling to any spccified level. In contrast,in an analytic problem, it is impossible to compute therisk of making a wrong decision. The author providps anumber of examples, and pleads for greater care in thpwriting and teaching of statistical theory and inferencp.

A im and scope of th1's paper. The aim hpre is to try tocontribute something to the improvempnt of statisticalpractice. The basic supposition here is that any statistical investigation is carried out for purposes of action.New knowledge modifips existing knowledge.Urgent needs for stal1'stical work. Challenges face

statisticians today as nevpr before. The \\'hole world istalking about safety in mechanical and plectricaldevices (in automobiles, for example), safety in drugs,reliability, due care, pollution, poverty, nutrition,improvement of medical practice, improvement ofagricultural practice, improvement in quality ofproduct, brpak-down of service, break-down of equipmerit, tardy busses, trains, and mail, need for greatrroutput in industry and in agriculture, enrichment ofjobs. The consumer requires month by month evergreater and greater safety, and he expects better andbetter performance of manufactured articles. Themanufacturer has the satne problems in his purchasesof materials, assemblies, machines, and use of manpower. He must, in addition, know more and moreabout his own product. What is due care in manu

I am indebted to many critics of earlier drafts of the manuscript for this paper; al80 to questions from the audience atlectures at a number of universities, including the Princetonmeeting. of the Biopharmaceutical Section of the AmericanStatistical A880ciation 4 Dec. 1974; the Universities of Mainz,Colorado, Wyoming, George Washington University, NorthCarolina, Inter-American Statistical Institute in Santiago deChile. Consultant in Statistical Surveys, 4924 Butterworth Pl.,Washington 20016.

W. EDWARDS DEMING**

facturing? What is malpractice in medicine? Statisticalwork in consumpr rpsparch is in a sorry statp, moremoney bping sppnt 011 it ypar by year, with evpr\\'orsening examplps of practic(' and prespntation.These probl('ms ('an not be undprstood and can noteven be stated, nor ('an the


3/8

the frame i,;; indl'finitl', and a prohahility P of sl:'!ectionis applil'd to l'ach sampling unit as it is pn'sl'llted en].A 100% sample (complell' cen,.;us) cannot be dl'filll'd inthl' ahsence of a frume of .V sampling units.Enlllnel'Qtire studies and annlytic studies contrasted.

Th e distinctioll hdll"l'('n enunll'r,ltiv(' an d analyticstudil's is vital in thl' dl'sign of stud iI's and in tlwintl'rprctation of rl>slilts.

E:'\"UMEHATIVE: in which action will he taken 011the matf'rial in the framc studil'd. Th e action to betaken on the frame depl'lIds purely all l'stimates orcomplde counts of onc or lIlorl' specific populations ofthe franw. Thl' aim of a statistical study in all enUlIll'rative probll'm is dl'scriptivl'. How 1I1any farms or howmany people hf'long to this or that catl'gory? What isth e expected out-turn of II"hl'at for this rl'gioll? Howmani units in the lot arf' defective? The aim is not tofind out why thprl' are so many or so few units in this orthat catrgory: ml'rely ho\\' many.

Examples: 1. Data of Census-type: ag(', sex, education, occupation hy arca. 2. Figures on the utilizationof out-patient psychiatric s f ' r v i c l ~ . 3. Prevalence ofdiabl'tes. 4. Assays of samples takell from a shipload of01'1', to estimatr what the shipload is worth alld todecide how much to offf'r for it. 5. Tl'sts of manufacturl'd product. (The Bureau of Customs will calculate also from the sampll's how much duty to pay, ifthe ore conws from ahroad.) Th e Census for Congrf'ssional rrprps('ntation in thl' Unitrd Statl's is a priml'I'x:.lI11ple of an l'nunwrative study. Congressionalrepresentation ill an area depl'nds on how manypmp!e are in it, no t why th('y are therl'.

ANALYTIC: in which actioll will he takell on th eprOl;I'SS or C a l l S ( ~ - s y s t l ' l I l that producl'd the framestudil'J, thl' aim heing to improvl' practicl' in the futun.Exampll's: tesls of varif'ties of wheat. comparison ofmachirl!'s, corr.parison of ways to advc'rti,;;l' a product orservice, comparison of drugs, actitlll on an industrialprocl'SS (changl' in speed, change in tcmperaturl',change in i n ~ r ( ' ( l i l ' n t s ) . Intf'rcst ccntn's in futureproduct, not in the matl'rial studied. Action: adoptMethod B over A, or hold on to A, or continue th eexperimen t.

There is a simple critNion hy which to distinguishbetween enullll'rative an d analytic studies. A 100 percent sample of thl' frallle providl's the complete allsw


4/8

data to predict the environmental conditions of the letter to the author from one of the largest airlinesfuture, nor the performance therein. in the country.Comparison of two treatments is not a uniformitytest by which to estimate the variance between plots How could tests of the past cover all conditions to bcwithin blocks subjected to the same treatment, or thevariance between patimts under the same treatment.It is not an investigation to ask whether the data conform to some specific genetic law of inheritance, whichmight generate the ratio 3: 1 for light hair and darkhair in the offspring.

Limitations of statistical inference. All results arcconditional on (a) the frame whence came the units fortest; (b) the method of investigation (the questionnaire or the test-method and how it was used); (c) thf'people that carry out the interviews or measurements.

( In addition (d), the results of an analytic s t - ~ d y arcconditional also on certain environmcntal states, suchas the" geographic locations of the comparison, the dateand duration of the tf'st, the soil, rainfall, climate,description and medical histories of t h c ~ patients orsubjects that took part in the test, the observers, thehospital or hospitals, duration of test, levcls of radiation, range of voltagc, speed, rang(' of temperature,range of pressure, thickness (as of plating), number offlexures, number of jolts, maximum thrust, maximumgust, maximum load.

The exact environmental conditions for any experimc:nt will ncver be seen again. Two treatmc'nts thatshow little difference under one set of environmentalcircumstance's or even within a r a n g c ~ of conditions,may diffe'r greatly undf'r other conditions-other soils,other climate, etc. The' converse may also bf' true: twotreatments that show a large diffc:renc(' \ I n d c ~ r one se tof conditions may be n c ~ r l y f'qual und('r other conditions.

There is no statis tical mf'thod by which to ( ~ x t r a p o l a t e ~ to longer u s a g c ~ of a drug bcyond the period of test, norto other patic'nts, soils, climates, highc'r v o l t a g c ~ s , nor toother limits of severity outside th(' r a n g c ~ studil)d. S i d c ~ effects may develop later on. Problc'ms of maintenance

" of machinery that show up well in a t('st that covers_ three weeks may cause gripf and wgrl't after a fewmonths. A compditor may step in with a n ( ~ w product,or pu t on a blast of advertising. Economic conditionschange, and upset pr('dictions and plans. These arcsome of the rf'asons why information on an a lalyt icproblcm can n c ~ v e r be complete, and why c o m p ~ t a t i o n s by usc of a 10ss-fun


5/8

The appropriate statistical design depends on the limits on the margin of uncertainty of the estimate.value of D, which must be stated in advance. It s mag Probability samples have an advantage in an enumeranitude is the responsibility of the expert in the subjcct tive problem, as they remove one important arca ofmatter. doubt; they enable one to evaluate the uncertainty in a

The problem is one of estimation. What is the mag,-. nitude of B - A?It is important to remember that the mean, thevariance, the standard error, likelihood, and many

other functions of a lWt of numbers, arc symmetric.Interchange of any two obsC'rvations Xi and Xj leavesunchanged the mean, the variance, and even the distribution itself. Obviously, then, use of variance andelaborate methods of estimation buriC's the informationcontainl'


6/8

enough strata an d conditions to l'stahlish tlw arl'aS nndcondition8 und('r whieh the supl'riority of B over .4 isequal to or grl':\ter than D, or in which thl' d i f f p r f ' n c ! ~ isinconsequf'ntial. Omission of a stratulII of specialintfOrpst may impilir an experinlPnt. Example:

The mid-portion of pregnancy may be as vulnerahl!' to environmentnl agl'nts as early p:'egnancy . . . hut the middle part is not included indrug-tp.sting routinps. - : ' iew York Timl's, 19 July1975: page 24, quoting Dr. Andrew G. Hpndrickx.I t is fairly easy now to understand why it is that a

prohability sample of a whole frame would be ineffieientfor an analytic study. Thus, to tpst two trl'atm!'nts inan agriculturul exp


7/8

distance acuity without correction than girls of thatage in each of the four regions of the country.However, only in the Midwest and in the Southwere the differencf's . . . large enough to be stat istically significan t.

U. S. 196&-70

Sex20/20 or better 20/70 or poorer

~ o r t h - ~ t i d - South Westeast west

72.1 70.0 80.0 74.52.63 1.96 1.67 2.70

66.3 60.9 72.6 66.73.21 2.82 2.10 5.96

North- Mid- South Westeast west

16.7 18.2 10.6 13.82.46 1.37 0.82 2.14

18.3 23.6 13.3 21.22.85 2.57 1.42 4.-10

Boy:!Per centStandarderror

Girls'Per centStandard

error

Comments. (a) The diffpn'ncps bptwf'en boys andgirls appt>ar to be pf'rsistl'nt from region to region, andto be suhstantial, of scipntific importance, worthy offurthpr study. (b) Examination of the detailpd tablesfor the U. S. as a whole (not includf'd hl're) givf'evidl'nce in conflict with thl' c o n c l u ~ i o n quotrd.Actually, more girls pt'r 1000 than boys p"r 1000 atevery age 12, 13, 14, 15, 16, 17, have vision 20/17 andlikewise 20/20, but more boys than girls have vision20/15 and 20/12 or brttpL (c) What appl'ars to be ahighrr percl'ntage of boys in the accompanying tablewith vision 20/20 or bettcr comps frolll th r fortuitousconsolidation and confounding of lop-siupu proportionsat the different ages and levels of vision, of the kindjust describrd. (d) These lop-sided proportions maywell be the most important result of the study, but thetext by-passes this possibility. (e) The high proportionsof both boys and girls in the South with vision 20/20or better (hot shown here) compared with the rest ofthe country, may have its origin in differences betweenthe visions of black and white boys and girls, bu t thedetailed tables do not show figures separately by color,possihly because of small samples for blacks. (f ) Differences betweE.'n examiners would, in my experience, beworth investigation, but the text gives no indication ofhow the boys and girls were allotted to the examiners,nor any summary of differences between examiners.(g) The standard errors shown in the table are meaningless; they apparently obscured the vision of thewriter of the text.

More on the teaching of statistics. Little advancementin the teaching of statistics is possible, and little hopefor statistical methods to be useful in the frightfulproblems that face man today, until the literature andclassroom be rid of terms so deadening to scientificenqui ry as" null hypothesis, population (in place offrame), true valU'e, level of significance for comparison

of treatments, representative sample. There is no truevalue of any concept that is measured. There may be,of course, an accepted operational definition (questionnaire, method of measurement) and an acceptedvalue-accepted until it is replaced with one that ismore acceptable to the experts in the subject-m!ltter[6J, [12J, [18].Here are three suggestions to replace topics thatshould be thrown out. First, non-sampling errors:their detection and measurement by statistical controls,and their possible effects on uses of the results [6].Second, the contrast between enumerative and analyticstudies, their purposes and contrast in design andanalysis. This would automatically bring in the use ofjudgment-samples where they are indicaled for bestefficiency. Third, every student should try his hand atstatistical inference, given a set of original data,together with the necessary non-statistical informationabout the environmental conditions of the experimentor survey. Cross-examination by other members of theclass would teach a student to be careful.

Studcnts of statistics need some teachers that areengaged in practice. What would happen in medicine ifmedical students studif'd s u r g , ~ r y and internal medicinefrom physicians none of whom had ever been in practice?

A teacher that gets involved in statistical problemshas a basis for making a choice of what theory to teach,and he has il lustrations of his own for the classroom andfor his book, and he will understand the illustrations.His trliching inspires students to thinK.We also need some teachers of theory that are not inpractice. The student can only learn theory. To learntheory, though, most studf'nts require examples of goodpractice and example:; of bad practice, with explanation,in terms of theory, of what was right and what waswrong about the procedure. Faultless, skillful teachingof statistic:; is unhappily too often undone by examplesof design and inference that mislead the student, as Ihave tried to ill ust rate here.Unfortunately, involvement in a problem carriesresponsibilities. The statistician in practice must writea report for management or for legal purposes, on thestatistical reliability of the results, what they maymean and what they don't mean. Such a report willstate the possible m a r g i ~ s of uncertainty from samplingvariation alld from operational blemishes big and littlediscovered in the controls, the nonresponse, illegibleentries, missing entries, inconsistencies found in thecoding, with a careful statement of the conditions of theexperiment (duration, locality, reasons for choicethereof, voltage, range of stress, etc.), the method ofmeasurement or the questionnaire, and the difficultiesencountered. (An example is in reference [6].) Involvement in a problem means the possibility of facinga board of directors, or facing cross-examination. I tmeans tedious work, such as studying the data invarious forms, making tables and charts and re-makingthem, trying to use and preserve the evidence in theresults and to be clear enough to the reader: to endure

151


8/8

disappointment and discouragement. Desultory adviceon possible ways to attack a double integral docs notconstitu te involv('mcnt in a problcm.REFERENCES

III Anscombe, F. J. : "Graphs in Statistical Analysis," TheA mmcan Slalislician, Feb. 1973, vol. 27, 1'\0. I.

12J Blythe, Hichard H.: "The economics of sample-size appliedto the Rcalinp: of SI1wlop:s," The Biomrlrirs BIIl/etin, Washinll:ton, vol. I, 1945: pp. 67-70.

131 Berkson, Joseph: "Tests of sip:nificance considereu asevidence," Journal of Ihe American Statistical Associa/ion,vol. 37, 1942: pp. 325-335.

141 Rorinp:, Euwin G.: "!\lathematical versus scientific significance," Psychological Bullrtin, vol. 15, 1919: pp. 335-338.

151 Uutl.erworth, Alfred T. , and Watls, Rohert D.: .. Doubleblind comparisons," Psychomatirs xv, 1974: pp . 8:}-87.

(61 Deminp:, W. Edwards: Some Theory of Sampling (Wiley,'1953; Dover 1960), Ch. 5. An example of a statistical report

appears in this book on pap:es 159-163. See also "Boundariesof statistical inference," beinp: Chapler 31 in the book hvNorman L. Johnson an d Harry Smith: NrIV Drvrlop'"l'lItsin Survey Sampling (Wiley, 19(9).

171 Ehrenberp:, A. S. C.: Data Redurtion (Wiley, 197.1). pp. vii,88, 350 in particular.181 r':l'harut, elirl: "Slatisti(s. Ii trap for the unwary," OIJMl'tnrs

an d G'/M.olog'l, vol. 14, Oct. 1959: pp. 5 4 ~ h ' i 5 4 . 191 Hanllen, ~ f o r r i s H., anu Deminll:, W. Euwards: "On an

important limitation to the use of uata from samples,"Hill/e/in de {,ins/itlt/e international de s/alistiqlll', !3prn 1 9 . ~ : pp. 2(,\-219.

1101 Koller, S.: "Use of non-representative surveys for etiolop;i('alpl'Oblems." II. ('hapter in the book euiteu by :\ormlln I..Johnson and Harry Smith: Ne w Dr"l'lopmen/s ill S,m'I"/Sampling (Wiley-Inlerscienee, 19(9): pp. 235-246.

I l l ] Kruskal, William H. : "Tests of sill:nificance," InternatioF:nrl/c1opedia of the Sorial SrirnCfs, vol. 14, pp. 238-2"Chromosomal effect an d LSD: samples of 4," Science, v162,27 Dec. 1968, pp. 1508-1509.

1121 Lewis, C. 1.: ~ I i n d an d the World-Order (Scribner, 192p.238.

1131 ~ t o r r i s o n , Denton E., an d Henkel, Ramon E. : The Sign'IInrl' 'frst Conlrrn'er .!! (Aldine I'uhlishinll: Company, 197Excellent for references lind uiscussion.

1141 Savall:e, 1. Richaru: Statistics: l 'ncrrtainty an d Behav(lloull:hton :\Iiffiin, 19(8), Ch. 4.

1151 Schlaifer, Robert: l'robabi/ill/ and Stalist ics for BusinDrcisions (:\tcGraw-lIill. 1959), Ch. 33.1161 Schlaifer. Hobert: loc. cit., p. 491.

1171 Shewhart, \\'alter A.: Statistical :\lI'/hod from the Vieu'Poof Quality Control jrauuate School, Department of A ~ r i cture, 1939), p. 120.

1181 Shewhart. Walter A.: ]oc. cit., pp. 88, 89, Ill, 135.1191 Stephan, F. F. : American Sociolt>ll:ical neview, vol. t, 19

pp.569-580.1201 Tornqvist, Leo: "An attempt to analyze the problem ofeconomical productioll of ,tatistical uata," :\ ord

Tidsskrift for T('knish 0konomi, vol. 37, 1948: pp. 263-21211 Tukey, John: "Some I/;rllphi(' linu semigraphic ui,play

heinll: Ch. 18 in Stll/is/iml J'apfTs In Honor of GrorgeSflP/lpmr, euit.ed by T. .\. Banl'roft (\owa ~ t a t e lJniversiI'r(',s, 19721. Erplnl'(l/nr.'l Uatll .1 flal,/sis (AUUISllll-W",lPres,. 19751.

1:l'21 Wolfowitz, J. : "Hemarks 011 thl' theory of testml/; hypot'es," The /"'ew rork StatlS/iriafl, ~ I a r c h 1967, vol. 18, :\0

[23[ Yates. Frank: Sall/pllng .Ifl'thnds and S"rrfVs (,riffin, 1919(0), Ch. 3. Jolly, (; . :\1., "Exphcit e'timates from capturel'apture uata," Hiollletrikll, vol. ;>2, 196;-,: pp. 22.')-2W. Euwards Deminl/;: "On vanant'es and ('stimators ototal population under st'vt'ml prOl'edllreS of samplinp.l/ethod.. in Bioml'try, fl'stsl'hrift flir Dr. A. I.inu('r (BihiiuSl'r, Ba,('I, 1975).

152 The Anurican Statiatician, November 1975, Vol. 19, NO.4

On Probability as a Basis for Action On Probability as a Basis for Action

Documents

Transcript of On Probability as a Basis for Action On Probability as a Basis for Action