STATISTICSW I L L I A M L . H A Y S

University of Texas at Austin

F I F T H

E D I T I O N

Harcourt Brace College Publishers

Fort Worth Philadelphia San Diego New York Orlando Austin San AntonioToronto Montreal London Sydney Tokyo

CONTENTS

Preface v

Introduction 1O.I On the Nature and the Role of Inferential Statistics 10.2 About This Book 50.3 The Organization of the Text 80.4 Statistical Packages for the Computer 90.5 Statistical Packages for Microcomputers 110.6 Why Learn Statistics in the Computer Age? 110.7 A Word on Rounding 12

Chapter 1

Elementary Probability Theory 141.1 Simple Experiments 151.2 Events 151.3 Events as Sets of Possibilities 161.4 Probabilities 211.5 Some Simple Rules of Probability 221.6 Equally Probable Elementary Events 251.7 "In the Long Run" 281.8 Example of Simple Statistical Inference 311.9 Probabilities and Betting Odds 341.10 Other Interpretations of Probability 35

ViH CONTENTS

1.11 More about Joint Events 371.12 Conditional Probability 421.13 Bayes's Theorem 451.14 Independence of Events 481.15 Representing Joint Events in Tables 501.16 Random Samples and Random Sampling 531.17 Random Numbers 54Terms You Should Know 55Practice Exercises 56Exercises 59

Chapter 2

Frequency and Probability Distributions 712.1 Measurement Scales 712.2 Frequency Distributions 772.3 Frequency Distributions with a Small Number of Measurement Classes 782.4 Grouped Distributions 802.5 Class Interval Width and Class Limits 812.6 Interval Size and the Number of Class Intervals 832.7 Midpoints of Class Intervals 852.8 Another Example of a Grouped Frequency Distribution 862.9 Frequency Distributions with Open or Unequal Class Intervals 872.10 Graphs of Distributions: Histograms 882.11 Frequency Polygons 912.12 Cumulative Frequency Distributions 932.13 Probability Distributions 942.14 Random Variables 972.15 Discrete Random Variables 982.16 Graphs of Probability Distributions 1022.17 Function Rules for Discrete Random Variables 1042.18 Continuous Random Variables 1072.19 Cumulative Distribution Functions 1112.20 Graphic Representations of Continuous Distributions 1122.21 Joint Distributions of Random Variables 1132.22 Frequency and Probability Distributions in Use 114Terms You Should Know 115Practice Exercises 115Exercises 120

Chapter 3

A Discrete Random Variable: The Binomial 1283.1 Calculating Probabilities 1283.2 Sequences of Events 1303.3 Counting Rule 1: Number of Possible Sequences for N Trials 130

CONTENTS JX

3.4 Counting Rule 2: For Sequences 1313.5 Counting Rule 3: Permutations 1313.6 Counting Rule 4: Ordered Combinations 1333.7 Counting Rule 5: Combinations 1343.8 Some Examples: Poker Hands 1353.9 Bernoulli Trials 1373.10 Sampling from a Bernoulli Process 1373.11 Number of Successes as a Random Variable: The Binomial Distribution 1403.12 The Binomial Distribution and the Binomial Expansion 1433.13 Probabilities of Intervals in a Binomial Distribution 1433.14 The Binomial Distribution of Proportions 1443.15 Preview of a Use of the Binomial Distribution 1453.16 The Sign Test for Two Groups 1483.17 The Geometric and Pascal Distributions 1493.18 The Poisson Distribution 1523.19 The Multinomial Distribution 1543.20 The Hypergeometric Distribution 156Terms You Should Know 157Practice Exercises 158Exercises 159

Chapter 4

Central Tendency and Variability 164

4.1 The Summation Notation 1644.2 Measures of Central Tendency 1654.3 The Mean as the Center of Gravity of a Distribution 1704.4 "Best Guess" Interpretations of Central Tendency 1724.5 Central Tendency in Discrete Probability Distributions 1734.6 The Mean of a Random Variable as the Expectation 1744.7 Expectation as Expected Value 1754.8 Theoretical Expectations: The Mean of the Binomial Distribution 1774.9 The Mean as a Parameter of a Probability Distribution 1784.10 Relations Between Central Tendency Measures and the "Shapes" of

Distributions 1794.11 Measures of Dispersion in Frequency Distributions 1824.12 The Standard Deviation 1834.13 The Computation of the Variance and Standard Deviation 1844.14 Some Meanings of the Variance and Standard Deviation 1864.15 The Mean as the "Origin" for the Variance 1884.16 The Variance and Other Moments of a Probability Distribution 1894.17 Standardized Scores 1914.18 Tchebycheff's Inequality 1934.19 Percentiles and Percentile Ranks 194Terms You Should Know 196Practice Exercises 197

X CONTENTS

Exercises 200

Chapter 5Sampling Distributions and Point Estimation 2045.1 Populations, Parameters, and Statistics 2045.2 Sampling Distributions 2065.3 Characteristics of Single-variate Sampling Distributions 2075.4 Sample Statistics as Estimators 2085.5 The Principle of Maximum Likelihood 2095.6 Other Desirable Properties of Estimators 2115.7 The Sampling Distribution of the Mean 2135.8 Standardized Scores Corresponding to Sample Means 2155.9 Correcting the Bias in the Sample Variance as Estimator 2165.10 Parameter Estimates Based on Pooled Samples 2185.11 Sampling from Relatively Small Populations 2205.12 The Idea of Interval Estimation 2215.13 Other Kinds of Sampling 2245.14 To What Populations Do Our Inferences Refer? 2255.15 Linear Combinations of Random Variables 227Terms You Should Know 228Practice Exercises 229Exercises 231

Chapter 6

Normal Population and Sampling Distributions 2376.1 Normal Distributions 2376.2 Cumulative Probabilities and Areas for the Normal Distribution 2406.3 Importance of the Normal Distribution 2436.4 The Normal Approximation to the Binomial 2446.5 The Theory of the Normal Distribution of Error 2476.6 Two Important Properties of Normal Population Distributions 2496.7 The Central Limit Theorem 2506.8 Confidence Intervals for the Mean 2546.9 Sample Size and Accuracy of Estimation of the Mean 2566.10 Use of a Confidence Interval in a Question of Inference 2576.11 Confidence Intervals for Proportions 258Terms You Should Know 260Practice Exercises 260Exercises 262

Chapter 7

Hypothesis Testing 2677.1 Statistical Hypotheses 269

CONTENTS XI

7.2 Testing a Hypothesis in the Light of Sample Evidence 2707.3 A Problem in Decision Theory 2737.4 Expected Loss as a Criterion for Choosing a Decision Rule 2767.5 Factors in Scientific Decision Making 2797.6 Type I and Type II Errors 2827.7 Conventional Decision Rules 2837.8 Power of a Statistical Test 2847.9 Power of Tests Against Various True Alternatives 2877.10 Power and the Size of a 2897.11 Effect of Sample Size on Power 2907.12 Power and Error Variance 2927.13 One-tailed Rejection Regions 2937.14 Two-tailed Rejection Regions 2957.15 Relative Merits of One- and Two-tailed Tests 2977.16 Interval Estimation and Hypothesis Testing 2997.17 Evidence and Change in Personal Probability 2997.18 Significance Tests and Common Sense 302Terms You Should Know 303Practice Exercises 303Exercises 305

Chapter 8

Inferences About Population Means 3118.1 Large-sample Problems with Unknown Population Variance 3118.2 The Distribution of t 3138.3 The t and the Standardized Normal Distribution 3168.4 Tables of the t Distribution 3178.5 The Concept of Degrees of Freedom 3188.6 Significance Tests and Confidence Limits for Means Using the t

Distribution 3198.7 Questions about Differences Between Population Means 3218.8 An Example of a Large-sample Significance Test for a Difference Between

Means 3238.9 Using the t Distribution to Test Hypotheses about Differences 3258.10 The Importance of the Assumptions in a t Test of a Difference Between

Means 3278.11 The Power of t Tests 3288.12 Strength of Association 3318.13 Strength of Association, Power, and Sample Size 3338.14 Strength of Association and Significance 3358.15 Estimating the Strength of a Statistical Association from Data 3368.16 Paired Observations 3388.17 Comparing More Than Two Means 340Terms You Should Know 342Practice Exercises 342

Xii CONTENTS

Exercises 345

Chapter 9

The Chi-Square and the F Distributions 350

9.0 Overview 3509.1 The Chi-Square Distribution 3519.2 Tables of the Chi-Square Distribution 3549.3 The Distribution of the Sample Variance from a Normal Population 3559.4 Testing Exact Hypotheses about a Single Variance 3569.5 Confidence Intervals for the Variance and Standard Deviation 3579.6 The Normal Approximation to the Chi-Square Distribution 3589.7 The Importance of the Normality Assumption in Inferences about

Variances 3599.8 The F Distribution 3609.9 Use of F Tables 3619.10 Using the F Distribution to Test Hypotheses about Two Variances 3629.11 Relationships among the Theoretical Distributions 3639.12 A Preview of the Pearson Chi-Square: An Important Special Use of the

Chi-Square Distribution 366Terms You Should Know 370Practice Exercises 370Exercises 373

Chapter 10

The General Linear Model and the Analysis of Variance 376

10.1 Linear Models 37710.2 A General Linear Model 37810.3 Analysis of Variance Models 38010.4 Least Squares and the Idea of an Effect 38110.5 Population Effects 38310.6 Model I: Fixed Effects 38510.7 Data Generated by a Fixed-Effects Model 38610.8 Partition of the Sum of Squares for Any Set of J Distinct Samples 38810.9 Assumptions Underlying Inferences about Treatment Effects 39010.10 Mean Square Between Groups 39110.11 Mean Square Within Groups 39310.12 The F Test in the Analysis of Variance 39410.13 Computational Forms for One-Way Analysis of Variance 39610.14 A Computational Outline for Analysis of Variance 39710.15 The Analysis of Variance Summary Table 39810.16 An Example of Simple, One-Way Analysis of Variance 39910.17 The Descriptive Statistics of Analysis of Variance 40110.18 Example with Unequal Groups 40310.19 The F Test and the t Test 405

CONTENTS XIII

10.20 The Importance of the Assumptions in the Fixed-Effects Model 40610.21 Strength of Association and the Power of F Tests in Analysis of Variance 40810.22 A Brief Look at Effect Size as Used in Meta-Analysis 410Terms You Should Know 412Practice Exercises 412Exercises 416

Chapter 11

Comparisons Among Means 42311.1 Asking Specific Questions of Data 42411.2 Planned Comparisons 42611.3 Statistical Properties of Comparisons 42811.4 Tests and Interval Estimates for Planned Comparisons 43111.5 Independence of Planned Comparisons 43311.6 Illustration of Independent and Nonindependent Planned Comparisons 43611.7 Independence of Sample Comparisons and the Grand Mean 43811.8 The Number of Possible Independent Comparisons 43911.9 Planned Comparisons and the Analysis of Variance 44011.10 Pooling the Sums of Squares for "Other Comparisons" 44311.11 Complete Example Using Planned Comparisons 44411.12 The Choice of the Planned Comparisons 44711.13 Error Rates: Per Comparison and Familywise 44911.14 Using Bonferroni Tests for Comparisons 45111.15 Incidental or Post-Hoc Comparisons in Data 45411.16 Scheffe Comparisons 45511.17 Pair Comparisons Among Means 45811.18 Planned or Post-Hoc Comparisons? 462Terms You Should Know 463Practice Exercises 463Exercises 467

Chapter 12

Factorial Designs and Higher-Order Analysis of Variance 47212.1 Factorial Designs 47212.2 Effects in a Factorial Design 47512.3 Partition of the Sum of Squares for a Two-Factor Design 48012.4 Population Effects for a Two-Way Analysis 48312.5 The Mean Squares and Their Expectations 48512.6 Computational Forms for a Two-Way Analysis of Variance 48812.7 An Example 49112.8 The Interpretation of Interaction Effects 49412.9 Proportion of Variance Accounted for in Two-Factor Experiments 49812.10 Planned Comparisons and Tests in Factorial Designs 50012.11 Post-Hoc Tests in a Factorial Design 504

XiV CONTENTS

12.12 Three-Factor and Higher Analysis of Variance 50512.13 General Algorithm for Analyzing Designs with Many Factors 51112.14 Examining and Interpreting Second-Order and Higher Interactions 51312.15 Assumptions in Two-Way (or Higher) Analysis of Variance with Fixed

Effects 51612.16 The Analysis of Variance as a Summary of Data 517Terms You Should Know 519Practice Exercises 519Exercises 522

Chapter 13

Analysis of Variance Models II and III: Random Effects andMixed Models 527

13.1 Randomly Selected Treatment Levels 52713.2 Random Effects and Model II 52813.3 Mean Squares for Model II 53013.4 An Example 53213.5 Estimation of Variance Components in a One-Way Analysis 53313.6 Testing Other Hypotheses and Calculating Power Under Model II 53713.7 Importance of the Assumptions in Model II 53813.8 Two-Factor Experiments with Sampling of Levels 53913.9 Model II for Two-Factor Experiments 54013.10 Mean Squares ' 54013.11 Hypothesis Testing in the Two-Way Analysis under Model II 54313.12 An Example of a Two-Factor Model II Experiment 54613.13 Estimation of Variance Components 54813.14 Model III: A Mixed Model 34913.15 Expected Mean Squares in a Mixed Model 55013.16 An Example Fitting Model III 55413.17 Randomization and Controls 55613.18 Randomized Blocks Designs 55713.19 Additivity and Generalized Randomized Blocks 56313.20 Some Pros and Cons of Randomized Blocks Designs 56613.21 Repeated Measures (or Within-Subjects) Experiments 56713.22 Special Assumptions in Randomized Blocks and Repeated Measures

Designs 57113.23 The Box Adjustment to Degrees of Freedom 57513.24 An Example of the Three-Step Procedure with Box Adjustment in

Repeated Measures 57713.25 Post-Hoc Tests of Means of a Within-Blocks or Within-Subjects Factor in

a Randomized Blocks or Repeated Measures Design 57913.26 The General Problem of Experimental Design 583Terms You Should Know 585Practice Exercises 585

CONTENTS XV

Exercises

Chapter 14Problems in Regression and Correlation14.1 Simple Linear Relations14.2 The Regression Equation for Predicting Y from X14.3 The Standard Error of Estimate for Raw Scores14.4 The Regression Equation for Standardized Scores14.5 Some Properties of the Correlation Coefficient in a Sample14.6 The Proportion of Variance Accounted for by a Linear Relationship14.7 The Idea of Regression Toward the Mean14.8 The Regression of zx on zy

14.9 Computational Forms for r^ and by,x14.10 Problems Encountered in Calculating Regression and Correlation

Coefficients for Sample Data14.11 An Example of a "Classic" Regression Problem14.12 Population Regression14.13 Assumptions in a Regression Problem14.14 Interval Estimation in a Regression Problem14.15 A Test for the Regression Coefficient14.16 Regression Problems in Analysis of Variance Format .14.17 Another Problem in Regression14.18 Errors in the Predictor Variable X14.19 Population Correlation14.20 Correlation in Bivariate and Multivariate Normal Populations14.21 Inferences about Correlations14.22 Confidence Intervals for p^,14.23 Other Inferences in Correlation Problems14.24 Examining Residuals14.25 An ExampleTerms You Should KnowPractice ExercisesExercises

Chapter 15

Partial and Multiple Regression15.1 Questions Involving More Than Two Variables15.2 Partial Correlations15.3 Higher-Order Partial Correlations15.4 Part or Semipartial Correlations15.5 Explaining Variance Through Part and Partial Correlations15.6 Inferences about Partial and Part Correlations15.7 Testing Significance for Intercorrelations15.8 Multiple Regression l

589

597

598599606608611613615616618

620622625627628630631637640644645647651652652655660660664

673

673674679680682684685687

XVI CONTENTS

Multiple Regression ModelsStandardized Multiple Regression EquationsFinding the Standardized Regression WeightsA Simple Example of Multiple RegressionMultiple Regression Equations for Raw ScoresThe Coefficient of Multiple CorrelationThe Proportion of Variance Accounted For and the Standard Error of

Multiple EstimateAn Example with Three PredictorsMultiple Regression Summarized in Analysis of Variance FormatInferences in Multiple Regression ProblemsThe F Test for Multiple RegressionSome Useful NotationTests and Interval Estimates for Regression CoefficientsThe Fisher Transformation for a Multiple CorrelationMultiple Regression in Terms of Part and Partial CorrelationsAn Important Special Case: Uncorrelated PredictorsIncrements in Predictive Ability Through Addition of More VariablesSome Hazards of Multiple Regression AnalysisThe Sweep-out Method for Multiple RegressionHierarchical and Stepwise RegressionSignificance as a Criterion for InclusionBackward Elimination of Variables

15.915.1015.1115.1215.1315.1415.15

15.1615.1715.1815.1915.2015.2115.2215.2315.2415.2515.2615.2715.2815.2915.30Terms You Should KnowPractice ExercisesExercises

Chapter 16

Further Topics in Regression16.1 Multiple Regression with Dummy Variables16.2 Example of Simple Analysis of Variance in Multiple Regression Format16.3 Orthogonal Comparisons and Multiple Regression16.4 An Example of a Factorial Experiment Analyzed Through Multiple

Regression16.5 Handling Unbalanced Data Through Multiple Regression Methods: Method

H, the Hierarchical Approach16.6 Handling Unbalanced Designs Through Multiple Regression: Method R,

the Regression Approach16.7 Two-way (or Higher) Fixed-effects Regression Analysis16.8 An Experiment with Two Quantitative Factors16.9 Curvilinear Regression16.10 Model for Linear and Curvilinear Regression16.11 Another Look at the Partition; of the Sum of Squares for Regression

Problems16.12 Example of Tests for Linear and Curvilinear Regression

688689690692694696

698700703705707708708711713716717721724731735737738738741

745

745748751

755

758

763766768770771

774776

CONTENTS XVM

16.13 Planned Comparisons for Trend: Orthogonal Polynomials 77816.14 Example of Planned Comparisons for Trend 78216.15 Estimation of a Curvilinear Prediction Function 78316.16 Looking at Curvilinear Regression in Other Ways 78716.17 Trend Analysis in Post-Hoc Comparisons 78916.18 Trend Analysis in Multifactor Designs 78916.19 An Example of a Two-factor Design with Trend Analysis 79416.20 Analyzing Trends in Repeated Measures Designs 79916.21 Computer Programs for Unbalanced Designs 800Terms You Should Know 801Practice Exercises 801Exercises 805

Chapter 17

The Analysis of Covariance 81017.1 The Analysis of Covariance as Statistical Control 81017.2 Partitioning Sums of Squares and Sums of Products 81217.3 Finding the Adjusted Sums of Squares 81417.4 Computations in the Analysis of Covariance 81617.5 A Simple Example of Covariance Analysis . 81817.6 Testing for Homogeneity of Regression 82017.7 Calculating Adjusted Means for the Groups 82317.8 Post-Hoc Comparisons in Analysis of Covariance 82417.9 Use of Analysis of Covariance in Place of Difference Scores 82517.10 Analysis of Covariance for a Factorial Design 82617.11 Finding Adjusted Means Following Factorial Analysis of Covariance 82817.12 Example of Analysis of Covariance in a Two-factor Design 82817.13 Analysis of Covariance with Two or More Covariates 83217.14 An Example with Two Covariates 83317.15 Assumptions and Problems in Analysis of Covariance 83617.16 Alternatives to Analysis of Covariance 838Terms You Should Know 838Practice Exercises 838Exercises 842

Chapter 18

Analyzing Qualitative Data: Chi-Square Tests 84818.1 Comparing Sample and Population Distributions: Goodness of Fit 84918.2 A Special Situation: Goodness of Fit for a Normal or Other Theoretical

Distribution 85318.3 Pearson Chi-Square Tests of Assocation 85618.4 The Special Case of a Fourfold Table 86118.5 Assumptions in Pearson Chi-Square Tests of Association 86218.6 The Possibility of Exact Tests for Association 863

xviii CONTENTS

18.7 A Test for Correlated Proportions in a Two-by-two Table 86518.8 Measures of Association in Contingency Tables 86618.9 A Measure of Predictive Association for Categorical Data 87018.10 Chi-Square Methods and Multiple Regression . 87218.11 Chi-Square and Multiple Regression in a J x K Table 87718.12 Partitioning a J x K Table into (J - I)(K - I) Distinct "Shadow Tables" 88118.13 A Measure of Interjudge Agreement 88518.14 Analyzing Larger Tables Through Log-Linear Models 887Terms You Should Know 893Practice Exercises 893Exercises 895

Appendix A: Rules of Summation 901Appendix B: The Algebra of Expectations 912Appendix C: Joint Random Variables and Linear Combinations 921Appendix D: Some Principles and Applications of Matrix Algebra 945Appendix E: Sets and Functions 973Appendix F: Tables 1005Appendix G: Solutions to Selected Exercises 1041

References and Suggestions for Further Reading 1030

Glossary of Symbols 1093

Index 1103