Peter Whittle

Probability via Expectation Fourth Edition

With 22 Illustrations

Springer

Contents

Preface to the Fourth Edition vii Preface to the Third Edition ix Preface to the Russian Edition of Probability (1982) xiii Preface to Probability (1970) xv

1 Uncertainty, Intuition, and Expectation 1 1 Ideas and Examples 1 2 The Empirical Basis 3 3 Averages over a Finite Population 5 4 Repeated Sampling: Expectation 8 5 More on Sample Spaces and Variables 10 6 Ideal and Actual Experiments: Observables 11

2 Expectation 13 1 Random Variables 13 2 Axioms for the Expectation Operator 14 3 Events: Probability 17 4 Some Examples of an Expectation 18 5 Moments 21 6 Applications: Optimization Problems 22 7 Equiprobable Outcomes: Sample Surveys 24 8 Applications: Least Square Estimation of Random Variables . . . 28 9 Some Implications of the Axioms 32

xviii Contents

3 Probability 39 1 Events, Sets and Indicators 39 2 Probability Measure 43 3 Expectation as a Probability Integral 46 4 SomeHistory 47 5 Subjective Probability 49

4 Some Basic Models 51 1 A Model ofSpatial Distribution 51 2 The Multinonüal, Binomial, Poisson and Geometrie

Distributions 54 3 Independence 58 4 Probability Generating Functions 61 5 The St. Petersburg Paradox 66 6 Matching, and Other Combinatorial Problems 68 7 Conditioning 71 8 Variables on the Continuum: The Exponential and

Gamma Distributions 76

5 Conditioning 80 1 Conditional Expectation 80 2 Conditional Probability 84 3 A Conditional Expectation as a Random Variable 88 4 Conditioning ona er-Field 92 5 Independence 93 6 Statistical Decision Theory 95 7 Information Transmission 97 8 Acceptance Sampling 99

6 Applications of the Independence Concept 102 1 Renewal Processes 102 2 Recurrent Events: Regeneration Points 107 3 A Result in Statistical Mechanics: The Gibbs

Distribution 111 4 Brandung Processes 115

7 The Two Basic Limit Theorems 121 1 Convergence in Distribution (Weak Convergence) 121 2 PropertiesoftheCharacteristic Function 124 3 TheLawofLargeNumbers 129 4 Normal Convergence (the Central Limit Theorem) 130 5 The Normal Distribution 132 6 The Law of Large Numbers and the Evaluation

of Channel Capacity 138

r\

Contents xix

8 Continuous Random Variables and Their Transformations 141 1 Distributions with a Density 141 2 Functions of Random Variables 144 3 Conditional Densities 148

9 Markov Processes in Discrete Time 150 1 Stochastic Processes and the Markov Property 150 2 The Case of a Discrete State Space: The Kolmogorov

Equations 156 3 Some Examples: Ruin, Survival and Runs 162 4 Birth and Death Processes: Detailed Balance 165 5 Some Examples We Should Like to Defer 167 6 Random Walks, Random Stopping and Ruin 168 7 Auguries of Martingales 174 8 Recurrence and Equilibrium 175 9 Recurrence and Dimension 179

10 Markov Processes in Continuous Time 182 1 The Markov Property in Continuous Time 182 2 The Case ofa Discrete State Space 183 3 The Poisson Process 186 4 Birth and Death Processes 187 5 Processes on Nondiscrete State Spaces 192 6 The Filing Problem 195 7 Some Continuous-Time Martingales 196 8 Stationarity and Reversibility 197 9 The Ehrenfest Model 200 10 Processes of Independent Increments 203 11 Brownian Motion: Diffusion Processes 207 12 First Passage and Recurrence for Brownian Motion 211

11 Action Optimisation; Dynamic Programming 215 1 Action Optimisation 215 2 Optimisation over Time: the Dynamic Programming Equation . . 216 3 State Structure 217 4 Optimal Control Under LQG Assumptions 220 5 Minimal-Length Coding 221 6 Discounting 223 7 Continuous-Time Versions and Infinite-Horizon Limits 225 8 Policy Improvement 227

12 Optimal Resource Allocation 229 1 Portfolio Selection in Discrete Time 229 2 Portfolio Selection in Continuous Time 232

xx Contents

3 Multi-Armed Bandits and the Gittins Index 232 4 Open Processes 236 5 Tax Problems 238

13 Finance: 'Risk-Free' Trading and Option Pricing 241 1 Options and Hedging Strategies 241 2 Optimal Targeting of the Contract 243 3 An Example 245 4 A Continuous-Time Model 246 5 HowS/ioj/WitBeDone? 248

14 Second-Order Theory 253 1 Backte L2 253 2 Linear Least Square Approximation 256 3 Projection: Innovation 257 4 The Gauss-Markov Theorem 260 5 The Convergence of Linear Least Square Estimates 262 6 Direct and Mutual Mean Square Convergence 264 7 Conditional Expectations as Least Square Estimates:

Martingale Convergence 266

15 Consistency and Extension: The Finite-Dimensional Case 268 1 Thelssues 268 2 ConvexSets 269 3 The Consistency Condition for Expectation Values 274 4 The Extension of Expectation Values 275 5 Examples of Extension 277 6 Dependence Information: Chernoff Bounds 280

16 Stochastic Convergence 282 1 The Characterization of Convergence 282 2 Types of Convergence 284 3 Some Consequences 286 4 Convergence in rth Mean 287

17 Martingales 290 1 The Martingale Property 290 2 Kolmogorov's Inequality: the Law of Large Numbers 294 3 Martingale Convergence: Applications 298 4 The Optional Stopping Theorem 301 5 Examples of Stopped Martingales . . . . , 303

18 Large-Deviation Theory 306 1 The Large-Deviation Property 306 2 Some Preliminaries 307 3 Cramer's Theorem 309

Contents xxi

4 Some Special Cases 310 5 Circuit-Switched Networks and Boltzmann Staüstics 311 6 Multi-Class Traffic and Effective Bandwidth 313 7 Birth and Death Processes 314

19 Extension: Examples of the Infinite-Dimensional Case 317 1 Generalities on the Infinite-Dimensional Case 317 2 Fields and or-Fieldsof Events 318 3 Extension on a Linear Lattice 319 4 Integrable Functions of a Scalar Random Variable 322 5 Expectations Derivable from the Characteristic Function:

Weak Convergence 324

20 Quantum Mechanics 329 1 The Static Case 329 2 The Dynamic Case 335

References 341

Index 345