Kolmogorov and Probability Theory

7/27/2019 Kolmogorov and Probability Theory

1/13

^ r b o rKolmogorov and Probabi l i ty TheoryDavid Nualart

607

Arbor CLXXVIII, 704 (Agosto 2004), 607-619 pp.

1 F ou nda t i ons of P r o ba b i l i t yThere is no doubt that the most famous and influential work by AndreiNikolaevitch Kolmogorov (1903-1987) is a monograph of around 60 pagespublished in 1933 by Springer in a collection of texts devoted to the moderntheo ry of proba bility [16]. This mono graph changed the charac ter of thecalculus of probability, moving it from a collection of calculations into amathematical theory.The basic elements of Kolmogorov^s formulation are the notion of probability space associated with a given random experience and the notion ofrandom variable. These notions were formulated in the context of measuretheory. More precisely, a proba bility space is a measure space with to tal massequal to one and a random variable is a real-valued measurable function:

(i) A probability space ( 0 , ^ , F ) is a triple formed by a set 0 , a cr-fieldof subse ts of O, denoted by ^ , and a mea sure P on the measurablespace (0,J^) such that P(0) = 1. The set ft has no s tructure andrepresents the set of all possible outcomes of the random experience.The elements of J^ are the events related with the experience, andfor any event A E J^, P{^) is a number in [0,1] which represents itsprobability. Th e set fi was also called by Kolmogorov the space ofelementary events.

(ii) A random variable is a map pin g X : fi M such t h at for an y realnumber a, the set {a; G fi : X{u) < a} is an event, that is, it belongs

(c) Consejo Superior de Investigaciones Cientficas

Licencia Creative Commons 3.0 Espaa (by-nc)

http://arbor.revistas.csic.es


2/13

David Nualart608to the cr-field J^ . A random variable X induces a probability in theBorei a-eld of the real line denoted by Px and given by

Px{B) = P{X-HB))for any Borei subet B of the real line. The probability Px is called thelaw or distribution of the random variable X.

The Russian t ranslat ion of Kolmogorov's monograph appeared in 1936,and the first Eng hsh version was published in 1950: Foundations of theTheory of Probability. The delay in the English translat ion shows tha t t heformulation proposed by Kolmogorov was not immed iately accepted. Th isfact may seem surprising in view of the noncontroversial na tur e of Kolmogorov's approach and its great influence in the development of probabilitytheory. In addition, Kolm ogorov's axioms were mo re practic al and usefulthan other formalizations of probability, l ike the theory of "collectives" introduced by von Mises in 1919 (see [19]). Von Mises attempted to formalizethe typical properties of a sequence obtained by sampling a sequence of independent random variables with a common dis tr ibut ion. Although this isan appealing conceptual problem, this construction is too awkward and limited to provide a basis for mo dern pro bability theory. So, in spite of someobjections on Kolmogorov's approach that appeared at the beginning, it wasdefinitely adopted by the young generation of probabilists of the fifties, andmea sure theo ry was proved to be a fruitful and powerful tool to describe theprobability of events related to a random experience.One of the main features of Kolmogorov's formulation is to provide aprecise probability space for each random experience, and this permitted toeliminate the ambiguity caused by the multiple paradoxes in the calculus ofprob ability like those of Be rtran d a nd B orei. As an illustration of the power ofhis formalism, Kolmogorov solves in his mo nogra ph BoreVs paradox abouta rando m po int on the sphere. Such a point is specified by its longitude6 G [0,7r), so th at 9 determines a complete meridian circle, and its latitude() G (7r,7r]. If we con dition b y the info rma tion th at th e poin t lies on aconcrete meridian {9 is fixed), its latitude is not uniform over (7r,7r], andit has the conditional density | |cos


3/13

Kolmogorov and Probability TheoryKolmogorov's construction of conditional probabilities using the techniquesof measure theory avoids these contradictions.The strength of Kolmogorov's monograph lies on the use of a totallyabstract framework, in particular, the set or possible outcomes O is notequipped with any topological s t ructure. This does not imply that in someparticular problems, like the convergence or probability laws, it is convenientto work on better spaces through the use of image measures. In that sense,Kolmogorov picks up the heritage of Borei who was the pioneer in the use ofmeasure theory and Lebesgue integral in dealing with probability problems.We will now describe some of the main contributions of Kolmogorov'smonograph:1.1 Co ns t ruc t io n o f a p ro ba b i l i ty on a n in f in i te p r od uc t o f spacesAt the beginning of the thirties, a great number of works of the Russian probability school were oriented to the study of stochastic processes in continuoustime. In this context, the following theorem proved by Kolmogorov providesa fundamental ingredient for the formalization of stochastic processes. Werecall that a stochastic process is a continuous family of random variables{ X ( ) , > 0 } .T h e o r e m 1 Consider a family of probability measures pt^^^^^^tn onW^, n>l,0 1, and each 0 < ii < < in; Pti,...,tn concides withthe image of P by the natural projection:

R [ 0 , + O O ) ^ j ^ n

In the above theore m an element function X : [0, +o o) ^M th at can be interpreted as a trajec tory of a given stochastic process. Asa consequence, the law of stochastic process {X(), > 0} is determined bythe marginal laws

609

PiXit^),...Mtn))-Po{X{ti),...,X{tn)) - 1





4/13

David Nualart6 1 0that can be chosen in an arbitrary way.As precedents of this theorem we can first mention the construction of aprobability on E^ as the product of a countable family of probabilities on thereal line, done by Danieli in 1919, corresponding to th e proba bility co ntext ofindependent trials, not necessarily with a common distribution. On the otherhand, using the techniques developed by Danieli, Wiener [22] constructed theprobability law of the Brownian motion on the space of continuous functions.

1.2 C on t ru c t io n o f c ond i t ion a l p rob a b i l i t i e sApplying the techniques of measure theory, Kolmogorov constructed the conditional probability by a random variable X. We present here this construction using the modern notation of conditional probabilities.We recall first the classical definition of the conditional probability of anevent C by an event D such tha t P{D) > 0:

P{C\D) = ^ ^ ~ ^ . (1)Suppose we are given an event A E J^ and a random variable X. We wouldlike to compute the conditional probability P{A\X = x). If the randomvariable X is continuous, the event {X = x} has proba bility zero and theconditional probability P{A\X = x) is not well-defined by formula (1). Theconditional probability P{A\X = x) should be a function /A^) defined onthe range of the rand om variable X , such th at for any Borei subset 5 C Mwith P{X eB)>0

P{A\X eB)= [ fA{X)dP{-\X e B). (2)The solution to this problem is obtained by choosing A as the Radon-Nikodym densi ty of the measure B -^ P{A f) X~^{B)), with respect to Px,that isIn fact, for any Bo rei set 5 C M

P{AnX-\B))= I fA{x)dPx{x)^ I l{xmA{X)dP ,JB Jn





5/13

Kolmogorov and Probability Theoryand, hence, dividing both members of this equality by P{X G B) we obtain(2). In part icular , li A = X~^{[a,h]), then /^ = l[a,6]- Using the modernlanguage of conditional expectation we can write

fA{X) = E{1A\X).

The use of measure theory allowed Kolmogorov to formulate in a rigorousway the conditioning by events of probability zero like {X = x}. Prom theabove definition, Kolmogorov proved all classical properties of conditionalprobabilities.1 . 3 T h e 0 - 1 l a wKolmogorov's precise definitions made it possible for him to prove the so-called 0-1 law. Con sider a sequenc e {Xn,n > 1} of independent randomvariables. For each n > 1 we deno te by Gn = a{Xn^Xn+i,...) th e cr-fieldgenerated by the random variables {X^, k > n}. The sequence of cr-fields Gnis decreasing and its intersection is called the asymptotic a-field:

Q=f]Gn.n> l

T h e o r e m 2 ( 0 - 1 L a w ) Any event in the asymptotic a-field G has probability zero or one.A simple proof of this result, using mo dern nota tion, is as follows. Let A GG J and suppose that P{A) > 0. For any n > 1, the cr-fields c r( X i, . . . , Xn)and Gn+i are independent . As a consequence, if B G cr (X i, . . . ,X ^) , the

events A and B are independent because A e G C Gn+i- Hence,

Therefore, the probabilities P{'\A) and P coinc ide on th e cr-field a{Xi,..., Xn)for each n > 1, and this implies that they coincide on the a-field generated by al l the random variables X^. So, F(A |A ) = P( A ), which impliesP{Ay = P{A) an d P{A) = 1.

6 1 1





6/13

David Nualart6 1 2T h e o r e m 4 ( T h r e e S e r i e s T h e o r e m ) Let {Xn,n > 1} be a sequence ofindependent random variables. For any constant K > Q define the truncatedsequence

Yn = Xnl{\Xn\ 0 almost surely,n^ nn> l

(c) Consejo Superior de Investigaciones Cient ficas




7/13

Kolmogorov and Probab i l i t y Theo ry

The condition on the convergence of the variances is optimal. In the caseof independent and identically distributed random variables, Kolmogorovprovides in [16] a definitive answer to the problem of finding necessary andsufficient conditions for the validity of the strong law of large numbers.T h e o r e m 6 ( S t r o n g L a w o f L a r g e N u m b e r s ) Let {Xn-, n > 1} 6e a se quence of independent random variables with the same distribution. Then,

SE{\Xi\) < oo = > ^ > E(Xi) almost surely,n \S I"(1X11) = 00 = ^ lims up = + 0 0 almost surely.

Suppose that {X^, n> 1} is a sequence of centered, independent randomvariables with the same distribution. Set Sn = Xi-\ l-X^, for each n > 1.The Strong Law of Large Numbers says thatn

almost surely. On the other hand, if E{Xl) < 00, the Central Limit Theoremasserts tha t % converges in distributio n to the norm al law Ar(0,cr^), wherecr^ = E{Xl), that is, for any real numbers a / , exp -77 dx .Taking into account these results, one may wonder about the asymptoticbehaviour of Sn as n tend s to infinity. Th e Law of Itera ted L ogarithm ,established by Khintchine in 1924 ([9]), precises this behavior:

limsup^ = an->oo V 2n log log nalmost surely. In 1929 Kolmogorov proved in [12] the following version of thelaw of iterated logarithm for non identically distributed random variables.T h e o r e m 7 Let { X ^ , n > 1} be a sequence of independent random variableswith zero mean and finite variance. Set Sn = Xi-\ h Xn, for each n > 1.Then,

l im sup = 1n- .oo ^/2Bn\0gl0g Bn

613





8/13

David Nualart6 1 4almost surely, where

n

k= land \Xn\ 0} with values in a state space Eis called a Markov process if for any s < t and any measurable set of statesAc E it holds that

P {Xt e A\Xr, 0


9/13

Kolmogorov and Probability Theory

called the transition probabilities of the process. They satisfy the so-calledChapman-Kolmogorov equat ion:P{s, X , t , A ) = / P ( s , X , u, dy)P{u, y , , A), (4)JE

for any s < u < t^ and they allow to describe all probabilistic propertiesof the process. Ch apm an has mentioned this equation in the work [4] onBrownian motion in 1928.Kolmogorov's approach to Markov process developed in [15] is purelyanalytic and the main goal is to find regularity conditions on the transition probabilities F(s, x, i , dy ) in order to handle the Chapman-Kolmogorovequation (4). The central ideal of Kolmogorov's paper is the introduction oflocal characteristics at t ime t and the construction of transition probabilitiesby solving certain differential eq uations involving these characteristics. Inthe case of real-valued processes (that is, E = R), Kolmogorov considers theclass of transition functions for which the following limits exist

A{t,x) = l i m - / ( y - x ) F ( i , x , t + 5 ,d y) ,B{t,x) = l i m / ( y - x f P ( , x , + ,d y ) .

dio Z J^Feller suggested the names drift and diffusion coefficients for these limits.A property on the third moments is also needed to exclude the possibilityof jumps. Assuming, in addition, that the density function of the measureP ( s , X, , ), denote d by

./ , N P{s,x,t,dy)/ ( 5 , X, , y) = '- ,dyis sufficiently smooth, Kolmogorov proved that it satisfies the forward differential equation: df _ d[A{t,y)f] d'[B\t,y)f]dt dy dy^ ^ '

and the backw ard differential equation:

6 1 5





10/13

David Nualart6 1 6Equation (5) arises if the study of the time evolution of the probability distribution of the process and a special form of this equation appeared earlierin papers of Fokker [8] and Planck [20]. Kolmogorov called Equation (5)the Fokker-Planck equation since 1934. When the coefl&cients depend onlyon time (processes homogeneous in space), these equations appeared first in1900 in a paper of Bachelier [1].The construction of transition functions from the drift and diffusion coefficients motivated the works on fundamental solutions to parabolic partialdifferential equations and were the starting point on the fruitful relationshipbetween Markov processes and parabolic equations.Although his point of view on the theory of stochastic processes wasmainly analytical, Kolmogorov also developed a certain number of tools forthe study of the properties of the paths of stochastic processes. Among thesetools, the most famous and most used is the criterion that guarantees thecontinuity of the trajectories of a given stochastic process from conditionson the moments of its increments. This criterion was proved by Kolmogorovin 1934 and presented in the Seminar of Moscow University. However, Kolmogorov never published this result, and it was Slutsky who stated and giveth e first proof in [21] in 1934, at tri bu tin g it to K olmog orov.Def in i t i on 9 We say that two stochastic processes {Xt^O < < 1} an d{^ ,0 < < 1} are equivalent (or that X is a version ofY) if for an y t,P{Xt ^ Y t) = 0.

Two equivalent processes may have different trajectories.T h e o r e m 1 0 ( K o l m o g o r o v c o n t i n u i t y c r i t e r i o n ) Consider a stochasticprocess {Xt^O < < 1}. Suppose that there exists constants p, c, e > 0 suchthat Ei\Xt-Xsn


11/13

Kolmogorov and Probability TheoryAs an example of the application of this theorem, consider the case ofthe Brownian motion {Bt,0 < < 1}, defined as a Gaussian process (its finite dimensional distributions are Gaussian) with zero mean and covariancefunction E{BtBs) = m in (s, i ) . This process has s ta t ionary and independent increments and the law of an increment Bt Bg is N{0,t s) . As aconsequence, for any integer k >1

and there is a version of the Brownian motion with Holder continuous trajectories of order a, for any a < |.4 C o n c l u s i o n sA ) Kolmogorov may be considered as the founder of proba bility theory. Th emonograph by Kolmogorov published in 1933 transformed the calculusof probabil i ty into a mathem atical discipline. Some authors com parethis role of Kolmogorov with the role played by Euclides in geometry.B) The results on limit theorems for sequences and series of independentrandom variables established by Kolmogorov were definitive and constitute a basic core of results on any text course in probability.C) Kolmogorov ideas influenced decisively almost all the work on Markovprocesses and make possible the posterior development of stochasticanalysis.

R e f e r e n c e s[1] L. Bachelier. Thorie de la spculation. Ann. Sci. Ecole Norm. Sup. 17,21-86 (1900).[2] P. Billingsley. Probability and Measure^ John Wiley 1979.[3] L. Chaumont, L. Mazliak and M. Yor: A. N. Kolmogorov. Quelquesaspects de Voeuvre probabiliste. In "L'hritage de Kolmogorov enmathmatiques". Ed. Berlin, collection Echelles, 2003.

6 1 7





12/13

6 1 8 David Nualart

[4] S. Chapman. On the Brownian displacement and thermal diffusion ofgrains suspended in a non-uniform fluid. Proc. Roy. Soc. London, Ser.A 119 , 34-54 (1928).[5] J. L. Doob. Kolmogorov's early work on convergence theory and foundat ions. Ann. Probab. 17, 815-821 (1989).[6] E. B. Dynkin. Kolmogorov and the theory of Markov processes. Ann.Probab. 17, 822-832 (1989).[7] A. Einstein. Zur Thorie des Brownschen Bewegung. Ann. Physik 19,371-381 (1906).[8] A. D. Fokker. Die mittlere Energie rotierende elektrischer Dipole imStrahlungsfeld. Ann. Physik 4 3 810-820 (1914).[9] A. Y. Khinchin. U ber einen Satz der W ahrscheinlichkeitrechnung. Fund.Mat. 6, 9-20 (1924).

[10] A. Y. Khinchin et A. N. Kolmogorov. Uber Konvergenz von Reihen,deren Glieder durch den Zufall bestimmt werden. Mat Sb. 32, 668-677(1925).[11] A. N. Kolmogorov. Uber dir Summen durch den Zufall bestimmter un-abhngiger Grssen. Math. Ann. 99 , 309-319 (1928).[12] A. N. Kolmogorov. Uber das Gesetz des iterierten Logarithmus. Math.Ann. 1 0 1 , 126-135 (1929).[13] A. N. Kolmogorov. Bermerkungen zu meiner Arbeit "Uber die Summenzufalliger Grssen". Math. Ann. 100 , 4844-488 (1930).[14] A. N. Kolmogorov. Sur la loi forte des grands nombres. CRAS Paris

191 , 910-912 (1930).[15] A. N. Kolmogorov. Uber die analytischen Methoden in der Wahrschein-lichkeitsrechnung. Math.Ann. 104 , 415-458 (1931).[16] A. N. Kolmogorov. Grundbegriffe der Wahrscheinlichkeitsrechnung,Springer 1933.





13/13

Kolmogorov and Probability Theory[17] A. A. Markov. Extension of the law of large numbers to independentevents. Bull. Soc. Phys. Math. Kazan 15, 135-156 (1906). (In Russian).[18] A. A. Markov. Extensio n of limit theorem s of proba bility theo ry to a sumof variables connected in a chain. Zap. Akad. Nauk. Fiz.-Mat. Otdel,Ser. VIII 22. Presented Dec. 5 1907.[19] R. von Mises. Grundlag en der W ahrscheinlichkeitsrechnung. Math. Z. 5,

52-99 (1919).[20] M. Plank. Uber einen Satz der statistischen Dynamik und seine Er-weiterung in der Quantentheorie . Sitzungsber. Preuss. Akad. Wiss.Phys.-Math. Kl. 324-341 (1917).[21] E. B. Slutsky. Qualche proposizione relativa alla teoria delle funzionialeatorie. Giorna le deW Instituto Italiano dei Attuari 8, 183-199 (1937).[22] N. Wiener. Differential space. Jour. Math, and Phys. 58, 131-174 (1923).

6 1 9




Kolmogorov and Probability Theory

Documents

Transcript of Kolmogorov and Probability Theory