utda

llas.edu /~m

etinPage 1Outcomes, Counting, Measures and Probability

Outline Experiments, Outcomes, Events Counting

– Finite Outcomes» Number of distinct strings, sequences, subsets and multisets

– Infinite but Countable Outcomes– Uncountable Outcomes

Probability

Prof. Metin Çakanyıldırım used various resources to prepare this document for teaching/training. To use this in your own course/training, please obtain permission from Prof. Çakanyıldırım.

If you find any inaccuracies, please contact metin@utdallas.edu for corrections.Updated in Fall 2019

utda

llas.edu /~m

etinPage 2Probability

Why Probability? – Limited information about experiments / events caused by … – Information collection

» Costly: Surveying all customers, voters, » Inaccurate: Measuring devices for speed, humidity, » Impossible: Location of an electron around the nucleus, causes of interior tumors

– Some uncertainty remains and is incorporated into models Probability or Statistics? – Both and interactively.

Real life

StatisticsDescription

ProbabilityPrescription

Observations

Decisions

EstimatesModels

utda

llas.edu /~m

etinPage 3Experiments, Outcomes, Events, Sample Space Experiment Action, activity, experiment, happening Outcomes Results of the experiment, 𝜔𝜔. Events Collection of some outcomes, 𝐴𝐴,𝐵𝐵. Sample space Collection of all outcomes, Ω.

Ex: Financial Accounting Standards Board (fasb.org) suggests reporting requirements of financial activities. – In the next meeting,

» it can Keep (K) the status quo of the requirements, » Increase (I) the requirements or » Decrease (D) them.

– Experiment = the next meeting; Outcomes = {K, I, D}; – Some events are {K,I}, {D}, {I,D}, {K,I,D}; sample space is Ω ={K,I,D}.

Ex: 3 traffic lights on the route to the office tomorrow morning

Disjoint events have no common outcome.

utda

llas.edu /~m

etinPage 4Example of Rolling a Dice Twice

Consider the product & sum of the numbers on the two rolls.

C is intersection of A and B is empty: 𝐶𝐶 = 𝐴𝐴 ∩ 𝐵𝐵 = ∅. Sample space is union of A, B and D: Ω = 𝐴𝐴 ∪ 𝐵𝐵 ∪ 𝐷𝐷.

(product, sum) ∈ eventSecond roll

1 2 3 4 5 6

First roll

1 (1,2)∈A (2,3)∈B (3,4)∈A (4,5)∈B (5,6)∈A (6,7)∈B

2 (2,3)∈B (4,4)∈D (6,5)∈B (8,6)∈D (10,7)∈B (12,8)∈D

3 (3,4)∈A (6,5)∈B (9,6)∈A (12,7)∈B (15,8)∈A (18,9)∈B

4 (4,5)∈B (8,6)∈D (12,7)∈B (16,8)∈D (20,9)∈B (24,10)∈D

5 (5,6)∈A (10,7)∈B (15,8)∈A (20,9)∈B (25,10)∈A (30,11)∈B

6 (6,7)∈B (12,8)∈D (18,9)∈B (24,10)∈D (30,11)∈B (36,12)∈D

Events: A={product is odd}, B={sum is odd}, C={product & sum are odd}, D={product & sum are even}

Product odd ⇒ Each odd ⇒ Sum even, hence product and sum are never odd togetherProduct either odd or even. Product odd ⇒ Sum even. Product even ⇒ Sum either odd or even

- Dice is plural of singular die. - But many use dice as singular.

A B D

Ω

utda

llas.edu /~m

etinPage 5

Outcome of a Coin Flip ExperimentRandom or Not?

Flipped with a vertical speed of v|

Stays in the air for 𝑡𝑡𝑎𝑎𝑎𝑎𝑟𝑟 = 2 𝑣𝑣|𝑔𝑔 seconds

– 𝑔𝑔 is the gravitational acceleration constant. Has the angular speed of 𝑣𝑣𝑜𝑜 revolutions/second.

– Then the coin makes 𝑣𝑣𝑜𝑜𝑡𝑡𝑎𝑎𝑎𝑎𝑟𝑟 revolutions in the air.

If the coin has head (𝐻𝐻) up & makes 1 revolution in the air, it comes down as head up.

– If it makes 14 , 34 revolutions in the air, it comes down as tail (𝑇𝑇).

– In general, it comes down 𝑇𝑇 after 𝑘𝑘+ 14 , 𝑘𝑘 + 3

4 revolutions for 𝑘𝑘 = 0,1,2 ….

» 𝑣𝑣𝑜𝑜𝑡𝑡𝑎𝑎𝑎𝑎𝑟𝑟 ∈ 𝑘𝑘 + 14 , 𝑘𝑘 + 3

4 for 𝑘𝑘 = 0,1,2 … guarantees an outcome of 𝑇𝑇

» 𝑣𝑣𝑜𝑜𝑣𝑣|𝑔𝑔 ∈

12 𝑘𝑘 + 1

4 , 𝑘𝑘 + 34 = 4𝑘𝑘+1

8 , 4𝑘𝑘+38 guarantees 𝐻𝐻 → 𝑇𝑇.

– 𝑣𝑣𝑜𝑜𝑣𝑣| ∈ 1 𝑔𝑔8 , 3 𝑔𝑔

8 ∪ 5 𝑔𝑔8 , 7 𝑔𝑔

8 ∪ 9 𝑔𝑔8 , 11 𝑔𝑔

8 … guarantees 𝐻𝐻 → 𝑇𝑇: Switch

– 𝑣𝑣𝑜𝑜𝑣𝑣| ∈ 0 𝑔𝑔8 , 1 𝑔𝑔

8 ∪ 3 𝑔𝑔8 , 5 𝑔𝑔

8 ∪ 7 𝑔𝑔8 , 9 𝑔𝑔

8 … guarantees 𝐻𝐻 → 𝐻𝐻: Maintain

Push

𝑣𝑣𝑜𝑜𝑣𝑣|

Push

𝑣𝑣|

𝑔𝑔

2𝑣𝑣|

𝑔𝑔

utda

llas.edu /~m

etinPage 6Controlling the Outcome of a Coin Flip

If the speeds 𝑣𝑣| & 𝑣𝑣𝑜𝑜 can be adjusted by a well-calibrated thumb (or a machine under some ideal conditions), the outcome of the coin flip experiment is deterministic (and controllable).

Ang

ular

spe

ed

Time to the max height = Vertical speed/gravitational constant

Time to max heightis 1/4 second

Too long in the air, practically irrelevant

Usually time to max height is ¼ second, or 𝑡𝑡𝑎𝑎𝑎𝑎𝑟𝑟 = 1/2 seconds

– 𝑣𝑣𝑜𝑜𝑣𝑣|

𝑔𝑔∈ 12𝑘𝑘 + 1

4,𝑘𝑘 + 3

4becomes

– 𝑣𝑣𝑜𝑜 ∈ 2𝑘𝑘 + 12

, 2𝑘𝑘+ 32

yields 𝐻𝐻 → 𝑇𝑇

If a coin has 𝑘𝑘 = 20 revolutions– 𝑣𝑣𝑜𝑜 ∈ 40.5,41.5 guarantees 𝐻𝐻 → 𝑇𝑇– But 𝑣𝑣𝑜𝑜 ∈ 39.5,40.5 ∪ (41.5,42.5)

yields 𝐻𝐻 → 𝐻𝐻– Flip a coin with at most 2% variation

in angular speed?» Yes ⇒ the outcome is NOT random

utda

llas.edu /~m

etinPage 7Counting the Sample Space Counting is assigning natural numbers to objects

– Natural numbers 0, 1, 2, 3, 4, 5, …. are infinitely many. When we are assigning numbers to infinite objects, we are not going to run out of numbers as long as we can imagine a process (sequence, order) for assigning numbers.

A set is countable when its elements can be assigned to natural numbers.

Countable sets Uncountable setsFinite sets

Eg: Cars in …

Infinite butCountable setsEg: Natural numbers

Infinite andUncountable setsEg: ??

Ex: All sets with finite cardinality are countable. Eg, UTD students, leaves on a tree, hair strings on your head are all countable.

How many marbles are there in the jar? 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Counting: Assigning numbers to marbles

utda

llas.edu /~m

etinPage 8Finite Sets – 1. Multiplication Principle

Multiplication principle: When the outcome of an experiment is defined by – K characteristics that are independent of each other, use the multiplication principle.– Start by enumerating the number 𝑛𝑛𝑘𝑘 of ways characteristic 𝑘𝑘 ∈ [1:𝐾𝐾] can materialize. – Then the number of outcomes is 𝑛𝑛1𝑛𝑛2 …𝑛𝑛𝐾𝐾 .

Ex: An online dress retailer carries 3 styles of lady dresses: Night dress, Corporate dress and Sporty dress. Each style has 20 cuts, 8 sizes and 5 colors. A stock keeping unit (sku) for the online retailer is defined by the dress style, cut, size and color as these four characteristics fully describe the dress item and are used to buy dresses from the suppliers. The number of skus for this retailer is 3 × 20 × 8 × 5 = 2400.

If the characteristics are not all independent of each other, we can still use the multiplication principle with some adjustments.

Ex: After a market research study, the online dress retailer decides to customize its offerings. It offers 22 cuts of Night dresses, 18 cuts of Corporate dresses and 34 cuts of Sporty dresses. Night dresses need to fit more closely so they have 10 sizes, while Corporate and Sporty dresses have respectively 8 and 6 sizes. The number of skus become (22 × 10 + 18 × 8 + 34 × 6) × 5. The color is independent of other characteristics. Within each style, the cut and the size characteristics are independent.

Ex: How many of the four-digit integer numbers contain no repeated digits? – Digits are {0,1,2,3,…,8,9} but 0 cannot appear as the first digit in a four-digit integer. Because 0987 is not a

four-digit integer, it is a three-digit integer. 1. The first digit say 𝑎𝑎 can take one of the nine values from {1,2,3,…,8,9}. 2. The second digit 𝑏𝑏 can take one of the nine values from {0,1,2,3,…,8,9}\{𝑎𝑎}. 3. The third digit 𝑐𝑐 can take 8 values from from {0,1,2,3,…,8,9}\{𝑎𝑎,𝑏𝑏}. 4. The fourth digit 𝑑𝑑 can take 7 values from from {0,1,2,3,…,8,9}\{𝑎𝑎, 𝑏𝑏, 𝑐𝑐}. The answer is 9x9x8x7.

utda

llas.edu /~m

etinPage 9Finite Sets – 2. Permutations

Permutation of k objects out of n distinct objects is picking up k objects out of n objects and sequencing these k objects. Each sequence is a different permutation.

Objectn …….Object

n-1Object

n-2Object

n-3Objectn-k+2

Objectn-k+1

k objects picked out of n

Ex: A professor picks 5 chapters out of 10 chapters of a textbook and sequences these 5 chapters arbitrarily to teach them in his course. How many different sequences of 5 chapters (syllabi) can be made from 10 chapters? 𝑃𝑃510.

The number of permutation of k objects out of n distinct objects is n(n-1)(n-2)…(n-k+1)=:𝑃𝑃𝑘𝑘𝑛𝑛.

1: 2: 3: 4: 5: 6:

Ex: Two marbles are picked out of three marbles with colors green, blue and red. How many distinct sequences are possible with these two marbles: 𝑃𝑃23 =3(2).

Ex: A professor hands out 6 exams papers back to 6 students. What is the number of ways exam papers can be sequenced? 𝑃𝑃66.

…Objectn-k

Object1

Not picked

utda

llas.edu /~m

etinPage 10Finite Sets – 2. Permutations

Ex: A university network requires passwords that are strings made out as follows:– Lowercase letters {a,…,z} and uppercase letters {A,…,Z} and digits {0,1,2,…,9} can be used in a string. A total

of 52 distinct letters and 10 distinct digits.– The length of the string must be exactly 6. No letter or digit can be repeated.– At least 2 digits and at least 2 letters must be used in the string.

How many passwords are possible? [A]. If exactly 2 digits in the password and 4 letters. Number of digit strings is 𝑃𝑃210and number of letter strings 𝑃𝑃452.

Merging letter string abcd and digit string xy: 5 mergers: xyabcd; xaybcd; xabycd; xabcyd; xabcdy 4 mergers: axybcd; axbycd; axbcyd; axbcdy 3 mergers: abxycd; abxcyd; abxcdy 2 mergers: abcxyd; abcxdy 1 mergre: abcdxy

6!/(4!2!)=15 total mergers of 𝑃𝑃210digit strings and 𝑃𝑃452 letter strings: 15𝑃𝑃210 𝑃𝑃452. [B]. If exactly 3 digits in the password and 3 letters. Number of digit strings is 𝑃𝑃310and number of letter strings 𝑃𝑃352.

Merging letter string abc and digit string xyz: 4 mergers: xyzabc; xyazbc; xyabzc; xyabcz 3 mergers: xayzbc; xaybzc; xaybcz 2 mergers: xabyzc; xabycz 1 merger: xabcyz

3 mergers: axyzbc; axybzc; axybcz 2 mergers: axbyzc; axbycz 1 merge: axbcyz 2 mergers: abxyzc; abxycz 1 merge: abxcyz

1 merge: abcxyz

6!/(3!3!)=20 total merges of 𝑃𝑃310digit strings and 𝑃𝑃352 letter strings: 20𝑃𝑃310 𝑃𝑃352. [C]. If exactly 4 digits in the password and 2 letters. Numbers of mergers is the same as in [A]. 15𝑃𝑃410 𝑃𝑃252. The total number of passwords: 15𝑃𝑃210 𝑃𝑃452+20𝑃𝑃310 𝑃𝑃352+15𝑃𝑃410 𝑃𝑃252.

utda

llas.edu /~m

etinPage 11Finite Sets – 3. Combinations

Combination of k objects out of n distinct objects is picking up k objects out of n objects without paying attention to sequencing of these k objects.

𝑛𝑛objects ….𝑛𝑛 − 1

objects𝑛𝑛 − 2

objects𝑛𝑛 − 3

objects𝑛𝑛 − 𝑘𝑘 + 2

objects𝑛𝑛 − 𝑘𝑘 + 1

objects

k objects but their k! sequence does not matter

The number of combinations of k objects out of n distinct objects is 𝑃𝑃𝑘𝑘𝑛𝑛

𝑘𝑘! =:𝐶𝐶𝑘𝑘𝑛𝑛.– Sequence 1, 2, …, k! are all counted as one combination

Ex: A professor picks 5 chapters out of 10 chapters of a textbook to teach them in his course. How many different ways are there to pick 5 chapters from 10 textbook chapters? 𝐶𝐶510.

1: 1: 2: 2: 3: 3:

Ex: Two marbles are picked out of three marbles with colors green, blue and red. How many distinct sequences subsets are possible with these two marbles: 𝐶𝐶23 =3(2)/2=3.

The same subset The same subset The same subset

…Objectn-k

Object1

Not picked

The number of distinct subsets of size k that can be made from a set of size n is also 𝐶𝐶𝑘𝑘𝑛𝑛.

utda

llas.edu /~m

etinPage 12Finite Sets – 4. Combinations with Repeating 𝐶𝐶𝑘𝑘𝑛𝑛 is the number of subsets. Each subset can have each element at most once, so elements

cannot be repeated.

If the elements are repeated in a collection, the collection is called a multiset. In a multiset,

– the total number of elements, including repetitions, is the cardinality of the multiset – the number of times an element appears is the multiplicity of that element.

Ex: Suppose that we are to pick colors B, R, G to create multisets of cardinality 3. – <B,R,G> is the unique multiset whose elements do not repeat, so <B,R,G > is also a set. – As repetition is allowed in a multiset, some elements can be used twice or thrice while the others are not used at

all. If G is not used, we can still construct multisets <B,R,R>, <B,B,R>, <B,B,B>, <R,R,R>. – If G must be used, we can construct some other multisets <B,G,G>, <R,G,G>, <G,R,R>, <G,B,B>, <G,G,G>. – The multiplicity of B in <B,R,R> is 𝑥𝑥𝐵𝐵 = 1, while the multiplicity of B in <B,B,R> is 𝑥𝑥𝐵𝐵 = 2.

From members of a cardinality-𝑛𝑛 underlying set, how many cardinality k multisets can we create? Each multiset is uniquely identified by the multiplicities {𝑥𝑥1,𝑥𝑥2, … ,𝑥𝑥𝑛𝑛} of members of its

underlying set. – Since we set the cardinality equal to k, we need to insist on 𝑥𝑥1 + 𝑥𝑥2 + ⋯+ 𝑥𝑥𝑛𝑛 = 𝑘𝑘. – Also each multiplicity must be a natural number (non-negative integer). – The number of multisets with cardinality k = The number of natural number solutions to

𝑥𝑥1 + 𝑥𝑥2 +⋯+ 𝑥𝑥𝑛𝑛 = 𝑘𝑘.

utda

llas.edu /~m

etinPage 13Finite Sets – 4. Combinations with Repeating The number of natural number solutions to 𝑥𝑥1 + 𝑥𝑥2 + ⋯+ 𝑥𝑥𝑛𝑛 = 𝑘𝑘. First consider a seemingly different problem with the same number of solutions. Suppose that we have 𝑛𝑛 + 𝑘𝑘 − 1 objects denoted by “+”:

Encircle n-1 of these + objects to obtain exactly n segments containing no/some + objects. – If 𝑗𝑗 − 1st & 𝑗𝑗th encircled +s are next to each other ⇒ the 𝑗𝑗th segment has no + or 𝑥𝑥𝑗𝑗 = 0. – In general, 𝑥𝑥𝑗𝑗 is the number +s in the 𝑗𝑗th segment.

+ +1st 2nd

+ +3rd 4th

+ +𝑛𝑛 + 𝑘𝑘 − 2nd 𝑛𝑛 + 𝑘𝑘 − 1st

…...…...

+ +1st 𝑥𝑥1𝑡𝑡𝑡 1st circle

……....……....

…...…...

+ + +1st 𝑥𝑥2𝑛𝑛𝑑𝑑 2nd circle

…...…...

+ + +1st 𝑥𝑥𝑛𝑛𝑡𝑡𝑡n-1st circle

…...…...

+

1st segment, 𝑥𝑥1elements

2nd segment, 𝑥𝑥2 elements

nth segment, 𝑥𝑥𝑛𝑛 elements

The number of ways of encircling 𝑛𝑛 − 1 objects out of 𝑛𝑛 + 𝑘𝑘 − 1 objects is 𝐶𝐶𝑛𝑛−1𝑛𝑛+𝑘𝑘−1 = 𝐶𝐶𝑘𝑘𝑛𝑛+𝑘𝑘−1.

𝑥𝑥1 + 𝑥𝑥2 +⋯+ 𝑥𝑥𝑛𝑛 = 𝑘𝑘 “+”s remains

𝑘𝑘 + 𝑛𝑛 − 1 “+”s to begin with𝑛𝑛 − 1 “+”s circled

utda

llas.edu /~m

etinPage 14Finite Sets – Counting Summary

Number of ways of making cardinality k– permutations, – strings, – sets,– multisets

from n distinct objects.Sequence matters?

Yes No

Repeat?No 2. 𝑃𝑃𝑘𝑘𝑛𝑛 permutations 3. 𝐶𝐶𝑘𝑘𝑛𝑛 setsYes 1. 𝑛𝑛𝑘𝑘 strings 4. 𝐶𝐶𝑘𝑘𝑛𝑛+𝑘𝑘−1 multisets

Numbers 1 − 4 indicate order of discussion above

utda

llas.edu /~m

etinPage 15Infinite Sets – Countable Sets

Some experiments have an infinite sample space (set with cardinality of infinity).

Ex: Throwing a coin, rolling a dice and calling a call center are experiments that can be repeated. Each time they are performed, they generate outcomes: {Head, Tail} for throwing a coin, {1,2,3,4,5,6} for rolling a dice, {Busy, Available} for calling a call center.

– If we perform these experiments independently m times, the outcomes respectively become {H,T}m, {1,2,3,4,5,6}m, {B, A}m.

– Here superscript m denotes the Cartesian product applied m times, e.g., {H,T} 2:={H,T} × {H,T}. If 𝑚𝑚 → ∞, above sets become infinite but are countable.

Ex: Throwing a coin until heads show up has the following elements (outcomes) in the sample space: {H, TH, TTH, TTTH, TTTTH, …}. This sample space has infinite elements but it is countable, i.e., the number of Ts in an element is the number assigned to that element: 0 for H, 1 for TH, …

In general, if we are waiting for a particular outcome from an experiment (e.g., winning lottery), the number of sample space elements (purchasing lottery tickets) is infinite but countable.

When an outcome takes countable values, it is (can be represented by) a discrete variable.

utda

llas.edu /~m

etinPage 16Uncountable - Infinite Sets Any interval of real numbers is uncountable.

– Ex: Demand forecasters tend to consider the interval 𝑎𝑎 , 𝑏𝑏 as the forecast. So the sample space for the (experiment of) demand (realization) is the interval and uncountable.

When an outcome takes uncountable values, it is a continuous variable. Where are continuous variables coming from?

– Quantum (discrete) physics, accounting and others say variables are countable.– Time is a continuous variable but it is an invention of humanity.– Although analysts observe discrete variables in real-life, they invent continuous analogs for the

purpose of ease of analysis/communication. » An accountant says the cost is in [10,11] rather than {10.0, 10.1, 10.2, …, 10.9, 11.0}

What is in an interval? It is a collection of Real numbers! Unlike integers or rational numbers, Real numbers are not countable;

– To convince yourself, see appendix in the notes. Limiting the analysis to integers or rational numbers gets rid of calculus (continuity,

integral, etc.) and makes the analysis tedious.

Can we deal with an uncountable set without counting it but by focusing on some collections of subsets of it?

utda

llas.edu /~m

etinPage 17Are All Sets or Only Some Measurable?Can we assign probabilities to All or Some? Borel 𝜎𝜎 −algebra on ℜ is obtained by taking complements & countable unions of intervals. A measure assigns numbers from [0,∞) to sets

– It needs to be countable additive (measure of a set is the sum of measures of its countable partition) and assign 0 to the empty set

– Ex: Lebesgue measure on (ℜ, Borel 𝜎𝜎 −algebra) assigns the interval length to every interval Question 1: Is there a set (⊆ ℜ∩ [0,1]) that is not Lebesgue measurable?

– Answer: Yes. Construct a set to contradict countable additivity of Lebesgue measure: Theorem 1.4.9 of Cohn (2013). The set is uncountable union of meticulously picked reals.

1. Consider a relation ~ on ℜ that it holds if and only if 𝑥𝑥 − 𝑦𝑦 is rational; this is an equivalence relation 2. Each equivalence class has the form 𝑥𝑥 +{Set of rationals} for 𝑥𝑥 ∈ ℜ and intersects the interval (0,1)3. Consider the set made by taking exactly one element from (𝑥𝑥 + {Set of rationals}) ∩ (0,1) for each 𝑥𝑥 ∈ ℜ

– All subsets of (0,1) is not measurable ⇒ consider only a certain collection of subsets. Question 2: Is there a set in Borel 𝜎𝜎 −algebra that is not Lebesgue measurable?

– Answer: No. All sets in Borel 𝜎𝜎 −algebra are measurable: Theorem 1.3.8 of Cohn (2013).– Remark: Countable unions are in Borel 𝜎𝜎 −algebra whereas the counterexample in Q1 has uncountable unions.

To achieve measurability, consider only 𝜎𝜎 −algebras.

Measurablesets Non-

Measurablesets

Faulty model: (Ω, all sets, measure)

Correction: (Ω, measurable sets, measure)

All subsets of Ω

utda

llas.edu /~m

etinPage 18σ-algebra

Consider a collection of subsets of Ω to assign probability. We like to assign probability to Ω itself, assign probability to complements of events and consider the unions of the events. These requirements map to the following three properties required from the collection, specifically called σ-algebra ℑ:

– Include the sample space: Ω ∈ ℑ– Closed under complements: 𝐴𝐴 ∈ ℑ implies 𝐴𝐴𝑐𝑐 ∈ ℑ– Closed under countable unions: 𝐴𝐴𝑎𝑎 ∈ ℑ implies ∪𝑎𝑎=1∞ 𝐴𝐴𝑎𝑎 ∈ ℑ.

Ex: The σ-algebra also includes intersection of events. This can be shown by taking the complement of the unions. Hence we can measure the probability of two events in the algebra happening together.

When the sample space is – countable, we can index the members of Ω and specify the probability 𝑃𝑃(𝜔𝜔𝑎𝑎) for each 𝜔𝜔𝑎𝑎 ∈ Ω.

– uncountable, indexing cannot be done: 𝜔𝜔𝑤𝑤𝑤𝑎𝑎𝑤𝑤? . » Which objects are assigned probabilities?

utda

llas.edu /~m

etinPage 19Probability (Measure) A probability (measure) on Ω,ℑ satisfies the following properties:

– Measure the sets in ℑ: For every 𝐴𝐴 ∈ ℑ, there exists a real number P 𝐴𝐴 ∈ 0,1 .– Sample space always happens: P(Ω) = 1.– Countable additivity: P ∪𝑎𝑎=1∞ 𝐴𝐴𝑎𝑎 = ∑𝑎𝑎=1∞ P 𝐴𝐴𝑎𝑎 for disjoint 𝐴𝐴1,𝐴𝐴2 , … .

These three properties can also be called axioms of probability. It is easy to justify them when P(⋅) is interpreted as frequency.

Alternative additivity assumptions are either insufficient or incorrect.– Finite additivity: P ∪𝑎𝑎=1

𝑁𝑁 𝐴𝐴𝑎𝑎 = ∑𝑎𝑎=1𝑁𝑁 P 𝐴𝐴𝑎𝑎 . Some experiments have infinite sample space Ω, we

want to able to consider P ∪𝑎𝑎=1∞ 𝜔𝜔𝑎𝑎 for 𝜔𝜔𝑎𝑎 ∈ Ω. This cannot be done with finite additivity!

– Uncountable additivity: P(∪𝐴𝐴∈ℵ 𝐴𝐴) = ∑𝐴𝐴∈ℵ P(𝐴𝐴) fails with a counterexample: Uniform probability over [0,1], i.e., P [𝑎𝑎, 𝑏𝑏] = 𝑏𝑏 − 𝑎𝑎 so P 𝜔𝜔 = [𝜔𝜔,𝜔𝜔] = 𝜔𝜔 − 𝜔𝜔 = 0.» The left-hand side of proposed uncountable additivity yields 𝑃𝑃 ∪𝜔𝜔∈Ω 𝜔𝜔 = 𝑃𝑃 Ω = 1. » The right-hand side of proposed uncountable additivity yields ∑𝜔𝜔∈Ω𝑃𝑃 𝜔𝜔 = 0.

Countable sets, Discrete variables Uncountable setsContinuous variablesFinite sets Infinite but

Countable sets

Up to hereinsufficient

Up to hereincorrectUp to here

sufficient & correct

Increasingly strict additivity assumptions

utda

llas.edu /~m

etinPage 20Axiomatic Framework Countable additivity based framework is due to Kolmogorov's short book with a long-title:

Grundbegriffe der Wahrscheinlichkeitsrechnung (probabilistic computation), from the early 1930s. Before Kolmogorov's work, others were experimenting with different set of axioms.

– Borel introduced countable additivity to mathematics and carried it over to probability. – Borel & Hilbert motivated mathematicians in the first half of the 20th century to develop a framework. – Some researchers made more assumptions than necessary, referring to this, Kolmogorov wrote in his preface:

» “In the pertinent mathematical circles it has been common for some time to construct probability theory in accordance with this general point of view. But a complete presentation of the whole system, free from superfluous complications, has been missing.”

– Simultaneously with Kolmogorov, Frechet was writing a book to provide a similar framework. Kolmogorovwrote faster & a shorter book, which was noted later as a significant achievement in Frechet's book:

» “It is not enough to have all the ideas in mind, to recall them now and then; one must make sure that their totality is sufficient, bring them together explicitly, and take responsibility for saying that nothing further is needed …. This is what Mr. Kolmogorov did. This is his achievement.”

Frechet was debating the necessity of countable additivity with de Finetti. – de Finetti was against countable additivity & asking Frechet to explain its meaning. – Kolmogorov on the other hand was a pragmatic and said:

» “Since the new axiom [countable additivity] is essential only for infinite fields of probability, it is hardly possible to explain its empirical meaning ….”

– One of the examples used by de Finetti to attack countable additivity is the infinite lottery. » An integer is chosen at random among all integers (countable set). One would like to argue that the

probability of choosing each integer is the same, which is impossible under countable additivity: » i) Assign 0 to this probability for every integer ⇒ the sum of all probabilities = zero,

but the probability of choosing an integer (probability of countable union of choosing an integer) = 1. » ii) Assign 𝜖𝜖 > 0, the sum of all probabilities →∞. de Finetti wanted to rule out countable additivity and to develop probability with only finite additivity.

Kolmogorov

Frechet

Borel

Hilbert

de Finetti

utda

llas.edu /~m

etinPage 21Summary

Experiments, Outcomes, Events Counting

– Finite Outcomes» Number of distinct strings, sequences, subsets and multisets

– Infinite but Countable Outcomes– Uncountable Outcomes

Probability