Expectation Continued: Tail Sum, Coupon Collector, and ... · Expectation Continued: Tail Sum,...
Transcript of Expectation Continued: Tail Sum, Coupon Collector, and ... · Expectation Continued: Tail Sum,...
Expectation Continued: TailSum, Coupon Collector, and
Functions of RVs
CS 70, Summer 2019
Lecture 20, 7/29/19
1 / 26
Last Time...
I Expectation describes the weightedaverage of a RV.
I For more complicated RVs, use linearity
Today:
I Proof of linearity of expectation
I The tail sum formula
I Expectations of Geometric and Poisson
I Expectation of a function of an RV
2 / 26
Sanity CheckLet X be a RV that takes on values in A.Let Y be a RV that takes on values in B.Let c 2 R be a constant.
Both c · X and X + Y are also RVs!
3 / 26
Xty
Values a,
a Values :{ atb,
AEA,bEB}
Probe : probs :
IPCC X = ca ]=p[X=a ] ipfxty - Atb ]=P[X=a,7=b ]
Proof of Linearity of Expectation IRecall linearity of expectation:
E[X1 + . . .+ Xn] = E[X1] + . . .+ E[Xn]
For constant c , E[cXi ] = c · E[Xi ]
First, we show E[cXi ] = c · E[Xi ]:
4 / 26
Xi values in A.
Efc Xi ) = I ¢a) IPCX = a ]A E A
= c ÷ ,
a . IP EX - a ] = c IE [ X i ]-
EC Xi ]
Proof of Linearity of Expectation IINext, we show E[X + Y ] = E[X ] + E[Y ].
Two variables to n variables?
5 / 26
X has values MA, y has values in B
.
Efxty ]= I ( at b) p[X=a , Y - b ]AEA ,
BEB
rewrite" 4*94 . b)ate,£⇒b
. PCx=aY=b ]
AFAFEB -#
of= ¥aaq⇒pCx=a ,
't b) tfepbff.PK/--a.Y--b]
EE 7¥ ¥=b]BEBAEA
= qfaa.ipcx-ay-qf.gg b - PLY - b ]
TEXT TEEInduction .
The Tail Sum FormulaLet X be a RV with values in {0, 1, 2, . . . , n}.We use “tail” to describe P[X � i ].
What doesP1i=1 P[X � i ] look like?
Small example: X only takes values {0, 1, 2}:
6 / 26
Anon - neg
integers
team
€7PEX Z i ] = PEX 21 ] t PEX 22 ]/ \
=
p Cx - I] t PIX= 2) t¥ EX = 2 ]
Or PEX +1. IPCX -
- I ] -12 . PCX -
- 2 ]
EF
The Tail Sum FormulaThe tail sum formula states that:
E[X ] =1X
i=1
P[X � i ]
Proof: Let pi = P[X = i ].
7 / 26
x Ronningers=
€,9P[ x Z i ] = PEX Z IT t P EX Z 2) t PEX 233?
=
( p , t¥pzt. . . ) t (¥713t Pat . . . ) !
+ I p ,t Pg t
. . . )O ' P
I . p , t 2 ' p a t 3 . Pz t. - -
= E EXT
Expectation of a Geometric ILet X ⇠ Geometric(p).
P[X � i ] =
Apply the tail sum formula:
8 / 26
i - I
H - p )
÷,
-
pfxzi ] = It ( I -
p ) th - pit. . .
-Geometric series
firstEsma =L,
r -- H - P )
-
- Fr -
- Ifpi
Expectation of a Geometric IIUse memorylessness: the fact that thegeometric RV “resets” after each trial.
Two Cases:
9 / 26
X - aeomcp)
ft art trials failedP first trial .succeedfall 1st trial .
"
win in2day .
" " win in1€E[X] days
"
" reset,
"
day 2 looks identicalto day I .
ECX ] =p (1) t ( I - p ) f ITEM ]Solve
for ECX ].
PIECX I ' HttpEfxtipt
Expectation of a Geometric IIILastly, an intuitive but non-rigorous idea.
Let Xi be an indicator variable for success in asingle trial. Recall trials are i.i.d.
Xi ⇠
E[X1 + X2 + . . .+ Xk ] =
10 / 26
X - Geom Cp)
Ber ( p )use linearity of expectation
Efx , It ECX.IT . ..tECXn ]
= KEEX , ]= Kp
Need K = pt in order for ECX ,t
. . .
t Xx ) = I-
# of successes .
Coupon Collector I(Note 19.) I’m out collecting trading cards.
There are n types total. I get a random tradingcard every time I buy a cereal box.
What is the expected number of boxes I need tobuy in order to get all n trading cards?
High level picture:
11 / 26
Time T , Tz -13
=Lxz Xz X4
get get get1St 2nd 3rd
card ! card ! card .
Coupon Collector II
Let Xi =
What is the dist. of X1?
What is the dist. of X2?
What is the dist. of X3?
In general, what is the dist. of Xi?
12 / 26
time / " # boxes"
between Ci-Dth card
and the
ithGeom CL)X ,
=L always .
Xz ~ Geom ( ht )
Xz ~ Geom ( MT )
Xi - Geom ( nifty
Coupon Collector IIILet X =
X =
E[X ] =
13 / 26
total # boxes to get all n cards
X ,t Xzt . -
t XnIE [ Xitxzt
. . . txn ] linearity= Efx , ] HECK ) t
. - TECXN] d
Xi n Geom I n-ci ,
It IT t # t. . . thy
%= nEi e- ? ?
Aside: (Partial) Harmonic SeriesHarmonic Series:
P1k=1
1
k
Approximation forPnk=1
1
k in terms of n?
14 / 26
Diverges
E.it#fIdx=lnx/7--lnn-In1/"
↳x inn .
⇒ coupon collector : E EXT F N log N.
¥¥IE¥. EE
BreakA Bad Harmonic Series Joke...
A countably infinite number of mathematicians
walk into a bar. The first one orders a pint of
beer, the second one orders a half pint, the third
one orders a third of a pint, the fourth one orders
a fourth of a pint, and so on.
The bartender says ...
15 / 26
Expectation of a Poisson IRecall the Poisson distribution: values 0, 1, 2, . . . ,
P[X = i ] =�i
i !e��
We can use the definition to find E[X ]!
16 / 26
X - poi CX )
ECXI -
- Eto if¥1e-
a ) touswrwiesfor
⇒
te.
=
= Het -
- DD
Expectation of a Poisson IIOptional but intuitive / non-rigorous approach:
Think of a Poisson(�) as a Bin(n, �n ) distribution,taken as n !1.
Let X ⇠ Bin(n, �n ).
X =
17 / 26
* I 71¥
Rest of Today: Functions of RVs!Recall X from Lecture 19:
X =
8><
>:
1 wp 0.4
1
2wp 0.25
�12
wp 0.35
Refresh your memory: What is X 2?
18 / 26
XI ftWp . 0.4
-4 Wp 0.25
¥ Wp 0.35
= { ¥WP 0.4
Wp 0.6
Example: Functions of RVs
X 2 =
(1 wp 0.4
1
4wp 0.6
What is E[X 2]?
What is E[3X 2 � 5]?
19 / 26
Efx 21 -
- I . PC XIII t 4 PIX '-
- I ]= I .0.4t t . 0.6=0.55
linearity of exp .
IE [3×2] - ECS ) 3 EHF- 5
310.55 ) - 5
In General: Functions of RVs
Let X be a RV with values in A.Distribution of f (X ):
E[f (X )] =
20 / 26
f Cx ) = {f !!)
wppxf-ajac.CAHa ) pfX=a ]
Square of a Bernoulli
Let X ⇠ Bernoulli(p).Write out the distribution of X .
What is X 2? E[X 2]?
21 / 26
X -
- { f wppWp tp
X'
-- { to IfIp ECM =p .
Product of RVs
Let X be a RV with values in A.Let Y be a RV with values in B.
XY is also a RV! What is its distribution?(Use the joint distribution!)
22 / 26
Exercise
Product of Two Bernoullis
Let X ⇠ Bernoulli(p1), and Y ⇠ Bernoulli(p2).X and Y are independent.
What is the distribution of XY ?
What is E[XY ]?
23 / 26
Exercise
Square of a Binomial I
Let X ⇠ Bin(n, p).Decompose into Xi ⇠ Bernoulli(p).
X =
E[X ] =
24 / 26
X , t X zt
. . .
t X n
IE [ X ,t X at . . .
t X n ]
= Efx . I t IEC X a It . . .t Ef X n ]
= n p .
Square of a Binomial IIRecall, E[X 2i ] = 1, and E[XiXj ] = p2.
25 / 26
PECXZ ]=Ef( X ,tXzt .
.
.tt/n)2J=tEfCXi2tXz2t...tXn4t(XiXztXiXst . . . ))- -
square terms cross terms
n of them MIN - 1) of
= Ef square terms ] TEAMS terms ] them .I ::*::*: " "
:i÷i :*.
ECXIZ ] - P Efx ,Xz3=PZ> = nptnln - 1) P2
SummaryToday:
I Proof of linearity of expectation: did not use
independence, but did use joint distribution
I Tail sum for non-negative int.-valued RVs!
I Coupon Collector: break problem down into a
sum of geometrics.
I Expectation of a function of an RV: canapply definition and linearity of expectation
(after expanding) as well!!
26 / 26