Lecture 5: Estimation · ¥ Basic concepts of estimation ¥ Nonparametric interval estimation...

Lecture 5: Estimation

Goals

• Parametric interval estimation

• Statistical approaches for estimating parameters

• Basic concepts of estimation

• Nonparametric interval estimation (bootstrap)

Population

Sample

Inferential Statistics

DescriptiveStatistics

Probability

“Central Dogma” of Statistics

Estimation

• Estimator: Statistic whose calculated value is usedto estimate a population parameter,

• Estimate: A particular realization of an estimator,

• Types of Estimators:

!

ˆ "

!

"

- point estimate: single number that can be regarded as themost plausible value of

!

"

- interval estimate: a range of numbers, called a confidenceinterval indicating, can be regarded as likely containing thetrue value of

!

"

Properties of Good Estimators

• In the Frequentist world view parameters arefixed, statistics are rv and vary from sample tosample (i.e., have an associated sampling distribution)

• In theory, there are many potential estimators for apopulation parameter

• What are characteristics of good estimators?

Statistical Jargon for Good Estimators

• Consistent: As the sample size increases gets closer to

• Unbiased:

• Precise: Sampling distribution of should have a smallstandard error

Good Estimators Are:

!

"

!

ˆ "

!

E[ ˆ " ] = "

!

ˆ "

!

limn"#

Pˆ $ %$ > &( ) = 0

Bias Versus Precision

Precise Imprecise

Biased

Unbiased

Methods of Point Estimation

1.Method of Moments

2.Maximum Likelihood

3.Bayesian

Methods of Moments

• Advantage: simplest approach for constructing anestimator

• Disadvantage: usually are not the “best”estimators possible

• Principle:

Equate the kth population moment E[Xk] with the kth sample

moment and solve for the unknown parameter

!

1

nXi

k

n

"

Method of Moments Example

• How can I estimate the scaled population mutation rate:

!

" = 4Neµ

• Brief (very brief) expose of coalescent theory:

tim

e

T4

T3

T2

Coalescent times follow a geometric distribution

!

E[Ti] =

4N

i(i "1)

!

Tc

= iTi

i= 2

n

"


!

E[Tc] = iE[T

i]

i= 2

n

"


!

E[Tc] = iE[T

i] =

i= 2

n

"4Ni

i(i #1)i= 2

n

"


!

E[Tc] = iE[T

i] =

i= 2

n

"4Ni

i(i #1)i= 2

n

" = 4N1

i #1i= 2

n

"


!

E[Tc] = iE[T

i] =

i= 2

n

"4Ni

i(i #1)i= 2

n

" = 4N1

i #1i= 2

n

"

!

E[Sn] = µE[T

c]

!

E[Sn] = µ • 4N

1

i "1i= 2

n

#

!

E[Sn] = "

1

i #1i= 2

n

$

!

ˆ " =Sn

1

i #1i= 2

n

$mom


1. Method of Moments

2.Maximum Likelihood

3. Bayesian

Introduction to Likelihood

• Before an experiment is performed the outcome is unknown.Probability allows us to predict unknown outcomes basedon known parameters:

!

P(Data |")

• For example:

!

P(x | n, p) = ( )px (1" p)n"xn

x

• After an experiment is performed the outcome is known.Now we talk about the likelihood that a parameter wouldgenerate the observed data:

!

L(" |Data)

!

L(" |Data) = P(Data |")

Introduction to Likelihood

!

L(p | n,x) = ( )px (1" p)n"xn

x

• For example:

• Estimation proceeds by finding the value of that makes theobserved data most likely

!

"

Let’s Play T/F

• True or False: The maximum likelihood estimate (mle) ofgives us the probability of

!

"

!

ˆ "

• False - why?

• True or False: The mle of is the most likely value of

!

"

!

ˆ "

• False - why?

• True or False: Maximum likelihood is cool

Formal Statement of ML

• Let x1, x2, …, xn be a sequence of n observed variables

• Joint probability:

P(x1, x2, …, xn | ) = P(X1=x1)P(X2=x2)… P(Xn=xn)

!

= P(Xi= x

i)

i=1

n

"!

"

• Likelihood is then:

L( | x1, x2, …, xn )

!

"

!

= P(Xi= x

i)

i=1

n

"

Log L( | x1, x2, …, xn )

!

"

!

= log[P(Xi= x

i)]

i=1

n

"

MLE Example

• I want to estimate the recombination fraction between locusA and B from 5 heterozygous (AaBb) parents. I examine 30gametes for each and observe 4, 3, 5, 6, and 7 recombinantgametes in the five parents. What is the mle of therecombination fraction?

Probability of observing X = r recombinant gametes for a singleparent is binomial:

!

P(X = r) = ( )" r(1#")n#r

n

r

MLE Example: Specifying Likelihood

P(r1, r2, …, rn | , n) = P(R1 = r1)P(R2 = r2)… P(R5 = r5)

Probability:

P(r1, r2, …, rn | , n) =

!

( )" r1 (1#")n#r1 • ( )" r1 (1#")n#r2 • ...• ( )" r1 (1#")n#r5n

r1

n

r2

n

r5

!

"

!

"

Likelihood:

n

ri

!

= ( )i=1

5

" # ri (1$#)n$riL( | r1, r2, …, rn , n)

!

= log( )i=1

5

" + ri• log# + (n $ r

i)log(1$#)n

riLog L!

"

MLE Example: Maximizing the Likelihood

!

= log( )i=1

5

" + ri• log# + (n $ r

i)log(1$#)n

riLog L

• Want to find p such that Log L is maximized

• How?

1. Graphically

2. Calculus

3. Numerically

MLE Example: Finding the mle of p

!

"


1. Method of Moments

2. Maximum Likelihood

3.Bayesian

World View According to Bayesian’s

• The classic philosophy (frequentist) assumes parametersare fixed quantities that we want to estimate as preciselyas possible

• Bayesian perspective is different: parameters are randomvariables with probabilities assigned to particular valuesof parameters to reflect the degree of evidence for thatvalue

Revisiting Bayes Theorem

!

P(A |B) =P(B | A)P(A)

P(B)

!

P(B) = P(B | Ai)P(A

i)

i=1

n

"

!

P(B) = P(B | A" )P(A)dA

Discrete

Continuous

Bayesian Estimation

• In order to make probability statements about given some observeddata, D, we make use of Bayes theorem

!

"

!

f (" |D) =f (") f (D |")

f (D)=

f (")L(" |D)

f (") f (D |")d""#

!"#$%&'"&(!()'*%)'+"",(-(.&'"&

The likelihood is the probability of the data given the parameter andrepresents the data now available.

The prior is the probability of the parameter and represents what wasthought before seeing the data.

The posterior represents what is thought given both prior information andthe data just seen.

Bayesian Estimation: “Simple” Example

• I want to estimate the recombination fraction between locusA and B from 5 heterozygous (AaBb) parents. I examine 30gametes for each and observe 4, 3, 5, 6, and 7 recombinantgametes in the five parents. What is the mle of therecombination fraction?

• Tedious to show Bayesian analysis. Let’s simplify and ask whatthe recombination fraction is for parent three, who had 5observed recombinant gametes.

Specifying The Posterior Density

!

f (" | n = 30, r = 5) =f (") f (r = 5 |", n = 30)

f (r = 5 |", n = 30) f (") d"0

0.5

#

!

f (")= uniform[0, 0.5] = 0.5prior =

!

P(r = 5 |", n = 30)likelihood =

!

( )" ri (1#")n#ri30

5=

!

P(r = 5 |", n = 30) f (") d"0

0.5

#normalizing

constant =

!

0.5• ( ) " 5(1-")

25 d"

0

0.5

#30

5= ! 6531

Interval Estimation

• In addition to point estimates, we also want to understandhow much uncertainty is associated with it

• One option is to report the standard error

• Alternatively, we might report a confidence interval

• Confidence interval: an interval of plausible values forthe parameter being estimated, where degree of plausibilityspecifided by a “confidence level”

Interpreting a 95% CI

• We calculate a 95% CI for a hypothetical sample mean to bebetween 20.6 and 35.4. Does this mean there is a 95%probability the true population mean is between 20.6 and 35.4?

• NO! Correct interpretation relies on the long-rang frequencyinterpretation of probability

µ

• Why is this so?

Lecture 5: Estimation · ¥ Basic concepts of estimation ¥ Nonparametric interval estimation...

Documents

Transcript of Lecture 5: Estimation · ¥ Basic concepts of estimation ¥ Nonparametric interval estimation...