Point Estimation
description
Transcript of Point Estimation
Point Point EstimationEstimation
Notes of STAT 6205 by Dr. Fan
OverviewOverview• Section 6.1• Point estimation• Maximum likelihood estimation• Methods of moments• Sufficient statistics
o Definitiono Exponential familyo Mean square error (how to choose an estimator)
6205-Ch6 2
Big Picture• Goal: To study the unknown distribution of a
population• Method: Get a representative/random sample and
use the information obtained in the sample to make statistical inference on the unknown features of the distribution
• Statistical Inference has two parts: o Estimation (of parameters)o Hypothesis testing
• Estimation:o Point estimation: use a single value to estimate a parametero Interval estimation: find an interval covering the unknown parameter
6205-Ch6 3
Point EstimatorPoint Estimator• Biased/unbiased: an estimator is called unbiased
if its mean is equal to the parameter of estimate; otherwise, it is biased
• Example: X_bar is unbiased for estimating mu
6205-Ch6 4
Maximum Likelihood Estimation
• Given a random sample X1, X2, …, Xn from a distribution f(x; ) where is a (unknown) value in the parameter space
• Likelihood function vs. joint pdf
• Maximum Likelihood Estimator (m.l.e.) of , denoted as is the value which maximizes the likelihood function, given the sample X1, X2, …, Xn.
n
iixfxL
1
);();(
6205-Ch6 5
Examples/Exercises• Problem 1: To estimate p, the true probability of heads
up for a given coin.• Problem 2: Let X1, X2, …, Xn be a random sample
from a Exp(mu) distribution. Find the m.l.e. of mu.• Problem 3: Let X1, X2, …, Xn be a random sample
from a Weibull(a=3,b) distribution. Find the m.l.e. of b.
• Problem 4: Let X1, X2, …, Xn be a random sample from a N(,^2) distribution. Find the m.l.e. of and .
• Problem 5: Let X1, X2, …, Xn be a random sample from a Weibull(a,b) distribution. Find the m.l.e. of a and b.
6205-Ch6 6
Method of MomentsMethod of Moments• Idea: Set population moments = sample
moments and solve for parameters
• Formula: When the parameter is r-dimensional, solve the following equations for :
6205-Ch6 7
n
i
ki
k ,...r,knXXE1
21for /)(
Examples/ExercisesExamples/ExercisesGiven a random sample from a population
•Problem 1: Find the m.m.e. of for a Exp() population.
•Exercise 1: Find the m.m.e. of and for a N(^2) population.
6205-Ch6 8
Sufficient StatisticsSufficient Statistics• Idea: The “sufficient” statistic contains all
information about the unknown parameter; no other statistic can provide additional information as to the unknown parameter.
• If for any event A, P[A|Y=y] does not depend on the unknown parameter, then the statistic Y is called “sufficient” (for the unknown parameter).
• Any one-to-one mapping of a sufficient statistic Y is also sufficient.
• Sufficient statistics do not need to be estimators of the parameter.
6205-Ch6 9
Sufficient StatisticsSufficient Statistics
6205-Ch6 10
Examples/ExercisesExamples/ExercisesLet X1, X2, …, Xn be a random sample from f(x)
Problem: Let f be Poisson(a). Prove that1.X-bar is sufficient for the parameter a2.The m.l.e. of a is a function of the sufficient statistic
Exercise: Let f be Bin(n, p). Prove that X-bar is sufficient for p (n is known). Hint: find a sufficient statistic Y for p and then show that X-bar is a function of Y
6205-Ch6 11
Exponential FamilyExponential Family
6205-Ch6 12
Examples/ExercisesExamples/Exercises
Example 1: Find a sufficient statistic for p for Bin(n, p)
Example 2: Find a sufficient statistic for a for Poisson(a)
Exercise: Find a sufficient statistic for for Exp()
6205-Ch6 13
Joint Sufficient StatisticsJoint Sufficient Statistics
Example: Prove that X-bar and S^2 are joint sufficient statistics for and of N(, ^2)
6205-Ch6 14
Application of SufficienceApplication of Sufficience
6205-Ch6 15
ExampleExampleConsider a Weibull distribution with parameter(a=2, b)
1)Find a sufficient statistic for b
2)Find an unbiased estimator which is a function of the sufficient statistic found in 1)
6205-Ch6 16
Good Estimator?Good Estimator?• Criterion: mean square error
6205-Ch6 17
ExampleExample• Which of the following two estimator of variance
is better?
6205-Ch6 18