Austral. J. Statist., 22 (3), 1980, 328-331

MINIMUM VARIANCE UNBIASED ESTIMATION OF P R 0 B A B I L I TY DEN SIT I E S'

J A G B ~ SINGH School of Business Administration, Temple University

summary The purpose of this note is to point out an elementary method for

deriving the minimum variance unbiased (mvu) estimators of probabil- ity densities. The method is illustrated by estimating the densities of some well known important distributions. We believe that this note would be of interest especially to both teachers and students of a first course in mathematical statistics.

1. Introduction

Our method is based upon the following fact: Let X and T be two random variables having joint density function f ( x , t ) and marginal densities g ( . ) and h ( . ) respectively. For any x in the space of X, the average (expected value) of the conditional density g ( x 1 t) of X given T = t with respect to the distribution of T is indeed g ( x ) . That is,

The preceding fact is simple and can be exploited to estimate densities. We use it as follows: Consider independently and identically distributed (iid) random variables X , , X,, . . . , X,, from a distribution which admits a complete sufficient statistic T ( X , , X,, . . . , X,,). By virtue of the famous Rao-Blackwell theorem, the mvu estimate of the underlying density function is the conditional density function of XI given T = r.

2. Examples

Poisson. Let X , , X,, . . . , X,, be iid random variables having Pois- son density function f ( x ; e) = exp (- e ) e = / X ! ; x = 0, 1,2,. . . , 0 > O . Since T = 1 X, is a complete sufficient statistic, we find the conditional density g ( x I t ) of XI given T = t. It is a matter of simple calculations to

Manuscript received December 18, 1978; revised September 12, 1979.

ESTIMATION OF PROBABILITY DENSlTES 329

verify that,

LO, x > t. It may be said that the computation of g(x/t) is really the Rao-

Blackwellization of an indicator function I x ( X , ) which is one if X, = x, and zero otherwise. Regardless, g(x I t ) is the rnvu estimate of the Poisson density f(x; 0 ) for any x in its range. Patil (1963) derived the same estimator and it is a special case of the rnvu estimation of the generalized power series distribution by Patil & Wani (1965).

Two more observations are in order. First, for any x, it is not g(x I t ) but f(x; X) which is usually used to estimate f(x; O), where X = t /n is the maximum likelihood estimate of 8. But the two may not differ greatly since the binomial probability g(x I t ) approaches the Poisson probability f(x; 2 ) as both the parameters n and t increase in such a manner that X remains constant. Statistically also, it is a fact that both the estimates converge almost surely to f(x; O ) , for any given x, since.i converges to 8 almost surely.

Our second observation pertains to a characterization of the Poisson probability f(x; Z) as both the parameters n and t increase in Xi, X,. . . . . X , follow a Poisson distribution if and only if P ( X , = x / c y = t ) is g(x 1 t ) given in (l), for all t. The preceding characteriza- tion is really a variant of a rather well known result (see Kagan, Linnik & Rao (1973), page 425).

Normal. Let 4(x; 0, a’) denote the density of a normal distribu- tion with mean 8 and variance a’. We want to find the mvu estimator of the density when both the parameters are unknown but first consider the two important cases where one parameter is unknown.

(i) Let XI, X 2 , . . . , X,, be iid random variables with density 4 ( x ; 8, a’), a2 being known. A complete sufficient statistic for 8 is T=C Xi. Thus the conditional density of X, given T = t is +(x; t /n, n - l/na’) which is the mvu of the density.

(ii) Let X, , X 2 , . . . , X,, be iid random variables with density 4 ( x ; 0, a’), and note that T = 1 Xf is a complete sufficient statistic for a’. If h( . ) and h ( - I x ) denote respectively the density of T and the conditional density of T given X, = x, then

g(x 1 t ) = h( t 1 x)+(x; 0, a’)/h(t) (for t?x’), (2) and is zero otherwise. To find g(x I t ) , we note that both T and Y = cr=z X f follow gamma distributions. Also, since Y is independent of XI, h( t I x) is the density function of Y evaluated at (t-x2). We find,

(3) K(1- x’/t)‘”-3’’*/J;: x 2 < t

otherwise,

330 JAGBJR SINGH

where K = T(n/2) / f iT((n - 1)/2). Thus the mvu estimate of 4(x; 0, a’) is g ( x I t) as given in (3). Note that g ( x I t) is symmetric about x = 0 and has the shape of + ( x ; 0, a’), except that the former is truncated at -& and +h. Here also, to find g ( x I t). we must know both its parameters n and t. Let us treat g ( x 1 t ) as a function of these parameters for any fixed x and consider its limit as n-m, t - m . Assuming that the sample variance t /n=s2 remains constant, we can show that limit g ( x 1 t) = + ( x ; 0, s’), for m < x <m, which is the usual estimate obtained from c#~(x ;O,a ‘ ) if a’ is replaced by the sample variance s2. Also, g ( x I t ) is consistent.

If e f o , but known, then g ( ( x - O ) / t ) is the mvu estimate of

(iii) Let X I , X,, . . . , X,, be iid random variables with density 4 ( x ; 8, a’); and note that T = Xi, C (Xi -x)’] is a complete suffi- cient statistic for (6, a’). We can find the conditional density of X , given T = (tl, f2 ) in two stages. From the discussion in (i), 4(x; tJn, (n - l) /n a’), even though not a statistic. is unbiased for 4 ( x ; 6, a’)., Now look upon X , , X,, . . . , X,, as a random sample from the conditional distribution 4(x; t,/n, (n - l ) / n a‘). We may use rep- resentation (2). where 4 ( x : 0, a’) is replaced by 4(x; t,/n, (n - l)/na*), to fhd the conditional density g(x I tl, tJ of XI given T = ( t l , t2). Of course, in the representation now h( . ) and h( . I x ) are the density of T, and the conditional density of T2 given X , = x . Let Y = C; (Xi - XI)’, where .‘7, = 1; Xi / (n - 1). It can be shown that T2 = Y + n/(n - 1)(X, - t,/n)’. We know that both T2 and Y have gamma distributions. Also, since Y is independent of X , , h ( . I x ) is indeed the density function of Y evaluated at [ f2- n/(n - l ) ( x - t,/n)’]. We can now show that

4 k 8, D’).

’ n - 1 l2 *

otherwise,

where K = m ( ( n - 1)/2)/r((n - 2)/2) JlFiiBj. Once again, g ( x I t , , t,) is symmetric in x around the sample mean

2 = t,/n, and has the shape of the normal density only that it is truncated at X - m s and 2 + m s , where s =a is the sample standard deviation. Furthermore, if we assume that the sample size n increases and t,, t2 also increase in such a manner that i and s remain constant, then for any x, limit g ( x 1 t , , t2) = + ( x ; i, s2), --oo<x < z, the usual estimate. Also g ( x I r , , t2) is consistent.

Exponential. Let X , . X 2 , . . . , X,, be iid with density f(x; 8) = e-x’e/d, 8 > 0 , x > 0, and zero otherwise. A complete sufficient statistic

ESTIMAnON OF PROBABILITY DENSlTES 331

for 6 is T = 1 Xi. Following the basic arguments used previously, the conditional density of X , given T = t is

for x > t. If n increases and X = t/n remains fixed, then g(x I t ) approaches

f ( x : Z), the usual estimate. Geometric. Let X , , X,, . . . , X,, be iid random variables with

density f ( x ; 6) = 6 ( l - 6)x, O < 6 < 1 , x = 0, 1,2, . . . . It is easy to verify that the conditional density of XI given sufficient statistic T=C y = t is

otherwise.

If the sample size increases and t increases in such a way that t/n remains fixed then g ( x I t ) can be approximated by 6 ( l - 6)x, x = 0 , 1,2, .’. . , where 6 is the maximum likelihood estimator of 6.

Even though only a few illustrations have been given, we have used the method to estimate the densities of many interesting discrete distributions. Various mathematical, statistical, and characterizing properties of the mvu estimators for finite sample size are under investigation, and will be reported elsewhere.

Acknowledgements

I thank both the referee and the editor for their suggestions resulting in an improved version. I am grateful to the referee for pointing out a mistake in the expression for g ( x I t , , t2) and an alterna- tive derivation.

References

KAGAN, A. M., LINNIK, Yu. V. & RAO, C. R. (1973). Characterization Problems in Mathematical Statistics. New York; Wiley.

PA- G. P. (1963). Minimum variance unbiased estimation and certain problems of number theory. Ann. Math. Statist. 34, 1050-56.

PATIL, G. P. & WANI, J. K. (1966). Minimum variance unbiased estimation of the distribution function admitting a sufficient statistic. Ann. Inst. Statist. Math. 18, 39-47.