How Do You Find Unbiased Estimators for an Exponential Distribution?
At a glance
- 1. Identify the parameter
Check whether the exponential distribution is written using its rate or its mean (scale); the unbiased estimators differ.
- 2. Write the sample total
For independent observations with rate λ, their sum has a gamma distribution with shape n and rate λ.
- 3. Use inverse moments
Compute the expectation of the reciprocal sample total, which exists when n is greater than one.
- 4. Adjust the estimator
Multiply the reciprocal total by n minus one to obtain an unbiased estimator of the rate.
An exponential distribution can be parameterized by its rate or by its mean (scale) . Those parameters have different unbiased estimators, so first check which convention your problem uses.
Suppose are independent observations from an exponential distribution with rate . For , an unbiased estimator of the rate is
If the unknown parameter is the mean , the sample mean is unbiased. The derivation below explains why the rate estimator has the factor rather than .
Rate and scale parameterizations
In the rate convention, the density is
and . In the scale convention, the density is , and . The two notations describe the same family with .
An estimator of a parameter is unbiased if for every allowed value of that parameter. An estimator can be unbiased even though its value from one particular sample is not equal to the parameter.
Deriving an unbiased estimator of the rate
Let . The sum of independent exponential variables with rate has a gamma distribution with shape and rate . For , its reciprocal has expectation
Therefore, multiplying by gives
So is unbiased for the rate. The restriction matters: the expectation of is not finite for a sample of size one.
Worked example
Suppose observations have a total of . The rate estimate from this sample is
This is the estimate for these observed data. Unbiasedness is a different claim: if you repeatedly collected samples of size four from the same exponential distribution and calculated each time, the average of those estimates would be .
The maximum-likelihood estimator is , which here would be . Its expectation is , so it overestimates the rate on average. The adjustment from to removes that bias.
If the parameter is the mean
For the scale or mean parameter , each observation has expectation . By linearity of expectation,
Thus is unbiased for the mean. But taking its reciprocal does not generally give an unbiased estimator of the rate: in fact, has expectation when . This is why simply inverting an unbiased estimate does not usually preserve unbiasedness.
Estimating higher powers of the rate
The gamma distribution also gives unbiased estimators of powers of . For a positive integer ,
so an unbiased estimator of is
For example, when , is unbiased for . The condition ensures the required inverse moment is finite.
Common mistakes
- Mixing up rate and mean. The rate is ; the mean is . Their unbiased estimators are not interchangeable.
- Using for the rate and calling it unbiased. It is the maximum-likelihood estimator, but its expectation is for .
- Using the rate formula with one observation. The inverse-moment derivation requires ; does not have a finite expectation under an exponential distribution.
- Thinking unbiased means exact for one sample. It means the estimator’s expectation over repeated samples equals the target parameter.
References
- Statistical Inference, George Casella and Roger L. Berger
- Introduction to Mathematical Statistics, Robert V. Hogg, Joseph W. McKean, and Allen T. Craig
Spotted an error? Send a correction. Read how we write and check articles in our editorial policy.