St · Statistics & Probability

How Do You Find Unbiased Estimators for an Exponential Distribution?

Published By the 9to5Science editorial team

At a glance

  1. 1. Identify the parameter

    Check whether the exponential distribution is written using its rate or its mean (scale); the unbiased estimators differ.

  2. 2. Write the sample total

    For independent observations with rate λ, their sum has a gamma distribution with shape n and rate λ.

  3. 3. Use inverse moments

    Compute the expectation of the reciprocal sample total, which exists when n is greater than one.

  4. 4. Adjust the estimator

    Multiply the reciprocal total by n minus one to obtain an unbiased estimator of the rate.

An exponential distribution can be parameterized by its rate λ\lambda or by its mean (scale) θ=1/λ\theta=1/\lambda. Those parameters have different unbiased estimators, so first check which convention your problem uses.

Suppose X1,…,XnX_1,\ldots,X_n are independent observations from an exponential distribution with rate λ\lambda. For n>1n>1, an unbiased estimator of the rate is

λ^=n−1∑i=1nXi.\widehat{\lambda}=\frac{n-1}{\sum_{i=1}^n X_i}.

If the unknown parameter is the mean θ\theta, the sample mean X‾\overline X is unbiased. The derivation below explains why the rate estimator has the factor n−1n-1 rather than nn.

Rate and scale parameterizations

In the rate convention, the density is

f(x;λ)=λe−λx,x≥0,f(x;\lambda)=\lambda e^{-\lambda x},\qquad x\geq 0,

and E[Xi]=1/λ\mathbb{E}[X_i]=1/\lambda. In the scale convention, the density is f(x;θ)=1θe−x/θf(x;\theta)=\frac{1}{\theta}e^{-x/\theta}, and E[Xi]=θ\mathbb{E}[X_i]=\theta. The two notations describe the same family with θ=1/λ\theta=1/\lambda.

An estimator TT of a parameter ψ\psi is unbiased if E[T]=ψ\mathbb{E}[T]=\psi for every allowed value of that parameter. An estimator can be unbiased even though its value from one particular sample is not equal to the parameter.

Deriving an unbiased estimator of the rate

Let S=∑i=1nXiS=\sum_{i=1}^n X_i. The sum of nn independent exponential variables with rate λ\lambda has a gamma distribution with shape nn and rate λ\lambda. For n>1n>1, its reciprocal has expectation

E[1S]=λn−1.\mathbb{E}\left[\frac{1}{S}\right] =\frac{\lambda}{n-1}.

Therefore, multiplying by n−1n-1 gives

E[n−1S]=(n−1)λn−1=λ.\mathbb{E}\left[\frac{n-1}{S}\right] =(n-1)\frac{\lambda}{n-1} =\lambda.

So (n−1)/S(n-1)/S is unbiased for the rate. The restriction n>1n>1 matters: the expectation of 1/S1/S is not finite for a sample of size one.

Worked example

Suppose n=4n=4 observations have a total of S=10S=10. The rate estimate from this sample is

λ^=4−110=0.3.\widehat{\lambda}=\frac{4-1}{10}=0.3.

This is the estimate for these observed data. Unbiasedness is a different claim: if you repeatedly collected samples of size four from the same exponential distribution and calculated 3/S3/S each time, the average of those estimates would be λ\lambda.

The maximum-likelihood estimator is n/Sn/S, which here would be 0.40.4. Its expectation is nλ/(n−1)n\lambda/(n-1), so it overestimates the rate on average. The adjustment from nn to n−1n-1 removes that bias.

If the parameter is the mean

For the scale or mean parameter θ=1/λ\theta=1/\lambda, each observation has expectation θ\theta. By linearity of expectation,

E[X‾]=E[1n∑i=1nXi]=θ.\mathbb{E}[\overline X] =\mathbb{E}\left[\frac{1}{n}\sum_{i=1}^n X_i\right] =\theta.

Thus X‾\overline X is unbiased for the mean. But taking its reciprocal does not generally give an unbiased estimator of the rate: in fact, 1/X‾=n/S1/\overline X=n/S has expectation nλ/(n−1)n\lambda/(n-1) when n>1n>1. This is why simply inverting an unbiased estimate does not usually preserve unbiasedness.

Estimating higher powers of the rate

The gamma distribution also gives unbiased estimators of powers of λ\lambda. For a positive integer r<nr<n,

E[S−r]=λrΓ(n−r)Γ(n),\mathbb{E}[S^{-r}] =\lambda^r\frac{\Gamma(n-r)}{\Gamma(n)},

so an unbiased estimator of λr\lambda^r is

(n−1)(n−2)⋯(n−r)Sr.\frac{(n-1)(n-2)\cdots(n-r)}{S^r}.

For example, when n>2n>2, (n−1)(n−2)/S2(n-1)(n-2)/S^2 is unbiased for λ2\lambda^2. The condition n>rn>r ensures the required inverse moment is finite.

Common mistakes

  • Mixing up rate and mean. The rate is λ\lambda; the mean is 1/λ1/\lambda. Their unbiased estimators are not interchangeable.
  • Using n/Sn/S for the rate and calling it unbiased. It is the maximum-likelihood estimator, but its expectation is nλ/(n−1)n\lambda/(n-1) for n>1n>1.
  • Using the rate formula with one observation. The inverse-moment derivation requires n>1n>1; 1/X11/X_1 does not have a finite expectation under an exponential distribution.
  • Thinking unbiased means exact for one sample. It means the estimator’s expectation over repeated samples equals the target parameter.

References

  1. Statistical Inference, George Casella and Roger L. Berger
  2. Introduction to Mathematical Statistics, Robert V. Hogg, Joseph W. McKean, and Allen T. Craig

Spotted an error? Send a correction. Read how we write and check articles in our editorial policy.