How to Find the UMVUE of an Exponential Parameter

There is no single formula for the UMVUE until you specify what the parameter θ means. An exponential distribution is commonly written using either its scale (the mean) or its rate (the reciprocal of the mean); the estimators for these parameters differ.

Assume throughout that $X_1,\ldots,X_n$ are independent observations from the same exponential distribution. The sum of the observations is a complete sufficient statistic, which lets you identify the UMVUE once you find an unbiased estimator based on that sum.

The two common parameterizations

If θ is the scale, the density is

$$f(x;\theta)=\frac{1}{\theta}e^{-x/\theta},\qquad x\geq 0,\quad \theta>0.$$

Here $\mathbb{E}[X_i]=\theta$, so the sample mean is unbiased for θ. The UMVUE is

$$\widehat{\theta}_{\mathrm{UMVUE}}=\overline{X}=\frac{1}{n}\sum_{i=1}^n X_i.$$

If θ is instead the rate, the density is

$$f(x;\theta)=\theta e^{-\theta x},\qquad x\geq 0,\quad \theta>0.$$

Now $\mathbb{E}[X_i]=1/\theta$. For $n>1$, the UMVUE of the rate is

$$\widehat{\theta}_{\mathrm{UMVUE}}=\frac{n-1}{\sum_{i=1}^n X_i}.$$

The factor $n-1$ is necessary for unbiasedness; simply taking the reciprocal of the sample mean gives a biased estimator of the rate.

Why the estimators are UMVUEs

Let

$$S=\sum_{i=1}^n X_i.$$

Under the rate parameterization, the joint density is proportional to $\theta^n e^{-\theta S}$. By the factorization theorem, $S$ is sufficient for θ. Also, $S$ has a gamma distribution with shape $n$ and rate θ. This gamma family, with its shape fixed and rate ranging over positive values, is complete. The scale parameterization has the same property: its joint density depends on the sample through $S$, and the resulting gamma family is complete.

The Lehmann–Scheffé theorem says that an unbiased estimator which is a function of a complete sufficient statistic is the unique UMVUE. For the scale, $S/n$ is unbiased because $\mathbb{E}[S]=n\theta$. For the rate, use the gamma density of $S$ to calculate, when $n>1$,

$$\begin{aligned} \mathbb{E}\!\left[\frac{1}{S}\right] &=\frac{\theta^n}{\Gamma(n)}\int_0^\infty s^{n-2}e^{-\theta s}\,ds\\ &=\frac{\theta^n\Gamma(n-1)}{\Gamma(n)\theta^{n-1}}\\ &=\frac{\theta}{n-1}. \end{aligned}$$

Therefore $(n-1)/S$ is unbiased for the rate. Both estimators are functions of the complete sufficient statistic, so the theorem establishes that they are UMVUEs—not just unbiased estimators.

Worked example

Suppose four observations have sum $S=10$. If θ is the scale, the estimate is

$$\widehat{\theta}=\frac{S}{n}=\frac{10}{4}=2.5.$$

If θ is the rate, the estimate is instead

$$\widehat{\theta}=\frac{n-1}{S}=\frac{3}{10}=0.3.$$

The units help check the interpretation: a scale has the same units as the observations, while a rate has reciprocal units.

What if the rate sample has size one?

For $n=1$, the formula $(n-1)/S$ is zero and is not an unbiased estimator of a positive rate. In fact, there is no ordinary unbiased estimator of the rate based on a single exponential observation. If $T(X)$ were unbiased, then under the rate parameterization it would have to satisfy

$$\int_0^\infty T(x)e^{-\theta x}\,dx=1\qquad\text{for every }\theta>0.$$

No ordinary integrable function has this Laplace transform: the transform equal to 1 corresponds to a point mass at zero, not a function of the observation. For $n>1$, the reciprocal of the sum has a finite expectation and the stated correction works.

Common mistakes

• Not checking the parameterization. The sample mean estimates the scale, not the rate.
• Using $1/\overline{X}$ as an unbiased rate estimator. It is generally biased; the unbiased correction is $(n-1)/S$.
• Stopping after finding an unbiased estimator. Unbiasedness alone does not prove minimum variance. The complete sufficient statistic and Lehmann–Scheffé theorem supply that step.
• Forgetting the sample size condition. The rate formula requires $n>1$.