What Is the Fisher Information for an Exponential Distribution?

For an exponential random variable, Fisher information depends on which parameter you use: the rate or the mean (scale). With the rate parameter λ, one observation contains information $1/\lambda^2$ about λ. A sample of $n$ independent observations contains $n/\lambda^2$.

The calculation comes from the likelihood: differentiate its logarithm to get the score, then find the score’s variance or the expected negative second derivative. Both methods give the same result here.

Rate parameter: one observation

Using λ as the rate, the exponential density is

$$f(x;\lambda)=\lambda e^{-\lambda x},\qquad x\ge 0,\quad \lambda>0.$$

The log-density is

$$\ell(\lambda;x)=\log\lambda-\lambda x.$$

Differentiate with respect to λ to get the score:

$$U(\lambda;X)=\frac{\partial}{\partial\lambda}\ell(\lambda;X)=\frac{1}{\lambda}-X.$$

For an exponential variable with rate λ, $\mathbb{E}[X]=1/\lambda$ and $\operatorname{Var}(X)=1/\lambda^2$. The score therefore has mean zero, and its variance is

$$\mathcal{I}_1(\lambda)=\operatorname{Var}\!\left(\frac{1}{\lambda}-X\right)=\operatorname{Var}(X)=\frac{1}{\lambda^2}.$$

Here $\mathcal{I}_1(\lambda)$ denotes the Fisher information from one observation. Equivalently, the second derivative of the log-density is $-1/\lambda^2$, so the expected negative second derivative is also $1/\lambda^2$.

Fisher information for a sample

Let $X_1,\ldots,X_n$ be independent observations from the same exponential distribution. Their joint log-likelihood is

$$\ell(\lambda;\mathbf{x})=n\log\lambda-\lambda\sum_{i=1}^{n}x_i.$$

The sample score and its variance are

$$U(\lambda;\mathbf{X})=\frac{n}{\lambda}-\sum_{i=1}^{n}X_i, \qquad \operatorname{Var}(U)=n\operatorname{Var}(X_i)=\frac{n}{\lambda^2}.$$

Thus the sample’s Fisher information is

$$\boxed{\mathcal{I}_n(\lambda)=\frac{n}{\lambda^2}}.$$

The second-derivative method gives the same answer: the second derivative of the sample log-likelihood is $-n/\lambda^2$, so its negative expectation is $n/\lambda^2$. The information adds across independent observations.

If the parameter is the mean or scale

Some sources write the exponential density using its mean θ, also called its scale:

$$f(x;\theta)=\frac{1}{\theta}e^{-x/\theta},\qquad x\ge 0,\quad \theta>0.$$

In this convention, $\theta=1/\lambda$. The information is

$$\mathcal{I}_1(\theta)=\frac{1}{\theta^2}, \qquad \mathcal{I}_n(\theta)=\frac{n}{\theta^2}.$$

The formulas look alike, but they refer to different parameters. Do not substitute a rate into a formula intended for the mean or scale. For example, a rate of 2 corresponds to a mean of 0.5.

Worked example

Suppose you observe $n=5$ independent exponential values and use the rate parameter $\lambda=2$. The Fisher information in the full sample is

$$\mathcal{I}_5(2)=\frac{5}{2^2}=\frac{5}{4}=1.25.$$

The Cramér–Rao lower bound says that an unbiased estimator of λ, under the usual regularity conditions, has variance at least the reciprocal of this information:

$$\operatorname{Var}(\widehat{\lambda})\geq \frac{1}{\mathcal{I}_5(2)}=0.8.$$

This is a lower bound, not a claim that every estimator has variance 0.8. In particular, the maximum-likelihood estimator $\widehat{\lambda}=n/\sum_i X_i$ is generally biased for finite samples, so the unbiased-estimator version of the bound does not apply to it directly. For large samples, the inverse information also gives the usual asymptotic variance approximation for the maximum-likelihood estimator.

Common mistakes

• Mixing up rate and scale. The rate is λ; the mean and scale are $1/\lambda$. State which parameter you mean before using a formula.
• Forgetting the sample size. The information is $1/\lambda^2$ for one observation and $n/\lambda^2$ for $n$ independent observations.
• Using the score instead of its variance. Fisher information is not the score itself. It is the score’s expected squared value (equivalently, its variance when the score has mean zero), or the expected negative second derivative under regularity conditions.
• Treating the Cramér–Rao bound as an exact variance. It is a lower bound for unbiased estimators, not a formula for the variance of every estimator.