What Is the Fisher Information of the Rayleigh Distribution?

The Fisher information of a Rayleigh distribution depends on which parameter you use. With the common scale parameterization, the density is $f(x;\sigma)=\frac{x}{\sigma^2}\exp\left(-\frac{x^2}{2\sigma^2}\right)$ for $x\geq 0$ and $\sigma>0$. In this parameterization, one observation contains $4/\sigma^2$ units of Fisher information about $\sigma$.

For $n$ independent observations, information adds, giving $4n/\sigma^2$. The alternative parameter $\theta=\sigma^2$ is also common; its information is $1/\theta^2$ per observation. The derivation below shows where these results come from and how to avoid mixing the parameterizations.

Fisher information for the scale parameter

For a density $f(x;\sigma)$, the Fisher information in one observation can be calculated as either the variance of the score or the expected negative second derivative of the log-density:

$$I(\sigma)=\mathbb{E}\left[\left(\frac{\partial}{\partial\sigma}\log f(X;\sigma)\right)^2\right]=-\mathbb{E}\left[\frac{\partial^2}{\partial\sigma^2}\log f(X;\sigma)\right].$$

These two formulas agree under the usual regularity conditions. For the Rayleigh scale family, the support $x\geq 0$ does not depend on $\sigma$, so the standard calculation applies.

For one observation $X$, take the logarithm of the density:

$$\ell(\sigma;x)=\log x-2\log\sigma-\frac{x^2}{2\sigma^2}.$$

Differentiate once to get the score, and again to get the second derivative:

$$\frac{\partial\ell}{\partial\sigma}=-\frac{2}{\sigma}+\frac{x^2}{\sigma^3},\qquad \frac{\partial^2\ell}{\partial\sigma^2}=\frac{2}{\sigma^2}-\frac{3x^2}{\sigma^4}.$$

The Rayleigh second moment is $\mathbb{E}[X^2]=2\sigma^2$. Substituting it into the expected negative second derivative gives

$$\begin{aligned} I(\sigma)&=-\mathbb{E}\left[\frac{2}{\sigma^2}-\frac{3X^2}{\sigma^4}\right]\\ &=-\frac{2}{\sigma^2}+\frac{3(2\sigma^2)}{\sigma^4}\\ &=\frac{4}{\sigma^2}. \end{aligned}$$

Thus, for one Rayleigh observation parameterized by its scale $\sigma$, the Fisher information is $4/\sigma^2$.

Check using the score variance

The score is $U(\sigma;X)=-2/\sigma+X^2/\sigma^3$. Its expected value is zero because $\mathbb{E}[X^2]=2\sigma^2$. Also, the fourth moment is $\mathbb{E}[X^4]=8\sigma^4$, so

$$\operatorname{Var}(X^2)=8\sigma^4-(2\sigma^2)^2=4\sigma^4.$$

The constant term in the score does not affect its variance, hence

$$\operatorname{Var}(U)=\frac{\operatorname{Var}(X^2)}{\sigma^6}=\frac{4}{\sigma^2},$$

which confirms the result.

Information in a sample of size n

If $X_1,\ldots,X_n$ are independent Rayleigh observations with the same scale $\sigma$, their log-likelihood is the sum of the individual log-likelihoods. Independent observations contribute additive Fisher information, so

$$I_n(\sigma)=nI(\sigma)=\frac{4n}{\sigma^2}.$$

For example, suppose $\sigma=2$ and the sample contains $n=10$ observations. Then

$$I_{10}(2)=\frac{4(10)}{2^2}=10.$$

For an unbiased estimator of $\sigma$ under the usual regularity conditions, the Cramér–Rao lower bound is therefore $\operatorname{Var}(\hat\sigma)\geq 1/I_n(\sigma)=\sigma^2/(4n)$. In this numerical example, that bound is $0.1$.

If the parameter is the squared scale

Some sources write the same family using $\theta=\sigma^2$:

$$f(x;\theta)=\frac{x}{\theta}\exp\left(-\frac{x^2}{2\theta}\right),\qquad \theta>0.$$

The information must be recalculated for this parameter, or transformed using the chain rule. Since $d\sigma/d\theta=1/(2\sqrt{\theta})$, information transforms as $I_\theta=I_\sigma(d\sigma/d\theta)^2$. Therefore,

$$I_\theta=\frac{4}{\sigma^2}\cdot\frac{1}{4\theta}=\frac{1}{\theta^2},$$

because $\sigma^2=\theta$. For $n$ observations, $I_n(\theta)=n/\theta^2$. The two answers describe the same distribution but refer to different parameters; Fisher information is not unchanged by a change of parameter.

Common mistakes

• Using a different scale convention. In the standard density above, $\sigma$ is the Rayleigh scale and $\mathbb{E}[X^2]=2\sigma^2$. Check the density before using a formula from another source.
• Forgetting to square the score. The score-variance formula uses the expected square of the score, not just its expected value; the score itself has expectation zero.
• Confusing one observation with a sample. The information for $n$ independent observations is $n$ times the single-observation information.
• Changing parameters without transforming information. The formula $4/\sigma^2$ is for $\sigma$, whereas $1/\theta^2$ is for $\theta=\sigma^2$.