What Is the Fisher Information of a Normal Distribution?

The Fisher information for a normal model depends on how you parameterize its spread. Using the mean $\mu$ and variance $v=\sigma^2$, the information matrix for $n$ independent observations is diagonal: its entries are $n/v$ and $n/(2v^2)$. The off-diagonal entries are zero.

Below is the derivation, along with the equivalent result when the parameter is standard deviation $\sigma$.

Fisher information for one observation

Let $X$ have a normal distribution with unknown mean $\mu$ and unknown variance $v>0$. Its log-density is

$$\ell(\mu,v;x)=-\frac{1}{2}\log(2\pi v)-\frac{(x-\mu)^2}{2v}.$$

The score is the vector of derivatives of the log-density with respect to the parameters:

$$\begin{aligned} \frac{\partial\ell}{\partial\mu}&=\frac{x-\mu}{v},\\ \frac{\partial\ell}{\partial v}&=-\frac{1}{2v}+\frac{(x-\mu)^2}{2v^2}. \end{aligned}$$

Fisher information is the expected outer product of the score with itself. Equivalently, under the usual regularity conditions, it is the negative expected Hessian of the log-likelihood. For this normal model, use the centered normal moments

$$\mathbb{E}[X-\mu]=0,\quad \mathbb{E}[(X-\mu)^2]=v,\quad \mathbb{E}[(X-\mu)^3]=0,\quad \mathbb{E}[(X-\mu)^4]=3v^2.$$

The mean-mean entry is

$$I_{\mu\mu}=\mathbb{E}\left[\left(\frac{X-\mu}{v}\right)^2\right]=\frac{1}{v}.$$

The cross entry is zero. In the score-product calculation, it is proportional to the third centered moment, which is zero for a normal distribution. For the variance-variance entry, write the variance score as $((X-\mu)^2-v)/(2v^2)$. Since the variance of $(X-\mu)^2$ is $3v^2-v^2=2v^2$, this gives

$$I_{vv}=\frac{2v^2}{4v^4}=\frac{1}{2v^2}.$$

Therefore, for one observation and parameter order $(\mu,v)$,

$$I_1(\mu,v)=\begin{pmatrix}\frac{1}{v}&0\\[4pt]0&\frac{1}{2v^2}\end{pmatrix}.$$

Information for n independent observations

For independent observations, the log-likelihood is a sum of the individual log-likelihoods. Expected Fisher information therefore adds across observations. For $X_1,\ldots,X_n$ drawn independently from the same normal distribution,

$$I_n(\mu,v)=nI_1(\mu,v)=\begin{pmatrix}\frac{n}{v}&0\\[4pt]0&\frac{n}{2v^2}\end{pmatrix}.$$

The result does not depend on the observed sample values. That is a feature of expected Fisher information; the observed information, calculated from a particular sample’s Hessian, can depend on the data.

If the parameter is standard deviation

Some sources use $\sigma$ rather than $v=\sigma^2$. Reparameterizing gives, for parameter order $(\mu,\sigma)$,

$$I_n(\mu,\sigma)=\begin{pmatrix}\frac{n}{\sigma^2}&0\\[4pt]0&\frac{2n}{\sigma^2}\end{pmatrix}.$$

The variance entry changes because $v=\sigma^2$: a change in $\sigma$ produces a change of $2\sigma$ in $v$. Thus the variance-parameter entry $n/(2v^2)$ is not the same as the standard-deviation-parameter entry $2n/\sigma^2$. Check which parameter a problem calls the variance or scale before using a formula.

Worked example

Suppose there are $n=100$ independent observations and the model has variance $v=4$ (so $\sigma=2$). With parameter order $(\mu,v)$, the information matrix is

$$I_{100}(\mu,v)=\begin{pmatrix}100/4&0\\0&100/(2\cdot 4^2)\end{pmatrix} =\begin{pmatrix}25&0\\0&3.125\end{pmatrix}.$$

Its inverse is

$$I_{100}^{-1}=\begin{pmatrix}0.04&0\\0&0.32\end{pmatrix}.$$

For a regular model, the inverse information gives the Cramér–Rao lower-bound matrix for unbiased estimators, and it also gives the asymptotic covariance matrix of the maximum-likelihood estimator under standard conditions. Here the corresponding asymptotic standard errors are $\sqrt{0.04}=0.2$ for the mean and $\sqrt{0.32}\approx0.566$ for the variance. These are asymptotic approximations, not a claim that every finite-sample estimator has exactly this covariance.

Common mistakes

• Mixing up variance and standard deviation. If the parameter is $v$, use $n/(2v^2)$; if it is $\sigma$, use $2n/\sigma^2$.
• Forgetting the sample size. The displayed one-observation matrix must be multiplied by $n$ for $n$ independent observations.
• Assuming zero cross-information means the parameters are known independently in every sense. It means the Fisher information matrix has no cross term in this parameterization. It does not make the unknown mean or variance known, and it does not guarantee zero covariance for every finite-sample estimator.
• Treating inverse information as an exact finite-sample covariance. It is a bound for unbiased estimators and, under regularity conditions, an asymptotic covariance approximation for maximum-likelihood estimators.