What Is the Fisher Information of a Binomial Distribution?

For a binomial random variable $X\sim\operatorname{Bin}(n,p)$, the usual question is how much information the observed count provides about the unknown success probability $p$. Assume throughout that the trial count $n$ is known and that $0<p<1$.

The Fisher information in one binomial count is

$$I(p)=\frac{n}{p(1-p)}.$$

This is also the information in $n$ independent Bernoulli trials with the same success probability. The derivation below shows why, and explains how the answer changes if you use a different parameterization.

Derive the information

The probability mass function is

$$P(X=x)=\binom{n}{x}p^x(1-p)^{n-x},\qquad x=0,1,\ldots,n.$$

For an observed count $x$, the log-likelihood as a function of $p$ is

$$\ell(p;x)=\log\binom{n}{x}+x\log p+(n-x)\log(1-p).$$

The score is the derivative of the log-likelihood:

$$U(p)=\frac{\partial\ell}{\partial p}=\frac{x}{p}-\frac{n-x}{1-p}.$$

Fisher information can be calculated as the expected negative second derivative of the log-likelihood:

$$\begin{aligned} I(p) &=-\mathbb{E}\left[\frac{\partial^2\ell}{\partial p^2}\right]\\ &=\mathbb{E}\left[\frac{X}{p^2}+\frac{n-X}{(1-p)^2}\right]\\ &=\frac{np}{p^2}+\frac{n(1-p)}{(1-p)^2}\\ &=\frac{n}{p}+\frac{n}{1-p}\\ &=\frac{n}{p(1-p)}. \end{aligned}$$

The expectation uses $\mathbb{E}[X]=np$. Under the usual regularity conditions, the same result comes from the variance of the score: $I(p)=\mathbb{E}[U(p)^2]$. In this case the score has mean zero and can be written as $(X-np)/[p(1-p)]$, so its variance is $np(1-p)/[p^2(1-p)^2]=n/[p(1-p)]$.

One count versus many trials

A binomial count is the sum of $n$ independent Bernoulli outcomes. Each Bernoulli trial contributes $1/[p(1-p)]$ information about $p$, and independent observations contribute additively. Thus, $n$ trials contribute $n/[p(1-p)]$.

If instead you observe $m$ independent binomial counts, each based on $n$ trials and sharing the same $p$, their total information is $mn/[p(1-p)]$. More generally, if the counts have different known trial sizes $n_1,\ldots,n_m$, it is $(n_1+\cdots+n_m)/[p(1-p)]$.

Worked example

Suppose $X\sim\operatorname{Bin}(20,p)$ and you want the information about $p$ evaluated at $p=0.3$. Substitute $n=20$ and $p=0.3$:

$$I(0.3)=\frac{20}{0.3(1-0.3)}=\frac{20}{0.21}\approx95.24.$$

The inverse information is $1/I(0.3)=0.0105$. The Cramér–Rao lower bound says that an unbiased estimator of $p$ has variance at least this large, under the theorem’s regularity conditions. The sample proportion $\hat p=X/20$ has variance $p(1-p)/20$, which equals $1/I(p)$, so it attains the bound.

The observed negative second derivative is sometimes called the observed information. For a particular observed count $x$, it is

$$J(p;x)=\frac{x}{p^2}+\frac{n-x}{(1-p)^2}.$$

It depends on the observed data; Fisher information is its expectation and equals $n/[p(1-p)]$.

If you use a different parameter

Information depends on the parameter used to describe the model. If $\eta=\log(p/(1-p))$ is the log-odds parameter, then $dp/d\eta=p(1-p)$. The change-of-parameter rule gives

$$I(\eta)=I(p)\left(\frac{dp}{d\eta}\right)^2=np(1-p).$$

So $n/[p(1-p)]$ is the information about $p$, not about every possible parameterization. Check which parameter the problem treats as unknown before applying the formula.

Common mistakes

• Leaving out $n$: $1/[p(1-p)]$ is the information from one Bernoulli trial. A binomial count based on $n$ trials has $n$ times that information.
• Confusing observed and expected information: the observed value depends on $x$; Fisher information averages over the distribution of $X$.
• Using the formula at the boundary: the standard derivation assumes $0<p<1$. At $p=0$ or $p=1$, the model is degenerate and regularity conditions fail; do not treat the formula’s limiting divergence as ordinary Fisher information at those boundary points.
• Changing the parameter without transforming the information: the numerical expression changes under transformations such as probability to log-odds.

For the standard binomial model with known $n$ and unknown interior probability $p$, the result is $I(p)=n/[p(1-p)]$.