What Is the Fisher Information of a Geometric Distribution?

The Fisher information of a geometric distribution depends on how the distribution is parameterized and what the random variable counts. Using the usual success-probability parameter $p$, the information per observation is $1/[p^2(1-p)]$. For $n$ independent observations, multiply this by $n$.

Here, a geometric random variable counts the total number of trials up to and including the first success. The answer is unchanged if you instead count failures before the first success, provided $p$ still denotes the success probability.

Define the geometric model

Take $0<p<1$ and let $X$ be the number of trials up to and including the first success. Its probability mass function is

The parameter of interest is $p$. Fisher information measures how much the distribution of the data can tell you about that parameter. For a regular model, it can be calculated either as the variance of the score or as the negative expected second derivative of the log-likelihood.

Derive the information for one observation

For one observed value $x$, the log-likelihood is

Differentiate with respect to $p$:

The geometric distribution has $\mathbb{E}[X-1]=(1-p)/p$. Therefore, the expected Fisher information is

Thus one geometric observation contributes $1/[p^2(1-p)]$ units of Fisher information about $p$.

Extend the result to a sample

Let $X_1,\ldots,X_n$ be independent geometric observations with the same success probability $p$. Write $S=\sum_{i=1}^n X_i$. Their log-likelihood is

The second derivative is $-n/p^2-(S-n)/(1-p)^2$. Since $\mathbb{E}[S-n]=n(1-p)/p$, taking the negative expectation gives

This additivity is a general property of Fisher information for independent observations: each observation contributes the same amount, so the total is $nI(p)$.

Worked example

Suppose you observe $n=10$ independent geometric counts and want information about a success probability of $p=0.5$. Each observation contributes

The sample therefore has Fisher information $I_{10}(0.5)=10\times 8=80$. The Cramér–Rao lower bound says that the variance of any unbiased estimator of $p$ is at least $1/80=0.0125$, under the usual regularity conditions.

If the parameter is the failure probability

Some definitions write the geometric distribution in terms of the failure probability $q=1-p$. If $X$ counts failures before the first success, then $P(X=x)=(1-q)q^x$ for $x=0,1,2,\ldots$. In that parameterization, the information per observation is

This is the same model expressed with a different parameter. Fisher information changes under reparameterization; it is not a parameter-free number. For example, if you use the mean number of trials $\mu=1/p$ as the parameter, the information per observation is $1/[\mu(\mu-1)]$ for $\mu>1$.

Common mistakes

• Mixing up the two support conventions: The total-trials version takes values $1,2,\ldots$; the failures-count version takes values $0,1,\ldots$. Their information agrees when both use the same success probability $p$.
• Forgetting the sample size: The formula $1/[p^2(1-p)]$ is for one observation. For $n$ independent observations, use $n/[p^2(1-p)]$.
• Using the wrong parameter: If a formula uses failure probability rather than success probability, substitute $q=1-p$ or derive the information in that parameter directly.
• Confusing expected and observed information: The negative second derivative for a particular sample depends on the data. Fisher information is its expectation under the model.