What Is Wald’s Second Equation?

Wald’s second equation, also called Wald’s second identity, relates the variability of a stopped sum to the expected number of terms. It is useful when a process accumulates independent increments and stops according to what has happened so far.

The key is to center each increment first. The identity applies to the sum after subtracting the mean contribution associated with the stopping time; it does not usually give the uncentered second moment of the sum.

The equation and its assumptions

Let $X_1,X_2,\ldots$ be independent and identically distributed random variables with mean $\mu$ and finite variance $\sigma^2$. Define the sum of the first $n$ terms by

$$S_n = \sum_{i=1}^{n} X_i.$$

Let $N$ be a nonnegative integer-valued stopping time: deciding whether to continue after step $n$ can use the observations through step $n$, but not future observations. Assume $\mathbb{E}[N] < \infty$. Then Wald’s second equation is

$$\mathbb{E}\left[(S_N - \mu N)^2\right] = \sigma^2\,\mathbb{E}[N].$$

Equivalently, if $Y_i=X_i-\mu$ are the centered increments, the equation says

$$\mathbb{E}\left[\left(\sum_{i=1}^{N}Y_i\right)^2\right] = \sigma^2\,\mathbb{E}[N].$$

The stopping-time condition matters. A rule that looks ahead at future increments can select unusually large or small terms, breaking the identity. More generally, the same result holds when each centered increment has conditional mean zero and conditional variance $\sigma^2$ given the information available before that increment.

Why the identity holds

Write $Y_i=X_i-\mu$. The stopped centered sum can be written as

$$S_N-\mu N = \sum_{i=1}^{\infty} \mathbf{1}_{\{N\ge i\}}Y_i,$$

where $\mathbf{1}_{\{N\ge i\}}$ is 1 if the process takes at least $i$ steps and 0 otherwise. The event $\{N\ge i\}$ is determined before observing $X_i$.

When the square of this sum is expanded, distinct-increment terms have expectation zero. For $i<j$, whether the process reaches step $j$ is already known before $Y_j$ is observed, and $Y_j$ has conditional mean zero. The diagonal terms contribute

$$\mathbb{E}\left[\mathbf{1}_{\{N\ge i\}}Y_i^2\right] = \sigma^2\,\mathbb{P}(N\ge i).$$

Adding these contributions gives

$$\mathbb{E}[(S_N-\mu N)^2] = \sigma^2\sum_{i=1}^{\infty}\mathbb{P}(N\ge i) = \sigma^2\,\mathbb{E}[N].$$

The last equality is the tail-sum formula for a nonnegative integer-valued random variable. Finite $\mathbb{E}[N]$ ensures the sum is finite and the stopped centered sum is well-defined in mean square.

Worked example: stop at the first head

Toss a fair coin repeatedly. Let $X_i=1$ for heads and $X_i=-1$ for tails, and stop at the first head. Then $\mu=0$ and $\sigma^2=1$. The number of tosses $N$ has a geometric distribution with success probability $1/2$, so

$$\mathbb{E}[N]=2, \qquad \operatorname{Var}(N)=2.$$

If the first head occurs on toss $N$, the preceding $N-1$ tosses are tails. Thus $S_N=1-(N-1)=2-N$. Since $\mu=0$, the centered stopped sum is just $S_N$. Therefore

$$\mathbb{E}[(S_N-\mu N)^2] = \mathbb{E}[(2-N)^2] = \operatorname{Var}(N)+(2-\mathbb{E}[N])^2 =2.$$

Wald’s second equation gives the same result: $\sigma^2\mathbb{E}[N]=1\cdot 2=2$.

Why it is not usually a formula for $\mathbb{E}[S_N^2]$

Since $S_N=\mu N+(S_N-\mu N)$, expanding the square gives

$$\mathbb{E}[S_N^2] = \mu^2\mathbb{E}[N^2] +2\mu\,\mathbb{E}[N(S_N-\mu N)] +\sigma^2\mathbb{E}[N],$$

when these moments exist. The middle term need not be zero: the stopping time and the centered stopped sum can be dependent. So replacing $\mathbb{E}[(S_N-\mu N)^2]$ with $\mathbb{E}[S_N^2]$ is generally incorrect.

For a fixed number $N=n$ of terms, the identity reduces to the familiar variance formula $\operatorname{Var}(S_n)=n\sigma^2$. If $N$ is random but independent of all the increments, additional simplifications may be possible; do not assume them for a data-dependent stopping rule.

Common mistakes

• Using the uncentered sum: The left side is $(S_N-\mu N)^2$, not generally $S_N^2$.
• Allowing a look-ahead rule: $N$ must be a stopping time; it cannot depend on increments that have not yet been observed.
• Ignoring finiteness: The standard version assumes finite increment variance and $\mathbb{E}[N]<\infty$.
• Confusing the two Wald equations: Wald’s first equation concerns $\mathbb{E}[S_N]=\mu\mathbb{E}[N]$ under suitable conditions. Wald’s second equation concerns the mean square of the centered stopped sum.