Ma · Mathematics

What Are the Main Properties of the Spectral Norm?

Published By the 9to5Science editorial team

At a glance

  1. 1. Define the norm

    The spectral norm is the largest amount a matrix can stretch a unit vector in Euclidean length.

  2. 2. Use singular values

    Compute it as the largest singular value, or as the square root of the largest eigenvalue of A* A.

  3. 3. Apply the identities

    Use unitary invariance, submultiplicativity, and the bounds relating the spectral and Frobenius norms.

  4. 4. Check an example

    Find A* A, compute its largest eigenvalue, and take the square root.

Advertisement

The spectral norm measures the greatest amount a matrix can stretch a vector in Euclidean length. It is also called the matrix 2-norm or the operator norm induced by the Euclidean norm. Its key computational feature is that it equals the largest singular value of the matrix.

This article gives the definition, the most useful properties, and a worked example. The formulas apply to real or complex matrices; for a complex matrix, use the conjugate transpose, written A∗A^*.

Definition and equivalent formulas

For a matrix A∈Cm×nA\in\mathbb{C}^{m\times n}, the spectral norm is

∥A∥2=max⁡x≠0∥Ax∥2∥x∥2=max⁡∥x∥2=1∥Ax∥2.\|A\|_2=\max_{x\ne 0}\frac{\|Ax\|_2}{\|x\|_2} =\max_{\|x\|_2=1}\|Ax\|_2.

In words, among all unit-length input vectors, find the one whose image under AA has the greatest Euclidean length. The spectral norm is that greatest length.

If the singular values of AA are σ1≥σ2≥⋯≥0\sigma_1\geq\sigma_2\geq\cdots\geq 0, then

∥A∥2=σ1.\|A\|_2=\sigma_1.

A second useful formula is

∥A∥2=λmax⁡(A∗A),\|A\|_2=\sqrt{\lambda_{\max}(A^*A)},

where λmax⁡(A∗A)\lambda_{\max}(A^*A) is the largest eigenvalue of the positive semidefinite matrix A∗AA^*A. This works for rectangular matrices too: A∗AA^*A is always square, of size n×nn\times n.

Important properties

The spectral norm is a norm, so it satisfies nonnegativity, definiteness, absolute homogeneity, and the triangle inequality. In particular, ∥A∥2=0\|A\|_2=0 only when A=0A=0, and ∥cA∥2=∣c∣∥A∥2\|cA\|_2=|c|\|A\|_2 for a scalar cc.

Several further properties are especially useful:

  • Submultiplicativity: If the matrix product ABAB is defined, then ∥AB∥2≤∥A∥2∥B∥2\|AB\|_2\leq\|A\|_2\|B\|_2. This follows because applying BB and then AA cannot stretch a vector by more than the product of their maximum stretch factors.
  • Transpose or adjoint: ∥A∗∥2=∥A∥2\|A^*\|_2=\|A\|_2. For a real matrix, this says ∥AT∥2=∥A∥2\|A^T\|_2=\|A\|_2.
  • Unitary invariance: If UU and VV are unitary matrices of compatible sizes, then ∥UAV∥2=∥A∥2\|UAV\|_2=\|A\|_2. Orthogonal matrices are the real version of unitary matrices. They preserve Euclidean lengths, so they do not change the maximum stretch.
  • Comparison with the Frobenius norm: If r=rank⁡(A)r=\operatorname{rank}(A), then
∥A∥2≤∥A∥F≤r ∥A∥2.\|A\|_2\leq\|A\|_F\leq\sqrt{r}\,\|A\|_2.

The Frobenius norm is the square root of the sum of the squared singular values, while the spectral norm is their maximum.

For a normal square matrix, including a real symmetric or complex Hermitian matrix, the spectral norm equals the largest absolute value of its eigenvalues. For a general matrix, that shortcut is not valid: the spectral norm is determined by singular values, not simply by eigenvalues.

Worked example

Find the spectral norm of

A=[1100].A=\begin{bmatrix}1&1\\0&0\end{bmatrix}.

First compute the transpose (here the matrix is real) and multiply:

ATA=[1010][1100]=[1111].A^T A= \begin{bmatrix}1&0\\1&0\end{bmatrix} \begin{bmatrix}1&1\\0&0\end{bmatrix} = \begin{bmatrix}1&1\\1&1\end{bmatrix}.

The eigenvalues of ATAA^T A are 22 and 00. Therefore the largest singular value of AA is the square root of its largest eigenvalue:

∥A∥2=2.\|A\|_2=\sqrt{2}.

You can also verify the maximum directly. For the unit vector x=(1,1)T/2x=(1,1)^T/\sqrt{2}, Ax=(2,0)TAx=(\sqrt{2},0)^T, whose Euclidean length is 2\sqrt{2}. No unit vector can be stretched by more than the largest singular value, so this achieves the maximum.

Common mistakes

  • Using the largest eigenvalue of AA for every matrix. The norm equals the largest absolute eigenvalue only for normal matrices. For example, N=[0100]N=\begin{bmatrix}0&1\\0&0\end{bmatrix} has only zero eigenvalues, but ∥N∥2=1\|N\|_2=1.
  • Forgetting the square root. The largest eigenvalue of A∗AA^*A is the square of the spectral norm, not the norm itself.
  • Confusing it with the Frobenius norm. The Frobenius norm combines the sizes of all singular values; the spectral norm keeps only the largest one. They are equal for a matrix of rank one, but not in general.
  • Assuming multiplication always preserves the norm. Unitary factors preserve the spectral norm. General factors need not; submultiplicativity provides an upper bound instead.

When to use the spectral norm

Use the spectral norm when you need a worst-case bound on how a linear transformation affects Euclidean error. For example, for every vector xx, ∥Ax∥2≤∥A∥2∥x∥2\|Ax\|_2\leq\|A\|_2\|x\|_2. This makes the norm useful in numerical error analysis, optimization, and bounds for products of matrices. For large matrices, numerical software usually estimates the largest singular value rather than computing a full singular value decomposition.

References

  1. Matrix Analysis — Roger A. Horn and Charles R. Johnson
  2. Matrix Computations — Gene H. Golub and Charles F. Van Loan

Spotted an error? Send a correction. Read how we write and check articles in our editorial policy.