How to Compute a Covariance Matrix from an SVD in NumPy
At a glance
- 1. Arrange observations
Use rows for observations and columns for variables, then subtract each column’s mean.
- 2. Compute the SVD
Factor the centered data matrix as X = UΣVᵀ with NumPy’s reduced SVD.
- 3. Square singular values
Divide each squared singular value by n − 1 for the usual sample covariance.
- 4. Reconstruct and check
Form V diag(s²/(n − 1)) Vᵀ and compare it with numpy.cov.
If your data matrix has observations in rows and variables in columns, the covariance matrix can be calculated directly from the singular values and right singular vectors of the centered data. The main things to check are the data orientation and the divisor: NumPy’s default covariance uses the sample divisor .
This article derives the formula, shows how to implement it with numpy.linalg.svd, and checks the result against numpy.cov.
The SVD covariance formula
Let be a real data matrix, with observations and variables. Center each variable by subtracting its mean across observations:
The sample covariance matrix is
Now take the singular value decomposition of the centered matrix:
Since , multiplying out gives
The diagonal entries of are the squared singular values. If s is the vector of singular values and Vh is NumPy’s returned , then
This form works with NumPy’s reduced SVD, including when there are more variables than observations. In that case, Vh has only as many rows as there are singular values, but the reconstructed covariance still has shape .
NumPy implementation
import numpy as np
A = np.array([
[1.0, 2.0],
[2.0, 4.0],
[3.0, 6.0],
])
n = A.shape[0]
X = A - A.mean(axis=0) # center each variable
U, s, Vh = np.linalg.svd(X, full_matrices=False)
# For real data this is V diag(s**2 / (n - 1)) V.T;
# the conjugation also matches np.cov for complex data.
cov_svd = (Vh.T * (s**2 / (n - 1))) @ Vh.conj()
# Independent check using NumPy's default sample covariance
cov_numpy = np.cov(A, rowvar=False)
print(cov_svd)
print(cov_numpy)
print(np.allclose(cov_svd, cov_numpy))
Both covariance calculations give, up to floating-point rounding,
[[1. 2.]
[2. 4.]]
True
The centered matrix in this example is
Its nonzero singular value is , and its right singular vector is . The covariance is therefore
The second singular value is zero because the two columns are perfectly dependent. A rank-deficient data set is not a problem: zero singular values contribute zero to the covariance.
Match NumPy’s conventions
By default, np.cov treats rows as variables and uses the sample divisor . With observations in rows, specify rowvar=False:
np.cov(A, rowvar=False)
For the population covariance, divide by instead. Equivalently, use np.cov(A, rowvar=False, bias=True) or ddof=0. If you use ddof=1, the default sample convention is explicit. With only one observation, sample covariance is undefined because its divisor is zero.
The SVD expression assumes you centered the data first. Applying it directly to uncentered data computes a second-moment matrix about zero, not the usual covariance around the sample means.
Common mistakes
- Using the wrong orientation: If rows are observations,
np.cov(A)interprets them as variables. Userowvar=Falsewhen comparing results. - Forgetting to center: Compute the SVD of
A - A.mean(axis=0), not generally ofAitself. - Using the wrong divisor: Divide squared singular values by to match NumPy’s default sample covariance; divide by for population covariance.
- Reconstructing the wrong matrix: The feature covariance is , so use the right singular vectors,
Vh.TandVh. The expression involvingUinstead describes the observation-space matrix , which has a different shape. - Expecting exact equality: SVD and covariance routines can accumulate rounding error differently. Compare with
np.allclose, not==.
For complex-valued data, NumPy’s covariance convention conjugates the second variable. With observations in rows, the centered covariance is X.T @ X.conj() / (n - 1). If X = UΣVh, this gives the SVD expression Vh.T diag(s**2 / (n - 1)) Vh.conj(). The code above uses this form; for real-valued data it reduces to the transpose formula derived earlier. NumPy’s SVD returns Vh as the conjugate transpose of the right singular-vector matrix.
References
Spotted an error? Send a correction. Read how we write and check articles in our editorial policy.