How Is the Multivariate Gaussian Density Derived?

A multivariate Gaussian, or multivariate normal, distribution describes a random vector using a mean vector and a covariance matrix. Its density is the higher-dimensional counterpart of the familiar one-dimensional normal density.

The formula follows directly from a change of variables: start with independent standard normal coordinates, transform them to have the desired covariance, and shift them to the desired mean. The determinant in the density comes from the change in volume under that transformation.

The multivariate Gaussian density

Let $X$ be a multivariate normal random vector in $\mathbb{R}^d$ with mean vector $\mu$ and positive-definite covariance matrix $\Sigma$. Its density at $x\in\mathbb{R}^d$ is

$$f_X(x)=\frac{1}{(2\pi)^{d/2}\sqrt{\det\Sigma}}\exp\left(-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)\right).$$

Here, $d$ is the number of dimensions, $\Sigma^{-1}$ is the inverse covariance matrix, and $\det\Sigma$ is its determinant. The exponent contains the squared Mahalanobis distance from $x$ to the mean. Unlike ordinary squared distance, this measure accounts for the variables’ different scales and their correlations.

The condition that $\Sigma$ be positive definite ensures it is invertible and that the multivariate normal distribution has an ordinary density over all of $\mathbb{R}^d$.

Derivation by change of variables

Let $Z=(Z_1,\ldots,Z_d)^T$, where the coordinates are independent standard normal variables. Its joint density is the product of the one-dimensional densities:

$$f_Z(z)=(2\pi)^{-d/2}\exp\left(-\frac12 z^Tz\right).$$

Choose a nonsingular matrix $A$ such that $AA^T=\Sigma$. Such a matrix exists for every positive-definite covariance matrix; a Cholesky factor is a common choice. Define

$$X=\mu+AZ.$$

This transformation gives $X$ mean $\mu$ and covariance $A I A^T=\Sigma$. To find its density, solve for $z$:

$$z=A^{-1}(x-\mu).$$

The change-of-variables formula multiplies the density by the absolute Jacobian determinant. Since the derivative of $z$ with respect to $x$ is $A^{-1}$, that factor is $|\det A^{-1}|=1/|\det A|$. Also,

$$\det\Sigma=\det(AA^T)=(\det A)^2,$$

so $|\det A|=\sqrt{\det\Sigma}$. The exponent transforms as

$$z^Tz=(x-\mu)^T(A^{-1})^TA^{-1}(x-\mu)=(x-\mu)^T\Sigma^{-1}(x-\mu),$$

because $(A^{-1})^TA^{-1}=(AA^T)^{-1}=\Sigma^{-1}$. Substituting the exponent and Jacobian into the standard normal density gives the stated formula.

Worked two-dimensional example

Consider a bivariate normal vector with

$$\mu=\begin{pmatrix}1\\-1\end{pmatrix},\qquad \Sigma=\begin{pmatrix}4&1\\1&2\end{pmatrix}.$$

The determinant is $4\cdot2-1\cdot1=7$, and the inverse is

$$\Sigma^{-1}=\frac17\begin{pmatrix}2&-1\\-1&4\end{pmatrix}.$$

Writing $u=x_1-1$ and $v=x_2+1$, the density becomes

$$f_X(x_1,x_2)=\frac{1}{2\pi\sqrt7}\exp\left(-\frac{2u^2-2uv+4v^2}{14}\right).$$

For example, at $(x_1,x_2)=(2,1)$, we have $u=1$ and $v=2$. The numerator of the quadratic expression is $2-4+16=14$, so the exponent is $-1$. Thus the density there is $e^{-1}/(2\pi\sqrt7)$. This is a density value, not the probability of observing that exact point; for a continuous distribution, probabilities are calculated over regions.

Why the determinant and inverse appear

The determinant measures how the transformation scales volume. A covariance matrix that spreads probability across a larger volume has a larger determinant, so the density’s peak is scaled down by $1/\sqrt{\det\Sigma}$. This factor ensures the density integrates to one.

The inverse covariance matrix determines the shape of the exponent’s contours. Directions with large variance contribute less to the quadratic distance, while directions with small variance contribute more. Off-diagonal entries of $\Sigma$ encode relationships between coordinates and produce cross-terms, such as $-2uv$ in the example.

If $\Sigma$ is diagonal, the coordinates are uncorrelated; for a multivariate normal vector, they are then independent. The multivariate density reduces to the product of the corresponding one-dimensional normal densities.

Common mistakes

• Using $\det\Sigma$ instead of its square root. The Jacobian is $1/|\det A|$, which equals $1/\sqrt{\det\Sigma}$.
• Putting $\Sigma$ in the quadratic form. The correct matrix is $\Sigma^{-1}$; covariance itself describes spread, not the inverse weighting used in the exponent.
• Leaving out the factor $1/2$. It is inherited from the standard normal density.
• Assuming the formula works unchanged for singular covariance. If $\Sigma$ is singular, there is no ordinary density over all of $\mathbb{R}^d$; the distribution is concentrated on a lower-dimensional subspace and needs a different treatment.