How to Find a Gaussian Classifier’s Decision Boundary

A Gaussian model does not determine a binary decision boundary by itself. You need a probability model for each class, along with the class prior probabilities and a rule for deciding which kinds of errors matter.

For the usual Bayes classifier with equal costs for misclassification, the boundary consists of points where the two class posterior probabilities are equal. The calculation also shows why a shared covariance matrix gives a straight boundary, while different covariance matrices usually produce a quadratic one.

Set up the two class models

Let the observed feature vector be $x \in \mathbb{R}^d$, and let the class label be $Y \in \{0,1\}$. Assume the class-conditional distributions are

$$x \mid Y=k \sim \mathcal{N}(\mu_k,\Sigma_k), \qquad k\in\{0,1\},$$

with prior probabilities $\pi_k=P(Y=k)$. Each covariance matrix is assumed to be positive definite so that the usual Gaussian density is defined.

Bayes’ rule says that the posterior probability of class $k$ is proportional to its prior times its class-conditional density:

$$P(Y=k\mid x) \propto \pi_k p(x\mid Y=k).$$

With equal costs for the two types of classification error, predict class 1 when $P(Y=1\mid x)>P(Y=0\mid x)$. The decision boundary is where these probabilities are equal.

General Gaussian boundary

Taking logarithms is convenient because it turns products into sums. After dropping the Gaussian constant $-\frac{d}{2}\log(2\pi)$, the score for class $k$ is

$$g_k(x)=\log\pi_k-\frac{1}{2}\log|\Sigma_k|-\frac{1}{2}(x-\mu_k)^T\Sigma_k^{-1}(x-\mu_k).$$

The boundary is the set of points satisfying

$$g_1(x)=g_0(x).$$

Predict class 1 where $g_1(x)>g_0(x)$, and class 0 where $g_0(x)>g_1(x)$. When $\Sigma_0$ and $\Sigma_1$ differ, the inverse-covariance terms generally leave squared and cross-product terms in $x$. The boundary is therefore generally quadratic (though special parameter choices can simplify it).

Shared covariance gives a linear boundary

If both classes have the same covariance matrix, $\Sigma_0=\Sigma_1=\Sigma$, their quadratic terms in $x$ cancel when you subtract the scores. The boundary becomes

$$(\mu_1-\mu_0)^T\Sigma^{-1}x -\frac{1}{2}\left(\mu_1^T\Sigma^{-1}\mu_1-\mu_0^T\Sigma^{-1}\mu_0\right) +\log\frac{\pi_1}{\pi_0}=0.$$

This is a linear equation in the features. The normal vector to the boundary is $\Sigma^{-1}(\mu_1-\mu_0)$; the priors shift its position. If the priors are equal, the log-prior term vanishes. This shared-covariance model is often called linear discriminant analysis (LDA). The unequal-covariance Gaussian classifier is often called quadratic discriminant analysis (QDA).

Worked one-dimensional example

Suppose

$$X\mid Y=0\sim\mathcal{N}(-1,4),\qquad X\mid Y=1\sim\mathcal{N}(2,4),$$

and the class priors are $\pi_0=0.6$ and $\pi_1=0.4$. The variance is shared, so there is a single threshold. Setting the two class scores equal gives

$$x=\frac{\mu_0+\mu_1}{2}+\frac{\sigma^2}{\mu_1-\mu_0}\log\frac{\pi_0}{\pi_1} =\frac{1}{2}+\frac{4}{3}\log(1.5)\approx 1.041.$$

The score for class 1 increases with $x$, so predict class 1 when $x>1.041$ and class 0 when $x<1.041$. The more common class 0 pushes the threshold to the right of the midpoint, making the classifier require stronger evidence for class 1. At the threshold, the posterior probabilities are equal.

If the error costs differ

Equal posterior probabilities are the boundary only when the two kinds of error have equal cost. Let $C_{10}$ be the cost of predicting 1 when the true class is 0, and $C_{01}$ the cost of predicting 0 when the true class is 1. Predict class 1 when

$$\frac{P(Y=1\mid x)}{P(Y=0\mid x)}>\frac{C_{10}}{C_{01}}.$$

The boundary is then where the posterior odds equal this cost ratio, rather than 1. The equal-cost rule is recovered when $C_{10}=C_{01}$.

Common mistakes

• Using only one class distribution: A boundary compares classes. Knowing $p(x\mid Y=1)$ without $p(x\mid Y=0)$ does not determine where to classify.
• Ignoring priors: The means and covariances alone are not enough when the classes are not equally likely. Priors can move the boundary.
• Assuming every Gaussian boundary is linear: Linearity requires shared covariance (or another special simplification). Different covariance matrices usually yield a quadratic boundary.
• Confusing a density with a posterior: A point can have a larger likelihood under one class but still have a larger posterior under another if the prior probabilities differ.

If a problem gives just one Gaussian mean and covariance matrix, check whether it also specifies the competing class model, priors, and decision costs. Without those, there is no unique binary decision boundary.