How to Minimize a Sum-of-Squares Error Function

A sum-of-squares error measures how far a model’s predictions are from observed values. To minimize it, define the residuals, form the objective, and find parameter values that minimize that objective. The exact method depends on the model: linear models lead to a system of least-squares equations, while nonlinear models may require iterative numerical optimization.

Without a specific function or dataset, there is no single numerical minimizer to report. The method below covers the standard least-squares setup and shows a complete example.

Define the error function

Suppose you have observations $y_1,\ldots,y_m$ and a model prediction $f_i(\theta)$ for observation $i$, where $\theta$ is the vector of parameters. The residual is the difference between the prediction and the observed value:

$$r_i(\theta)=f_i(\theta)-y_i.$$

The sum-of-squares error, also called the sum of squared residuals, is

$$E(\theta)=\sum_{i=1}^{m} r_i(\theta)^2.$$

Minimizing $E$ means choosing parameters that make the total squared discrepancy as small as possible. Squaring prevents positive and negative residuals from canceling and gives larger errors more influence.

Differentiate and find candidate minima

For a differentiable model, differentiate the objective with respect to its parameters and set the gradient equal to zero. If $J(\theta)$ is the Jacobian matrix of the residuals, then

$$\nabla E(\theta)=2J(\theta)^{\mathsf T}r(\theta).$$

Thus, any interior differentiable minimum must satisfy

$$J(\theta)^{\mathsf T}r(\theta)=0.$$

This condition finds stationary points, not automatically global minima. For a linear model, the objective is a convex quadratic, so any solution of the normal equations is a global minimizer. For a nonlinear model, a stationary point could instead be a local minimum, a local maximum, or a saddle point; further checks or an optimization algorithm may be needed.

Linear least squares

If the model is linear in its parameters, write its predictions as $A\theta$, where $A$ is the design matrix. With observation vector $b$, the objective is

$$E(\theta)=\|A\theta-b\|_2^2.$$

Differentiating and setting the gradient to zero gives the normal equations:

$$A^{\mathsf T}A\theta=A^{\mathsf T}b.$$

If the columns of $A$ are linearly independent, the minimizer is unique and can be written as

$$\hat\theta=(A^{\mathsf T}A)^{-1}A^{\mathsf T}b.$$

The inverse formula is useful for theory, but is often not the best way to compute a result. In numerical work, QR factorization or singular value decomposition (SVD) is generally more stable, especially when columns of $A$ are nearly dependent. If the columns are dependent, a minimizer still exists, but there may be multiple parameter vectors giving the same minimum error.

Worked example: fit a straight line

Fit a line $y=a+bx$ to the three observations $(1,1)$, $(2,2)$, and $(3,2)$. The error function is

$$E(a,b)=(a+b-1)^2+(a+2b-2)^2+(a+3b-2)^2.$$

For a straight-line least-squares fit, the slope and intercept can be calculated from the sample means:

$$\bar{x}=2,\qquad \bar{y}=\frac{5}{3},$$ $$b=\frac{\sum_i(x_i-\bar{x})(y_i-\bar{y})}{\sum_i(x_i-\bar{x})^2} =\frac{1}{2}, \qquad a=\bar{y}-b\bar{x}=\frac{2}{3}.$$

So the fitted line is $\hat{y}=\frac{2}{3}+\frac{1}{2}x$. Its predictions are $\frac{7}{6}$, $\frac{5}{3}$, and $\frac{13}{6}$. The residuals (prediction minus observation) are $\frac{1}{6}$, $-\frac{1}{3}$, and $\frac{1}{6}$, giving

$$E\left(\frac{2}{3},\frac{1}{2}\right)=\left(\frac{1}{6}\right)^2+\left(-\frac{1}{3}\right)^2+\left(\frac{1}{6}\right)^2=\frac{1}{6}.$$

The objective is a convex quadratic in $a$ and $b$, and the design matrix has independent columns, so these parameters give the unique global minimum.

Nonlinear models and variants

When $f_i(\theta)$ is nonlinear in the parameters, the gradient equation usually cannot be solved in a single step. Methods such as gradient descent, Gauss–Newton, and Levenberg–Marquardt search for a minimum iteratively. Their result can depend on the initial parameter values, so it is sensible to check convergence and, when practical, try more than one starting point.

If observations should have different importance, use a weighted sum of squares, $E(\theta)=\sum_i w_i r_i(\theta)^2$, with nonnegative weights $w_i$. The weights change the objective and therefore can change the minimizing parameters.

Common mistakes

• Forgetting to square the residuals: The sum $\sum_i r_i$ is a different objective and allows errors to cancel.
• Assuming every stationary point is a global minimum: This follows for linear least squares because the objective is convex, but not for every nonlinear model.
• Assuming the minimizer is always unique: Uniqueness for a linear model requires the design matrix to have full column rank.
• Computing with an explicit matrix inverse by default: QR or SVD is usually a more reliable numerical approach.
• Confusing squared error with root-mean-square error: Dividing the sum by the number of observations, or taking its square root, changes the reported scale; for an unweighted fixed dataset, those transformations do not change the minimizing parameters.