Section 22

Least squares

advancedviz

\overset{x}{^} = ar g x min ∥ A x - b ∥_{2}^{2}

Seeks the best approximation when Ax=b has no exact solution.

View detail →

Normal equations

advancedviz

A^{T} A \overset{x}{^} = A^{T} b

Characterize the projection of b onto the column space of A.

View detail →

The closed-form expression $(A^{T} A)^{- 1} A^{T} b$ requires the columns of $A$ to be linearly independent. For general cases, QR or SVD is preferred.

Pseudoinverse

advancedviz

\overset{x}{^} = A^{+} b; A^{+} = V Σ^{+} U^{T}

Produces minimum-norm least-squares solutions.

View detail →