Section 24

Matrix norms and conditioning

avanzadoviz

\|A\|_p = \max_{\mathbf x\neq\mathbf0} \frac{\|A\mathbf x\|_p}{\|\mathbf x\|_p}.

Equivalently

\|A\|_p = \max_{\|\mathbf x\|_p=1}\|A\mathbf x\|_p.

Measures the maximum factor by which a matrix can amplify vectors under a given norm.

View detail →

Frobenius norm

intermedioviz

For $A=(a_{ij})\in\mathbb R^{m\times n}$

\|A\|_F = \sqrt{\sum_{i=1}^{m}\sum_{j=1}^{n}|a_{ij}|^2}.

Via the singular values

\|A\|_F = \sqrt{\sum_i\sigma_i^2}.

Extends the Euclidean norm to the set of all matrix entries.

View detail →

Matrix 1-norm

intermedioviz

\|A\|_1 = \max_j\sum_i|a_{ij}|.

It is the maximum absolute column sum.

View detail →

Matrix infinity norm

intermedioviz

\|A\|_\infty = \max_i\sum_j|a_{ij}|.

It is the maximum absolute row sum.

View detail →

Spectral norm

avanzadoviz

\|A\|_2 = \sigma_{\max}(A).

Also

\|A\|_2 = \sqrt{\lambda_{\max}(A^TA)}

It is the largest Euclidean stretching factor produced by a matrix.

View detail →

Submultiplicativity

avanzadoviz

For a submultiplicative matrix norm

\|AB\| \le \|A\|\,\|B\|.

The amplification of a composition does not exceed the product of the maximum amplifications of each transformation.

View detail →

Condition number

avanzadoviz

For an invertible matrix and a compatible norm

\kappa(A) = \|A\|\,\|A^{-1}\|.

In the 2-norm

\kappa_2(A) = \frac{\sigma_{\max}(A)}{\sigma_{\min}(A)}.

Measures the sensitivity of a linear problem to small perturbations in the data.

The formula \|A\|\|A^{-1}\| requires A to be invertible.
For a singular matrix, the condition number is considered infinite in this context.

View detail →

Section 24

Matrix norms and conditioning

Induced matrix norm

avanzadoviz

\|A\|_p = \max_{\mathbf x\neq\mathbf0} \frac{\|A\mathbf x\|_p}{\|\mathbf x\|_p}.

Equivalently

\|A\|_p = \max_{\|\mathbf x\|_p=1}\|A\mathbf x\|_p.

Measures the maximum factor by which a matrix can amplify vectors under a given norm.

View detail →

Frobenius norm

intermedioviz

For $A=(a_{ij})\in\mathbb R^{m\times n}$

\|A\|_F = \sqrt{\sum_{i=1}^{m}\sum_{j=1}^{n}|a_{ij}|^2}.

Via the singular values

\|A\|_F = \sqrt{\sum_i\sigma_i^2}.

Extends the Euclidean norm to the set of all matrix entries.

View detail →

Matrix 1-norm

intermedioviz

\|A\|_1 = \max_j\sum_i|a_{ij}|.

It is the maximum absolute column sum.

View detail →

Matrix infinity norm

intermedioviz

\|A\|_\infty = \max_i\sum_j|a_{ij}|.

It is the maximum absolute row sum.

View detail →

Spectral norm

avanzadoviz

\|A\|_2 = \sigma_{\max}(A).

Also

\|A\|_2 = \sqrt{\lambda_{\max}(A^TA)}

It is the largest Euclidean stretching factor produced by a matrix.

View detail →

Submultiplicativity

avanzadoviz

For a submultiplicative matrix norm

\|AB\| \le \|A\|\,\|B\|.

The amplification of a composition does not exceed the product of the maximum amplifications of each transformation.

View detail →

Condition number

avanzadoviz

For an invertible matrix and a compatible norm

\kappa(A) = \|A\|\,\|A^{-1}\|.

In the 2-norm

\kappa_2(A) = \frac{\sigma_{\max}(A)}{\sigma_{\min}(A)}.

Measures the sensitivity of a linear problem to small perturbations in the data.

The formula \|A\|\|A^{-1}\| requires A to be invertible.
For a singular matrix, the condition number is considered infinite in this context.

View detail →