Understanding Condition Numbers
The condition number of a matrix is a measure of how sensitive the solution of a linear system Ax = b is to small changes in the matrix A or the vector b. A matrix with a low condition number is said to be well-conditioned, while a matrix with a high condition number is ill-conditioned. The condition number is denoted κ(A) and is always greater than or equal to 1.
In numerical linear algebra, the condition number is one of the most important quantities because it directly determines the accuracy that can be expected when solving linear systems computationally.
How to Calculate the Condition Number
The condition number is defined as the product of the norm of the matrix and the norm of its inverse:
κ(A) = ||A|| · ||A¹||
This requires computing the matrix inverse and then calculating the chosen matrix norm for both A and A¹.
Matrix Norms
1-Norm (Column Sum)
Maximum absolute column sum of the matrix.
Infinity-Norm (Row Sum)
Maximum absolute row sum of the matrix.
Frobenius Norm
Square root of the sum of squared elements.
Condition Number
Product of the matrix norm and its inverse norm.
2x2 Inverse
The inverse of a 2x2 matrix uses the adjugate formula.
Singular Matrix
If det(A) = 0, the matrix is singular and κ = ∞.
Interpreting the Condition Number
- κ(A) = 1: The matrix is perfectly conditioned (e.g., the identity matrix or an orthogonal matrix).
- κ(A) < 100: Generally considered well-conditioned. Solutions are reliable.
- κ(A) = 100 to 10,000: Moderately ill-conditioned. Some loss of accuracy may occur.
- κ(A) > 10,000: Ill-conditioned. Numerical solutions may be highly inaccurate.
- κ(A) = ∞: The matrix is singular (non-invertible).
Why Condition Number Matters
When solving Ax = b computationally, the relative error in the solution x is bounded by κ(A) times the relative error in b (or A). This means that if κ(A) = 10k, you can expect to lose approximately k digits of accuracy in the computed solution compared to the exact solution.
Applications
Condition numbers are essential in numerical analysis, scientific computing, structural engineering (stiffness matrices), electrical engineering (circuit analysis), machine learning (feature scaling), and optimization. Engineers and scientists use condition numbers to assess whether their computational results are trustworthy and to decide whether preconditioning techniques should be applied to improve the numerical stability of their computations.
Tips for Reducing Condition Numbers
- Scale the matrix rows and columns to have similar magnitudes.
- Use preconditioning techniques before solving linear systems.
- Consider using higher precision arithmetic for ill-conditioned problems.
- Reformulate the problem if possible to avoid ill-conditioned matrices.