Understanding Singular Value Decomposition
Singular Value Decomposition (SVD) is a fundamental matrix factorization technique in linear algebra. For any m x n matrix A, SVD decomposes it into three matrices: A = U Σ VT, where U and V are orthogonal matrices and Σ is a diagonal matrix containing the singular values.
How SVD Works
The Decomposition
Any matrix A can be written as A = UΣVT where U and V are orthogonal and Σ is diagonal.
Finding Singular Values
Singular values are the square roots of the eigenvalues of ATA (or AAT).
Properties
Singular values are always non-negative and conventionally ordered from largest to smallest.
Applications of SVD
SVD is widely used in data science for dimensionality reduction (PCA), image compression, recommendation systems, solving least-squares problems, computing matrix pseudoinverses, and determining matrix rank. It is one of the most important decompositions in numerical linear algebra.
Key Facts
- The number of non-zero singular values equals the rank of the matrix.
- The largest singular value is the matrix 2-norm (spectral norm).
- The ratio of the largest to smallest singular value gives the condition number.
- SVD always exists for any real or complex matrix.