Chapter 2
Systems of Linear Equations
Outline
Systems of Linear Equations
Singularity and Nonsingularity
# of linearly independent columns or rows
Always column rank=row rank
Existence and Uniqueness
Geometric Interpretation
Example: Nonsingularity
Example: Singularity
Vector Norms
Example: Vector Norms
Norms - Graphical Example
Equivalence of Norms
Properties of Vector Norms
Matrix Norms
Matrix Norms
Limiting case: |2x1+3x2|+|5x1+4x2| => |7x1|+|7x2|
therefore: |7x1|+|7x2| / (|x1 | + | x2|) can be at most 7
Limiting case x1=x2: max(|2x1+3x2|,|5x1+4x2|) => max(|5x1|,|9x1|)
therefore: |9x1|/ max(|x1|,|x2|) can be at most 9
Properties of Matrix norms
Condition Number
Properties of condition number
Matrix Condition Number - Interactive Example
Computation of Condition Number
Computing Condition Number, continued
Error Bounds
∥ 𝑏 ∥=∥ 𝐴𝑥 ∥≤∥ 𝐴 ∥'∥ 𝑥 ∥
∥ 𝑥 ∥≥∥ 𝑏 ∥/∥ 𝐴 ∥
∥ ∆𝑥 ∥=∥ 𝐴+, ∆𝑏 ∥≤∥ 𝐴+, ∥'∥ ∆𝑏 ∥
Error Bounds - Interactive example
Error Bounds, continued
Error Bounds, illustration
Residual an easy way to verify a found approximate solution to a problem
Residual, continued
Solving Linear Systems
Example: Permutations
Triangular Linear Systems
Triangular Matrices
Forward-Substitution
Back-Substitution
Triangular Linear System
Elimination
Elementary Elimination Matrices
Elementary Elimination Matrices, continued
Elementary Elimination Matrices, continued
Example: Elementary Elimination Matrices
Example, continued
Gaussian Elimination
Gaussian Elimination, continued
LU Factorization
LU Factorization, continued
Example: Gaussian Elimination
Example, continued
Example, continued
Example, continued
Row Interchanges
Partial Pivoting
Gaussian Elimination Interactive Illustration
LU Factorization with Partial Pivoting
Complete Pivoting
Example: pivoting
Example: Small pivots
Example, continued
Pivoting, continued
Example: Small Residual
Example, continued
Implementation of Gaussian Elimination
Storage Management
Complexity of Solving Linear Systems
Inversion vs. Factorization
Gauss-Jordan Elimination
Gauss-Jordan Elimination, continued
Gauss-Jordan interactive example
Scaling
e.g., perturbing the right hand side by [0 –ε] T
changes solution from [1 1]T to [1 0]T
Special Types of Linear Systems
Symmetric Positive Definite Matrices
Cholesky Factorization
Cholesky Factorization, continued
Cholesky Factorization - Interactive Example
LINPACK and LAPACK
Basic Linear Algebra Subprograms
Examples of BLAS
