Scribd
Upload
351 views

Mathematical Foundations For Data Science: BITS Pilani

This document discusses numerical methods for solving systems of linear equations. It introduces LU decomposition and Cholesky decomposition methods. It then motivates iterative methods and defines matrix norms. It describes the Gauss-Seidel and Gauss-Jacobi iterative methods for solving systems of linear equations and criteria for determining their convergence. Examples are provided to illustrate diagonal dominance conditions and cases of convergence and divergence.

Uploaded by

Tanissh Khanna
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
351 views

Mathematical Foundations For Data Science: BITS Pilani

This document discusses numerical methods for solving systems of linear equations. It introduces LU decomposition and Cholesky decomposition methods. It then motivates iterative methods and defines matrix norms. It describes the Gauss-Seidel and Gauss-Jacobi iterative methods for solving systems of linear equations and criteria for determining their convergence. Examples are provided to illustrate diagonal dominance conditions and cases of convergence and divergence.

Uploaded by

Tanissh Khanna
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 23

Mathematical Foundations for

Data Science
BITS Pilani MFDS Team
Pilani Campus
BITS Pilani
Pilani Campus

DSECL ZC416, MFDS

Lecture No. 2
Agenda

• LU Decomposition (continued)

• Motivation for numerical methods


• Matrix Norms
• Iterative Solution

• Gauss Seidel Method


• Gauss Jacobi Method
• Convergence Criteria

BITS Pilani, Pilani Campus


Crout’s Method

BITS Pilani, Pilani Campus


Crout’s Method

BITS Pilani, Pilani Campus


Cholesky Method

BITS Pilani, Pilani Campus


Cholesky Method

• Decompose A such that A = UTU. Hence, we may have UTUx = b

• Set up and solve UTd = b, where d can be obtained by using


forward substitution

• Set up and solve Ux = d, where x can be obtained by using


backward substitution

BITS Pilani, Pilani Campus


Computational Complexity
The LU decomposition is computed directly without solving simultaneous
equations

• It is more economical to produce the LU Factorization

• This is followed by solving two simpler linear systems

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956


Motivation

Problem: Solve 7x = x + 18. (Solution x = 3)


Iterative Procedure: xi+1 = f(xi), with x1 given
7xi +1 = xi + 18 , x1 = 1 xi+1 = 7xi – 18 , x1 = 1
x2 = 2.7143
x2 = -11
x3 = 2.9592
x3 = -95
x4 = 2.9942
x4 = -683
x5 = 2.9992
x5 = -4799
x6 = 2.9999
x6 = -33611
Converges to x = 3
Diverges
a) Extension to Linear Systems?
b) Criteria for convergence (Similar to )

BITS Pilani, Pilani Campus


Matrix Norms

Some of the commonly used matrix norms are given below

BITS Pilani, Pilani Campus


Iterative Methods for Systems

Consider the normalized system Ax = b


Write A = I + L + U, where
• I is an identity matrix
• L is lower triangular matrix with diagonal entries 0
• U is upper triangular matrix with diagonal entries 0
Ax = (I + L + U)x = b
• xk+1 = -Lxk – Uxk + b (Gauss Jacobi)
• xk+1 = -(I + L)-1Uxk +(I + L)-1b (Gauss Seidel)
Converges if <1

BITS Pilani, Pilani Campus


Gauss Seidel Method

BITS Pilani, Pilani Campus


Gauss Seidel Method

BITS Pilani, Pilani Campus


Gauss Seidel Method

Note: Always check the convergence using specified tolerance

BITS Pilani, Pilani Campus


Diagonally Dominant Matrix

Theorem

BITS Pilani, Pilani Campus


Gauss Jacobi Method

BITS Pilani, Pilani Campus


Convergence of Jacobi Method

BITS Pilani, Pilani Campus


Convergence of Jacobi Method

BITS Pilani, Pilani Campus


Convergence of Jacobi Method
Diagonal dominance is not
necessary but sufficient
condition

Column sum

BITS Pilani, Pilani Campus


Example of Divergence
Ex 1

BITS Pilani, Pilani Campus


Example of Divergence
JACOBI METHOD

GAUSS SEIDEL METHOD

BITS Pilani, Pilani Campus


Diagonally Dominant Matrix
Ex 1

BITS Pilani, Pilani Campus


Diagonally Dominant Matrix

Do not conclude that strict diagonal dominance is a necessary condition


for convergence of the Gauss Jacobi or Gauss- Seidel methods. For
instance, the coefficient matrix of the system

is not a strictly diagonally dominant matrix, and yet both methods converge
to the solution x1 = 1 and x2 = 1 when you use an initial approximation of
(x1, x2) = (0,0).

BITS Pilani, Pilani Campus

You might also like