Simultaneous Linear Equations

Topic: Gauss-Seidel Method

1
 Gauss-Seidel Method

An iterative method.

Basic Procedure:
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.

2
 Gauss-Seidel Method

Why?
The Gauss-Seidel Method allows the user to control round-off error.

Elimination methods such as Gaussian Elimination and LU

Decomposition are prone to round-off error.

Also: If the physics of the problem are understood, a close initial

guess can be made, decreasing the number of iterations needed.

3
 Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
If: the diagonal elements are
a11 x1  a12 x2  a13 x3  ...  a1n xn b1 non-zero
a21 x1  a22 x2  a23 x3  ...  a2n xn b2 Rewrite each equation solving
. .
. . for the corresponding unknown
. .
ex:
an1 x1  an 2 x2  an 3 x3  ...  ann xn bn
First equation, solve for x1
Second equation, solve for x2

4
 Gauss-Seidel Method
Algorithm
Rewriting each equation
c  a12 x 2  a13 x3   a1n x n From Equation 1
x1  1
a11

c2  a21 x1  a23 x3   a2 n xn
x2  From equation 2
a22
  
cn  1  an  1,1 x1  an  1, 2 x2   an  1,n  2 xn  2  an  1,n xn From equation n-1
xn  1 
an  1,n  1
cn  an1 x1  an 2 x2    an ,n  1 xn  1 From equation n
xn 
ann

5
 Gauss-Seidel Method
Algorithm
General Form of each equation
n n

c1  a 1j xj cn  1  a
j 1
n  1, j xj
j 1
j n  1
x1 
j 1 xn  1 
a11 an  1,n  1
n

c2 
n

 a2 j x j cn  a j 1
nj xj
j 1
j n
x2 
j 2
xn 
a 22 a nn
6
 Gauss-Seidel Method
Algorithm
General Form for any row ‘i’

n
ci  a
j 1
ij xj
j i
xi  , i 1,2,  , n.
aii

How or where can this equation be used?

7
 Gauss-Seidel Method
Solve for the unknowns

Assume an initial guess for [X]

 x1  Use rewritten equations to solve for

x  each value of xi.
 2 Important: Remember to use the
  most recent value of xi. Which
  means to apply values calculated to
 xn-1  the calculations remaining in the
current iteration.
 xn 

8
 Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
x inew  x iold
a i
 new
100
xi

So when has the answer been found?

The iterations are stopped when the absolute relative

approximate error is less than a prespecified tolerance for all
unknowns.

9
Gauss-Seidel Method: Example 1
The upward velocity of a rocket
is given at three different times
Time, t Velocity, v

s m/s
5 106.8
8 177.2
12 279.2

The velocity data is approximated by a polynomial as:

vt  a1t 2  a 2 t  a3 , 5  t 12.

10
Gauss-Seidel Method: Example 1
 t12 t1 1  a1   v1 
 2 
Using a Matrix template of the form
 t2 t 2 1  a2   v2 
 t32 t3 1  a3   v3 

 25 5 1  a1   106.8 
 64 8 1  a   177.2 
The system of equations becomes    2  
144 12 1  a3   279.2

 a1   1 
 a    2
Initial Guess: Assume an initial guess of
 2  
 a3   5
11
 Gauss-Seidel Method: Example 1

Rewriting each equation

106.8  5a 2  a3
a1 
25
 25 5 1  a1   106.8 
 64 8 1  a   177.2 
   2   177.2  64a1  a3
144 12 1  a3   279.2
a2 
8

279.2  144a1  12a 2

a3 
1

12
 Gauss-Seidel Method: Example 1
Applying the initial guess and solving for ai

 a1   1  106.8  5(2)  (5)

a1  3.6720
 a    2 25
 2  
 a3   5 177.2  643.6720  5
a2   7.8510
Initial Guess 8
279.2  1443.6720  12 7.8510 
a3   155.36
1

When solving for a2, how many of the initial guess values were used?

13
 Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error

x inew  x iold At the end of the first iteration

a i
 new
100
xi  a1   3.6720 
 a     7.8510
3.6720  1.0000  2  
a 1  x100 72.76%  a3    155.36 
3.6720
The maximum absolute
 7.8510  2.0000 relative approximate error is
a 2  x100 125.47% 125.47%
 7.8510

 155.36  5.0000
a 3  x100 103.22%
 155.36
14
 Gauss-Seidel Method: Example 1
Iteration #2
Using
 a1   3.6720  the values of ai are found:
 a     7.8510 106.8  5 7.8510 155.36
 2   a1  12.056
 a3    155.36  25

from iteration #1
177.2  6412.056 155.36
a2   54.882
8

279.2  14412.056 12 54.882

a3   798.34
1

15
 Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error

12.056  3.6720 At the end of the second iteration

a 1  x100 69.542%
12.056  a1   12.056 
 a     54.882
 2  
 54.882   7.8510   a3    798.34
a 2  x100 85.695%
 54.882
The maximum absolute
 798.34   155.36  relative approximate error is
a 3  x100 80.54% 85.695%
 798.34

16
 Gauss-Seidel Method: Example 1
Repeating more iterations, the following values are obtained
Iteration
a1 a 1 % a2 a 2 % a3 a 3 %
1 3.672 72.767 -7.8510 125.47 -155.36 103.22
2 12.056 67.542 -54.882 85.695 -798.34 80.540
3 47.182 74.448 -255.51 78.521 -3448.9 76.852
4 193.33 75.595 -1093.4 76.632 -14440 76.116
5 800.53 75.850 -4577.2 76.112 -60072 75.962
6 3322.6 75.907 -19049 75.971 -249580 75.931

! Notice – The relative errors are not decreasing at any significant rate
 a1   0.29048
Also, the solution is not converging to the true solution of  a   19.690 
 2  
 a3   1.0858 
17
 Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.

Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.

Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:

n n
aii  aij for all ‘i’ and aii   aij for at least one ‘i’
j 1 j 1
j i j i 18
 Gauss-Seidel Method: Pitfall
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other coefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
Which coefficient matrix is diagonally dominant?

 2 5.81 34 124 34 56 

A  45 43 1  [B]  23 53 5 
123 16 1   96 34 129

Most physical systems do result in simultaneous linear equations that

have diagonally dominant coefficient matrices.

19
 Gauss-Seidel Method: Example 2

Given the system of equations The coefficient matrix is:

12x 1  3x 2 - 5x 3 1
x 1  5x 2  3x 3 28 12 3  5

3x1  7x2  13x3 76

 A  1 5 3 
 3 7 13 
With an initial guess of
Will the solution converge using the
 x1   1
 x    0 Gauss-Siedel method?
 2  
 x3   1

20
 Gauss-Seidel Method: Example 2
Checking if the coefficient matrix is diagonally dominant

x11 12 12  x12  x13  3   5 8

12 3  5
 A  1 5 3  x22  5 5  x21  x23 1  3 4
 3 7 13 
x33 13 13  x31  x32  3  7 10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method

21
 Gauss-Seidel Method: Example 2

Rewriting each equation With an initial guess of

12 3  5  x1   1   x1   1
1 5 3  x   28  x    0
   2    2  
 3 7 13   x 3   76  x3   1

1  3 x 2  5 x3 1  30  51
x1  x1  0.50000
12 12
28  x1  3 x3 28  0.5 31
x2  x2  4.9000
5 5
76  3 x1  7 x 2
x3  76  30.50000 74.9000
13 x3  3.0923
13
22
Gauss-Seidel Method: Example 2
The absolute relative approximate error

0.50000  1.0000
a 1  100 67.662%
0.50000

4.9000  0
a 2
 100 100.00%
4.9000

3.0923  1.0000
a 3
 100 67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%

23
 Gauss-Seidel Method: Example 2
After Iteration #1
 x1   0.5000
 x   4.9000
 2  
 x3   3.0923
Substituting the x values into the equations After Iteration #2
1  34.9000   53.0923  x1   0.14679
x1  0.14679  x   3.7153 
12  2  
 x3   3.8118 
28  0.14679  33.0923
x2  3.7153
5

76  30.14679  74.900
x3  3.8118
13
24
Gauss-Seidel Method: Example 2
Iteration #2 absolute relative approximate error
0.14679  0.50000
a 1  100 240.62%
0.14679
3.7153  4.9000
a 2  100 31.887%
3.7153
3.8118  3.0923
a 3  100 18.876%
3.8118

The maximum absolute relative error after the first iteration is 240.62%

This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
25
 Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
Iteration a1 a2 a3
a 1 a a
2 3

1 0.50000 67.662 4.900 100.00 3.0923 67.662

2 0.14679 240.62 3.7153 31.887 3.8118 18.876
3 0.74275 80.23 3.1644 17.409 3.9708 4.0042
4 0.94675 21.547 3.0281 4.5012 3.9971 0.65798
5 0.99177 4.5394 3.0034 0.82240 4.0001 0.07499
6 0.99919 0.74260 3.0001 0.11000 4.0001 0.00000

 x1   0.99919
The solution obtained    
x
 2   3.0001 
 x3   4.0001 
 x1   1 
 x    3
is close to the exact solution of  2  
 x3   4 26
 Gauss-Seidel Method: Example 3
Given the system of equations Rewriting the equations
3x1  7x2  13x3 76
76  7 x2  13 x3
x1  5x2  3x3 28 x1 
3
12x1  3x2 - 5x3 1
28  x1  3 x3
With an initial guess of
x2 
5
 x1   1
 x   0  1  12 x1  3 x 2
 2  
 x3   1 x3 
5
27
Gauss-Seidel Method: Example 3
Conducting six iterations
Iteration a1 a a2 a a3 a
1 2 3

1 21.000 110.71 0.80000 100.00 5.0680 98.027

2 -196.15 109.83 14.421 94.453 -462.30 110.96
3 -1995.0 109.90 -116.02 112.43 4718.1 109.80
4 -20149 109.89 1204.6 109.63 -47636 109.90
5 2.0364x105 109.90 -12140 109.92 4.8144x105 109.89
6 -2.0579x105 1.0990 1.2272x10 109.89 -4.8653x106 109.89
5

The values are not converging.

Does this mean that the Gauss-Seidel method cannot be used?

28
 Gauss-Seidel Method
The Gauss-Seidel Method can still be used
 3 7 13 
The coefficient matrix is not
diagonally dominant
 A  1 5 3 
12 3  5

But this is the same set of 12 3  5

equations used in example #2,  A  1 5 3 
which did converge.
 3 7 13 

If a system of linear equations is not diagonally dominant, check to see if

rearranging the equations can form a diagonally dominant matrix.

29
 Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.

Observe the set of equations

x1  x 2  x3 3
2 x1  3 x2  4 x3 9
x1  7 x 2  x3 9

Which equation(s) prevents this set of equation from having a

diagonally dominant coefficient matrix?
30
 Example 4

first iteration third iteration

second iteration

After five more iterations, we

obtain the final values for x, y
and z as x = 3, y = 1 and z = 1
Example 5

32
 Gauss-Seidel Method
Summary

-Advantages of the Gauss-Seidel Method

-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss-Seidel Method

33