Scribd
Upload
24 views

Accuracy, Condition Numbers and Pivoting: The Effect of Rounding

This document discusses two issues related to accuracy in solving linear systems: pivoting and condition number. Pivoting ensures Gaussian elimination proceeds accurately by switching rows to always eliminate with the largest pivot element. Condition number measures how sensitive a matrix is to small changes in inputs - a high condition number means small input errors can cause large output errors. The document provides an example showing how pivoting improves accuracy and defines condition number mathematically.

Uploaded by

aleyhaider
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
24 views

Accuracy, Condition Numbers and Pivoting: The Effect of Rounding

This document discusses two issues related to accuracy in solving linear systems: pivoting and condition number. Pivoting ensures Gaussian elimination proceeds accurately by switching rows to always eliminate with the largest pivot element. Condition number measures how sensitive a matrix is to small changes in inputs - a high condition number means small input errors can cause large output errors. The document provides an example showing how pivoting improves accuracy and defines condition number mathematically.

Uploaded by

aleyhaider
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Lecture 11

Accuracy, Condition Numbers and Pivoting


In this lecture we will discuss two separate issues of accuracy in solving linear systems. The first, pivoting, is
a method that ensures that Gaussian elimination proceeds as accurately as possible. The second, condition
number, is a measure of how bad a matrix is. We will see that if a matrix has a bad condition number, the
solutions are unstable with respect to small changes in data.

The effect of rounding


All computers store numbers as finite strings of binary floating point digits (bits). This limits numbers to
a fixed number of significant digits and implies that after even the most basic calculations, rounding must
happen.
Consider the following exaggerated example. Suppose that our computer can only store 2 significant
digits and it is asked to do Gaussian elimination on


.001 1 3
.
1
2 5
Doing the elimination exactly would produce

.001
0

1
998

3
2995

but rounding to 2 digits, our computer would store this as




.001
1
3
.
0
1000 3000
Backsolving this reduced system gives
x1 = 0

and

x2 = 3.

This seems fine until you realize that backsolving the unrounded system gives
x1 = 1 and

x2 = 3.001.

Row Pivoting
A way to fix the problem is to use pivoting, which means to switch rows of the matrix. Since switching rows
of the augmented matrix just corresponds to switching the order of the equations, no harm is done:


1
2 5
.
.001 1 3
Exact elimination would produce


1
2
0 .998

39

5
2.995

40

LECTURE 11. ACCURACY, CONDITION NUMBERS AND PIVOTING

Storing this result with only 2 significant digits gives



1 2
0 1

5
3

Now backsolving produces


x1 = 1 and

x2 = 3,

which is the true solution (rounded to 2 significant digits).


The reason this worked is because 1 is bigger than 0.001. To pivot we switch rows so that the largest
entry in a column is the one used to eliminate the others. In bigger matrices, after each column is completed,
compare the diagonal element of the next column with all the entries below it. Switch it (and the entire row)
with the one with greatest absolute value. For example in the following matrix, the first column is finished
and before doing the second column, pivoting should occur since | 2| > |1|:

4
1 2
3
0 1
6
7 .
0 2 10 10
Pivoting the 2nd and 3rd rows would produce

1 2
0 2
0 1

4
3
10 10 .
6
7

Condition number
In some systems, problems occur even without rounding. Consider the following augmented matrices:




1
1/2 3/2
1
1/2 3/2
and
.
1/2 1/3
1
1/2 1/3 5/6
Here we have the same A, but two different input vectors:
b1 = (3/2, 1)

and

b2 = (3/2, 5/6)

which are pretty close to one another. You would expect then that the solutions x1 and x2 would also be
close. Notice that this matrix does not need pivoting. Eliminating exactly we get




1 1/2 3/2
3/2
1 1/2
and
.
0 1/12 1/4
0 1/12 1/12
Now solving we find
x1 = (0, 3)

and

x2 = (1, 1)

which are not close at all despite the fact that we did the calculations exactly. This poses a new problem:
some matrices are very sensitive to small changes in input data. The extent of this sensitivity is measured
by the condition number. The definition of condition number is: consider all small changes A and b in
A and b and the resulting change, x, in the solution x. Then



Relative error of output
|x|/|x|
.
cond(A) max |A| |b| = max
Relative error of inputs
+
|A|

|b|

41
Put another way, changes in the input data get multiplied by the condition number to produce changes in
the outputs. Thus a high condition number is bad. It implies that small errors in the input can cause large
errors in the output.
In Matlab enter
> H = hilb(2)
which should result in the matrix above. Matlab produces the condition number of a matrix with the
command
> cond(H)
Thus for this matrix small errors in the input can get magnified by 19 in the output. Next try the matrix
> A = [ 1.2969 0.8648 ; .2161 .1441]
> cond(A)
For this matrix small errors in the input can get magnified by 2.5 108 in the output! (We will see this
happen in the exercise.) This is obviously not very good for engineering where all measurements, constants
and inputs are approximate.
Is there a solution to the problem of bad condition numbers? Usually, bad condition numbers in engineering contexts result from poor design. So, the engineering solution to bad conditioning is redesign.
Finally, find the determinant of the matrix A above:
> det(A)
which will be small. If det(A) = 0 then the matrix is singular, which is bad because it implies there
will not be a unique solution. The case here, det(A) 0, is also bad, because it means the matrix is
almost singular. Although det(A) 0 generally indicates that the condition number will be large, they
are actually independent things. To see this, find the determinant and condition number of the matrix
[1e-10,0;0,1e-10] and the matrix [1e+10,0;0,1e-10].

Exercises
11.1 Let
A=

1.2969
.2161

.8648
.1441

(a) Find the determinant and inverse of A (using Matlab).


(b) Let B be the matrix obtained from A by rounding off to three decimal places (1.2969 7 1.297).
Find the determinant and inverse of B. How do A1 and B 1 differ? Explain how this happened.
(c) Set b1 = [1.2969; 0.2161] and do x = A \ b1 . Repeat the process but with a vector b2
obtained from b1 by rounding off to three decimal places. Explain exactly what happened. Why
was the first answer so simple? Why do the two answers differ by so much?
11.2 Try
> B = [sin(sym(1)) sin(sym(2)); sin(sym(3)) sin(sym(4))]
> c = [1; 2]
> x = B \ c
> pretty(x)
Next input the matrix:
Cs =

1 2
2 4

symbolically as above. Create a numerical version via Cn = double(Cs) and define the two vectors d1 = [4; 8] and d2 = [1; 1]. Solve the systems Cs*x = d1, Cn*x = d1, Cs*x = d2, and
Cn*x = d2. Explain the results. Does the symbolic or non-symbolic way give more information?

42

LECTURE 11. ACCURACY, CONDITION NUMBERS AND PIVOTING

11.3 Recall the matrix A that you saved using A_equil_truss in exercise 9.1. (Enter > load A_equil_truss
or click on the file A_equil_truss.mat in the folder where you saved it; this should reproduce the
matrix A.)
Find the condition number for this matrix. Is it good or bad? Now change any of the entries in A
and recalculate the condition number and compare. What does this say about the equilateral truss?

You might also like