CME 200/ME 300A

Autumn 2012
Problem Set 1 Solution
1. Elementary Operations
(a) Let A =
_
a b
c d
_
A
2
=
_
a b
c d
__
a b
c d
_
=
_
a
2
+ bc ab + bd
ac + cd bc + d
2
_
=
_
1 0
0 1
_
a
2
+ bc = 1 (1)
ab + bd = b (a + d) = 0 (2)
ac + cd = c (a + d) = 0 (3)
bc + d
2
= 1 (4)
From equation 1 and equation 4,
(a + d) (a d) = 0 (5)
Case 1, a + d = 0 or a = d and a
2
+ bc = 1.
Case 2, a d = 0 or a = d.
From equation 2 and 3,
ab = ac = 0 (6)
From equation 6 and 1, if a = 0, then d = 0 and bc = 1. This can
be included in case 1.
From equation 6, 1 and 4, if a = 0, then b = c = 0 and a
2
= d
2
= 1.
This means a = d = 1 or a = d = 1.
Hence there are three possible congurations for A.
i) A = I
ii) A = I
iii) A =
_
a b
c d
_
where a = d and a
2
+ bc = 1
1
(b) Let B =
_
b
1,1
b
1,2
b
2,1
b
2,2
_
B
2
=
_
b
1,1
b
1,2
b
2,1
b
2,2
__
b
1,1
b
1,2
b
2,1
b
2,2
_
=
_
0 0
0 0
_
b
2
1,1
+ b
1,2
b
2,1
= 0 (7)
b
1,1
b
1,2
+ b
1,2
b
2,2
= b
1,2
(b
1,1
+ b
2,2
) = 0 (8)
b
1,1
b
2,1
+ b
2,1
b
2,2
= b
2,1
(b
1,1
+ b
2,2
) = 0 (9)
b
1,2
b
2,1
+ b
2
2,2
= 0 (10)
Case 1, b
1,1
+b
2,2
= 0. By equation 8 and 9, b
1,2
= b
2,1
= 0. And
by equation 7 and 10, b
1,1
= b
2,2
= 0. B = 0 in this case.
Case 2, b
1,1
+ b
2,2
= 0 or b
1,1
= b
2,2
. From equation 7 and
10,
b
1,1
=
_
b
1,2
b
2,1
(11)
.
b
2,2
=
_
b
1,2
b
2,1
(12)
Example: B =
_
2 1
4 2
_
(c) X =
_
1 1
0 1
_
. Y =
_
1 0
1 1
_
X + Y =
_
2 1
1 2
_
. X Y =
_
0 1
1 0
_
(X + Y ) (X Y ) =
_
1 2
2 1
_
(13)
X
2
=
_
1 2
0 1
_
2
Y
2
=
_
1 0
2 1
_
X
2
Y
2
=
_
0 2
2 0
_
(14)
As it is evident, equation 13 = 14.
The inequality doesnt hold in general because matrix multipli-
cation sometimes (but rarely) commutes.
(X + Y ) (X Y ) = X
2
Y
2
XY + Y X (15)
If XY = Y X, then
(X + Y ) (X Y ) = X
2
Y
2
(16)
(d) C C
1
= I
(C C
1
)
T
= I
T
= I
(C
1
)
T
C
T
= I
By denition of the inverse matrix,
_
C
T

1
= [C
1
]
T
(e) By the denition of the skew-symmetric matrix,
A
T
= A (17)
If the size of A is n n, the size of x is n 1 and the size of x
T
is 1 n, then the size of x
T
Ax is 1 1, which is a scalar.
x
T
Ax =
_
x
T
Ax
_
T
(18)
3
By equation 17,
x
T
Ax =
_
x
T
Ax
_
T
= x
T
A
T
x = x
T
Ax (19)
Therefore, the only scalar that is equal to its own negative is 0,
which proves x
T
Ax = 0
(f) Assuming A, B, A + B are invertible,
_
B
1
+ A
1
_
= A
1
_
AB
1
+ I
_
= A
1
(A + B) B
1
(20)
_
A
1
(A + B) B
1
_
B(A + B)
1
A

= I (21)
Therefore A
1
(A + B) B
1
is invertible and so is B
1
+ A
1
.
4
2. The matrix exponential
(a) Sample code:
S
5
=
_
1.2833 2.35
2.35 1.2833
_
norm of error = 0.1948
S
10
=
_
1.1314 2.4718
2.4718 1.1314
_
norm of error = 1.88 10
4
S
15
=
_
1.1312 2.4717
2.4717 1.1312
_
norm of error = 1.97 10
8
expm(A) =
_
1.1312 2.4717
2.4717 1.1312
_
exp(A) =
_
2.7183 7.3891
0.1353 2.7183
_
, which is a totally dierent matrix
(b) A =
_
0 1
0 0
_
. Note A
2
, A
3
...A
n
= 0.
e
At
= I + tA (22)
5
e
At
=
_
1 t
0 1
_
(c) It may be easier to solve for the exponential matrix and to search
for the sucient condition rst to avoid doing too much algebra.
For example, lets look at the coecient of the square term.
For exp(A+B), it is
1
2
(A + B)
2
=
1
2
A
2
+
1
2
B
2
+
1
2
AB +
1
2
BA.
For exp(A)exp(B), it is
1
2
A
2
+
1
2
B
2
+ AB.
As it is evident, if AB = BA, the two will be equal.
The same is true for all other terms. AB = BA is a sucient
condition for the equality.
For the two sets of matrices given, lets examine the coecient
of the square term.
If A =
_
0 1
1 0
_
and B =
_
1 1
1 1
_
1
2
(A + B)
2
=
_
1.5 2
2 1.5
_
(23)
1
2
A
2
+
1
2
B
2
+ AB =
_
1.5 2
2 1.5
_
(24)
The coecient of the other terms should also match because AB =
BA. In other words, the matrix multiplication is commutative.
If A =
_
1 0
0 0
_
and B =
_
0 1
1 0
_
The same process can be repeated but the coecients in the given
series wont match. This is because AB = BA.
6
3. Simple dierential equation
(a) Dirichlet boundary conditions
i. Integrating twice,
u = sin (x) + c
1
x + c
2
Applying BCs, u(0) = u(2) = 0
u = sin (x)
ii. Discretizing,
d
2
u
dx
2

u
i+1
2u
i
+ u
i1
h
2
(25)
For n=8, h =
2
8
.
The vector b is found by discretizing the LHS of the dieren-
tial equation given (or the sine function).
A =
_
_
_
_
_
_
_
_
_
_
2 1 0 0 0 0 0
1 2 1 0 0 0 0
0 1 2 1 0 0 0
0 0 1 2 1 0 0
0 0 0 1 2 1 0
0 0 0 0 1 2 1
0 0 0 0 0 1 2
_
_
_
_
_
_
_
_
_
_

b = h
2
[0.7071, 1, 0.7071, 0, 0.7071, 1, 0.7071]
T
iii. Inverse of A is not sparse. It is not banded. The structure is
gone. All the entries are non-zero and there is no discernible
pattern. This is one of the reasons why calculating the inverse
is a bad idea. Most of the matrices encountered are sparse and
calculating the inverse of such a matrix is a waste of memory.
7
iv. Sample code:
8
v. Sample code:
Convergence rate = 1.99
9
(b) Periodic boundary conditions
i. u(0) = u(2)
c
2
= 2c
1
+ c
2
c
1
= 0 and c
2
is a free variable.
u = sin (x) + c
2
ii. A =
_
_
_
_
_
_
_
_
_
_
_
_
2 1 0 0 0 0 0 1
1 2 1 0 0 0 0 0
0 1 2 1 0 0 0 0
0 0 1 2 1 0 0 0
0 0 0 1 2 1 0 0
0 0 0 0 1 2 1 0
0 0 0 0 0 1 2 1
1 0 0 0 0 0 1 2
_
_
_
_
_
_
_
_
_
_
_
_

b = h
2
[0, 0.7071, 1, 0.7071, 0, 0.7071, 1, 0.7071]
T
The number 1 in the last row is present because the u
N
in
the discretization for u
N1
, which was previously 0 is now u
0
.
The coecient for that u
0
is 1 and hence there is a 1 in the
rst column of the last row.
To deal with the added u
0
unknown, we need to gure out
what the additional equation is. We cant just say u
0
= u
N
because u
N
is not a known value. So we can use the dis-
cretization in equation 25 for i = 0. The u
1
term that will
appear for that discretization equals u
N1
because u
0
= u
N
.
This implication of x
1
equaling x
N1
is a property of peri-
odic boundary conditions. This type of boundary condition
is often used in simulations to mirror the results of a portion
of a domain on the other parts of the domain.
iii. Yes, all the non-zero solutions to Au =

0 are [1, 1 . . . 1]
T
,
where is not 0. A is not invertible(using MATLAB is suf-
cient). When you have a singular system, you either have
innite solutions or no solution. In this case you will have
innite solution. This agrees with the previous part as the
analytic solution has one free variable (i.e. innite solutions).
10
4. Finite dierence solution of the coupled set of equations
(a) Using central-dierence,
d
2
u
dx
2

1
h
2
(u
i1
2u
i
+ u
i+1
) (26)
du
dx

1
2h
(u
i+1
u
i1
) (27)
Thus, discretizing the equations for 2 i N 1 gives,
1
h
2
u
i1
+
_

2
h
2
_
u
i
+
1
h
2
u
i+1

1
2h
v
i1
+
1
2h
v
i+1
= x
i
(28)

cos (2x
i
)
2h
u
i1
+
cos (2x
i
)
2h
u
i+1
+
1
h
2
v
i1
+
_

2
h
2
_
v
i
+
1
h
2
v
i+1
= x
2
i
(29)
Lets dene, a =
1
h
2
, b =
2
h
2
, c =
1
2h
, d
i
=
cos(2x
i
)
2h
and
f =
2
h
2
Let the system of equations be set up as Ax =

b, where x is the
column vector of unknowns.
If the equations are ordered such that all the equations 1 from the
problem set come before all the equation 2 (which is not what
the problem asks for), A consists 4 blocks. A =
_
I II
III IV
_
I =
_
_
_
_
_
_
_
_
_
_
_
1 0 0 0
a b a 0 0
0 a b a 0 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 a b a 0
0 0 0 a b a
0 0 0 1
_
_
_
_
_
_
_
_
_
_
_
II =
_
_
_
_
_
_
_
_
_
_
_
0 0 0
c 0 c 0 0
0 c 0 c 0 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 c 0 c 0
0 0 0 c 0 c
0 0 0
_
_
_
_
_
_
_
_
_
_
_
11
III =
_
_
_
_
_
_
_
_
_
_
_
0 0 0
d
2
0 d
2
0 0
0 d
3
0 d
3
0 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 d
N2
0 d
N2
0
0 0 0 d
N1
0 d
N1
0 0 0
_
_
_
_
_
_
_
_
_
_
_
IV =
_
_
_
_
_
_
_
_
_
_
_
1 0 0 0
a f a 0 0
0 a f a 0 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 a f a 0
0 0 0 a f a
0 0 0 1
_
_
_
_
_
_
_
_
_
_
_
b = [0, x
2
, x
3
x
N2
, x
N1
, 1, 2, x
2
2
, x
2
3
x
2
N2
, x
2
N1
, 2]
T
If the equation 2 comes after 1 for each point i (as the homework
asks), then
A =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 0 0
0 1 0 0
a b a 0 c 0 c 0 0
d
2
0 d
2
0 a f a 0 0
0 a b a 0 0 c 0 c 0
.
.
.
0 d
3
0 d
3
0 0 a f a 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 a b a 0 0 c 0 c 0
0 d
N2
0 d
N2
0 0 a f a 0
0 0 a b a 0 0 c 0 c
0 d
N1
0 d
N1
0 0 a f a
0 0 1 0 0
0 1
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
b = [0, 2, x
2
, x
2
2
, x
3
, x
2
3
x
N2
, x
2
N2
, x
N1
, x
2
N1
, 1, 2]
T
12
This is a worse matrix to deal with compared to the previous case
because the previous case had blocks of matrices that are banded.
However, for this case, the matrix is not banded and there are
many zeros at the diagonal which requires a lot of re-ordering.
(b) A =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 0 0
0 1 0 0
a c b 0 a c 0 0
d
2
a 0 f d
2
a 0 0
0 0 a c b 0 a c 0 0
0 0 d
3
a 0 f d
3
a 0 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 a c b 0 a c 0 0
0 0 d
N2
a 0 f d
N2
a 0 0
0 0 a c b 0 a c
0 0 d
N1
a 0 f d
N1
a
0 0 1 0
0 0 1
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
b = [0, 2, x
2
, x
2
2
, x
3
, x
2
3
x
N2
, x
2
N2
, x
N1
, x
2
N1
, 1, 2]
T
Matrix A is banded and the algorithm to solve is much faster.
(c) u
i+1
= u
i
+ hu

i
+
h
2
2
u

i
+ O(h
3
)
u
i+2
= u
i
+ 2hu

i
+
4h
2
2
u

i
+ O(h
3
)
4u
i+1
u
i+2
= 3u
i
+ 2hu

i
+ O(h
3
)
3u
i
+ 4u
i+1
u
i+2
2h
= u

i
+ O
_
h
2
_
(30)
Therefore, accurate to the second order.
(d) The rst row will change denoted as R
1
. The rst value in the b
column vector will also change, denoted as b
1
.
R
1
=
_
3c 0 4c 0 c 0 0
_
b
1
=
13
5. Penta-diagonal Matrix
This system is solved by using gaussian elimination to transform the
matrix into an upper triangular matrix and back substitution. Recall,
for a general Matrix A, where a
i,j
is the component in the ith row and
jth column, gaussian elmination, for the rst column, is performed by
the equation 31.
a

i,j
= a
i,j

a
i,1
a
1,1
a
1,j
(31)
a

i,j
is the updated value of a
i
j, i = 2...n and j = 1...n
Now, refer to the matrix A given in the problem to know the meaning
of the symbols used from here on. Lets look at the rst column. The
rst three rows have a non-zero value at the rst column. Hence, only
the second and third row need to be transformed. Since the rst row
has three non-zero values, for those two rows, only the second and third
values are updated as well as the b value. Also note, the pivot needs
to be used only once in the division for each row .
Hence, to eliminate b
1
, 1 division (D), 3 multiplications (M) and 3
additions (A) are required and the same for and a
1
, for a total of 2 D,
6 M and 6A.
The same logic can be used for the next n 3 columns.
For the next n 3 columns, 2(n 3) D, 6(n 3) M and 6(n 3) A.
After these operations, to complete forward elimination, you need to
transform the second last column dierently, as only one value is elim-
inated and only the values of c
n
and b
n
are changed. This requires 1
D, 2 M and 2 A operations.
Total forward elimination = (2n 3) D, (6n 10) M, (6n 10) A
Now to perform back substitution, for the matrix given, to solve for
the rst n 2 unknowns, we follow equation 32.
x
i
=
b
i
d

i
x
i+1
e

i
x
i+2
c

i
(32)
14
where d

i
, e

i
, c

i
are the updated upper triangular values.
This takes 1 (n 2) D, 2 (n 2) M and 2 (n 2) A.
The second to last unknown, requires 1D, 1 M and 1 A and the last
unknown requires 1 D. Hence for back substitution, we need a total of
n D, (2n 3) M, (2n 3) A.
Therefore, the total operations needed are (3n 3) D, (8n 13) M,
(8n 13) A.
A total of (19n 29) operations.
The number of operations required is O(n), similar to the tridiago-
nal matrix linear solver algorithm, like the famous Thomas algorithm.
Thats why banded matrices are very helpful. They also lend them-
selves to ecient parallel implementation. Often, you need to arrange
your unknowns to get a matrix like this, as you saw in problem no. 4.
15
6. Matrix A and matrix AB
A =
_

V
1
,

V
2
, ,

V
n
_
AB =
_

U
1
,

U
2
, ,

U
n
_
if B =
_

B
1
,

B
2
, ,

B
j
, ,

B
n
_
j is some number that is between 1 and n. Other than that column,
the V columns and U columns are the same.
Hence, B looks like an identity matrix for all the columns except for
column j.
A

B
j
=

U
j
= b
1,j

V
1
+ b
2,j

V
2
+ + b
j,j

V
j
+ + b
n,j

V
n

k
= b
k,j
for all k, 1 k j
B =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 0
1
0
0 1 0
2
0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 0 1
j1
0 0
.
.
.
.
.
.
.
.
. 0
j
0 0
.
.
.
.
.
.
.
.
.
.
.
. 0 1 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 1
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
16