Advanced Engineering Mathematics

Ch. 7 Linear Algebra :

Matrices, Vectors, Determinants. Linear Systems

 Systems of linear equations arise in electrical networks, mechanical frameworks, economic

models, optimization problems, numeric for differential equation.

 As main tools, linear algebra uses matrices and vectors.

Advanced Engineering Mathematics

7.1 Matrices, Vectors:

Addition and Scalar Multiplication

 Matrix : A rectangular array of numbers (or functions) enclosed in brackets

• Entry (or sometimes the element) : numbers ( or functions ) in the matrix

• Row : Horizontal lines of entries

• Column : Vertical lines

 General Concepts and Notations

 a11 a12 a1n 

a a22 a2 n 
A   a jk    21 : m  n matrix
 
 
 am1 am 2 amn 

• Matrices are denoted by capital boldface letters or by writing the general entry in brackets.
• An m  n matrix : A matrix with m rows and n columns
• Each entry has two subscripts. The first is the row number and the second is the column number.
Advanced Engineering Mathematics

7.1 Matrices, Vectors:

Addition and Scalar Multiplication

 Square Matrix : An n  n matrix

• Main diagonal : Diagonal containing the entries a11 , a22 , , ann .
 Vectors : A matrix with only one row or column
• Components : Entries in and vector
• Vectors are denoted by lowercase boldface letters or by its general component in brackets.

• Row Vector : Matrix having just one row is of the form a   a1 , a2 , an 

 b1 
b 
• Column Vector : Matrix having just one column is of the form b   2 

 
bm 
Advanced Engineering Mathematics

7.1 Matrices, Vectors:

Addition and Scalar Multiplication

 Matrix Addition and Scalar Multiplication

 Definition Equality of Matrices

Two matrices have the same size and the corresponding entries are equal.

 Definition Addition of Matrices

: The entries in the sum of two matrices are obtained by adding the corresponding entries
 Matrices of different sizes cannot be added.

 Definition Scalar Multiplication (Multiplication by a Number)

: The product of any matrix and any scalar is obtained by multiplying each entry of the
matrix by the scalar.
Advanced Engineering Mathematics

7.1 Matrices, Vectors:

Addition and Scalar Multiplication

Rules for Matrix Addition and Scalar Multiplication.

ABBA cA  B   cA  cB
A  B   C  A  B  C c  k A  cA  kA
A0A ckA   ck A
A   A   0 1A  A
Advanced Engineering Mathematics

7.2 Matrix Multiplication

 Definition Multiplication of a Matrix by a Matrix

The product C=AB of an m  n matrix A   a jk  times an r  p matrix B  b jk 

•r=n
n
• the m m  p matrix C  c jk  with entries c jk   a jl blk  a j1b1k  a j 2b2 k    a jnbnk
  j 1

 Matrix multiplication is not commutative, AB  BA in general

 Rules for Matrix multiplication.

(kA)B = k(AB) = A(kB)

A(BC) = (AB)C Associative law

(A + B)C = AC + BC Distributive law

C(A + B) = CA + CB
Advanced Engineering Mathematics

7.2 Matrix Multiplication

 Definition Transposition of Matrices and Vectors

Matrix obtained by exchanging rows and columns

 a11 a21 am1 

a a22 am 2 
A   a jk   A   akj    12
T
 
 
 a1n a2 n amn 

 Rules for transposition

A T T
A

 A  B   A T  BT
T

 cA   cAT
T

 AB   BT AT
T
Advanced Engineering Mathematics

7.2 Matrix Multiplication

 Special Matrices
 Symmetric Matrix : Matrix whose transpose equals the matrix itself A T
 A

 Skew- symmetric Matrix : Matrix whose transpose equals minus the matrix  AT  A 
 Triangular Matrix
: A square matrix where the entries either below or above the main diagonal are zero.
• Upper Triangular Matrix : A square matrices that can have nonzero entries only on and above the
main diagonal, whereas any entry below the diagonal must be zero.
• Lower Triangular Matrix : A square matrices that have nonzero entries only on and below the
main diagonal.
 Diagonal Matrix : A square matrix in which the entries outside the main diagonal are all zero
• Scalar Matrix : A diagonal matrix whose diagonal elements all contain the same scalar.
• Unit Matrix ( or Identity Matrix) : A scalar matrix whose entries on the main diagonal are all 1.
Advanced Engineering Mathematics

7.3 Linear Systems of Equations.

Gauss Elimination

 Linear System, Coefficient Matrix, Augmented Matrix

 A linear system of m equations in n unknowns :
a11 x1   a1n xn  b1  a11 a12 a1n   x1   b1 
a21 x1   a2 n xn  b2 a a2 n   x2  b2 
  21 a22 
    
    
am1 x1   amn xn  bm  am1 am 2 amn   xn  bn 
 Matrix Form of the Linear System : Ax=b

 a11 a12 a1n   x1   b1   a11 a12 a1n b1 

a a22 a2 n  x  b  a a22 a2 n b2 
A   21 , x   2, b   2, A   21
       
       
 am1 am 2 amn   xn  bn   am1 am 2 amn bn 
Coefficient Matrix Solution Vector Augmented matrix

• Homogeneous System : all the b j are zero

• Nonhomogeneous System : at least one b j is not zero

Advanced Engineering Mathematics

7.3 Linear Systems of Equations.

Gauss Elimination

 Gauss Elimination and Back Substitution

Linear system Augmented matrix

2 x1  5 x2  2 2 5 2 
4 x1  3x2  30  4 3 30
 
Step 1 Elimination of x1 : Add 2 times the first equation to the second equation.

2 x1  5 x2  2 2 5 2 
13x2  26  0 13 26 
 

Step 2 Back Substitution : Determination of x2 , x1

Working backward from the last to the first equation of this triangular system

 2  5x2    2  5  2    6
1 1
x2  26  2, x1 
13 2 2
Advanced Engineering Mathematics

7.3 Linear Systems of Equations.

Gauss Elimination

 Elementary Row Operations. Row-Equivalent Systems

<Elementary Operations for Equations> <Elementary Row Operations for Matrices>

• Interchange of two equations • Interchange of two rows
• Addition of a constant multiple of one • Addition of a constant multiple of one row
equation to another equation to another row
• Multiplication of an equation by a nonzero • Multiplication of a row by a nonzero
constant constant

 These operations are for rows, not for columns.

 Row-Equivalent
A linear system S1 row-equivalent to a linear system S 2
if S1 can be obtained from S 2 by row operations.
Advanced Engineering Mathematics

7.3 Linear Systems of Equations.

Gauss Elimination

 Theorem 1 Row-Equivalent Systems

Row-equivalent linear systems have the same set of solutions.

 Gauss Elimination : The Three Possible Cases of Systems

• With a unique solution (see Example 2)
• With infinitely many solutions( Example 3, below)
• Without solutions ( inconsistent systems; see Example 4)
Advanced Engineering Mathematics

7.3 Linear Systems of Equations.

Gauss Elimination

 Ex.3 Solve the following linear systems of three equations in four unknowns whose augmented matrix is

3.0 2.0 2.0 5.0 8.0  3.0 x1  2.0 x2  2.0 x3  5.0 x4  8.0
0.6 1.5 1.5 5.4 2.7  0.6 x1  1.5 x2  1.5 x3  5.4 x4  2.7

1.2 0.3 0.3 2.4 2.1 1.2 x1  0.3x2  0.3x3  2.4 x4  2.1

Step 1 Elimination of x1 :  0.6 / 3.0  0.2 times the first equation to the second equation,
1.2 / 3.0  0.4 times the first equation to the third equation.

3.0 2.0 2.0 5.0 8.0  3.0 x1  2.0 x2  2.0 x3  5.0 x4  8.0
0 1.1 1.1 4.4 1.1  Row 2   0.2   Row 1 1.1x2  1.1x3  4.4 x4  1.1

 0 1.1 1.1 4.4 1.1 Row 3   0.4   Row 1 1.1x2  1.1x3  4.4 x4  1.1

Step 2 Elimination of x2 : 1.1/1.1  1 times the second equation to the third equation.

3.0 2.0 2.0 5.0 8.0  3.0 x1  2.0 x2  2.0 x3  5.0 x4  8.0
 0 1.1 1.1 4.4 1.1  1.1x2  1.1x3  4.4 x4  1.1

 0 0 0 0 0  Row 3  Row 2 00
Advanced Engineering Mathematics

7.3 Linear Systems of Equations.

Gauss Elimination

Step 3 Substitution : From the second equation, x2  1  x3  4 x4

From this and the first equation, x1  2  x4

Since x3 and x4 remain arbitrary, we have infinitely many solution.

Advanced Engineering Mathematics

7.3 Linear Systems of Equations.

Gauss Elimination

 Ex.4 Gauss Elimination if no Solution Exists

3 2 1 3 3 x1  2 x2  x3  3
2 1 1 0  2 x1  x2  x3  0

6 2 4 6  6 x1  2 x2  4 x3  6

Step 1 Elimination of x1 :  2 3 times the first equation to the second equation,

 6  2 times the first equation to the third equation.

3
3 2 1 3 3x1  2 x2  x3  3
 
0  1 1  2 1 1
2  Row 2      Row 1  x2  x3  2
 3 3   3 3 3
0 2 2 
0  Row 3   2   Row 1 2 x2  2 x3  0

Step 2 Elimination of x2
3 2 1 3 3x1  2 x2  x3  3
 
0  1 1
2 
1 1
 x2  x3  2
 3 3  3 3
0 0 Row 3   6   Row 2
 0 12  0  12

The false statement 0 = 12 shows that the system has no solution.

Advanced Engineering Mathematics

7.3 Linear Systems of Equations.

Gauss Elimination

 Row Echelon Form and Information From It

 Row Echelon Form
: At the end of the Gauss elimination the form of the coefficient matrix, the augmented matrix
and the system itself

 Three possible cases :

• Exactly one solution : r = n and b r 1 , , b m , if present, are zero.
• Infinitely many solutions : r < n and b r 1 , , b m , if present, are zero.
• No solution : r < m and one of the entries b r 1 , , b m is not zero.
Advanced Engineering Mathematics

7.4 Linear Independence. Rank of a Matrix.

Vector Space

 Linear Independence and Dependence of Vectors

(1) c1a1  c2a 2   cma m  0 c , c ,

1 2 , cm : any scalars, a1 , , a m : vectors 
• Independent : (1) has only the trivial solution.

• Linearly Dependent : (1) holds with scalars not all zero.

 Rank of a Matrix

 Definition
Rank : The maximum number of linearly independent row vectors

 Theorem 1 Row-Equivalent Matrix

Row-equivalent matrices have the same rank.
Advanced Engineering Mathematics

7.4 Linear Independence. Rank of a Matrix.

Vector Space

 Theorem 2 Linear Independence and Dependence of Vectors

p vectors with n components each are linearly independent if the matrix with these vectors as
row vectors has rank p, but they are linearly dependent if that rank is less than p.

 Theorem 3 Rank in Terms of Column Vectors

The rank r equals the maximum number of linearly independent column vectors.
Hence the matrix and its transpose have the same rank.

 Theorem 4 Linear Dependence of Vectors

p vectors with n < p components are always linearly dependent.
Advanced Engineering Mathematics

7.4 Linear Independence. Rank of a Matrix.

Vector Space

 Vector Space : A set V of vectors such that with any two vectors in V all their linear
combinations are elements of V, and these vectors satisfy the following laws.
AB BA c  A  B   cA  cB
 A  B   C  A   B  C  c  k  A  cA  kA
A0 A c  kA    ck  A
A   A   0 1A  A

• Dimension : The maximum number of linearly independent vectors in V

• Basis : A linearly independent set in V consisting of a maximum possible number of vectors in V
• Span : The set of all linear combinations of given vectors with the same number of components.
• Subspace : A nonempty subset that forms itself a vector space with respect to the two algebraic
operations ( addition and scalar multiplication ) defined for the vectors,
Advanced Engineering Mathematics

7.4 Linear Independence. Rank of a Matrix.

Vector Space

 Theorem 5 Vector Space R n

n
The vector space R consisting of all vectors with n components ( n real numbers ) has
dimension n.

 Theorem 6 Row Space and Column Space

The row space and the column space of a matrix A have the same dimension, equal to rank A.

• Null Space of A : is the solution set of the homogeneous system Ax = 0

• Nullity of A : The dimension of the null space

rank A + nullity A = Number of columns of A

Advanced Engineering Mathematics

7.5 Solutions of Linear Systems: Existence,

Uniqueness

 Theorem 1 Fundamental Theorem for Linear Systems

• Existence : A linear system of m equations in n unknowns is consistent, that is, has solutions, if
and only if the coefficient matrix and the augmented matrix have same rank.
• Uniqueness : The linear system has precisely one solution if and only if this common rank r of
the coefficient matrix and the augmented matrix equals n.
• Infinitely Many Solutions : If this common rank r is less than n, the system has infinitely many
solutions. All of these solutions are obtained by determining r suitable unknowns
(whose submatrix of coefficients must have rank r ) in terms of the remaining n – r
unknowns, to which arbitrary values can be assigned.
• Gauss Elimination : If solutions exist, they can all be obtained by the Gauss elimination. ( This
method will automatically reveal whether or not solutions exist)
Advanced Engineering Mathematics

7.5 Solutions of Linear Systems: Existence,

Uniqueness

 Homogeneous Linear System

 Theorem 2 Homogeneous Linear System

• A homogeneous linear system of m equations in n unknowns always has the trivial solution.
• Nontrivial solutions exist if and only if rank A < n.
• If rank A = r < n, these solutions, together with x = 0, form a vector space of dimension n – r,
called the solution space.
• Linear combination of two solution vectors of the homogeneous linear system is a solution
vector.

 Theorem 3 Homogeneous Linear System with Fewer Equations Than Unknowns

A homogeneous linear system with fewer equations than unknowns has always nontrivial
solutions.
Advanced Engineering Mathematics

7.5 Solutions of Linear Systems: Existence,

Uniqueness

 Theorem 4 Nonhomogeneous Linear System

If a nonhomogeneous linear system is consistent, then all of its solutions are obtained as
x  x0  xh where x 0 is any solution of the nonhomogeneous linear system and x h runs through
all the solutions of the corresponding homogeneous system.
Advanced Engineering Mathematics

7.6 For Reference : Second- and Third-Order

Determinants

a11 a12
 Determinant of Second Order : D  det  A    a11a22  a12 a21
a21 a22

 Cramer’s rule
b1 a12
b2 a22 b1a22  a12b2
Cramer’s rule x1  
a11 x1  a12 x2  b1 D D
a21 x1  a22 x2  b2 D0 a11 b1
a b a b b a
x2  21 2  11 2 1 21
D D
Advanced Engineering Mathematics

7.6 For Reference : Second- and Third-Order

Determinants

 Determinant of Third Order :

a11 a12 a13
a a23 a a13 a a13
D  det(A)  a21 a22 a23  a11 22  a21 12  a31 12 
a32 a33 a32 a33 a22 a23
a31 a32 a33
 a11a22a33  a11a23a32  a21a12a33  a21a13a32  a31a12a23  a31a13a22
 Cramer’s rule

a11 x1  a12 x2  a13 x3  b1 Cramer’s rule

a21 x1  a22 x2  a23 x3  b2 D1 D D
x1  , x2  2 , x3  3
a31 x1  a32 x2  a33 x3  b3 D0 D D D

b1 a12 a13 a11 b1 a13 a11 a12 b1

D1  b2 a22 a23 , D2  a21 b2 a23 , D3  a21 a22 b2
b3 a32 a33 a31 b3 a33 a31 a32 b3
Advanced Engineering Mathematics

7.7 Determinants. Cramer’s Rule

a11 a12 a1n

a21 a22 a2 n
 Determinant of Third Order n : D  det  A  

an1 an 2 ann

• If n = 1 then D  a11

• If n  2 then D  a j1C j1  a j 2C j 2   a jnC jn  j  1, 2, , n

or D  a1k C1k  a2k C2k   ank Cnk  k  1, 2, , n ,

• The minor M jk of a jk in D : a determinant of order n - 1

• The cofactor C jk of a jk in D : C jk   1 j k M jk

Advanced Engineering Mathematics

7.7 Determinants. Cramer’s Rule

 General Properties of Determinants

 Theorem 1 Behavior of an nth-Order Determinant under Elementary Row Operations

• Interchange of two rows multiplies the value of the determinant by -1.

• Addition of a multiple of a row to another row does not alter the value of the determinant.

• Multiplication of a row by a nonzero constant c multiplies the value of the determinant by c.

Advanced Engineering Mathematics

7.7 Determinants. Cramer’s Rule

 Theorem 2 Further Properties of nth-Order Determinants

• Interchange of two columns multiplies the value of the determinant by -1.

• Addition of a multiple of a column to another column does not alter the value of the determinant.

• Multiplication of a column by a nonzero constant c multiplies the value of the determinant by c.

• Transposition leaves the value of a determinant unaltered.

• A zero row or column renders the value of a determinant zero.

• Proportional rows or columns render the value of a determinant zero.

Advanced Engineering Mathematics

7.7 Determinants. Cramer’s Rule

 Theorem 3 Rank in Terms of Determinants

Consider an m  n matrix A   a jk  :

• A has rank r  1 if and only if A has an r  r submatrix with nonzero determinant.

• The determinant of any square submatrix with more than r rows, contained in A has

a value equal to zero.

• An n  n square matrix A has rank n if and only if det  A   0 .

Advanced Engineering Mathematics

7.7 Determinants. Cramer’s Rule

 Cramer’s Rule

 Theorem 4 Cramer’s Rule(Solution of Linear Systems by Determinants)

a11 x1  a12 x2   a1n xn  b1
a21 x1  a22 x2   a2 n xn  b2 Cramer’s rule
D1 D Dn
x1  , x2  2 , , xn 
D0 D D D
an1 x1  an 2 x2   ann xn  bn

Dk is the determinant obtained from D by replacing in D the kth column by the column

with the entries b1 , , bn .

Advanced Engineering Mathematics

7.8 Inverse of a Matrix.

Gauss-Jordan Elimination

 Inverse Matrix
A1 : The inverse of an n  n matrix A  a jk   AA1  A1A  I

• Nonsingular Matrix : The matrix whose determinant is not equal to zero

• Singular Matrix : A square matrix whose determinant is zero

• If the matrix has an inverse, the inverse is unique.
 Theorem 1 Existence of the Inverse
• The inverse of an matrix A exists if and only if rank A = n, thus if and only if det A  0
• A is nonsingular if rank A = n
• A is singular if rank A < n
 Determination of the Inverse by the Gauss-Jordan Method

Gauss elimination
A  A I I K   A1  K
Advanced Engineering Mathematics

7.8 Inverse of a Matrix.

Gauss-Jordan Elimination

 1 1 2 
 
 Ex.1 Determine the inverse of A   3 1 1 
 1 3 4 

 1 1 2 1 0 0
A I    3 1 1 0 1 0 
 1 3 4 0 0 1 
 1 1 2 1 0 0  1 1 2 1 0 0
  0 2 7 3 1 0  Row 2  3  Row 1  0 2
 7 3 1 0 
 0 2 2 1 0 1  Row 3  Row 1  0 0 5 4 1 1  Row 3  Row 2
1 1 2 1 0 0  Row 1 1 1 0 0.6 0.4 0.4  Row 1  2  Row 3
 0 1 3.5 1.5 0.5 0  0.5  Row 2  0 1
 0 1.3 0.2 0.7  Row 2  3.5  Row 3
0 0 1 0.8 0.2 0.2  0.2  Row 3 0 0 1 0.8 0.2 0.2 
1 0 0 0.7 0.2 0.3  Row 1  Row 2
 0 1 0 1.3 0.2 0.7 
0 0 1 0.8 0.2 0.2 
Advanced Engineering Mathematics

7.8 Inverse of a Matrix.

Gauss-Jordan Elimination

 Theorem 2 Inverse of a Matrix

 C11 C21 Cn1 
 Cn 2 
1 1 C12 C22
The inverse of a nonsingular n  n matrix A   a jk  is A 
1
C jk  
det A   det A  
 
C1n C2 n Cnn 
where C jk is the cofactor of a jk in det A.

a a12  1 1  a22 a12 

In particular, the inverse of A   11 
a22  det A  a21 a11 
is A .
 a21
Advanced Engineering Mathematics

7.8 Inverse of a Matrix.

Gauss-Jordan Elimination

 1 1 2 
 
 Ex.3 Find the inverse of A   3 1 1 
 1 3 4 
det A  1 7   113  2  8  10,
1 1 1 2 1 2
C11   7, C21    2, C31   3,
3 4 3 4 1 1
 0.7 0.2 0.3 
1 2 1 2
A   1.3 0.2 0.7 
3 1 1
C12   13, C22   2, C32    7, 
1 4 1 4 3 1
 0.8 0.2 0.2 
3 1 1 1 1 1
C13   8, C23    2, C33   2
1 3 1 3 3 1
Advanced Engineering Mathematics

7.8 Inverse of a Matrix.

Gauss-Jordan Elimination

 The inverse of the diagonal matrices

A   a jk  , a jk  0 when j  k , have an inverse if and only if all a jj  0.
Then A 1 is diagonal, too, wigh entries 1 , , 1 .
a11 ann

 0.5 0 0   2 0 0
 Ex.4 The inverse of A   0 4 0  is A 1   0 0.25 0 

 0 0 1   0 0 1 

 Products :  AC 1  C1A 1

 AC PQ   Q 1P 1
1
C1A 1

 The inverse of the inverse :  A 1   A

1
Advanced Engineering Mathematics

7.8 Inverse of a Matrix.

Gauss-Jordan Elimination

 Unusual Properties of Matrix Multiplication. Cancellation Laws

• Matrix multiplication is not commutative, that is, in general we have AB  BA

1 1   1 1  0 0 
• AB = 0 does not generally imply A = 0 or B = 0; for example,   
2 2  1 1 0 0

• AC = AD does not generally imply C = D (even when A  0 ).

 Theorem 3 Cancellation Laws

Let A, B, C be n  n matrices. Then :
• If rank A = n and AB = AC, then B = C.
• If rank A = n, then AB = 0 implies B = 0.
Hence if AB = 0, but A  0 as well as B  0 , then rank A < n and rank B < n.
• If A is singular, so are BA and AB.
Advanced Engineering Mathematics

7.8 Inverse of a Matrix.

Gauss-Jordan Elimination

 Theorem 4 Determinant of Product of Matrices: for any n x n matrices A and B

det(AB)=det(BA)=detA detBx
Advanced Engineering Mathematics

7.9 Vector Spaces, Inner Product Spaces,

Linear Transformations

 Definition Real Vector Space

• Vector addition : a + b
Commutativity. a + b = b +a
Associativity. (u+v)+w=u+(v+w)
Zero Vector. a+0=a
a + ( -a ) = 0
• Scalar multiplication : ka
Distributivity. c (a + b) = ca + cb
Distributivity. (c + k ) a = ca + ka
Associativity. c ( ka ) = (ck ) a
1a = a
Advanced Engineering Mathematics

7.9 Vector Spaces, Inner Product Spaces,

Linear Transformations

 Definition Real Inner Product Space

Inner Product : ( a , b )
Linearity. ( qa + kb , c ) = q ( a , c ) + k ( b , c )
Symmetry. (a,b)=(b,a)
Positive-definiteness ( a , a ) r 0,
( a , a ) = 0 if and only if a = 0

• Orthogonal : Vectors whose product is zero

• Norm : a   a,a 
• Unit Vector : A vector of norm 1
Advanced Engineering Mathematics

7.9 Vector Spaces, Inner Product Spaces,

Linear Transformations

 Basic inequality

• Cauchy-Schwarz inequality.  a,b   a b

• Triangle inequality. ab  a  b

• Parallelogram equality. ab  ab  2 a  b

2 2
 2 2

Advanced Engineering Mathematics

7.9 Vector Spaces, Inner Product Spaces,

Linear Transformations

 Linear Transformations

• A mapping ( or transformation or operator ) of X into Y

: We assign a unique vector y in Y to each vector x in X

• Linear mapping or linear transformation F : for all vectors v and x in X and scalars c,

F(v+x)=F(v)+F(x)

F ( cx ) = c F ( x )

 Linear Transformation of Space Rn into Space Rm

• Linear transformation of space Rn into space Rm is linear

• Any real m  n matrix A  a jk  gives a transformation of space R n into space R m