2 Linear Equations and Matrices

In this chapter we explore the important connections between differential equations and
linear algebra. We seek a common language and set of tools that will help unify our discussion of
these two subjects. The key players in our study are vectors and matrices.

Model: Simple Harmonic Motion

Most of the remaining text concerns objects that engage in repetitive or periodic motion.
A typical example is the simple harmonic model (SHM), given by the second-order equation

that could describe the given vertical spring-mass system, where

is one possible solution of this linear equation.

2.1 Second-Order Equations

We may describe any process as linear if there is an operator L that obeys the following
properties for any two quantities
(1)
(2)
The derivative is a linear operator for any two functions ƒ and g since

Many embarrassing algebra mistakes occur when students don’t recognize that other operations
are not linear, which explains why (x + y)² x² + y².

The linear algebraic equation

has a solution point or vector x 0 ú6, given by (x1, x2,..., xn). If b = 0, we have the homogeneous
equation L(x) = 0 and the superposition properties

which are direct results of the linear operator L. Consider the equation

which has two solutions (verify)

along with the sum

The linear nth-order differential equation L(y) = ƒ(t) is formed when the operator

is applied to some function y(t). We had the first-order equation L(y) = yN + p(t)y = q(t). The
2.1 Second-Order Equations

SHM equation L(y) = yO + ù²y = 0 is an example of a second-order homogeneous equation. In

this case, the superposition of any two solutions

is also a solution, since the linear properties this operator permit us to confirm that

Example 1 By inspection, two possible solutions of the (SHM) equation yO + y =

0 are the cosine and sine functions (yellow and blue)

There are many other solutions (graphs) formed by the superposition of

these basic functions, which produce a general solution

The arbitrary parameters are determined by two initial conditions, say

y(0) = 3 and yN(0) = -2, which provide the solution (red)

# For practice, let y(0) = 1 and yN(0) = -2.

When we impose the initial conditions y(a) = b0 and yN(a) = b1 on the equation

with a general solution

we get a linear system of algebraic equations

The values of the c’s will determine the nature of any possible solutions y(x).

Example 2 The functions

form the general solution (superposition)

that satisfies the homogeneous equation yO - 4y = 0 since

The initial conditions y(0) = 7 and yN(0) = 6 produce the system

where A = 5, B = 2 and the solution (graph shown)

The general solution can be expressed in another form, where

2.1 Second-Order Equations

applying the definitions of the hyperbolic functions.

# For practice, let {y

The second-order equation yO + p(x)yN + q(x)y = 0 has a unique solution for any
conditions y(a) = b0 and yN(a) = b1 only if the functions {y1,y2} are linearly independent and not
multiples of each other, where y1/y2 = k (constant). The functions sin x and cos x are independent
since (sin x)/(cos x) = tan x does not have a constant value. On the other hand the functions

are clearly dependent since ƒ(x) = 2g(x).

Example 3 We can verify that the independent functions

are both solutions of the equation yO- 2yN+ y = 0. The general solution
is any function of the form (graphs shown)

while its derivative

and the initial conditions y(0) = 3 and yN(0) = 1 produce the system

with unique solutions c1 = 3, c2 = -2 and the graph (green)

# For practice, let

Example 4 Solve: yO + 2yN = 0.

Notice that we can reduce the order if we let v = yN and obtain the first-
order equation vN = -2v, which by inspection has a solution

This equation can be integrated immediately to produce a function

where the new constant B = -C/2. The initial values y(0) = A + B and
yN(0) = -2B form a simple linear system, where each solution (A,B)
2.1 Second-Order Equations

provides a unique solution (graph) of the IVP.

# For practice, try

The homogeneous linear system

clearly has one trivial solution (0,0), but it may also have other non-trivial solutions depending on
the values of the coefficients.

Example 5 The second-order equation with variable coefficients

has two possible solutions (verify)

so we should expect a general solution of the form

Consider the trivial initial conditions y(1) = yN(1) = 0 and the

homogeneous linear system

which only has a unique trivial solution (why?), so y = 0 is the only

solution of the IVP.
# For practice, let y(0) = yN(0) = 0.

PS2.1
 2.2 Linear Systems

We might expect that any higher-order equation

has a general solution

where the superposition principle is applied to n solutions {y1,y2,...,yn}. There is a unique solution
determined by n initial conditions that form a linear system

For example, we can verify that each one of the following functions

satisfies the third-order equation L(y) = y(3) + 3yO + 4yN + 12y = 0, so

The conditions y(0) = 0, yN(0) = 5 and yO(0) = -39 correspond to the unique solution (graph)

A linear system can have any number of unknowns and equations, for example

The second version is the augmented matrix [A b] with a coefficient matrix A. The size of any
matrix is specified by the number of rows × columns, thus the matrices

are 2 × 2, 1 × 4 and 3 × 1, respectively.

The solution of a linear system is found by reducing the augmented matrix using
elementary row operations: multiply any row by some constant and add the result to another
row. For example, consider this simple case:

Here we have used these operations to create row equivalent matrices. This method is called
Gaussian elimination, where we seek to write the matrix in a row-echelon form E such as

Notice that the leading entry or pivot in each row is 1 with only zeros below.
2.2 Linear Systems

Example 1 Solve:

Our strategy is to simplify the augmented matrix using the “leading 1"
or pivot in the upper left corner to create zeros below it and express the
matrix in row-echelon form:

The last row implies x3 = 3, which is substituted into the equations of

the reduced system

to obtain the unique solution (5, -2, 3).

# For practice, try (1,4,-2)

Example 2 The augmented matrix of the system

has the row echelon form

Since there are 5 unknowns, but only 3 equations, there are many
solutions, which are described by two free parameters (choices), say

The remaining unknowns are then expressed as functions of {s,t}, so

from the second and third equations (rows) we have

Finally, the first equation (row) gives

2.2 Linear Systems

# For practice, solve (2 - 5t,-1 + 3t,t)

An extension of this method is Gauss-Jordan elimination where the goal is to write the
matrix in reduced row-echelon form (RREF), where there are zeros above and below each pivot.

Example 3 We had the linear system and its reduced form

We continue performing row operations until only the pivot values

remain in each column:

The RREF immediately gives the solution as (5, -2, 3), without the
need for any back substitution.
# For practice, let (1,4,-2)

Observe in this example that a unique solution exists and the coefficient matrix is row
equivalent to the identity matrix

We can show this connection is valid for every n × n linear system.

We know that any homogeneous system [A 0] is always consistent since it has a trivial or
zero solution. Consider the system

which is the homogeneous version of the system solved in the previous example. Since the
coefficient matrix is row equivalent to the identity matrix, there is only one solution (0,0,0).

Example 4 Solve:
2.2 Linear Systems

This system also has more unknowns (4) than equations (3), so we
expect many solutions, which is confirmed by row reduction:

The reduced system has two equations and four unknowns, so the
solution is described by two parameters, say x3 = s and x4 = t, so

Each pair of values {s,t} yields a particular solution (x1, x2, x3, x4).

# For practice, solve (3-s-t,5+2s-3t,s,t)

Example 5 We had the third-order equation y(3) + 3yO + 4yN + 12y = 0 with a
general solution

and its derivatives

The initial conditions y(0) = 0, yN(0) = 5 and yO(0) = -39 produce the
following linear system and its RREF version:

Hence, we have the solution (shown)

# For practice, let

PS2.2
 2.3 Matrix Operations

The linear algebraic equation L(x) = b given by

has many solutions x 0 ú6, each given by the vector or point

We can also think of a row vector

which is the transpose of a column (proper) vector. The two basic vector operations, scalar
multiplication and addition each follow expected patterns. For example, consider the vectors

The third vector is a linear combination of the two given vectors.

Example 1 Find the solution of the homogeneous system

The coefficient matrix for this system has the RREF

We have more unknowns (4) than equations (2), so this system has
many solutions given by (say) the parametric form

so the general solution is a combination of two solution vectors:

Notice that we recover the expected trivial solution when s = t = 0.

# For practice, verify this solution.

2.3 Matrix Operations

A matrix is any array formed by stacking or adjoining vectors, for example

Matrix A consists of m row vectors, while matrix B is formed by p column vectors.

If two matrices have the same size, their sum is found by adding the corresponding row
(or column) vectors.

Example 2 Consider the following sums.

(A)

(B)

# For practice, compute A + C. {does not exist}

The third important vector operation involves the row and column vectors

The inner product of these vectors is the number

For example, we have

The rules of matrix algebra were originally developed to study linear systems, where we
recognize, for example, the following equivalent forms:
2.3 Matrix Operations

In the second version we “factor out” the common vector x, so that we may write the system as
the simple matrix equation Ax = b, where we apply the row × column inner product rule.

The matrix product AB is formed by computing the all inner products of the rows of
matrix A with the columns of matrix B. The matrices must therefore be compatible: each row
vector of A(m × p) must have the same number of components as each column vector of B(p ×
n), so that each entry or location of AB(m × n) is the product

Example 3 Consider the products.

(A)

(B)

# For practice, find BA and DC.

Some of the typical algebra rules also work with matrices, while others do not. We just
observed the failure of the commutative rule, AB BA, while the associative law ABC = (AB)C
= A(BC) and distributive law A(B + C) = AB + AC still hold.

Example 4 Consider

Notice that we have the equal matrix products

even though A B, so the cancellation rule is also invalid.

# For practice, verify A(BC) = (AB)C and A(B + C) = AB + AC.

The cancellation rule failed since there is no division of matrices. Instead we require the
existence of the inverse matrix defined by the matrix products

the identity matrix. For example, we have the pair of inverse matrices given by
2.3 Matrix Operations

The transpose of any matrix is formed if we interchange rows and columns, so that

A matrix is symmetric across the main diagonal if the matrix and its transpose are equal A = AT .
The product of a matrix and its transpose is always symmetric.

Example 5 Consider the following product:

# For practice, find AT A.

Project: Rotation Matrix

Matrices have other uses besides their role in working with systems of linear equations.
The linear system Ax = b can also be interpreted as the mapping of one vector x into another b,
an operation T: ú6÷ ú6 given by the linear function T(x) = Ax. One important mapping that we
use later in the course is the result of the rotation matrix in ú², given by

PS2.3
 2.4 Determinants

The linear system Ax = b has the unique solution

provided that the inverse matrix exists. We may verify for the simplest case, the 2 × 2 matrix

there is an inverse matrix

which exists only if the determinant *A* / ad - bc 0. We could also solve the corresponding
linear system

which may be reduced by the elimination of x2 to a single equation

so that one solution (component)

Observe that we have the ratio of two determinants: the bottom number is the determinant of the
coefficient matrix, while the top determinant is found by replacing the first column of A with the
vector b. The second solution will follow a similar pattern:

We have found an equivalent unique solution of the system Ax = b, called Cramer’s rule with
the same restriction *A* 0.

Example 1 Solve:

The determinant of the coefficient matrix is

The remaining determinants are found to be

So, by Cramer’s rule we have the solutions

# For practice, solve (1,2)

2.4 Determinants

We know that the second-order homogeneous equation yO + p(x)yN + q(x)y = 0 has only
two independent solutions {y1,y2} that are chosen to satisfy the linear system

For any constants c1 and c2 we can apply Cramer’s rule to show that these functions should satisfy
the Wronskian condition

For any larger square (n × n) matrix A the determinant

where M is the minor or determinant of the remaining (n - 1) × (n - 1) matrix if n > 2. Each

signed minor is the cofactor Cjk. The sign changes form an alternating grid pattern.

Example 2 Find each determinant.

(A)

(B)

There is only one non-zero minor to compute down the first column
of this triangular matrix, the product along the main diagonal.

# For practice, try {-9}

Example 3 Solve:

Applying Cramer’s rule as before, we compute the determinant after

first using row operations to create another triangular form:
2.4 Determinants

The value of (say) x is computed using the modified determinant

so x = -26/(-26) = 1, obtained without the remaining solutions which

are computed in a similar manner.
# For practice, find y and z. {0,1}

Our work with the linear system Ax = b is nicely summarized by the following list of
equivalent statements, where A is any n × n matrix:
(1) A is row-equivalent to In;
(2) Ax = b has a unique solution for every vector b;
(3) Ax = 0 has only the trivial (zero) solution;
(4) det(A) 0;
(5) A is invertible.

Linear Independence

Recall that two functions y1 and y2 are dependent if they are scalar multiples of each other,
so y1 = ky2 or y2 = ky1. The same definition applies for two vectors, such as v1 = (2,3) and v2 =
(4,6) since 2v1 = v2, which is the linear combination

By definition any dependent vector is a linear combination of other vectors so that

for some non-zero ci. For example, the vectors

are linearly dependent since we can verify the linear combination

so v3 = -4v1 + 3v2. There are many other non-trivial combinations. So, vectors are linearly
independent only if c1 = c2 = ... = ck = 0.

Example 4 Consider these vectors from ú³:

The condition

is equivalent to a linear system Ac = 0 with a coefficient matrix

2.4 Determinants

Row operations also reveal that *A* 0, so the system has only the
trivial solution c1 = c2 = c3 = 0, hence these vectors are linearly
independent.
# For practice, try (1,1,1), (-1,0,1) and (-2,1,4). {dependent}

The number of vectors will determine whether they are independent. It is easy to show
that if the number of vectors k from ú6 are linearly dependent if k > n since the system Ac = 0 has
more unknowns (n+1) than equations (n). So, adding one more vector (say) w 0 ú6 implies that

for some non-zero coefficient ci. If c 0 and the n vectors {v1,v2,...,vn} are independent, then w
is a linear combination of this spanning set that forms a basis in ú6. The standard basis in ú³ is
the set of special vectors

that span all vectors in ú³ since any vector

is a linear combination by the usual rules of matrix algebra. They form the linear system Ic = 0 so
c = 0, confirming that they are linearly independent. Any set of n independent vectors is a basis in
ú6. In the previous example the vectors (1,1,1), (1,2,-1) and (1,3,2) also form a basis for ú³.

If we treat the set of functions {ƒ1,ƒ2,...,ƒk} as one-dimensional vectors, then the definition
of linear independence can be extended so that

when c1 = c2 = ... = ck = 0 for all numbers x.

It should be clear that every quadratic polynomial

is spanned by the fundamental set {1, t, t²}. We know from basic algebra

is only valid for all t when each coefficient is zero. The set {1,t,t²} is the standard basis for P2
and forms a general solution of the third-order equation yNNN = 0.

Example 5 The set of “vectors”

will be independent by definition if

2.4 Determinants

This condition is valid for all t only when each expression in

parentheses is zero, so we have the system Ac = 0 with

This matrix has a determinant *A* = -4, so this system will have a trivial
solution. For any system Ac = b there is a unique These functions are
not only independent, but will also span every polynomial P2 and form
another basis.
# For practice, try {basis for P3}

We may extend the Wronskian determinant to test for the linear independence given a set
of n differentiable functions, where

Example 6 Earlier we encountered three functions

These functions are linearly independent since their Wronskian

Thus, they form a basis for the general solution

of the equation

# For practice, try {-8}

PS2.4