Linear Algebra

Selçuk Demir

February 5, 2025

1 / 24
Systems of Linear Equations

A linear equation is an equation of the form

a1 x1 + a2 x2 + · · · + an xn = b.

Here ai ’s are called the coefficients of the equation.

A system of linear equations (or a linear system) is a collection of one or more linear
equations involving the same variables—say, x1 , x2 , ..., xn . An example is

3x1 + 2x2 − 3x3 = 1

2x1 − x3 = 3.

A solution of the system is a list {s1 , s2 , ..., sn } of numbers that makes each equation
a true statement when the values s1 , s2 , ..., sn are substituted for x1 , x2 , ..., xn ,
respectively.

2 / 24
Systems of Linear Equations

A system of linear equations has

1. no solution, or
2. exactly one solution, or
3. infinitely many solutions.
A system of linear equations is said to be consistent if it has either one solution or
infinitely many solutions; a system is inconsistent if it has no solution.
The essential information of a linear system can be recorded compactly in a
rectangular array called a matrix.

3 / 24
Systems of Linear Equations
Consider the system

3x1 + 2x2 − 3x3 = 1

2x1 − x3 = 3
−x1 + 3x2 + 2x3 = 5.
The matrix  
3 2 −3
 2 0 −1
−1 3 2
is called the coefficient matrix of the system we have.
The matrix  
3 2 −3 1
 2 0 −1 3 
−1 3 2 5
is called the augmented matrix of the system.
4 / 24
Systems of Linear Equations

Example
Let’s solve the system

x1 − 2x2 + x3 = 0
2x2 − 8x3 = 8
5x1 − 5x3 = 10

The augmented matrix is  

1 −2 1 0
0 2 −8 8 
5 0 −5 10
The solution is given by (1, 0, −1).

5 / 24
Systems of Linear Equations

ELEMENTARY ROW OPERATIONS

1. (Replacement) Replace one row by the sum of itself and a multiple of another row.
2. (Interchange) Interchange two rows.
3. (Scaling) Multiply all entries in a row by a nonzero constant.
Row operations can be applied to any matrix, not merely to one that arises as the
augmented matrix of a linear system. Two matrices are called row equivalent if there
is a sequence of elementary row operations that transforms one matrix into the other.
Remark
The row operations are reversible!
FACT
If the augmented matrices of two linear systems are row equivalent, then the two
systems have the same solution set.

6 / 24
Systems of Linear Equations

TWO FUNDAMENTAL QUESTIONS ABOUT A LINEAR SYSTEM

1. Is the system consistent; that is, does at least one solution exist?
2. If a solution exists, is it the only one; that is, is the solution unique?

Example
Consider the system

x1 − 2x2 + x3 = 0
2x2 − 8x3 = 8
5x1 − 5x3 = 10

7 / 24
Systems of Linear Equations

The
 augmented matrix
  
1 −2 1 0 1 −2 1 0
0 2 −8 8  is row equivalent to the matrix 0 1 −4 4 
5 0 −5 10 0 0 1 −1
This shows that the system is consistent.

8 / 24
Systems of Linear Equations

Example
Determine whether the following system of equations is consistent.

x2 − 4x3 = 8
2x1 − 3x2 + 2x3 = 1
4x1 − 8x2 + 12x3 = 1
   
0 1 −4 8 2 −3 2 1
The augmented matrix 2 −3 2 1 is row equivalent to 0 1 −4 8 
4 −8 12 1 0 0 0 15
which is clearly incosistent.

9 / 24
Row Reduction

A rectangular matrix is in echelon form (or row echelon form) if it has the following
three properties:
1. All nonzero rows are above any rows of all zeros.
2. Each leading entry of a row is in a column to the right of the leading entry of the
row above it.
3. All entries in a column below a leading entry are zeros.
If a matrix in echelon form satisfies the following additional conditions, then it is
in reduced echelon form (or reduced row echelon form):
4. The leading entry in each nonzero row is 1.
5. Each leading 1 is the only nonzero entry in its column.

10 / 24
Row Reduction

Theorem (Uniqueness of the Reduced Echelon Form)

Each matrix is row equivalent to one and only one reduced echelon matrix.
If a matrix A is row equivalent to an echelon matrix U , we call U an echelon form
(or row echelon form) of A; if U is in reduced echelon form, we call U the reduced
echelon form of A.
Definition
A pivot position in a matrix A is a location in A that corresponds to a leading 1 in
the reduced echelon form of A. A pivot column is a column of A that contains a pivot
position.

11 / 24
 cepts in the first four chapters will be connected in one way or another with pivot posi-
columns
tions of .
in a matrix.
Row Reduction
EXAMPLE 2 Row reduce the matrix
Example below to echelon form, and locate the pivot
columns of . 2 3
0 3 6 4 9
SOLUTION Use the same basic 6 strategy
1 2 as1 in Section
3 17 71.1. The top of the leftmost
AD6
nonzero column is the first pivot position.
4 2 3A nonzero
0 3 entry, 1 5or pivot, must be placed in this
position. A good choice is to interchange
1 4rows51 and9 4 (because
7 the mental computations
in the next step will not involve fractions).
SOLUTION Use the same basic strategy as in Section 1.1. The top of the leftmost
nonzero column is the first pivot position. APivot
nonzero entry, or pivot, must be placed in this
position. A good choice is to interchange
2
1 4 51 and9 4 (because
 rows 7
3 the mental computations
in the next step will not involve6fractions).
6 1 2 1 3 17 7
4 2 3 0 3 15
Pivot
0 3 6 4 9
6 Pivot column

Create zeros below the pivot, 1, by adding multiples of the first row to the rows below,
12 / 24
 Create zeros below the pivot, 1, by adding multiples of the first row to the rows below,
possible—namely
and in the
obtain matrix (1) second
below. Thecolumn. Chooseinthe
pivot position the2 second
in this position
row mustas be
theasnext
far pivot.
left as
Row Reduction
possible—namely in the second column. Choose the 2 in this position as the next pivot.
Pivot

2 Pivot 3
1 4 5 9 7 (1)
60 2 4 6 67
6
40
7 (1)
5 10 15 15 5
0 3 Next
6 pivot4 column
9
6 Next pivot column
Add times row 2 to row 3, and add times row 2 to row 4.
Add times row
R 1 Linear Equations in Linear Algebra 2 to row 3,
2 and add times row32 to row 4.
1 4 5 9 7
60 2 4 6 67
6
40 0 0 0 05
7 (2)
destroy the echelon arrangement of the leading entries already produced.) However, if
(2)
we interchange rows 3 and 4, we0 can 0produce
0 a5leading
0 entry in column 4.

The matrix in (2) is different from

Pivotany encountered in Section 1.1. There is no way to
create a leading entry in column 3! (Weencountered
can’t use row 1 or 2 because doing soway
would
2 3 2 3
The matrix in (2) is different from any
1 4 5 9 7 in Section 1.1. There
 is no to
create a leading
6 0 2entry
4 in column
6 6 73! (We can’t
General
use row
form:
6 10 or 2 because
 doing
  so
7 would
4 0 0 0 5 0 5
6 7 6 7
40 0 0 5
0 0 0 0 0 0 0 0 0 0
6 6 6 Pivot columns

The matrix is in echelon form and thus reveals that columns 1, 2, and 4 of are pivot 13 / 24
Row Reduction
Pivot columns

The matrix is in echelon form and thus reveals that columns 1, 2, and 4 of are pivot
columns.
Pivot positions
0
2 3
3 6 4 9
6 1 2 1 3 17
AD6 (3)
3
7
4 2 3 0 15
1 4 5 9 7
6 6 6 Pivot columns

A pivot, as illustrated in Example 2, is a nonzero number in a pivot position that is

used as needed to create zeros via row operations. The pivots in Example 2 were 1, 2,
and . Notice that these numbers are not the same as the actual elements of in the
highlighted pivot positions shown in (3).
With Example 2 as a guide, we are ready to describe an efÏcient procedure for14 / 24
 by an
first example.
into echelon form and then into reduced echelon form:
Row Reduction Algorithm
EXAMPLE
Example 3 Apply elementary row operations to transform the following matrix
first into echelon form and then into reduced echelon form:
2 3
0 3 6 6 4 5
SOLUTION 43 7 8 5 8 95
3 9 12 9 6 15
Step 1
SOLUTION
Begin with the leftmost nonzero column. This is a pivot column. The pivot
Step 1
position
Begin is at the top.
Stepwith
1 the leftmost nonzero column. This is a pivot column. The pivot position is
at the top.
Begin with the leftmost 2 nonzero column. This is a 3
pivot column. The pivot
position is at the top. 0 3 6 6 4 5
43 7 8 5 8 95
3 9 12 9 6 15
6 Pivot column
15 / 24
 rows to move this entry into the pivot position.
Row Reduction Algorithm
Step 2
Interchange rows 1entry
Select a nonzero and 3.
in (We couldcolumn
the pivot have interchanged
as a pivot. If rows 1 andinterchange
necessary, 2 instead.)rows to
move this entry into the pivot position.
Step 3 Pivot

2 3
3 9 12 9 6 15
Use row replacement operations to create zeros in all positions below the pivot.
43 7 8 5 8 95
0 3 6 6 4 5
As a preliminary step, we could divide the top row by the pivot, 3. But with two 3’s in
Step 3
column 1, it is just as easy to add times row 1 to row 2.
Step
Use row3replacement operations to create zeros in all positions below the pivot.
Use row replacement operations to create zeros in all positions below the pivot.
Pivot
3  9 12 9 6 15
2 3
40 2 4 4 2 65
As a preliminary step, we could 0 divide
3 the
6 top6 row4by the5 pivot, 3. But with two 3’s in
column 1, it is just as easy to add times row 1 to row 2. 16 / 24
 With row 1 covered, step 1 shows that column 2 is the next pivot column; for step 2,
select
Row as a pivot the
Reduction “top” entry in that column.
Algorithm
Step 4 Cover (or ignore) the row containing the pivot position and cover all rows, if
With
any,row
above1 covered,
it. Apply step
steps11-3
shows that
to the column 2 is remains.
Pivot that
submatrix the next Repeat
pivot column; for until
the process step 2,
select
thereasare
a pivot the nonzero
no more “top” entry
rowsintothat column.
modify.

Pivot
2 3
3 9 New 12 pivot9 column
6 15
40 2 4 4 2 65
0 3 step
For step 3, we could insert an optional 6 of dividing
6 4 the 5 “top” row of the submatrix by
the pivot, 2. Instead, we add 6
times thepivot
New “top” row to the row below. This produces
column

For step 3, we could insert an2optional step of dividing the “top”

3 row of the submatrix by
3 9 12 9 6 15
the pivot, 2. Instead, we add4 times the “top” row to the row below. This produces
0 2 4 4 2 65
0 0 0 0 1 4

17 / 24
 Pivot
When we cover the row containing the second pivot position for step 4, we are left with
Row
new Reduction
a Steps 1–3 require
submatrix Algorithm
no work
having onlyfor
onethis submatrix, and we have reached an echelon form of
row:
the full matrix. If we want the reduced echelon form, we perform one more step.
2 3
3 9 12 9 6 15
40 2 4 4 2 65
Step 5
0 0 0 0 1 4
Beginning with the rightmost pivot and working upward Pivot
and to the left, create
zeros above each pivot. If a pivot is not 1, make it 1 by a scaling operation.
Steps 1–3 require no work for this submatrix, and we have reached an echelon form of
Stepmatrix.
the full 5 If we want the reduced echelon form, we perform one more step.
Beginning with the rightmost pivot and working upward and to the left, create zeros
The rightmost pivot is in row 3. Create zeros above it, adding suitable multiples of row
above each pivot. If a pivot is not 1, make it 1 by a scaling operation.
3 to rows 2 and 1.
Step 5 2 3
 Row 1 C . 6/
row 3
Beginning with3 the9rightmost
12 9 pivot 0 and 9working upward and to the left, create
40 2 4 4 0 14  Row 2 C . 2/
row 3
zeros above each pivot. If a pivot is not 1, make it 1 by a scaling operation.
5
0 0 0 0 1 4

Therightmost
The next pivotpivot
is inisrow 2. Scale
in row this row,
3. Create dividing
zeros byadding
above it, the pivot.
suitable multiples of row 18 / 24
 The next pivot is in row 2. Scale this row, dividing by the pivot.
Row The
Reduction
next pivotAlgorithm
is in row 2. Scale this row, dividing by the pivot.
Row scaled by
Row scaled by
2 3
3 9 12 9 0 9
40 1 2 2 0 75  Row scaled by 1
2
Create a zero in column
0 0 2 by 0 adding
0 91 times4 row 2 to row 1.
Create a zero in column 2 by adding 9 times row 2 to row 1.
Create a zero2 in column 2 by adding 9 times 3 row 2 to
Rowrow
1 1. row 2
3 0 6 9 0 72  Row 1 C .9/
row 2
40 1 2 2 0 75 Row 1 row 2
0 0 0 0 1 4
Finally, scale row 1, dividing by the pivot, 3.
Finally, scale row
2 1, dividing by the pivot, 3. 3
Finally, scale row
1 1, 0dividing
2 by 3 the0pivot,
243.  Row scaled by 31
40 1 2 2 0 75 Row scaled by
0 0 0 0 1 4 Row scaled by

This is the reduced echelon form of the original matrix.

19 / 24
 choice of
ced to determines a (different) solution of the system, and every solution of the
Row Reduction
system Algorithm
is determined by a choice of .

EXAMPLE
Example 4 Find the general solution of the linear system whose augmented matrix
Thehas beenisreduced
matrix to form,2but we want the reduced echelon
in echelon 3 form
1 6 is2completed
g for the basic variables. The row reduction 5 2 next. 4 The symbol
atrix indicates that the matrix is row0 equivalent
0 2 to 8the preceding
1 3 5 matrix.
4
0 0 0 0 1 7
SOLUTION The matrix 5 is 4in echelon form, but we want
5 the reduced echelon form
before solving echelon
The reduced for the basic
form variables. The rowmatrix
of the augmented reduction
is is completed next. The symbol
before a matrix indicates
3 that
2 the matrix is row equivalent
3 to the preceding matrix.
1 6 0 3 0 0
5  40 0 1 4 0 55
0 0 0 0 1 7

he term leading variables because they correspond to the columns containing leading
20 / 24
Row Reduction
There Algorithm
are five variables because the augmented matrix has six columns. The associated
There
Theare five variables because the augmented matrix has six columns. The associated
associated
system now is system now is
system now is
x1 C 6x2 C 3x4 D0
(6)
x3 4x4 D5 (6)
x5 D 7
The pivot columns of the matrix are 1, 3, and 5, so the basic variables are , , and .
The pivot columns of the matrix are 1, 3, and 5, so the basic variables are , , and .
The remaining variables, and , must be free. solution:
Solve for the basic variables to obtain
TheSolve for the variables,
remaining basic variablesand
to obtain thebegeneral
, must free. Solve for the basic variables to obtain
the general solution:
the general solution: 8
ˆ
ˆ x1 D 6x2 3x4
<x2isisfree
free
ˆ
ˆ
ˆ
ˆ
x3 D 5 C 4x4 (7)
(7)
ˆx4isisfree
free
ˆ
ˆ
ˆ
ˆ
x5 D 7
:̂

Note that the value of is already fixed by the third equation in system (6). 21 / 24
 just the right device for answering two fundamental questions posed in Section 1.1.
Although
Row Reductiona nonreduced echelon form is a poor tool for solving a system, this form is
Algorithm
just the right device for answering two fundamental questions posed in Section 1.1.
EXAMPLE
Example 5 Determine the existence and uniqueness of the solutions to the system
EXAMPLE 5 Determine the3xexistence and uniqueness of the solutions to the system
6x C 6x C 4x D 5
2 3 4 5
3x1 7x2 C 8x3 5x4 C 8x5 D 9
3x1 9x2 C 12x3 9x4 C 6x5 D 15
SOLUTION The augmented matrix of this system was row reduced in Example 3 to
By an example above, we know that the following matrix is an echelon form of the
augmented The
SOLUTION augmented
matrix matrix
of the given of this
system system was row reduced in Example 3 to
of equations.
2 3 (8)
3 9 12 9 6 15
40 2 4 4 2 65 (8)
The basic variables are , 0 , and 0 ;0 the 0free 1variables
4 are and . There is no
equation such as that would indicate an inconsistent system, so we could use back-
The basic variables are , , and ; the free variables are and . There is no
substitution
One can seeto that
find ax3solution. But
and x4 are thevariables.
free existence of a solution is already clear in (8). Also,
equation such as that would indicate an inconsistent system, so we could use back-
the solution is not unique because there are free variables. Each different choice of 22 / 24
Row Reduction Algorithm

Theorem (Existence and Uniqueness Theorem)

A linear system is consistent if and only if the rightmost column of the augmented
matrix is not a pivot column—that is, if and only if an echelon form of the augmented
matrix has no row of the form
[ ]
0 0 0 ··· b

with b nonzero If a linear system is consistent, then the solution set contains either (i)
a unique solution, when there are no free variables, or (ii) infinitely many solutions,
when there is at least one free variable.

23 / 24
Row Reduction Algorithm

USING ROW REDUCTION TO SOLVE A LINEAR SYSTEM

1. Write the augmented matrix of the system.
2. Use the row reduction algorithm to obtain an equivalent augmented matrix in
echelon form. Decide whether the system is consistent. If there is no solution,
stop; otherwise, go to the next step.
3. Continue row reduction to obtain the reduced echelon form.
4. Write the system of equations corresponding to the matrix obtained in step 3.
5. Rewrite each nonzero equation from step 4 so that its one basic variable is
expressed in terms of any free variables appearing in the equation.

24 / 24