Linear Algebra Fundamentals

Linear Algebra Fundamentals

Linear Algebra Fundamentals

By

Kartikeya Dutta

Linear Algebra Fundamentals

Kartikeya Dutta

ISBN - 9789361520433

COPYRIGHT © 2025 by Educohack Press. All rights reserved.

This work is protected by copyright, and all rights are reserved by the Publisher. This includes, but is not limited to, the rights to translate, reprint, reproduce, broadcast, electronically store or retrieve, and adapt the work using any methodology, whether currently known or developed in the future.

The use of general descriptive names, registered names, trademarks, service marks, or similar designations in this publication does not imply that such terms are exempt from applicable protective laws and regulations or that they are available for unrestricted use.

The Publisher, authors, and editors have taken great care to ensure the accuracy and reliability of the information presented in this publication at the time of its release. However, no explicit or implied guarantees are provided regarding the accuracy, completeness, or suitability of the content for any particular purpose.

If you identify any errors or omissions, please notify us promptly at [email protected] & [email protected] We deeply value your feedback and will take appropriate corrective actions.

The Publisher remains neutral concerning jurisdictional claims in published maps and institutional affiliations.

Published by Educohack Press, House No. 537, Delhi- 110042, INDIA

Email: [email protected] & [email protected]

Cover design by Team EDUCOHACK

Preface

Welcome to the fascinating world of Linear Algebra! This book is crafted with the undergraduate student in mind, aiming to provide a comprehensive yet accessible introduction to the fundamental concepts and techniques of linear algebra.

Linear algebra serves as the mathematical foundation for a wide range of disciplines, including mathematics, physics, engineering, computer science, and economics. Its applications are ubiquitous, from solving systems of linear equations to analyzing complex data sets and understanding geometric transformations.

In this text, we embark on a journey through the key topics of linear algebra, beginning with the basic concepts of vectors, matrices, and systems of linear equations. We then delve into foundational topics such as vector spaces, linear transformations, eigenvalues, and eigenvectors, building a solid understanding of the core principles.

Throughout the book, we emphasize both theoretical rigor and practical applications, providing numerous examples and exercises to reinforce learning and problem-solving skills. Real-world applications of linear algebra are woven into the narrative, illustrating its relevance in diverse fields and inspiring curiosity about its potential applications.

Whether you’re a mathematics major, an aspiring engineer, or a student exploring the connections between mathematics and your chosen field, this book aims to equip you with the mathematical tools and insights necessary to navigate the complexities of linear algebra with confidence and enthusiasm. So, let’s embark on this journey together and uncover the beauty and power of linear algebra!

Table of Contents

Chapter-1

Introduction to Linear Algebra 1

1.1 Matrices and Vectors 1

1.2 Systems of Linear Equations 2

1.3 Matrix Operations 4

1.4 Linear Transformations 6

1.5 Applications of Linear Algebra 8

1.6 Notation and Terminology 9

References 11

Chapter-2

Vectors and Vector Spaces 12

2.1 Introduction to Vectors 12

2.2 Vector Operations 13

2.3 Vector Spaces 15

2.4 Subspaces 17

2.5 Span and Linear Independence 19

2.6 Basis and Dimension 23

2.7 Coordinate Vectors 26

References 27

Chapter-3

Matrix Algebra 28

3.1 Types of Matrices 28

3.2 Matrix Operations 30

3.3 Matrix Multiplication 34

3.4 Inverse of a Matrix 36

3.5 Elementary Row Operations 40

3.6 Determinants 44

3.7 Properties of Determinants 46

Conclusion 49

References 49

Chapter-4

Systems of Linear Equations 50

4.1 Introduction to Systems of Linear

Equations 50

4.2 Gaussian Elimination 52

4.3 Gauss-Jordan Elimination 55

4.4 Homogeneous and Non-Homogeneous Systems 56

4.5 Existence and Uniqueness of

Solutions 57

4.6 Applications of Systems of Linear Equations 58

Conclusion 60

References 60

Chapter-5

Vector Spaces and Subspaces 61

5.1 Definition and Examples of Vector

Spaces 61

5.2 Subspaces 62

5.3 Linear Combinations and Span 64

5.4 Linear Independence 67

5.5 Basis and Dimension 68

5.6 Change of Basis 71

Conclusion 74

References 74

Chapter-6

Linear Transformations 76

6.1 Introduction to Linear

Transformations 76

6.2 Matrix Representation of Linear Transformations 77

6.3 Kernel and Range of a Linear

Transformation 78

6.4 Isomorphisms 79

6.5 Compositions of Linear

Transformations 81

6.6 Invertible Linear Transformations 83

Conclusion 85

References 85

Chapter-7

Eigenvalues and Eigenvectors 86

7.1 Introduction to Eigenvalues and

Eigenvectors 86

7.2 Characteristic Polynomial 88

7.3 Diagonalization 89

7.4 Similarity of Matrices 91

7.5 Applications of Eigenvalues and

Eigenvectors 93

Conclusion 94

References 95

Chapter-8

Inner Product Spaces 96

8.1 Definition and Examples of Inner

Product Spaces 96

8.2 Norm and Distance 97

8.3 Orthogonality 98

8.4 Orthogonal Complements 99

8.5 Gram-Schmidt Process 101

8.6 Orthogonal Matrices 101

Conclusion 103

References 103

Chapter-9

Orthogonal Projections and Least Squares 105

9.1 Orthogonal Projections 105

9.2 Least Squares Approximations 107

9.3 Normal Equations 110

9.4 Orthogonal Matrices and

Projections 110

9.5 Applications of Least Squares 112

Conclusion 117

References 117

Chapter-10

Determinants and their Properties 118

10.1 Definition and Evaluation of

Determinants 118

10.2 Properties of Determinants 120

10.3 Cramer’s Rule 122

10.4 Inverse of a Matrix using

Determinants 125

10.5 Applications of Determinants 128

Conclusion 129

References 129

Glossary 130

Index 132

Chapter-1

Introduction to

Linear Algebra

1.1 Matrices and Vectors

Matrices and vectors are the fundamental building blocks of linear algebra. They provide a concise way to represent and manipulate systems of linear equations, which arise frequently in various fields such as physics, engineering, computer science, economics, and statistics.

Vectors

A vector is an ordered collection of numbers, often representing a point or a directed line segment in space. Vectors can be expressed in different forms, including column vectors and row vectors.

Column Vector: A column vector is a rectangular array of numbers arranged vertically. It is denoted by a bold-faced lowercase letter or a letter with an arrow on top.

Example:

⃗x = [3, -2, 5]^T or x = [3, -2, 5]^T

Row Vector: A row vector is a rectangular array of numbers arranged horizontally. It is denoted by a bold-faced lowercase letter or a letter with an arrow on top.

Example:

y→ = [1, 4, -3] or y = [1, 4, -3]

Vectors can be added, subtracted, and multiplied by scalars (constants). These operations follow specific rules and have geometric interpretations.

Vector Addition: To add two vectors of the same dimension, we add their corresponding components.

Example:

If ⃗a = [2, 3, -1] and ⃗b = [-1, 4, 2], then ⃗a + ⃗b = [2 + (-1), 3 + 4, -1 + 2] = [1, 7, 1]

Vector Subtraction: To subtract two vectors of the same dimension, we subtract their corresponding components.

Example:

If ⃗a = [2, 3, -1] and ⃗b = [-1, 4, 2], then ⃗a - ⃗b = [2 - (-1), 3 - 4, -1 - 2] = [3, -1, -3]

Scalar Multiplication: To multiply a vector by a scalar, we multiply each component of the vector by the scalar.

Example:

If ⃗a = [2, 3, -1] and k = 4, then k⃗a = [4 × 2, 4 × 3, 4 × (-1)] = [8, 12, -4]

Matrices

A matrix is a rectangular array of numbers arranged in rows and columns. Matrices are denoted by bold-faced capital letters.

Example:

A = [1 2 3

4 5 6

7 8 9]

The dimensions of a matrix are typically expressed as m × n, where m is the number of rows, and n is the number of columns.

Types of Matrices:

•Square Matrix: A matrix with the same number of rows and columns (m = n).

•Rectangular Matrix: A matrix with a different number of rows and columns (m ≠ n).

•Diagonal Matrix: A square matrix with all non-diagonal entries equal to zero.

•Scalar Matrix: A diagonal matrix with all diagonal entries equal to the same value.

•Identity Matrix: A square matrix with ones on the main diagonal and zeros elsewhere.

•Zero Matrix: A matrix with all entries equal to zero.

Fig. 1.1 Linear Algebra

https://images.app.goo.gl/1DxPxRfZV7aKYavv8

Practice Problems:

(1) Represent the following vectors in column vector form and row vector form:

(a) The point (2, -3, 5) in 3D space.

(b) The vector from (1, 2, 3) to (4, 5, 6).

(2) Given the vectors ⃗a = [1, 2, 3], ⃗b = [-2, 4, 1], and ⃗c = [3, -1, 2], find:

(a) 2⃗a - 3⃗b

(b) ⃗a + ⃗b + ⃗c

(c) 4⃗c

(3) Determine the dimensions of the following matrices:

(a) A = [1 2 3

4 5 6]

(b) B = [7 8

9 0

1 2

3 4]

(c) C = [5]

Solutions:

(1) a) Column vector: [2, -3, 5]^T Row vector: [2, -3, 5]

(b) Column vector: [3, 3, 3]^T Row vector: [3, 3, 3]

(2) (a) 2⃗a - 3⃗b = [2(1) - 3(-2), 2(2) - 3(4), 2(3) - 3(1)] = [5, -6, 5]

(b) ⃗a + ⃗b + ⃗c = [1 + (-2) + 3, 2 + 4 + (-1), 3 + 1 + 2] = [2, 5, 6]

(c) 4⃗c = [4(3), 4(-1), 4(2)] = [12, -4, 8]

(3) (a) A is a 2 × 3 matrix (2 rows, 3 columns).

(b) B is a 4 × 2 matrix (4 rows, 2 columns).

(c) C is a 1 × 1 matrix (1 row, 1 column).

1.2 Systems of Linear Equations

A system of linear equations is a collection of two or more linear equations involving the same set of variables. Linear equations are equations in which each term is either a constant or a variable multiplied by a constant coefficient, and the variables are raised to the power of 1.

Example:

2x + 3y = 7

4x - y = 2

In this system, there are two equations and two variables, x and y.

Solving a system of linear equations means finding the values of the variables that satisfy all the equations simultaneously. There are various methods to solve systems of linear equations, including:

(1) Graphical Method (for two variables)

(2) Substitution Method

(3) Elimination Method

(4) Matrix Method (using invertible matrices)

The graphical method is limited to systems with two variables, as it involves plotting the lines represented by each equation on the same coordinate plane and finding the intersection point, which represents the solution.

The substitution method involves expressing one variable in terms of the other from one equation and substituting this expression into the remaining equations to solve for the other variables.

The elimination method involves multiplying one or more equations by constants and then adding or subtracting the resulting equations to eliminate one of the variables. This process is repeated until there is only one variable left, which can be solved for. The remaining variables are then found by back-substitution.

The matrix method involves representing the system of linear equations in matrix form and using matrix operations to find the solution, provided that the coefficient matrix is invertible.

Practice Problems:

(1) Solve the following system of linear equations using the substitution method:

2x + 3y = 7

x - y = 1

(2) Solve the following system of linear equations using the elimination method:

3x + 2y = 5

2x - y = 4

(3) Represent the following system of linear equations in matrix form and solve it using the matrix method:

x + 2y - z = 3

2x + 4y - 2z = 6

3x + 6y - 3z = 9

Solutions:

(1) Substitution method:

From the second equation, x - y = 1

x = y + 1

Substituting x = y + 1 into the first equation:

2(y + 1) + 3y = 7

2y + 2 + 3y = 7

5y = 5

y = 1

Substituting y = 1 into x = y + 1:

x = 1 + 1 = 2

Therefore, the solution is x = 2, y = 1.

(2) Elimination method:

Multiplying the first equation by 2 and the second equation by 3:

6x + 4y = 10

6x - 3y = 12

Subtracting the second equation from the first:

0x + 7y = -2

7y = -2

y = -2/7

Substituting y = -2/7 into the first equation:

3x + 2(-2/7) = 5

3x - 4/7 = 5

3x = 5 + 4/7

x = (5 + 4/7)/3

x = 3

Therefore, the solution is x = 3, y = -2/7.

(3) Matrix form:

[1 2 -1][x] [3]

[2 4 -2][y] = [6]

[3 6 -3][z] [9]

Let A = [1 2 -1

2 4 -2

3 6 -3]

and X = [x

y

z]

and B = [3

6

9]

Then, AX = B

To find X, we need to find the inverse of A, denoted as A^-1, and multiply both sides by A^-1:

A^-1 * AX = A^-1 * B

I * X = A^-1 * B

X = A^-1 * B

Using a calculator or computer software to find A^-1 and perform the multiplication:

A^-1 = [ 1 0 1

-1 1 0

1 -2 1]

X = A^-1 * B

= [ 1 0 1 [3

-1 1 0] * [6

1 -2 1] [9]

= [1

2

1]

Therefore, the solution is x = 1, y = 2, z = 1.

1.3 Matrix Operations

Matrices can be added, subtracted, multiplied, and operated on in various ways. These operations have specific rules and properties that must be followed for the operations to be valid and meaningful.

Matrix Addition and Subtraction

Two matrices can be added or subtracted only if they have the same dimensions (the same number of rows and columns). The addition or subtraction is performed element-wise, where corresponding elements are added or subtracted.

Example:

Let A = [1 2

3 4]

and B = [5 6

7 8]

Then, A + B = [1 + 5 2 + 6

3 + 7 4 + 8]

= [6 8

10 12]

And, A - B = [1 - 5 2 - 6

3 - 7 4 - 8]

= [-4 -4

-4 -4]

Scalar Multiplication

A scalar (a single number) can be multiplied with a matrix by multiplying the scalar with each element of the matrix.

Example:

Let A = [1 2

3 4]

and k = 3

Then, kA = [3(1) 3(2)

3(3) 3(4)]

= [3 6

9 12]

Matrix Multiplication

Matrix multiplication is a binary operation that produces a new matrix from two input matrices. However, matrix multiplication is defined only for matrices of compatible dimensions.

If A is an m × n matrix (m rows and n columns), and B is an n × p matrix (n rows and p columns), then the matrix product AB is defined and yields an m × p matrix.

The entries of the product matrix AB are calculated by multiplying each row of A with each column of B and summing the products.

Example:

Let A = [1 2

3 4]

and B = [5 6

7 8]

Then, AB = [1(5) + 2(7) 1(6) + 2(8)

3(5) + 4(7) 3(6) + 4(8)]

= [19 22

43 50]

However, BA is not defined because the number of columns in the first matrix (2) does not match the number of rows in the second matrix (2).

Matrix Transpose

The transpose of a matrix is obtained by interchanging the rows and columns of the matrix. It is denoted by A^T.

Example:

Let A = [1 2 3

4 5 6]

Then, A^T = [1 4

2 5

3 6]

Practice Problems:

(1) Given the matrices A = [1 2

3 4]

and B = [5 6

7 8],

find:

(a) A + B

(b) A - B

(c) 3A

(d) AB

e) BA (if possible)

(2) Find the transpose of the following matrix:

A = [1 2 3

4 5 6

7 8 9]

(3) Solve the following system of linear equations using the matrix method:

x + 2y = 5

3x + 4y = 11

Solutions:

(1) (a) A + B = [1 + 5 2 + 6

3 + 7 4 + 8]

= [6 8

10 12]

(b) A - B = [1 - 5 2 - 6

3 - 7 4 - 8]

= [-4 -4

-4 -4]

(c) 3A = [3(1) 3(2)

3(3) 3(4)]

= [3 6

9 12]

(d) AB = [1(5) + 2(7) 1(6) + 2(8)

3(5) + 4(7) 3(6) + 4(8)]

= [19 22

43 50]

(e) BA is not defined because the number of columns in A (2) does not match the number of rows in B (2).

(2) A^T = [1 4 7

2 5 8

3 6 9]

(3) Matrix form:

[1 2][x] = [5]

[3 4][y] [11]

Let A = [1 2

3 4]

and X = [x

y]

and B = [5

11]

Then, AX = B

To find X, we need to find the inverse of A, denoted as A^-1, and multiply both sides by A^-1:

A^-1 * AX = A^-1 * B

I * X = A^-1 * B

X = A^-1 * B

Using a calculator or computer software to find A^-1 and perform the multiplication:

A^-1 = [-2 1

1.5 -0.5]

X = A^-1 * B

= [-2 1 [5

1.5 -0.5] * [11]

= [1

3]

Therefore, the solution is x = 1, y = 3.

1.4 Linear Transformations

Linear transformations are functions that map vectors from one vector space to another while preserving the properties of vector addition and scalar multiplication. They play a crucial role in linear algebra and have numerous applications in various fields, such as computer graphics, physics, and data analysis.

Definition:

A linear transformation T from a vector space V to a vector space W is a function that satisfies the following two properties for all vectors u and v in V, and all scalars c:

1. T(u + v) = T(u) + T(v) (Preservation of vector addition)

2. T(c * u) = c * T(u) (Preservation of scalar multiplication)

In simpler terms, a linear transformation maps the sum of two vectors to the sum of their images, and it maps a scalar multiple of a vector to the scalar multiple of its image.

Matrix Representation of Linear Transformations:

Linear transformations can be represented by matrices, which provide a convenient way to perform calculations and analyze their properties. If the domain and codomain of a linear transformation are both R^n (n-dimensional vector spaces), then the linear transformation can be represented by an n × n matrix.

Given a linear transformation T: R^n → R^m, and a basis {v1, v2, ..., vn} for R^n and a basis {w1, w2, ..., wm} for R^m, the matrix representation of T is the m × n matrix whose ith column consists of the coordinates of T(vi) with respect to the basis {w1, w2, ..., wm}.

Example:

Let T: R^2 → R^2 be