Comprehensive Linear Algebra Notes

Overview

This document provides detailed class notes and worked problems in Linear Algebra. It includes key

definitions, examples, and practice exercises covering vector spaces, linear independence, basis

and dimension, inner products, linear transformations, and the theory of eigenvalues and

eigenvectors.

This extended document also covers advanced topics like determinants, matrix inverses, rank-nullity

theorem, Gaussian elimination, diagonalization, and applications in data science and engineering.

Solved examples and practice questions with answers are included to support deep understanding.

1. Vector Spaces

A vector space is a collection of objects called vectors, which may be added together and multiplied

by scalars (real or complex numbers), satisfying certain axioms (closure, associativity, identity, etc.).

Key Properties:

* Closure under addition and scalar multiplication

* Associativity and commutativity of addition

* Existence of zero vector and additive inverses

Examples:

* R^2 = {(x, y) | x, y in R}

* Set of polynomials of degree <= 2: {a + bx + cx^2}

* Space of continuous functions on [a, b]: C[a, b]

Practice:

Page 1
 Q1. Show that the set of 2x1 column vectors forms a vector space.

Q2. Is the set of vectors {(1,2), (2,4)} a vector space on its own?

2. Linear Independence

A set of vectors {v1, v2, ..., vn} is linearly independent if the only solution to:

c1*v1 + c2*v2 + ... + cn*vn = 0

is when all scalar coefficients c1, c2, ..., cn are zero.

Otherwise, the set is linearly dependent.

Examples:

* Independent: {(1, 0), (0, 1)}

* Dependent: {(1, 2), (2, 4)} (2nd is multiple of 1st)

Practice:

Q1. Are the vectors (1,0,1), (0,1,2), and (1,1,3) linearly independent?

Q2. Determine if {1, x, x^2} are linearly independent in P_2.

3. Basis and Dimension

A basis of a vector space is a linearly independent set of vectors that spans the entire space.

The number of vectors in a basis is called the dimension of the space.

Examples:

* Standard basis of R^3: {(1,0,0), (0,1,0), (0,0,1)}

* Basis of P_2: {1, x, x^2} -> Dimension = 3

Practice:

Q1. Find a basis for the space of 2x2 symmetric matrices.

Page 2
 Q2. Determine the dimension of the null space of the matrix A = [[1, 2], [2, 4]].

4. Inner Product

The inner product is a generalization of the dot product. It allows measuring angles and lengths.

For vectors u and v in R^n: <u, v> = u1*v1 + u2*v2 + ... + un*vn

For functions f and g: <f, g> = integral from a to b of f(x)g(x) dx

Properties:

* Symmetry: <u, v> = <v, u>

* Linearity: <au + bv, w> = a<u, w> + b<v, w>

* Positive-definite: <v, v> >= 0

Practice:

Q1. Compute the dot product of u = (1, 2, 3) and v = (4, -1, 2).

Q2. Determine if two vectors are orthogonal: u = (1, 2), v = (2, -1).

5. Linear Transformations and Matrix Representation

A linear transformation T: V -> W preserves vector addition and scalar multiplication.

Every linear transformation from R^n to R^m can be represented as matrix multiplication.

Example:

T(x, y) = (2x + y, x - y)

Matrix A = [[2, 1], [1, -1]]

T([x, y]) = A * [x, y]^T

Practice:

Q1. Define a transformation T: R^2 -> R^2 by T(x, y) = (x + y, x - y). Find its matrix.

Page 3
 Q2. Is T(x, y, z) = (x + 2y, y - z) linear? Justify.

6. Eigenvalues and Eigenvectors

For a square matrix A, an eigenvector v satisfies A*v = lambda*v, where lambda is the eigenvalue.

To find lambda, solve det(A - lambda*I) = 0.

Then solve (A - lambda*I)v = 0 for each lambda to find corresponding eigenvectors.

Example:

A = [[2, 1], [1, 2]]

Characteristic equation: (2 - lambda)^2 - 1 = 0 -> lambda = 1, 3

Practice:

Q1. Find eigenvalues and eigenvectors of A = [[4, 1], [2, 3]].

Q2. For A = [[1, 0], [0, 2]], what are the eigenvalues and eigenvectors?

7. Determinants

The determinant is a scalar value that can be computed from the elements of a square matrix and

encapsulates important properties of the matrix, such as invertibility.

Key Properties:

* det(AB) = det(A) * det(B)

* A matrix is invertible iff det(A) != 0

* For 2x2 matrix A = [[a, b], [c, d]]: det(A) = ad - bc

Example:

A = [[2, 3], [1, 4]]

det(A) = (2)(4) - (3)(1) = 8 - 3 = 5

Page 4
Practice:

Q1. Compute the determinant of A = [[1, 2], [3, 4]].

Solution: 1*4 - 2*3 = 4 - 6 = -2

8. Orthogonality and Gram-Schmidt Process

Two vectors are orthogonal if their inner product is zero.

The Gram-Schmidt process takes a linearly independent set and generates an orthonormal basis.

Steps of Gram-Schmidt for vectors v1, v2:

u1 = v1

u2 = v2 - proj_u1(v2) where proj_u1(v2) = (<v2,u1>/<u1,u1>)*u1

Normalize u1, u2 to get orthonormal set

Example:

v1 = (1, 1), v2 = (1, 0)

u1 = (1, 1), proj_u1(v2) = 0.5*(1,1) = (0.5,0.5)

u2 = (0.5, -0.5)

Practice:

Q1. Apply Gram-Schmidt to {(1,0,0), (1,1,0)}.

Solution: u1 = (1,0,0), u2 = (0,1,0)

9. Matrix Inverses

A square matrix A is invertible if there exists a matrix A^-1 such that AA^-1 = I.

For 2x2 matrix A = [[a, b], [c, d]]:

A^-1 = (1/det(A)) * [[d, -b], [-c, a]]

if det(A) != 0

Page 5
Example:

A = [[2, 1], [5, 3]]

det = 2*3 - 1*5 = 6 - 5 = 1

A^-1 = [[3, -1], [-5, 2]]

Practice:

Q1. Find the inverse of A = [[4, 7], [2, 6]].

10. Rank and Nullity

The rank of a matrix is the dimension of its row space (or column space).

The nullity is the dimension of the null space (solutions to Ax = 0).

Rank-Nullity Theorem:

rank(A) + nullity(A) = number of columns in A

Example:

A = [[1, 2], [2, 4]] -> rank = 1 (second row is multiple of first)

nullity = 2 - 1 = 1

Practice:

Q1. What is the rank and nullity of A = [[1, 2, 3], [2, 4, 6]]?

11. Systems of Linear Equations (Row Reduction)

We can solve linear systems using Gaussian elimination (row reduction to echelon form).

Steps:

* Use row operations to get leading 1s (pivots)

* Eliminate below and above pivot rows

* Back substitute to solve

Page 6
Example:

Solve: x + y = 2, 2x + 3y = 5

Augmented matrix: [[1,1,2],[2,3,5]] -> [[1,1,2],[0,1,1]] -> x = 1, y = 1

Practice:

Q1. Solve: x + 2y + z = 3, 2x + 5y + z = 8, x + y + 2z = 4.

12. Diagonalization

A matrix A is diagonalizable if there exists a matrix P such that P^-1*A*P = D, where D is diagonal.

This is possible if A has n linearly independent eigenvectors.

Steps:

* Find eigenvalues and eigenvectors

* Form P from eigenvectors, D from eigenvalues

Example:

A = [[4, 1], [0, 2]] -> eigenvalues 4 and 2 -> diagonalizable

Practice:

Q1. Is A = [[2, 1], [1, 2]] diagonalizable?

13. Applications of Linear Algebra

Linear algebra is fundamental to many fields:

* Computer Graphics: Transformations using matrices (rotation, scaling)

* Data Science: PCA for dimensionality reduction (uses eigenvectors)

* Engineering: Solving systems of equations in circuits, mechanics

Page 7
 * Machine Learning: Linear regression, SVD, matrix factorization

Example:

PCA: Principal Component Analysis uses eigenvectors of the covariance matrix to find principal

axes of data.

Practice:

Q1. Explain how eigenvectors are used in Principal Component Analysis (PCA).

Q2. What transformation matrix rotates points 90 degrees counterclockwise in 2D?

Page 8