Machine Learning Diploma

DSPrep1: Linear Algebra

Agenda
➔ Introduction
➔ Linear equation
➔ Vectors
➔ Matrices
➔ Linear algebra applications in machine learning.

2
 1. Introduction to Linear Algebra

3
Introduction:
➔ Linear algebra is the mathematics of data. Modern statistics is described
using the notation of linear algebra and modern statistical methods
harness the tools of linear algebra.

➔ Linear algebra is a large field of study that has tendrils into engineering,
physics and quantum physics. Only a specific subset of linear algebra is
required, though you can always go deeper once you have the basics.

4
Linear Equation:
➔ A linear equation is just a series of terms and mathematical operations
where some terms are unknown; for example:

➔ A mathematical statement that has an equal to the "=" symbol is called

an equation. Linear equations are equations of degree 1, meaning the
highest power is 1.
X square is not linear.

This equation describes a line on a two-dimensional graph. The line

comes from plugging in different values into the unknown x to find out
what the equation or model does to the value of y.
5
Linear Equation:
➔ Example:

6
Quiz:
➔ Which is linear and which is nonlinear?

7
Quiz(Solution):
➔ Which is linear and which is nonlinear?

8
 2. Vectors

9
 Vectors:
➔ Vectors are objects that can be added together to form new vectors and
that can be multiplied by scalars (i.e., numbers), also to form new
vectors.
➔ If you have the heights, weights, and ages of a large number of people,
you can treat your data as three-dimensional vectors [height, weight,
age]
➔ The simplest from-scratch approach is to represent vectors as lists of
numbers. A list of three numbers corresponds to a vector in three
dimensional space, and vice versa.

10
 Vectors addition/subtraction:
➔ We’ll also want to perform arithmetic on vectors. Because Python lists
aren’t vectors (and hence provide no facilities for vector arithmetic)
➔ To begin with, we’ll frequently need to add two vectors. Vectors add
componentwise. This means that if two vectors v and w are the same
length, their sum is just the vector whose first element is v[0] + w[0],
whose second element is v*1+ + w*1+, and so on. (If they’re not the same
length, then we’re not allowed to add them.)

➔ For example, [1, 2] + [2, 1] = [1 + 2, 2 + 1] = [3, 3]

11
 Quiz:
➔ Write a function that adds two vectors together

12
 Quiz (Solution):
➔ Write a function that adds two vectors together

13
 Quiz (Solution 2):
➔ Write a function that adds two vectors together
Notes:
zip() is a built-in function that pairs together items from the passed
sequences
assert() is a built-in function that raises error if the condition is false

14
 Quiz (Solution 2):
➔ Write a function that adds two vectors together
Notes:
We here checked first if the lengths are equal because we can’t add
two vectors with different sizes. (Data validation step)
Instead of printing the result of adding those two vectors we used
assert to test our function, to test it adds correctly.

15
 Quiz (Solution 2):
➔ Write a function that adds two vectors together
Notes:
We also have provided a sentence after the assertion condition to
explain what happened. For example “vectors must be the same length”

16
 Vectors Multiplication by a scalar:
➔ A Scaler is simply any number. Multiplying a vector by a scalar multiplies
all the elements by that scaler.

➔ For example:
2 * [2,4,6] = [4,8,12]

17
 Quiz:
➔ Write a function that multiplies a scalar and a vector.
○ Inputs: scaler_num , vector
○ Outputs: vector

18
 Quiz (Solution):
➔ Write a function that multiplies a scalar and a vector.
○ Inputs: scaler_num , vector
○ Outputs: vector

19
 Quiz (Solution2):
➔ Write a function that multiplies a scalar and a vector.
Notes:
We used here list Comprehension. It just makes it shorter
syntax instead of using for loops on lists. It’s just more pythonic but it’s the
same
Syntax: newlist = [expression for item in iterable if
condition == True]

20
 Vectors Multiplication Dot product:
➔ The dot product of two vectors is the sum of their componentwise
products
➔ For example:
[1,2,3] * [3,4,5] = 1*3 + 2*4 + 3*5 = 27

21
 Quiz:
➔ Write a function that multiplies two vectors.

22
 Quiz (Solution):
➔ Write a function that multiplies two vectors.

23
 3. Matrices

24
 Matrices:
➔ A matrix is a two-dimensional collection of numbers. We will represent
matrices as lists of lists, with each inner list having the same size and
representing a row of the matrix.
➔ If A is a matrix, then A[i][j] is the element in the ith row and the jth
column.

25
 Matrices:
➔ Given this list-of-lists representation, the matrix A has len(A) rows and
len(A[0]) columns. which we consider its shape.

➔ A has a shape of (2, 3)

➔ B has a shape of (3,2)

26
 Quiz:
➔ Write a function that takes a matrix and gives you its shape.

27
 Quiz (Solution):
➔ Write a function that takes a matrix and gives you its shape.

28
 Matrices:
➔ If a matrix has n rows and k columns, we will refer to it as an n × k
matrix. We can (and sometimes will) think of each row of an n × k matrix
as a vector of length k, and each column as a vector of length n:.

➔ A is 2*3 matrix
➔ B is 3*2 matrix

29
 Matrices:
➔ We can use a matrix to represent a dataset consisting of multiple
vectors, simply by considering each vector as a row of the matrix. For
example, if you had the heights, weights, and ages of 1,000 people, you
could put them in a 1,000 × 3 matrix

30
 Matrices Addition/Subtraction:
➔ Two matrices with the same dimensions can be added together to create
a new third matrix.

➔ The scalar elements in the resulting matrix are calculated as the addition
of the elements in each of the matrices being added.

31
 Quiz:
➔ Write a function that adds two matrices, using this notation:

32
 Quiz (Solution):
➔ Write a function that adds two matrices

33
 Matrices Multiplication:
➔ Two matrices with the same size can be multiplied together, and this is
often called element-wise matrix multiplication or the Hadamard
product.

➔ As with element-wise subtraction and addition, element-wise

multiplication involves the multiplication of elements from each parent
matrix to calculate the values in the new matrix.

34
 Quiz:
➔ Write a function that multiply two matrices element-wise, using this
notation:

35
 Quiz (Solution):
➔ Write a function that multiply two matrices element-wise:

36
 Matrices Multiplication Dot product:
➔ Matrix multiplication, also called the matrix dot product is more
complicated than the previous operations and involves a rule as not all
matrices can be multiplied together.

➔ The number of columns (n) in the first matrix (A) must equal the number
of rows (m) in the second matrix (B).
➔ This rule applies for a chain of matrix multiplications where the number
of columns in one matrix in the chain must match the number of rows in
the following matrix in the chain.

37
 Matrices Multiplication Dot product:

38
 Matrix-Vector Multiplication:
➔ A matrix and a vector can be multiplied together as long as the rule of
matrix multiplication is observed. Specifically, that the number of
columns in the matrix must equal the number of items in the vector.

39
 Quiz:
➔ Write a function that multiply a vector with a matrix. Using this notation:

40
 Quiz:
➔ Write a function that multiply a vector with a matrix.

41
 Matrix-Scalar Multiplication:
➔ A matrix can be multiplied by a scalar. This can be represented using the
dot notation between the matrix and the scalar.

➔ The result is a matrix with the same size as the parent matrix where each
element of the matrix is multiplied by the scalar value.

42
 Quiz:
➔ Write a function that multiplies a matrix by a scalar, using this notation:

43
 Quiz (Solution):
➔ Write a function that multiplies a matrix by a scalar:

44
 4. Types of Matrices

45
 Types of Matrices:
➔ Square Matrix
➔ Symmetric Matrix
➔ Triangular Matrix
➔ Diagonal Matrix
➔ Identity Matrix

46
 Square Matrix:
➔ A square matrix is a matrix where the number of rows (n) is equivalent
to the number of columns (m).
➔ The square matrix is contrasted with the rectangular matrix where the
number of rows and columns are not equal.
➔ Square matrices are readily added and multiplied together and are the
basis of many simple linear transformations, such as rotations (as in the
rotations of images)

47
 Symmetric Matrix:
➔ A symmetric matrix is a type of square matrix where the top-right
triangle is the same as the bottom-left triangle.
➔ To be symmetric, the axis of symmetry is always the main diagonal of the
matrix, from the top left to the bottom right. Below is an example of a 5
× 5 symmetric matrix.

48
 Triangular Matrix
➔ A triangular matrix is a type of square matrix that has all values in the
upper-right or lower-left of the matrix with the remaining elements filled
with zero values.
➔ Below is an example of a 3 × 3 upper triangular matrix.

➔ Below is an example of a 3 × 3 lower triangular matrix.

49
 Diagonal Matrix:
➔ A diagonal matrix is one where values outside of the main diagonal have
a zero value, where the main diagonal is taken from the top left of the
matrix to the bottom right.
➔ Diagonal matrices consist mostly of zeros and have non-zero entries only
along the main diagonal.
➔ A diagonal matrix does not have to be square. In the case of a
rectangular matrix, the diagonal would cover the dimension with the
smallest length; for example:

50
 Identity Matrix
➔ An identity matrix is a square matrix that does not change a vector when
multiplied. The values of an identity matrix are known. All of the scalar
values along the main diagonal (top-left to bottom-right) have the value
one, while all other values are zero.
➔ An identity matrix is a matrix that does not change any vector when we
multiply that vector by that matrix.

51
 5. Linear Algebra Examples

52
 Linear Algebra in ML Examples:
1. Dataset and Data Files
2. Images and Photographs
3. One Hot Encoding
4. Linear Regression
5. Regularization
6. Principal Component Analysis
7. Singular-Value Decomposition
8. Latent Semantic Analysis
9. Recommender Systems

53
 1- Dataset and Data Files
This is the iris flower dataset. the table
consists of a set of numbers where:
each row represents an observation
each column represents a feature of the
observation.

- This data is in fact a matrix (Data structure in linear algebra).

- when you split the data into inputs and outputs to fit a supervised
machine learning model, such as the measurements and the flower
species, you have a matrix (X) and a vector (y).
- The vector is another key data structure in linear algebra
54
 2- Images and Photographs
➔ Perhaps you are more used to working with images
or photographs in computer vision applications.
➔ Each image that you work with is itself a table
structure with a width and height and one pixel
value in each cell for black and white images or 3
pixel values in each cell for a color image. A photo
is yet another example of a matrix from linear
algebra.
➔ Operations on the image, such as cropping, scaling, shearing and so
on are all described using the notation and operations of linear
algebra
55
 2- Images and Photographs

56 Image reference link

 3- One Hot Encoding
Perhaps the class labels for classification
problems, or perhaps categorical (data type:
string) input variables. It is common to encode
categorical variables to make their easier to
work with and learn by some techniques.

- A one hot encoding is where a table is created to represent the variable

with one column for each category and a row for each example in the
dataset.
- This is an example of a sparse representation, a whole sub-field of linear
algebra.
57
 4- Linear Regression
Linear regression is an old method from
statistics for describing the relationships
between variables. It is often used in
machine learning for predicting numerical
values.

- The common way of summarizing the linear regression

equation uses linear algebra notation Where y is the
output variable A is the dataset and b are the model
coefficients

58
 5- Regularization
In applied machine learning, we often seek
the simplest possible models that achieve the
best skill on our problem. Simpler models are
often better at generalizing from specific to
unseen data.

- Simpler models are often characterized by models that have smaller

coefficient values.
- Common implementations include the L 2 and L 1 forms of regularization.
Both of these forms of regularization are in fact a measure of the
magnitude or length of the coefficients as a vector and are methods lifted
directly from linear algebra called the vector norm.
59
 6- Principal Component Analysis
Often a dataset has many columns, perhaps
thousands or more. Modeling data with many
features is challenging, and models built from data
that include irrelevant features are often less skillful
than models trained from the most relevant data.
Methods for automatically reducing the number
of columns of a dataset are called dimensionality
reduction.
- The core of the PCA method is a matrix factorization method from
linear algebra.

60
 7- Singular-Value Decomposition
Another popular dimensionality reduction method is the singular-value
decomposition method or SVD for short.
It is a matrix factorization method from the field of linear algebra.
It has wide use in linear algebra and can be used directly in applications
such as feature selection, visualization, noise reduction and more.

61
 8- Latent Semantic Analysis
In the sub-field of machine learning for working with text data called
natural language processing, it is common to represent documents as large
matrices of word occurrences.
For example, the columns of the matrix may be the known words in the
vocabulary and rows may be sentences, paragraphs, pages or documents of
text with cells in the matrix marked as the count or frequency of the number
of times the word occurred.

62
 8- Latent Semantic Analysis
Matrix factorization methods such as the SVD can be applied to this
sparse matrix which has the effect of distilling the representation down to its
most relevant essence.

Documents processed in this way are much easier to compare, query and
use as the basis for a supervised machine learning model.

This form of data preparation is called Latent Semantic Analysis or LSA for
short, and is also known by the name Latent Semantic Indexing or LSI.

63
 9- Recommender Systems
Predictive modeling problems that involve the recommendation of
products are called recommender systems, a sub-field of machine learning.
The development of recommender systems is primarily concerned with
linear algebra methods. Matrix factorization methods like the singular-value
decomposition are used widely in recommender systems to distill item and
user data to their essence for querying and searching and comparison.

64
 Any Questions?

65
THANK YOU!