Practical 07

Aim: Write a program to implement algebra of vectors like addition, subtraction, scalar multiplication, dot
product, cross product, scalar triple product.

Theory:

Vectors are a foundational element of linear algebra. Vectors are used throughout the field of machine
learning in the description of algorithms and processes such as the target variable (y) when training an
algorithm.

What a vector is and how to define one in Python with NumPy.

How to perform vector arithmetic such as addition, subtraction, multiplication and division. How to
perform additional operations such as dot product and multiplication with a scalar

What is a Vector?
A vector is a tuple of one or more values called scalars.

Vectors are built from components, which are ordinary numbers. You can think of a vector as
a list of numbers, and vector algebra as operations performed on the numbers in the list.

Vectors are often represented using a lowercase character such as “v”; for example:
v = (v1, v2, v3)

Vectors are also shown using a vertical representation or a column; for example:

1 v1
2v = ( v2 )
3 v3
t is common to represent the target variable as a vector with the lowercase “y” when
describing the training of a machine learning algorithm.

It is common to introduce vectors using a geometric analogy, where a vector represents a

point or coordinate in an n-dimensional space, where n is the number of dimensions, such as
2.

The vector can also be thought of as a line from the origin of the vector space with a direction
and a magnitude.
These analogies are good as a starting point, but should not be held too tightly as we often
consider very high dimensional vectors in machine learning. I find the vector-as-coordinate
the most compelling analogy in machine learning.

Now that we know what a vector is, let’s look at how to define a vector in Python.

Defining a Vector
We can represent a vector in Python as a NumPy array.
A NumPy array can be created from a list of numbers. For example, below we define a vector
with the length of 3 and the integer values 1, 2 and 3.

1# create a vector
2from numpy import array
3v = array([1, 2, 3])
4print(v)
The example defines a vector with 3 elements.

Running the example prints the defined vector.

1[1 2 3]

Vector Arithmetic
In this section will demonstrate simple vector-vector arithmetic, where all operations are
performed element-wise between two vectors of equal length to result in a new vector with
the same length

Vector Addition
Two vectors of equal length can be added together to create a new third vector.

1c = a + b

The new vector has the same length as the other two vectors. Each element of the new vector
is calculated as the addition of the elements of the other vectors at the same index; for
example:

1a + b = (a1 + b1, a2 + b2, a3 + b3)

Or, put another way:

1c[0] = a[0] + b[0]

2c[1] = a[1] + b[1]

3c[2] = a[2] + b[2]

We can add vectors directly in Python by adding NumPy arrays.

1# add vectors

2from numpy import array

3a = array([1, 2, 3])

4print(a)

5b = array([1, 2, 3])

6print(b)

7c = a + b

8print(c)

The example defines two vectors with three elements each, then adds them together.

Running the example first prints the two parent vectors then prints a new vector that is the
addition of the two vectors.

1[1 2 3]

3[1 2 3]

5[2 4 6]

Vector Subtraction
One vector can be subtracted from another vector of equal length to create a new third vector.

1c = a - b

As with addition, the new vector has the same length as the parent vectors and each element
of the new vector is calculated as the subtraction of the elements at the same indices.

1a - b = (a1 - b1, a2 - b2, a3 - b3)

Or, put another way:

1c[0] = a[0] - b[0]

2c[1] = a[1] - b[1]

3c[2] = a[2] - b[2]

The NumPy arrays can be directly subtracted in Python.

1# subtract vectors

2from numpy import array

3a = array([1, 2, 3])

4print(a)

5b = array([0.5, 0.5, 0.5])

6print(b)

7c = a - b

8print(c)

The example defines two vectors with three elements each, then subtracts the first from the
second.

Running the example first prints the two parent vectors then prints the new vector that is the
first minus the second.

1[1 2 3]

3[ 0.5 0.5 0.5]

5[ 0.5 1.5 2.5]

Vector Multiplication
Two vectors of equal length can be multiplied together.

1c = a * b

As with addition and subtraction, this operation is performed element-wise to result in a new
vector of the same length.

1a * b = (a1 * b1, a2 * b2, a3 * b3)

or

1ab = (a1b1, a2b2, a3b3)

Or, put another way:

1c[0] = a[0] * b[0]

2c[1] = a[1] * b[1]

3c[2] = a[2] * b[2]

We can perform this operation directly in NumPy.

1# multiply vectors

2from numpy import array

3a = array([1, 2, 3])

4print(a)

5b = array([1, 2, 3])

6print(b)

7c = a * b

8print(c)

The example defines two vectors with three elements each, then multiplies the vectors
together.

Running the example first prints the two parent vectors, then the new vector is printed.

1[1 2 3]

3[1 2 3]

5[1 4 9]

Vector Division
Two vectors of equal length can be divided.

1c = a / b

As with other arithmetic operations, this operation is performed element-wise to result in a

new vector of the same length.

1a / b = (a1 / b1, a2 / b2, a3 / b3)

or

1a / b = (a1b1, a2b2, a3b3)

Or, put another way:

1c[0] = a[0] / b[0]

2c[1] = a[1] / b[1]

3c[2] = a[2] / b[2]

We can perform this operation directly in NumPy.

1# divide vectors

2from numpy import array

3a = array([1, 2, 3])

4print(a)

5b = array([1, 2, 3])

6print(b)

7c = a / b

8print(c)

The example defines two vectors with three elements each, then divides the first by the
second.

Running the example first prints the two parent vectors, followed by the result of the vector
division.

1[1 2 3]

3[1 2 3]

5[ 1. 1. 1.]

Vector Dot Product

We can calculate the sum of the multiplied elements of two vectors of the same length to give
a scalar.

This is called the dot product, named because of the dot operator used when describing the
operation.

The dot product is the key tool for calculating vector projections, vector decompositions, and
determining orthogonality. The name dot product comes from the symbol used to denote it.

— Page 110, No Bullshit Guide To Linear Algebra, 2017

1c = a . b

The operation can be used in machine learning to calculate the weighted sum of a vector.
The dot product is calculated as follows:

1a . b = (a1 * b1 + a2 * b2 + a3 * b3)

or

1a . b = (a1b1 + a2b2 + a3b3)

We can calculate the dot product between two vectors in Python using the dot() function on
a NumPy array. It can also be calculated using the newer @ operator, since Python version
3.5. The example below demonstrates both methods.

1 # dot product vectors

2 from numpy import array

3 a = array([1, 2, 3])

4 print(a)

5 b = array([1, 2, 3])

6 print(b)

7 c = a.dot(b)

8 print(c)

9 d=a@b

10print(d)

The example defines two vectors with three elements each, then calculates the dot product.

Running the example first prints the two parent vectors, then the scalar dot product.

1[1 2 3]

3[1 2 3]

514

714

Vector-Scalar Multiplication
A vector can be multiplied by a scalar, in effect scaling the magnitude of the vector.

To keep notation simple, we will use lowercase “s” to represent the scalar value.
1c = s * v

or

1c = sv

The multiplication is performed on each element of the vector to result in a new scaled vector
of the same length.

1s * v = (s * v1, s * v2, s * v3)

Or, put another way:

1c[0] = a[0] * s

2c[1] = a[1] * s

3c[2] = a[2] * s

We can perform this operation directly with the NumPy array.

1# vector-scalar multiplication

2from numpy import array

3a = array([1, 2, 3])

4print(a)

5s = 0.5

6print(s)

7c = s * a

8print(c)

The example first defines the vector and the scalar then multiplies the vector by the scalar.

Running the example first prints the parent vector, then scalar, and then the result of
multiplying the two together.

1[1 2 3]

30.5

5[ 0.5 1. 1.5]

Similarly, vector-scalar addition, subtraction, and division can be performed in the same way.