WEEK 1:

What is Machine Learning?

Two definitions of Machine Learning are offered. Arthur Samuel described it as: "the field of
study that gives computers the ability to learn without being explicitly programmed." This is an
older, informal definition.

Tom Mitchell provides a more modern definition: "A computer program is said to learn from
experience E with respect to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with experience E."

Example: playing checkers.

E = the experience of playing many games of checkers

T = the task of playing checkers.

P = the probability that the program will win the next game.

In general, any machine learning problem can be assigned to one of two broad classifications:

Supervised learning and Unsupervised learning.

Supervised Learning
In supervised learning, we are given a data set and already know what our correct output should
look like, having the idea that there is a relationship between the input and the output.

Supervised learning problems are categorized into "regression" and "classification" problems. In
a regression problem, we are trying to predict results within a continuous output, meaning that
we are trying to map input variables to some continuous function. In a classification problem, we
are instead trying to predict results in a discrete output. In other words, we are trying to map input
variables into discrete categories.

Example 1:

Given data about the size of houses on the real estate market, try to predict their price. Price as
a function of size is a continuous output, so this is a regression problem.

We could turn this example into a classification problem by instead making our output about
whether the house "sells for more or less than the asking price." Here we are classifying the
houses based on price into two discrete categories.
Example 2:

(a) Regression - Given a picture of a person, we have to predict their age on the basis of the
given picture

(b) Classification - Given a patient with a tumor, we have to predict whether the tumor is
malignant or benign.

Unsupervised Learning
Unsupervised learning allows us to approach problems with little or no idea what our results
should look like. We can derive structure from data where we don't necessarily know the effect of
the variables.

We can derive this structure by clustering the data based on relationships among the variables in
the data.

With unsupervised learning there is no feedback based on the prediction results.

Example:

Clustering: Take a collection of 1,000,000 different genes, and find a way to automatically group
these genes into groups that are somehow similar or related by different variables, such as
lifespan, location, roles, and so on.

Non-clustering: The "Cocktail Party Algorithm", allows you to find structure in a chaotic
environment. (i.e. identifying individual voices and music from a mesh of sounds at a cocktail
party).

Model Representation
To establish notation for future use, we’ll use x^{(i)}x(i) to denote the “input” variables (living
area in this example), also called input features, and y^{(i)}y(i) to denote the “output” or target
variable that we are trying to predict (price). A pair (x^{(i)} , y^{(i)} )(x(i),y(i)) is called a
training example, and the dataset that we’ll be using to learn—a list of m training
examples (x(i),y(i));i=1,...,m—is called a training set. Note that the superscript “(i)” in the
notation is simply an index into the training set, and has nothing to do with exponentiation. We
will also use X to denote the space of input values, and Y to denote the space of output values.
In this example, X = Y = ℝ.
To describe the supervised learning problem slightly more formally, our goal is, given a training
set, to learn a function h : X → Y so that h(x) is a “good” predictor for the corresponding value of
y. For historical reasons, this function h is called a hypothesis. Seen pictorially, the process is
therefore like this:
When the target variable that we’re trying to predict is continuous, such as in our housing
example, we call the learning problem a regression problem. When y can take on only a small
number of discrete values (such as if, given the living area, we wanted to predict if a dwelling is a
house or an apartment, say), we call it a classification problem.

Cost Function
We can measure the accuracy of our hypothesis function by using a cost function. This takes
an average difference (actually a fancier version of an average) of all the results of the
hypothesis with inputs from x's and the actual output y's.

J(θ0,θ1)=12m∑i=1m(y^i−yi)2=12m∑i=1m(hθ(xi)−yi)2
To break it apart, it is \frac{1}{2}21 x¯ where x¯ is the mean of the squares of h_\theta
(x_{i}) - y_{i}hθ(xi)−yi , or the difference between the predicted value and the actual value.
This function is otherwise called the "Squared error function", or "Mean squared error". The
mean is halved \left(\frac{1}{2}\right)(21) as a convenience for the computation of the
gradient descent, as the derivative term of the square function will cancel out the \frac{1}
{2}21 term. The following image summarizes what the cost function does:
Cost Function - Intuition I
If we try to think of it in visual terms, our training data set is scattered on the x-y plane. We are
trying to make a straight line (defined by h_\theta(x)hθ(x)) which passes through these
scattered data points.

Our objective is to get the best possible line. The best possible line will be such so that the
average squared vertical distances of the scattered points from the line will be the least. Ideally,
the line should pass through all the points of our training data set. In such a case, the value
of J(\theta_0, \theta_1)J(θ0,θ1) will be 0. The following example shows the ideal situation where
we have a cost function of 0.

When \theta_1 = 1θ1=1, we get a slope of 1 which goes through every single data point in our
model. Conversely, when \theta_1 = 0.5θ1=0.5, we see the vertical distance from our fit to the
data points increase.
This increases our cost function to 0.58. Plotting several other points yields to the following
graph:

Thus as a goal, we should try to minimize the cost function. In this case, \theta_1 = 1θ1=1 is our
global minimum.

Cost Function - Intuition II

A contour plot is a graph that contains many contour lines. A contour line of a two variable
function has a constant value at all points of the same line. An example of such a graph is the
one to the right below.
Taking any color and going along the 'circle', one would expect to get the same value of the cost
function. For example, the three green points found on the green line above have the same value
for J(\theta_0,\theta_1)J(θ0,θ1) and as a result, they are found along the same line. The circled
x displays the value of the cost function for the graph on the left when \theta_0θ0 = 800
and \theta_1θ1= -0.15. Taking another h(x) and plotting its contour plot, one gets the following
graphs:

When \theta_0θ0 = 360 and \theta_1θ1 = 0, the value of J(\theta_0,\theta_1)J(θ0,θ1) in the

contour plot gets closer to the center thus reducing the cost function error. Now giving our
hypothesis function a slightly positive slope results in a better fit of the data.
The graph above minimizes the cost function as much as possible and consequently, the result
of \theta_1θ1 and \theta_0θ0 tend to be around 0.12 and 250 respectively. Plotting those values
on our graph to the right seems to put our point in the center of the inner most 'circle'.

Gradient Descent
So we have our hypothesis function and we have a way of measuring how well it fits into the
data. Now we need to estimate the parameters in the hypothesis function. That's where gradient
descent comes in.

Imagine that we graph our hypothesis function based on its

fields \theta_0θ0 and \theta_1θ1 (actually we are graphing the cost function as a function of the
parameter estimates). We are not graphing x and y itself, but the parameter range of our
hypothesis function and the cost resulting from selecting a particular set of parameters.

We put \theta_0θ0 on the x axis and \theta_1θ1 on the y axis, with the cost function on the
vertical z axis. The points on our graph will be the result of the cost function using our hypothesis
with those specific theta parameters. The graph below depicts such a setup.
We will know that we have succeeded when our cost function is at the very bottom of the pits in
our graph, i.e. when its value is the minimum. The red arrows show the minimum points in the
graph.

The way we do this is by taking the derivative (the tangential line to a function) of our cost
function. The slope of the tangent is the derivative at that point and it will give us a direction to
move towards. We make steps down the cost function in the direction with the steepest descent.
The size of each step is determined by the parameter α, which is called the learning rate.

For example, the distance between each 'star' in the graph above represents a step determined
by our parameter α. A smaller α would result in a smaller step and a larger α results in a larger
step. The direction in which the step is taken is determined by the partial derivative
of J(\theta_0,\theta_1)J(θ0,θ1). Depending on where one starts on the graph, one could end up
at different points. The image above shows us two different starting points that end up in two
different places.

The gradient descent algorithm is:

repeat until convergence:

θj:=θj−α∂∂θjJ(θ0,θ1)
where

j=0,1 represents the feature index number.

At each iteration j, one should simultaneously update the parameters \theta_1,

\theta_2,...,\theta_nθ1,θ2,...,θn. Updating a specific parameter prior to calculating another one
on the j^{(th)}j(th) iteration would yield to a wrong implementation.
Gradient Descent Intuition
In this video we explored the scenario where we used one parameter \theta_1θ1 and plotted its
cost function to implement a gradient descent. Our formula for a single parameter was :

Repeat until convergence:

\theta_1:=\theta_1-\alpha \frac{d}{d\theta_1} J(\theta_1)θ1:=θ1−αdθ1dJ(θ1)

Regardless of the slope's sign for \frac{d}{d\theta_1} J(\theta_1)dθ1dJ(θ1

), \theta_1θ1 eventually converges to its minimum value. The following graph shows that when
the slope is negative, the value of \theta_1θ1 increases and when it is positive, the value
of \theta_1θ1 decreases.

On a side note, we should adjust our parameter \alphaα to ensure that the gradient descent
algorithm converges in a reasonable time. Failure to converge or too much time to obtain the
minimum value imply that our step size is wrong.
How does gradient descent converge with a fixed step size \alphaα?

The intuition behind the convergence is that \frac{d}{d\theta_1} J(\theta_1)dθ1dJ(θ1

) approaches 0 as we approach the bottom of our convex function. At the minimum, the
derivative will always be 0 and thus we get:

\theta_1:=\theta_1-\alpha * 0θ1:=θ1−α∗0
Gradient Descent For Linear Regression
Note: [At 6:15 "h(x) = -900 - 0.1x" should be "h(x) = 900 - 0.1x"]

When specifically applied to the case of linear regression, a new form of the gradient descent
equation can be derived. We can substitute our actual cost function and our actual hypothesis
function and modify the equation to :

repeat until convergence: {θ0:=θ1:=}θ0−α1m∑i=1m(hθ(xi)−yi)θ1−α1m∑i=1m((hθ(xi)−yi)xi)

where m is the size of the training set, \theta_0θ0 a constant that will be changing
simultaneously with \theta_1θ1 and x_{i}, y_{i}xi,yiare values of the given training set (data).

Note that we have separated out the two cases for \theta_jθj into separate equations
for \theta_0θ0 and \theta_1θ1; and that for \theta_1θ1 we are multiplying x_{i}xi at the end
due to the derivative. The following is a derivation of ∂∂θjJ(θ) for a single example :

The point of all this is that if we start with a guess for our hypothesis and then repeatedly apply
these gradient descent equations, our hypothesis will become more and more accurate.

So, this is simply gradient descent on the original cost function J. This method looks at every
example in the entire training set on every step, and is called batch gradient descent. Note that,
while gradient descent can be susceptible to local minima in general, the optimization problem
we have posed here for linear regression has only one global, and no other local, optima; thus
gradient descent always converges (assuming the learning rate α is not too large) to the global
minimum. Indeed, J is a convex quadratic function. Here is an example of gradient descent as it
is run to minimize a quadratic function.
The ellipses shown above are the contours of a quadratic function. Also shown is the trajectory
taken by gradient descent, which was initialized at (48,30). The x’s in the figure (joined by
straight lines) mark the successive values of θ that gradient descent went through as it
converged to its minimum.

Matrices and Vectors

Matrices are 2-dimensional arrays:

⎡⎣⎢⎢⎢adgjbehkcfil⎤⎦⎥⎥⎥

The above matrix has four rows and three columns, so it is a 4 x 3 matrix.

A vector is a matrix with one column and many rows:

⎡⎣⎢⎢wxyz⎤⎦⎥⎥

So vectors are a subset of matrices. The above vector is a 4 x 1 matrix.

Notation and terms:

 A_{ij}Aij refers to the element in the ith row and jth column of matrix A.
 A vector with 'n' rows is referred to as an 'n'-dimensional vector.

 v_ivi refers to the element in the ith row of the vector.

 In general, all our vectors and matrices will be 1-indexed. Note that for some
programming languages, the arrays are 0-indexed.
 Matrices are usually denoted by uppercase names while vectors are lowercase.
 "Scalar" means that an object is a single value, not a vector or matrix.
  R refers to the set of scalar real numbers.
 Rn refers to the set of n-dimensional vectors of real numbers.
Run the cell below to get familiar with the commands in Octave/Matlab. Feel free to create
matrices and vectors and try out different things.

10

11

12

13

14

15

16

17

18

% The ; denotes we are going back to a new row.

A = [1, 2, 3; 4, 5, 6; 7, 8, 9; 10, 11, 12]

% Initialize a vector

v = [1;2;3]

% Get the dimension of the matrix A where m = rows and n = columns

[m,n] = size(A)

% You could also store it this way

dim_A = size(A)

% Get the dimension of the vector v

dim_v = size(v)

% Now let's index into the 2nd row 3rd column of matrix A

A_23 = A(2,3)

RunReset

Addition and Scalar Multiplication

Addition and subtraction are element-wise, so you simply add or subtract each corresponding
element:

[acbd]+[wyxz]=[a+wc+yb+xd+z]

Subtracting Matrices:

[acbd]−[wyxz]=[a−wc−yb−xd−z]

To add or subtract two matrices, their dimensions must be the same.

In scalar multiplication, we simply multiply every element by the scalar value:

[acbd]∗x=[a∗xc∗xb∗xd∗x]

In scalar division, we simply divide every element by the scalar value:

 [acbd]/x=[a/xc/xb/xd/x]

Experiment below with the Octave/Matlab commands for matrix addition and scalar
multiplication. Feel free to try out different commands. Try to write out your answers for each
command before running the cell below.

10

11

12

13

14

15

16

17

18
 19

20

21

22

% Initialize matrix A and B

A = [1, 2, 4; 5, 3, 2]

B = [1, 3, 4; 1, 1, 1]

% Initialize constant s

s = 2

% See how element-wise addition works

add_AB = A + B

% See how element-wise subtraction works

sub_AB = A - B

% See how scalar multiplication works

mult_As = A * s

% Divide A by s

div_As = A / s

% What happens if we have a Matrix + scalar?

add_As = A + s

RunReset

Matrix-Vector Multiplication
We map the column of the vector onto each row of the matrix, multiplying each element and
summing the result.

⎡⎣acebdf⎤⎦∗[xy]=⎡⎣⎢a∗x+b∗yc∗x+d∗ye∗x+f∗y⎤⎦⎥
The result is a vector. The number of columns of the matrix must equal the number of rows of
the vector.

An m x n matrix multiplied by an n x 1 vector results in an m x 1 vector.

Below is an example of a matrix-vector multiplication. Make sure you understand how the
multiplication works. Feel free to try different matrix-vector multiplications.

10

% Initialize matrix A

A = [1, 2, 3; 4, 5, 6;7, 8, 9]

% Initialize vector v

v = [1; 1; 1]

% Multiply A * v

Av = A * v

RunReset
Matrix-Matrix Multiplication
We multiply two matrices by breaking it into several vector multiplications and concatenating the
result.

⎡⎣acebdf⎤⎦∗[wyxz]=⎡⎣⎢a∗w+b∗yc∗w+d∗ye∗w+f∗ya∗x+b∗zc∗x+d∗ze∗x+f∗z⎤⎦⎥

An m x n matrix multiplied by an n x o matrix results in an m x o matrix. In the above example,

a 3 x 2 matrix times a 2 x 2 matrix resulted in a 3 x 2 matrix.

To multiply two matrices, the number of columns of the first matrix must equal the number
of rows of the second matrix.

For example:

10

% Initialize a 3 by 2 matrix

A = [1, 2; 3, 4;5, 6]

% Initialize a 2 by 1 matrix
B = [1; 2]

% We expect a resulting matrix of (3 by 2)*(2 by 1) = (3 by 1)

mult_AB = A*B

% Make sure you understand why we got that result

RunReset

Matrix Multiplication Properties

 Matrices are not commutative: A∗B≠B∗A
 Matrices are associative: (A∗B)∗C=A∗(B∗C)
The identity matrix, when multiplied by any matrix of the same dimensions, results in the
original matrix. It's just like multiplying numbers by 1. The identity matrix simply has 1's on the
diagonal (upper left to lower right diagonal) and 0's elsewhere.

⎡⎣100010001⎤⎦

When multiplying the identity matrix after some matrix (A∗I), the square identity matrix's
dimension should match the other matrix's columns. When multiplying the identity matrix before
some other matrix (I∗A), the square identity matrix's dimension should match the other
matrix's rows.

8
 9

10

11

12

13

14

15

16

17

18

19

20

21

22

% Initialize random matrices A and B

A = [1,2;4,5]

B = [1,1;0,2]

% Initialize a 2 by 2 identity matrix

I = eye(2)

% The above notation is the same as I = [1,0;0,1]

% What happens when we multiply I*A ?

IA = I*A

% How about A*I ?

AI = A*I

% Compute A*B

AB = A*B
% Is it equal to B*A?

BA = B*A

% Note that IA = AI but AB != BA

RunReset

Inverse and Transpose

The inverse of a matrix A is denoted A^{-1}A−1. Multiplying by the inverse results in the
identity matrix.

A non square matrix does not have an inverse matrix. We can compute inverses of matrices in
octave with the pinv(A)pinv(A) function and in Matlab with the inv(A)inv(A) function.
Matrices that don't have an inverse are singular or degenerate.

The transposition of a matrix is like rotating the matrix 90° in clockwise direction and then
reversing it. We can compute transposition of matrices in matlab with the transpose(A) function
or A':

A=⎡⎣acebdf⎤⎦

AT=[abcdef]

In other words:

A_{ij} = A^T_{ji}Aij=AjiT

5
 6

10

11

12

13

% Initialize matrix A

A = [1,2,0;0,5,6;7,0,9]

% Transpose A

A_trans = A'

% Take the inverse of A

A_inv = inv(A)

% What is A^(-1)*A?

A_invA = inv(A)*A;RunReset

WEEK-2:
Multiple Features
Note: [7:25 - \theta^TθT is a 1 by (n+1) matrix and not an (n+1) by 1 matrix]

Linear regression with multiple variables is also known as "multivariate linear regression".

We now introduce notation for equations where we can have any number of input variables.

x(i)jx(i)mn=value of feature j in the ith training example=the input (features) of the ith training examp

training examples=the number of features

The multivariable form of the hypothesis function accommodating these multiple features is as
follows:
hθ(x)=θ0+θ1x1+θ2x2+θ3x3+⋯+θnxn
In order to develop intuition about this function, we can think about \theta_0θ0 as the basic price
of a house, \theta_1θ1 as the price per square meter, \theta_2θ2 as the price per floor,
etc. x_1x1 will be the number of square meters in the house, x_2x2 the number of floors, etc.

Using the definition of matrix multiplication, our multivariable hypothesis function can be
concisely represented as:

hθ(x)=[θ0θ1...θn]⎡⎣⎢⎢⎢x0x1⋮xn⎤⎦⎥⎥⎥=θTx

This is a vectorization of our hypothesis function for one training example; see the lessons on
vectorization to learn more.

Remark: Note that for convenience reasons in this course we assume x(i)0=1 for (i∈1,

…,m). This allows us to do matrix operations with theta and x. Hence making the two vectors
'\thetaθ' and x^{(i)}x(i) match each other element-wise (that is, have the same number of
elements: n+1).]

Gradient Descent For Multiple Variables

Gradient Descent for Multiple Variables

The gradient descent equation itself is generally the same form; we just have to repeat it for our
'n' features:

}repeat until convergence:{θ0:=θ0−α1m∑i=1m(hθ(x(i))−y(i))⋅x(i)0θ1:=θ1−α1m∑i=1m(hθ(x(i))

−y(i))⋅x(i)1θ2:=θ2−α1m∑i=1m(hθ(x(i))−y(i))⋅x(i)2⋯

In other words:

}repeat until convergence:{θj:=θj−α1m∑i=1m(hθ(x(i))−y(i))⋅x(i)jfor j := 0...n

The following image compares gradient descent with one variable to gradient descent with
multiple variables:
Gradient Descent in Practice I - Feature Scaling
We can speed up gradient descent by having each of our input values in roughly the same
range. This is because θ will descend quickly on small ranges and slowly on large ranges, and
so will oscillate inefficiently down to the optimum when the variables are very uneven.

The way to prevent this is to modify the ranges of our input variables so that they are all roughly
the same. Ideally:

−1 ≤ x_{(i)}x(i) ≤ 1

or

−0.5 ≤ x_{(i)}x(i) ≤ 0.5

These aren't exact requirements; we are only trying to speed things up. The goal is to get all
input variables into roughly one of these ranges, give or take a few.

Two techniques to help with this are feature scaling and mean normalization. Feature scaling
involves dividing the input values by the range (i.e. the maximum value minus the minimum
value) of the input variable, resulting in a new range of just 1. Mean normalization involves
subtracting the average value for an input variable from the values for that input variable resulting
in a new average value for the input variable of just zero. To implement both of these techniques,
adjust your input values as shown in this formula:

x_i := \dfrac{x_i - \mu_i}{s_i}xi:=sixi−μi

Where μi is the average of all the values for feature (i) and s_isi is the range of values (max -
min), or s_isi is the standard deviation.
Note that dividing by the range, or dividing by the standard deviation, give different results. The
quizzes in this course use range - the programming exercises use standard deviation.

For example, if x_ixi represents housing prices with a range of 100 to 2000 and a mean value of
1000, then, x_i := \dfrac{price-1000}{1900}xi:=1900price−1000.

Gradient Descent in Practice II - Learning Rate

Note: [5:20 - the x -axis label in the right graph should be \thetaθ rather than No. of iterations ]

Debugging gradient descent. Make a plot with number of iterations on the x-axis. Now plot the
cost function, J(θ) over the number of iterations of gradient descent. If J(θ) ever increases, then
you probably need to decrease α.

Automatic convergence test. Declare convergence if J(θ) decreases by less than E in one

iteration, where E is some small value such as 10−3. However in practice it's difficult to choose
this threshold value.

It has been proven that if learning rate α is sufficiently small, then J(θ) will decrease on every
iteration.
To summarize:

If \alphaα is too small: slow convergence.

If \alphaα is too large: may not decrease on every iteration and thus may not converge.

Features and Polynomial Regression

We can improve our features and the form of our hypothesis function in a couple different ways.

We can combine multiple features into one. For example, we can

combine x_1x1 and x_2x2 into a new feature x_3x3 by taking x_1x1⋅x_2x2.

Polynomial Regression

Our hypothesis function need not be linear (a straight line) if that does not fit the data well.

We can change the behavior or curve of our hypothesis function by making it a quadratic,

cubic or square root function (or any other form).

For example, if our hypothesis function is h_\theta(x) = \theta_0 + \theta_1 x_1hθ(x)=θ0+θ1

x1 then we can create additional features based on x_1x1, to get the quadratic
function h_\theta(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_1^2hθ(x)=θ0+θ1x1+θ2x12 or
the cubic function h_\theta(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_1^2 + \theta_3
x_1^3hθ(x)=θ0+θ1x1+θ2x12+θ3x13

In the cubic version, we have created new features x_2x2 and x_3x3 where x_2 = x_1^2x2

=x12 and x_3 = x_1^3x3=x13.
To make it a square root function, we could do: h_\theta(x) = \theta_0 + \theta_1 x_1 +
\theta_2 \sqrt{x_1}hθ(x)=θ0+θ1x1+θ2x1

One important thing to keep in mind is, if you choose your features this way then feature scaling
becomes very important.

eg. if x_1x1 has range 1 - 1000 then range of x_1^2x12 becomes 1 - 1000000 and that
of x_1^3x13 becomes 1 - 1000000000

Normal Equation
Note: [8:00 to 8:44 - The design matrix X (in the bottom right side of the slide) given in the
example should have elements x with subscript 1 and superscripts varying from 1 to m because
for all m training sets there are only 2 features x_0x0 and x_1x1. 12:56 - The X matrix is m by
(n+1) and NOT n by n. ]

Gradient descent gives one way of minimizing J. Let’s discuss a second way of doing so, this
time performing the minimization explicitly and without resorting to an iterative algorithm. In the
"Normal Equation" method, we will minimize J by explicitly taking its derivatives with respect to
the θj ’s, and setting them to zero. This allows us to find the optimum theta without iteration. The
normal equation formula is given below:

\theta = (X^T X)^{-1}X^T yθ=(XTX)−1XTy

There is no need to do feature scaling with the normal equation.

The following is a comparison of gradient descent and the normal equation:

 Gradient Descent Normal Equation

Need to choose alpha No need to choose alpha

Needs many iterations No need to iterate

O (kn^2kn2) O (n^3n3), need to calculate inverse of X^TXXTX

Works well when n is large Slow if n is very large

With the normal equation, computing the inversion has complexity O(n3). So if we have a very
large number of features, the normal equation will be slow. In practice, when n exceeds 10,000 it
might be a good time to go from a normal solution to an iterative process.

Normal Equation Noninvertibility

When implementing the normal equation in octave we want to use the 'pinv' function rather than
'inv.' The 'pinv' function will give you a value of \thetaθ even if X^TXXTX is not invertible.

If X^TXXTX is noninvertible, the common causes might be having :

 Redundant features, where two features are very closely related (i.e. they are linearly
dependent)
 Too many features (e.g. m ≤ n). In this case, delete some features or use "regularization"
(to be explained in a later lesson).

Solutions to the above problems include deleting a feature that is linearly dependent with another
or deleting one or more features when there are too many features.

Week-3:
Classification
To attempt classification, one method is to use linear regression and map all predictions greater
than 0.5 as a 1 and all less than 0.5 as a 0. However, this method doesn't work well because
classification is not actually a linear function.
The classification problem is just like the regression problem, except that the values we now
want to predict take on only a small number of discrete values. For now, we will focus on
the binary classification problem in which y can take on only two values, 0 and 1. (Most of
what we say here will also generalize to the multiple-class case.) For instance, if we are trying to
build a spam classifier for email, then x^{(i)}x(i) may be some features of a piece of email, and
y may be 1 if it is a piece of spam mail, and 0 otherwise. Hence, y∈{0,1}. 0 is also called the
negative class, and 1 the positive class, and they are sometimes also denoted by the symbols “-”
and “+.” Given x^{(i)}x(i), the corresponding y^{(i)}y(i) is also called the label for the training
example.

Hypothesis Representation
We could approach the classification problem ignoring the fact that y is discrete-valued, and use
our old linear regression algorithm to try to predict y given x. However, it is easy to construct
examples where this method performs very poorly. Intuitively, it also doesn’t make sense
for h_\theta (x)hθ(x) to take values larger than 1 or smaller than 0 when we know that y ∈ {0,
1}. To fix this, let’s change the form for our hypotheses h_\theta (x)hθ(x) to satisfy 0 \leq
h_\theta (x) \leq 10≤hθ(x)≤1. This is accomplished by plugging \theta^TxθTx into the
Logistic Function.

Our new form uses the "Sigmoid Function," also called the "Logistic Function":

hθ(x)=g(θTx)z=θTxg(z)=11+e−z

The following image shows us what the sigmoid function looks like:

The function g(z), shown here, maps any real number to the (0, 1) interval, making it useful for
transforming an arbitrary-valued function into a function better suited for classification.

h_\theta(x)hθ(x) will give us the probability that our output is 1. For

example, h_\theta(x)=0.7hθ(x)=0.7 gives us a probability of 70% that our output is 1. Our
probability that our prediction is 0 is just the complement of our probability that it is 1 (e.g. if
probability that it is 1 is 70%, then the probability that it is 0 is 30%).

hθ(x)=P(y=1|x;θ)=1−P(y=0|x;θ)P(y=0|x;θ)+P(y=1|x;θ)=1
Decision Boundary
In order to get our discrete 0 or 1 classification, we can translate the output of the hypothesis
function as follows:

hθ(x)≥0.5→y=1hθ(x)<0.5→y=0

The way our logistic function g behaves is that when its input is greater than or equal to zero, its
output is greater than or equal to 0.5:

g(z)≥0.5whenz≥0

Remember.

z=0,e0=1⇒g(z)=1/2z→∞,e−∞→0⇒g(z)=1z→−∞,e∞→∞⇒g(z)=0

So if our input to g is \theta^T XθTX, then that means:

hθ(x)=g(θTx)≥0.5whenθTx≥0

From these statements we can now say:

θTx≥0⇒y=1θTx<0⇒y=0

The decision boundary is the line that separates the area where y = 0 and where y = 1. It is
created by our hypothesis function.

Example:

θ=⎡⎣5−10⎤⎦y=1if5+(−1)x1+0x2≥05−x1≥0−x1≥−5x1≤5

In this case, our decision boundary is a straight vertical line placed on the graph where x_1 =
5x1=5, and everything to the left of that denotes y = 1, while everything to the right denotes y =
0.

Again, the input to the sigmoid function g(z) (e.g. \theta^T XθTX) doesn't need to be linear, and
could be a function that describes a circle (e.g. z = \theta_0 + \theta_1 x_1^2 +\theta_2
x_2^2z=θ0+θ1x12+θ2x22) or any shape to fit our data.
LOGISTIC REGRESSION MODEL:

Cost Function
We cannot use the same cost function that we use for linear regression because the Logistic
Function will cause the output to be wavy, causing many local optima. In other words, it will not
be a convex function.

Instead, our cost function for logistic regression looks like:

J(θ)=1m∑i=1mCost(hθ(x(i)),y(i))Cost(hθ(x),y)=−log(hθ(x))Cost(hθ(x),y)=−log(1−hθ(x))if y = 1if y

When y = 1, we get the following plot for J(\theta)J(θ) vs h_\theta (x)hθ(x):

Similarly, when y = 0, we get the following plot for J(\theta)J(θ) vs h_\theta (x)hθ(x):

 Cost(hθ(x),y)=0 if hθ(x)=yCost(hθ(x),y)→∞ if y=0andhθ(x)→1Cost(hθ(x),y)→∞ if y=1andhθ(x)→0

If our correct answer 'y' is 0, then the cost function will be 0 if our hypothesis function also
outputs 0. If our hypothesis approaches 1, then the cost function will approach infinity.

If our correct answer 'y' is 1, then the cost function will be 0 if our hypothesis function outputs 1. If
our hypothesis approaches 0, then the cost function will approach infinity.

Note that writing the cost function in this way guarantees that J(θ) is convex for logistic
regression.

Simplified Cost Function and Gradient Descent

Note: [6:53 - the gradient descent equation should have a 1/m factor]

We can compress our cost function's two conditional cases into one case:

Cost(hθ(x),y)=−ylog(hθ(x))−(1−y)log(1−hθ(x))
Notice that when y is equal to 1, then the second term (1-y)\log(1-h_\theta(x))(1−y)log(1−hθ
(x)) will be zero and will not affect the result. If y is equal to 0, then the first term -y
\log(h_\theta(x))−ylog(hθ(x)) will be zero and will not affect the result.

We can fully write out our entire cost function as follows:

J(\theta) = - \frac{1}{m} \displaystyle \sum_{i=1}^m [y^{(i)}\log (h_\theta (x^{(i)})) + (1 - y^{(i)})\log (1 -

h_\theta(x^{(i)}))]J(θ)=−m1i=1∑m[y(i)log(hθ(x(i)))+(1−y(i))log(1−hθ(x(i)))]

A vectorized implementation is:

h=g(Xθ)J(θ)=1m⋅(−yTlog(h)−(1−y)Tlog(1−h))

Gradient Descent

Remember that the general form of gradient descent is:

Repeat{θj:=θj−α∂∂θjJ(θ)}

We can work out the derivative part using calculus to get:

Repeat{θj:=θj−αm∑i=1m(hθ(x(i))−y(i))x(i)j}
Notice that this algorithm is identical to the one we used in linear regression. We still have to
simultaneously update all values in theta.

A vectorized implementation is:

\theta := \theta - \frac{\alpha}{m} X^{T} (g(X \theta ) - \vec{y})θ:=θ−mαXT(g(Xθ)−y)

Advanced Optimization
Note: [7:35 - '100' should be 100 instead. The value provided should be an integer and not a
character string.]

"Conjugate gradient", "BFGS", and "L-BFGS" are more sophisticated, faster ways to optimize θ
that can be used instead of gradient descent. We suggest that you should not write these more
sophisticated algorithms yourself (unless you are an expert in numerical computing) but use the
libraries instead, as they're already tested and highly optimized. Octave provides them.

We first need to provide a function that evaluates the following two functions for a given input
value θ:

J(θ)∂∂θjJ(θ)
We can write a single function that returns both of these:

function [jVal, gradient] = costFunction(theta)

jVal = [...code to compute J(theta)...];

gradient = [...code to compute derivative of J(theta)...];

end
Then we can use octave's "fminunc()" optimization algorithm along with the "optimset()" function
that creates an object containing the options we want to send to "fminunc()". (Note: the value for
MaxIter should be an integer, not a character string - errata in the video at 7:30)

options = optimset('GradObj', 'on', 'MaxIter', 100);

initialTheta = zeros(2,1);

[optTheta, functionVal, exitFlag] = fminunc(@costFunction, initialTheta,

     options);

We give to the function "fminunc()" our cost function, our initial vector of theta values, and the
"options" object that we created beforehand.

Multiclass Classification: One-vs-all

Now we will approach the classification of data when we have more than two categories. Instead
of y = {0,1} we will expand our definition so that y = {0,1...n}.

Since y = {0,1...n}, we divide our problem into n+1 (+1 because the index starts at 0) binary
classification problems; in each one, we predict the probability that 'y' is a member of one of our
classes.

y∈{0,1...n}h(0)θ(x)=P(y=0|x;θ)h(1)θ(x)=P(y=1|x;θ)⋯h(n)θ(x)=P(y=n|x;θ)prediction=maxi(h(i)θ(x))

We are basically choosing one class and then lumping all the others into a single second class.
We do this repeatedly, applying binary logistic regression to each case, and then use the
hypothesis that returned the highest value as our prediction.

The following image shows how one could classify 3 classes:

To summarize:

Train a logistic regression classifier h_\theta(x)hθ(x) for each class to predict the probability

that  y = i .

To make a prediction on a new x, pick the class that maximizes h_\theta (x)hθ SO

SOLVING THE PROBLEM OF FITTING:

The Problem of Overfitting

Consider the problem of predicting y from x ∈ R. The leftmost figure below shows the result of
fitting a y = θ0+θ1x to a dataset. We see that the data doesn’t really lie on straight line, and so
the fit is not very good.

Instead, if we had added an extra feature x^2x2 , and fit y = \theta_0 + \theta_1x +

\theta_2x^2y=θ0+θ1x+θ2x2 , then we obtain a slightly better fit to the data (See middle
figure). Naively, it might seem that the more features we add, the better. However, there is also a
danger in adding too many features: The rightmost figure is the result of fitting a 5^{th}5th order
polynomial y = \sum_{j=0} ^5 \theta_j x^jy=∑j=05θjxj. We see that even though the fitted
curve passes through the data perfectly, we would not expect this to be a very good predictor of,
say, housing prices (y) for different living areas (x). Without formally defining what these terms
mean, we’ll say the figure on the left shows an instance of underfitting—in which the data
clearly shows structure not captured by the model—and the figure on the right is an example
of overfitting.

Underfitting, or high bias, is when the form of our hypothesis function h maps poorly to the trend
of the data. It is usually caused by a function that is too simple or uses too few features. At the
other extreme, overfitting, or high variance, is caused by a hypothesis function that fits the
available data but does not generalize well to predict new data. It is usually caused by a
complicated function that creates a lot of unnecessary curves and angles unrelated to the data.

This terminology is applied to both linear and logistic regression. There are two main options to
address the issue of overfitting:

1) Reduce the number of features:

 Manually select which features to keep.

 Use a model selection algorithm (studied later in the course).

2) Regularization

 Keep all the features, but reduce the magnitude of parameters \theta_jθj.

 Regularization works well when we have a lot of slightly useful features.

Cost Function
Note: [5:18 - There is a typo. It should be \sum_{j=1}^{n} \theta _j ^2∑j=1nθj2 instead
of \sum_{i=1}^{n} \theta _j ^2∑i=1nθj2]

If we have overfitting from our hypothesis function, we can reduce the weight that some of the
terms in our function carry by increasing their cost.

Say we wanted to make the following function more quadratic:

\theta_0 + \theta_1x + \theta_2x^2 + \theta_3x^3 + \theta_4x^4θ0+θ1x+θ2x2+θ3x3+θ4x4

We'll want to eliminate the influence of \theta_3x^3θ3x3 and \theta_4x^4θ4x4 . Without

actually getting rid of these features or changing the form of our hypothesis, we can instead
modify our cost function:

min_\theta\ \dfrac{1}{2m}\sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})^2 +

1000\cdot\theta_3^2 + 1000\cdot\theta_4^2minθ 2m1∑i=1m(hθ(x(i))−y(i))2+1000⋅θ32
+1000⋅θ42

We've added two extra terms at the end to inflate the cost of \theta_3θ3 and \theta_4θ4. Now, in
order for the cost function to get close to zero, we will have to reduce the values
of \theta_3θ3 and \theta_4θ4 to near zero. This will in turn greatly reduce the values
of \theta_3x^3θ3x3 and \theta_4x^4θ4x4 in our hypothesis function. As a result, we see that
the new hypothesis (depicted by the pink curve) looks like a quadratic function but fits the data
better due to the extra small terms \theta_3x^3θ3x3 and \theta_4x^4θ4x4.

We could also regularize all of our theta parameters in a single summation as:

min_\theta\ \dfrac{1}{2m}\ \sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})^2 + \lambda\ \sum_{j=1}^n \theta_j^2m

(x(i))−y(i))2+λ ∑j=1nθj2

The λ, or lambda, is the regularization parameter. It determines how much the costs of our
theta parameters are inflated.

Using the above cost function with the extra summation, we can smooth the output of our
hypothesis function to reduce overfitting. If lambda is chosen to be too large, it may smooth out
the function too much and cause underfitting. Hence, what would happen if \lambda = 0λ=0 or
is too small ?

Regularized Linear Regression

Note: [8:43 - It is said that X is non-invertible if m \leq≤ n. The correct statement should be that
X is non-invertible if m < n, and may be non-invertible if m = n.

We can apply regularization to both linear regression and logistic regression. We will approach
linear regression first.
Gradient Descent

We will modify our gradient descent function to separate out \theta_0θ0 from the rest of the
parameters because we do not want to penalize \theta_0θ0.

Repeat {    θ0:=θ0−α 1m ∑i=1m(hθ(x(i))−y(i))x(i)0    θj:=θj−α [(1m ∑i=1m(hθ(x(i))−y(i))x(i)j)+λm

j∈{1,2...n}

The term \frac{\lambda}{m}\theta_jmλθj performs our regularization. With some manipulation

our update rule can also be represented as:

\theta_j := \theta_j(1 - \alpha\frac{\lambda}{m}) - \alpha\frac{1}

{m}\sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})x_j^{(i)}θj:=θj(1−αmλ)−αm1∑i=1m(hθ
(x(i))−y(i))xj(i)

The first term in the above equation, 1 - \alpha\frac{\lambda}{m}1−αmλ will always be less

than 1. Intuitively you can see it as reducing the value of \theta_jθj by some amount on every
update. Notice that the second term is now exactly the same as it was before.

Normal Equation

Now let's approach regularization using the alternate method of the non-iterative normal
equation.

To add in regularization, the equation is the same as our original, except that we add another
term inside the parentheses:

θ=(XTX+λ⋅L)−1XTywhere  L=⎡⎣⎢⎢⎢⎢⎢⎢011⋱1⎤⎦⎥⎥⎥⎥⎥⎥

L is a matrix with 0 at the top left and 1's down the diagonal, with 0's everywhere else. It should
have dimension (n+1)×(n+1). Intuitively, this is the identity matrix (though we are not
including x_0x0), multiplied with a single real number λ.

Recall that if m < n, then X^TXXTX is non-invertible. However, when we add the term λ⋅L,
then X^TXXTX + λ⋅L becomes invertible.

Regularized Logistic Regression

We can regularize logistic regression in a similar way that we regularize linear regression. As a
result, we can avoid overfitting. The following image shows how the regularized function,
displayed by the pink line, is less likely to overfit than the non-regularized function represented by
the blue line:

Cost Function

Recall that our cost function for logistic regression was:

J(\theta) = - \frac{1}{m} \sum_{i=1}^m \large[ y^{(i)}\ \log (h_\theta (x^{(i)})) + (1 -

y^{(i)})\ \log (1 - h_\theta(x^{(i)})) \large]J(θ)=−m1∑i=1m[y(i) log(hθ(x(i)))
+(1−y(i)) log(1−hθ(x(i)))]

We can regularize this equation by adding a term to the end:

J(\theta) = - \frac{1}{m} \sum_{i=1}^m \large[ y^{(i)}\ \log (h_\theta (x^{(i)})) + (1 - y^{(i)})\ \log (1 - h_\theta(x
\frac{\lambda}{2m}\sum_{j=1}^n \theta_j^2J(θ)=−m1∑i=1m[y(i) log(hθ(x(i)))+(1−y(i)) log(1−hθ(x(i)))]+2

The second sum, \sum_{j=1}^n \theta_j^2∑j=1nθj2 means to explicitly exclude the bias

term, \theta_0θ0. I.e. the θ vector is indexed from 0 to n (holding n+1
values, \theta_0θ0 through \theta_nθn), and this sum explicitly skips \theta_0θ0, by running
from 1 to n, skipping 0. Thus, when computing the equation, we should continuously update the
two following equations:
Week-6:
Evaluating a Hypothesis
Once we have done some trouble shooting for errors in our predictions by:

 Getting more training examples

 Trying smaller sets of features
 Trying additional features
 Trying polynomial features
 Increasing or decreasing λ

We can move on to evaluate our new hypothesis.

A hypothesis may have a low error for the training examples but still be inaccurate (because of
overfitting). Thus, to evaluate a hypothesis, given a dataset of training examples, we can split up
the data into two sets: a training set and a test set. Typically, the training set consists of 70 %
of your data and the test set is the remaining 30 %.

The new procedure using these two sets is then:

1. Learn Θ and minimize Jtrain(Θ) using the training set

2. Compute the test set error Jtest(Θ)

The test set error

1. For linear regression: Jtest(Θ)=12mtest∑mtesti=1(hΘ(x(i)test)−y(i)test)2
2. For classification ~ Misclassification error (aka 0/1 misclassification error):
 err(hΘ(x),y)=10if hΘ(x)≥0.5 and y=0 or hΘ(x)<0.5 and y=1otherwise
This gives us a binary 0 or 1 error result based on a misclassification. The average test error for
the test set is:

Test Error=1mtest∑mtesti=1err(hΘ(x(i)test),y(i)test)
This gives us the proportion of the test data that was misclassified.

Model Selection and Train/Validation/Test Sets

Just because a learning algorithm fits a training set well, that does not mean it is a good
hypothesis. It could over fit and as a result your predictions on the test set would be poor. The
error of your hypothesis as measured on the data set with which you trained the parameters will
be lower than the error on any other data set.

Given many models with different polynomial degrees, we can use a systematic approach to
identify the 'best' function. In order to choose the model of your hypothesis, you can test each
degree of polynomial and look at the error result.

One way to break down our dataset into the three sets is:

 Training set: 60%

 Cross validation set: 20%
 Test set: 20%

We can now calculate three separate error values for the three different sets using the following
method:

1. Optimize the parameters in Θ using the training set for each polynomial degree.
2. Find the polynomial degree d with the least error using the cross validation set.

3. Estimate the generalization error using the test set with Jtest(Θ(d)), (d = theta from
polynomial with lower error);
This way, the degree of the polynomial d has not been trained using the test set.

Diagnosing Bias vs. Variance

In this section we examine the relationship between the degree of the polynomial d and the
underfitting or overfitting of our hypothesis.

 We need to distinguish whether bias or variance is the problem contributing to bad

predictions.
 High bias is underfitting and high variance is overfitting. Ideally, we need to find a golden
mean between these two.
The training error will tend to decrease as we increase the degree d of the polynomial.

At the same time, the cross validation error will tend to decrease as we increase d up to a point,
and then it will increase as d is increased, forming a convex curve.

High bias (underfitting): both Jtrain(Θ) and JCV(Θ) will be high. Also, JCV(Θ)≈Jtrain(Θ).

High variance (overfitting): Jtrain(Θ) will be low and JCV(Θ) will be much greater
than Jtrain(Θ).
The is summarized in the figure below:

Regularization and Bias/Variance

Note: [The regularization term below and through out the video should be \frac \lambda
{2m} \sum _{j=1}^n \theta_j ^22mλ∑j=1nθj2 and NOT \frac \lambda {2m} \sum
_{j=1}^m \theta_j ^22mλ∑j=1mθj2]
In the figure above, we see that as \lambdaλ increases, our fit becomes more rigid. On the other
hand, as \lambdaλ approaches 0, we tend to over overfit the data. So how do we choose our
parameter \lambdaλ to get it 'just right' ? In order to choose the model and the regularization
term λ, we need to:

1. Create a list of lambdas (i.e.

λ∈{0,0.01,0.02,0.04,0.08,0.16,0.32,0.64,1.28,2.56,5.12,10.24});
2. Create a set of models with different degrees or any other variants.

3. Iterate through the \lambdaλs and for each \lambdaλ go through all the models to
learn some Θ.
4. Compute the cross validation error using the learned Θ (computed with λ) on
the JCV(Θ) without regularization or λ = 0.
5. Select the best combo that produces the lowest error on the cross validation set.

6. Using the best combo Θ and λ, apply it on Jtest(Θ) to see if it has a good generalization
of the problem.

Learning Curves
Training an algorithm on a very few number of data points (such as 1, 2 or 3) will easily have 0
errors because we can always find a quadratic curve that touches exactly those number of
points. Hence:

 As the training set gets larger, the error for a quadratic function increases.
 The error value will plateau out after a certain m, or training set size.

Experiencing high bias:

Low training set size: causes Jtrain(Θ) to be low and JCV(Θ) to be high.

Large training set size: causes both Jtrain(Θ) and JCV(Θ) to be high
with Jtrain(Θ)≈JCV(Θ).
If a learning algorithm is suffering from high bias, getting more training data will not (by
itself) help much.

Experiencing high variance:

Low training set size: Jtrain(Θ) will be low and JCV(Θ) will be high.

Large training set size: Jtrain(Θ) increases with training set size and JCV(Θ) continues to
decrease without leveling off. Also, Jtrain(Θ) < JCV(Θ) but the difference between them
remains significant.
If a learning algorithm is suffering from high variance, getting more training data is likely to help.

Deciding What to Do Next Revisited

Our decision process can be broken down as follows:

 Getting more training examples: Fixes high variance

 Trying smaller sets of features: Fixes high variance

 Adding features: Fixes high bias

 Adding polynomial features: Fixes high bias

 Decreasing λ: Fixes high bias

 Increasing λ: Fixes high variance.

Diagnosing Neural Networks

 A neural network with fewer parameters is prone to underfitting. It is

also computationally cheaper.
  A large neural network with more parameters is prone to overfitting. It is
also computationally expensive. In this case you can use regularization (increase λ) to
address the overfitting.

Using a single hidden layer is a good starting default. You can train your neural network on a
number of hidden layers using your cross validation set. You can then select the one that
performs best.

Model Complexity Effects:

 Lower-order polynomials (low model complexity) have high bias and low variance. In this
case, the model fits poorly consistently.
 Higher-order polynomials (high model complexity) fit the training data extremely well and
the test data extremely poorly. These have low bias on the training data, but very high
variance.
 In reality, we would want to choose a model somewhere in between, that can generalize
well but also fits the data reasonably well.

Prioritizing What to Work On

System Design Example:

Given a data set of emails, we could construct a vector for each email. Each entry in this vector
represents a word. The vector normally contains 10,000 to 50,000 entries gathered by finding the
most frequently used words in our data set. If a word is to be found in the email, we would assign
its respective entry a 1, else if it is not found, that entry would be a 0. Once we have all our x
vectors ready, we train our algorithm and finally, we could use it to classify if an email is a spam
or not.

So how could you spend your time to improve the accuracy of this classifier?

 Collect lots of data (for example "honeypot" project but doesn't always work)
  Develop sophisticated features (for example: using email header data in spam emails)
 Develop algorithms to process your input in different ways (recognizing misspellings in
spam).

It is difficult to tell which of the options will be most helpful.

Error Analysis
The recommended approach to solving machine learning problems is to:

 Start with a simple algorithm, implement it quickly, and test it early on your cross
validation data.
 Plot learning curves to decide if more data, more features, etc. are likely to help.
 Manually examine the errors on examples in the cross validation set and try to spot a
trend where most of the errors were made.

For example, assume that we have 500 emails and our algorithm misclassifies a 100 of them.
We could manually analyze the 100 emails and categorize them based on what type of emails
they are. We could then try to come up with new cues and features that would help us classify
these 100 emails correctly. Hence, if most of our misclassified emails are those which try to steal
passwords, then we could find some features that are particular to those emails and add them to
our model. We could also see how classifying each word according to its root changes our error
rate:

It is very important to get error results as a single, numerical value. Otherwise it is difficult to
assess your algorithm's performance. For example if we use stemming, which is the process of
treating the same word with different forms (fail/failing/failed) as one word (fail), and get a 3%
error rate instead of 5%, then we should definitely add it to our model. However, if we try to
distinguish between upper case and lower case letters and end up getting a 3.2% error rate
instead of 3%, then we should avoid using this new feature. Hence, we should try new things, get
a numerical value for our error rate, and based on our result decide whether we want to keep the
new feature or not.