Scribd
Upload
8 views

MLS+1+-+Presentation

Uploaded by

golgothgolgoth039
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views

MLS+1+-+Presentation

Uploaded by

golgothgolgoth039
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

jacques.lethuaut@gmail.

com
R8L0PN473F Neural Networks

This file is meant for personal use by [email protected] only.


Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
Topics covered so far
● Neural Networks
○ General Introduction
○ Reminder of nonlinear features
○ Single unit and activation functions
○ Multiple layers
○ Architecture
[email protected]
R8L0PN473F○ Cross Entropy Loss
○ Gradient descent
○ Basic Training Algorithms, SGD, Minibatch

This file is meant for personal use by [email protected] only.


Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. 2
Discussion questions

1. What is a neural network and what do different layers in a neural network represent?

2. What is an activation function and what are the different types of activation functions?

3. What do forward and back propagation mean in neural networks?


[email protected]
4. What is the
R8L0PN473F gradient descent algorithm and how does it work?

This file is meant for personal use by [email protected] only.


Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. 3
Neural Networks
Artificial Neural Networks (ANNs) are inspired by biological neural networks and employ a collection of
interconnected artificial neurons to extract the patterns from the given data.

It consists of three types of layers:

● Input layer
○ Represents dimensions of the input vector (one node for
each dimension)
[email protected]
R8L0PN473F
● Hidden layer(s)
○ Represents the intermediary nodes that divide the input
space into regions with (soft) boundaries
○ Given enough hidden nodes, we can model any arbitrary
input-output relation
○ It takes in a set of weighted input and produces output
through an activation function
● Output layer
○ Represents the output of the neural network
○ Mostly, it doesn't have anisactivation
This file function
meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. Image Source 4
Activation functions
● An artificial neural network works in three steps:
○ First, it multiplies the input signals with corresponding
weights
○ Second, it adds the weighted signals together
○ Third, it converts the result into another value using a
mathematical transformation (activation function)
● For the third step, there are multiple mathematical functions
[email protected]
R8L0PN473F available that can be used for the activation function.

● The purpose of the activation function is to act like a switch for the neuron.
● The activation function is critical to the overall functioning of the neural network. Without it, the whole
neural network will mathematically become equivalent to one single neuron!
● The activation function is one of the critical components that give neural networks the ability to deal with
complex problems, by tackling the nonlinearity of the patterns in the data.

This file is meant for personal use by [email protected] only.


Source: Quora Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. 5
Types of activation functions
These are some activation functions that are generally used in neural networks:

1. The Sigmoid function


2. The Tanh function
3. The ReLU (Rectified Linear Unit) function

[email protected]
R8L0PN473F

Sigmoid Tanh ReLU

● Range : 0 to 1 ● Range : -1 to 1 ● Range : 0 to ∞


● Gives probabilities ● More steeper than sigmoid ● Less computationally expensive
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. Image Source 6
Forward Propagation
● In the forward propagation, the input data is propagated forward from the input layer through the hidden
layer until it reaches the final/output layer where predictions are made.
● At every layer, data gets transformed in three steps in every neuron:
○ Sum of weighted input at every neuron (by multiplying 𝑋 by the hidden weight 𝑊) and the bias
○ Apply the activation function on the sum
○ Pass the result to all the neurons in the next layer
● The last layer is the output layer, which may have a sigmoid function (for binary classification) or a softmax
[email protected]
R8L0PN473F function (if the network is a multi-class classifier). The output layer gives the predictions of the neural
network.

After getting the predictions, we use an optimization algorithm


that helps us to minimize the error (cost) function E(x) which is
simply a mathematical function dependent on the model’s
learnable parameters which are used in computing the target
values (Y) from the set of predictors (X) used in the model.

This file is meant for personal use by [email protected] only.


Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. 7
Back Propagation
● Back propagation is the process of learning that the neural network employs to re-calibrate the weights at
every layer and every node to minimize the error in the output layer.
● During the first pass of forward propagation, the weights are random numbers.
● The output of the first iteration is not always accurate. The difference between the actual value / class and
the predicted value / class is the error.
● All the nodes in all the preceding layers contribute to error
and hence need to get their share of the error and correct
[email protected]
R8L0PN473F their weights.
● This process of allocating a proportion of the error (error
gradient) to all the nodes in the previous layer is called the
back propagation.
● The goal of back propagation is to adjust weights in
proportion to the error contribution and in iterative process
identify the optimal combination of weights.
● At each layer, at each node, the gradient descent algorithm
is applied to adjust the weights.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. Image Source
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. 8
Gradient Descent Algorithm
● The goal of optimization is to find a set of weights that
minimizes the loss function.
● Optimization functions usually calculate the gradient, i.e.,
the partial derivative of the loss function with respect to
weights, and the weights are modified in the opposite
direction of the calculated gradient. This cycle is repeated
until we reach the minima of loss function.
[email protected]
● The procedure of repeatedly evaluating the gradient and
R8L0PN473F
then performing a parameter update is known as Gradient
Descent Algorithm.

Learning Rate:
● It is a hyperparameter which determines the step size (the
amount by which the weights are updated).
● We can try out different values of learning rate to improve
the results.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. Image Source
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. 9
Case Study
[email protected]
R8L0PN473F

This file is meant for personal use by [email protected] only.


Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited. 10
Happy Learning !
[email protected]
R8L0PN473F

This file is meant for personal use by [email protected] only.


Sharing or publishing the contents in part or full is liable for legal action. 11
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.

You might also like