Unit 4

Data Science Using R

Introduction to CNN
• A convolutional neural network (CNN) is a category of
machine learning model. Specifically, it is a type of
deep learning algorithm that is well suited to analyzing visual
data.
• CNNs are commonly used to process image and video tasks.
And, because CNNs are so effective at identifying objects,
they are frequently used for computer vision tasks, such as
image recognition and object recognition, with common use
cases including self-driving cars, facial recognition and
medical image analysis.
• CNN architecture is inspired by the connectivity patterns of
the human brain -- in particular, the visual cortex, which plays
an essential role in perceiving and processing visual stimuli.
• A convolutional neural network is a feed-forward neural
network that is generally used to analyze visual images by
processing data with grid-like topology. It’s also known as
a ConvNet. A convolutional neural network is used to detect
and classify objects in an image.
• CNNs also use principles from linear algebra, particularly
convolution operations, to extract features and identify
patterns within images. Although CNNs are predominantly
used to process images, they can also be adapted to work with
audio and other signal data.
 Feed Forward Network
• The mechanism of obtaining an input to generate an output for
predictive purposes is referred to as Feed Forward.
• Feed Forward neural networks serve as the foundational
structure for various significant neural network architectures,
including convolutional neural networks.
• In a feed-forward neural network, the architecture is
characterized by the absence of feedback loops or
interconnections within the network. It consists solely of an
input layer, a hidden layer, and an output layer.
• Neural networks may contain several hidden layers, contingent
upon the nature of the data being processed. The quantity of
these hidden layers is referred to as the network's depth.
Continued..

• A deep neural network is capable of learning from a greater

variety of functions. Initially, the input layer supplies data to
the neural network, while the output layer generates
predictions based on a sequence of functions applied to that
data.
• The Rectified Linear Unit (ReLU) function is the most
prevalent activation function utilized within deep neural
networks.
Layers Of Neural Network
 Layers Of Neural Network
• Input Layers: It’s the layer in which we give input to our model.
The number of neurons in this layer is equal to the total number of
features in our data (number of pixels in the case of an image).
• Hidden Layer: The input from the Input layer is then fed into the
hidden layer. There can be many hidden layers depending on our
model and data size. Each hidden layer can have different numbers
of neurons which are generally greater than the number of features.
The output from each layer is computed by matrix multiplication
of the output of the previous layer with learnable weights of that
layer and then by the addition of learnable biases followed by
activation function which makes the network nonlinear.
• Output Layer: The output from the hidden layer is then fed into a
logistic function like sigmoid or softmax which converts the output
of each class into the probability score of each class.
Activation Function
1. Linear Activation Function
2. Non-Linear Activation Function
• Sigmoid Activation Function
• Tanh Activation Function
3. ReLU(Rectified Linear Unit) Function
 Linear Activation Function
• Linear Activation
Function resembles straight line
define by y=x. No matter how
many layers the neural network
contains, if they all use linear
activation functions, the output is
a linear combination of the input.
• The range of the output spans
from (−∞ to +∞)(−∞ to +∞).
• Linear activation functions are
useful for specific tasks but must
be combined with non-linear
functions to enhance the neural
network’s learning and predictive
capabilities.
Non- linear Activation Function
Sigmoid Function
• Sigmoid Activation Function is
characterized by ‘S’ shape. It is
mathematically defined as
A=11+e−xA=1+e−x1​​. This
formula ensures a smooth and
continuous output that is essential
for gradient-based optimization
methods.
• It allows neural networks to
handle and model complex
patterns that linear equations
cannot.
• The output ranges between 0 and
1, hence useful for binary
classification.
Tanh Function
• Tanh function or hyperbolic
tangent function, is a shifted
version of the sigmoid,
allowing it to stretch across the
y-axis.

• The tanh function outputs

values in the range of -1 to +1.
This means that it can deal
with negative values more
effectively than the sigmoid
function, which has a range of
0 to 1.
ReLU(Rectified Linear Unit) Function
• The Rectified Linear Unit
(ReLU) is one of the most
popular activation functions
used in neural networks,
especially in deep learning
models.
• It has become the default choice
in many architectures due to its
simplicity and efficiency..
• The ReLU function is a
piecewise linear function that
outputs the input directly if it is
positive; otherwise, it outputs
zero.
• In simpler terms, ReLU allows positive values to pass through
unchanged while setting all negative values to zero. This helps
the neural network maintain the necessary complexity to learn
patterns while avoiding some of the pitfalls associated with
other activation functions, like the vanishing gradient problem.
• The ReLU function can be described mathematically as follows:
• f(x)=max(0,x)f(x)=max(0,x)
Where:
• x is the input to the neuron.
• The function returns x if x is greater than 0.
• If x is less than or equal to 0, the function returns 0.
 Back Propagation
• A back propagation
algorithm, or backward
propagation of errors, is
an algorithm that's used to
help train neural network
models. The algorithm
adjusts the network's
weights to minimize any
gaps referred to as errors
between predicted outputs
and the actual target
output.
 Why is Back Propogation Important?

• Effective Weight Adjustment: It calculates the gradient of the loss

function for each weight through the chain rule, enabling efficient
weight updates.
• Adaptability: The backpropagation method is well-suited for
networks with numerous layers and intricate structures, facilitating
deep learning applications.
• Self-Optimizing Learning: Backpropagation automates the learning
process, allowing the model to self-adjust for enhanced performance.
Working of Backpropagation Algorithm
Forward Pass Work:
• During the forward pass, input data is introduced to the input
layer, where it is combined with corresponding weights and
transmitted to the hidden layers.
• For Example In a network featuring two hidden layers (h1 and
h2), the output from h1 is utilized as the input for h2, with a
bias added to the weighted inputs prior to the activation
function application.
• Each hidden layer employs an activation function, such as
ReLU, to introduce non-linearity, enabling the model to
capture intricate data relationships, while the final layer uses
softmax to transform outputs into classification probabilities.
Forward Pass Work:
 Backward Pass Work:
• During the backward pass, the discrepancy between predicted
and actual outputs is transmitted through the network to
modify weights and biases. A prevalent method for
quantifying this error is the Mean Squared Error (MSE),
defined as
MSE = (Predicted Output − Actual Output)².
• After determining the error, the network updates its weights by
utilizing gradients derived from the chain rule, which indicate
the necessary adjustments for each weight and bias to reduce
the error in subsequent iterations.
• The backward pass progresses through each layer, facilitating
the network's learning and enhancement of performance, with
the activation function's derivative being essential for gradient
computation in the backpropagation process.