6. Deep Learning
6. Deep Learning
Semester – IV
1
Artificial Intelligence – SHIT601 - Syllabus
• Introduction to the course, What is AI, Fundamentals of AI, usage of AI in a Business Perspective
• Basic Programming and other Data structural (DS) concepts for AI
• Agents and Environments, Good Behaviour: The Concept of Rationality, the Nature of Environments, Structure of Agents.
• Constraint Satisfaction Problems, Backtracking Search for CSPs. Local Search for Constraint Satisfaction Problems, The
Structure of Problems.
• To Study Informed Search and Exploration: Heuristic Function, Hill Climbing Function, 8 queen problem
• To Study First order logic: Representation Revisited Syntax and Semantic of First-order Logic, Models for first order logic,
Symbols and interpretations Atomic Sentences Complex Sentences Quantifiers
• Inference in First-order Logic Forward and Backward Chaining
• Knowledge Representation
• Uncertain Knowledge and reasoning
• Probability Reasoning
• Deep Learning, Q Learning and applications, Neural Networks, Optimizing Processes
2
Resources for the course- Books
3
Expected Learning Outcomes:
4
Evaluation Timelines and Weightage
5
Introduction to AI
Definition
"It is a branch of computer science by which we can create intelligent machines which
can behave like a human, think like humans, and able to make decisions."
6
Deep Learning
Deep learning is a subset of machine learning, which is essentially a neural network with multiple
layers. These neural networks attempt to simulate the behaviour of the human brain.
Deep learning is a branch of machine learning which is based on artificial neural networks. It
is capable of learning complex patterns and relationships within data. It is designed to learn
from large amounts of data.
7
Basic Concept of AI, ML and DL
8
Machine Learning
• Supervised Learning: Start with the basics of supervised learning, where the
algorithm is trained on labeled datasets to make predictions.
9
Deep Learning architectures include
Convolutional Neural Networks (CNNs) - are specifically for image and video recognition
tasks. CNNs can automatically learn features from the images, which makes them well-suited
for tasks such as image classification, object detection, and image segmentation.
Recurrent Neural Networks (RNNs) - are a type of neural network that can process sequential
data, such as time series and natural language.
10
Relationship - AI, ML, NN and DS
11
Applications of Deep Learning :
Computer Vision - Deep learning models can enable machines to identify and understand
visual data. Some of the main applications of deep learning in computer vision include:
•Object detection and recognition: A deep learning model can be used to identify and
locate objects within images and videos, making it possible for machines to perform
tasks such as self-driving cars, surveillance, and robotics.
•Image classification: Deep learning models can be used to classify images such as
animals, plants, and buildings. This is used in applications such as medical imaging,
quality control, and image retrieval.
12
Natural language processing (NLP):
The Deep learning model can enable machines to understand and generate human language. Some of the
main applications of deep learning in NLP include:
•Automatic Text Generation – A deep learning model can learn the corpus of text and new text like
summaries, and essays can be automatically generated using these trained models.
•Language translation: Deep learning models can translate text from one language to another, making it
possible to communicate with people from different linguistic backgrounds.
•Sentiment analysis: Deep learning models can analyse the sentiment of a piece of text, making it possible
to determine whether the text is positive, negative, or neutral. This is used in applications such as customer
service, social media monitoring, and political analysis.
•Speech recognition: Deep learning models can recognize and transcribe spoken words, making it possible
to perform tasks such as speech-to-text conversion, voice search, and voice-controlled devices.
13
ANN:-
Similar to a human brain has neurons interconnected to each other, artificial neural
networks also have neurons that are linked to each other in various layers of the
networks.
14
Artificial Neural Network looks like:-
The artificial neural network takes input and computes the
weighted sum of the inputs and includes a bias. This
computation is represented in the form of a transfer function.
Weights control the signal between two neurons. In other words, a weight decides how much
influence the input will have on the output.
Biases, which are constant, are an additional input into the next layer that will always have the value of
1. Bias units are not influenced by the previous layer but they do have outgoing connections with their
own weights. The bias unit guarantees that even when all the inputs are zeros there will still be an
activation in the neuron.
Activation Function is a mathematical formula which helps the neuron to switch ON/OFF.
Each artificial neuron has the following main functions:
Perceptron is a function that maps its input “x,” which is multiplied with the learned weight coefficient;
an output value ”f(x)”is generated.
The output can be represented as “1” or “0.” It can also be represented as “1” or “-1” depending on
which activation function is used.
Consider a neuron with two inputs (x1,x2) as
The values of the two inputs(x1,x2) are 0.8 and 1.2
We have a set of weights (1.0,0.75) corresponding to the two inputs
Then we have a bias with value 0.5 which needs to be added to the sum
The input to activation function is then calculated using the
formula:-
This neural network is formed in three layers, called the input layer, hidden layer, and output
layer.
Each layer consists of one or more nodes, represented in this diagram by the small circles.
The lines between the nodes indicate the flow of information from one node to the next.
The nodes of the input layer are passive, means they do not modify the data. They receive a single
value on their input, and duplicate the value to their multiple outputs. In comparison, the nodes of
the hidden and output layer are active.
Connections
Nodes
Each value from the input layer is duplicated and sent to all of
the hidden nodes. This is called a fully
interconnected structure.
Input Layer:
It accepts inputs in several different formats provided by the programmer.
Hidden Layer:
The hidden layer presents in-between input and output layers. It performs all the
calculations to find hidden features and patterns.
Output Layer:
The input goes through a series of transformations using the hidden layer, which
finally results in output that is conveyed using this layer.
In this particular type of neural network, the information flows only from the input to the
output (from left-to-right).
Other types of neural networks have more intricate connections, such as feedback paths.
Models of neural networks:
Feed-forward ANNs allow signals to travel one way only: from input to output.
There is no feedback or loops. The output of any layer does not affect that same layer in such
networks.
Feed-forward ANNs tend to be straightforward networks that associate inputs with outputs.
This model is extensively used in pattern recognition. Also referred to as bottom-up or top-down.
They have fixed inputs and outputs. They are mostly used in pattern generation, pattern
recognition and classification.
A threshold transfer function is sometimes used to quantify the output of a neuron in the output
layer.
Feed-forward networks include Perceptron (linear and non-linear).
Feed-forward networks are often used in data mining.
Feedforward neural networks are ideally suitable for modelling relationships between a set of
predictor or input variables and one or more response or output variables.
In other words, they are appropriate for any functional mapping problem where we want to
know how a number of input variables affect the output variable.
The multilayer feed-forward neural networks, also called multi-layer perceptron (MLP), are the
most widely studied and used neural network model in practice.
Feed forward neural network
Feedback neural network
Signals can travel in both the directions in Feedback neural networks. Feedback neural
networks are very powerful and can get very complicated.
Feedback neural networks are dynamic. The ‘state’ in such network keep changing until
they reach an balance point.
Feedback neural network
Perceptron (Artificial Neuron)
Based on linear threshold unit (LTU)
Perceptron is the basic unit of the neural network. In simple terms, it is a mathematical function based on
a model of biological neurons. It can also be seen as a simple logic gate with binary outputs.
The Perceptron receives multiple input signals, and if the sum of the input signals exceeds a certain
threshold, it either outputs a signal or does not return an output. In the context of supervised learning
and classification, this can then be used to predict the class of a sample.
The threshold is one of the key components of the perceptron. It determines, based on the inputs,
whether the perceptron fires or not.
Types of Perceptron
The single-layer perceptron does not have a priori knowledge, so the initial weights are assigned
randomly.
SLP sums all the weighted inputs and if the sum is above the threshold (some predetermined
value), SLP is said to be activated (output=1).
The input values are presented to the perceptron, and if the predicted
output is the same as the desired output, then the performance is
considered satisfactory and no changes to the weights are made.
However, if the output does not match the desired output, then the
weights need to be changed to reduce the error.
Multiple layer Perceptron - Backpropagation algorithm
Multilayer Perceptron or feed-forward neural networks with two or more layers have the greater
processing power. A multilayer perceptron is a type of feed-forward artificial neural network that
generates a set of outputs from a set of inputs. An MLP is a neural network connecting multiple
layers in a directed graph, which means that the signal path through the nodes only goes one
way.
Multi-layer Perceptron – Back propagation algorithm
A multi-layer perceptron (MLP) has the same structure as a single-layer perceptron with one or
more hidden layers.
•the forward phase where the activations are propagated from the input to the output layer,
•and the backward phase, where the error between the observed actual and the requested
nominal value in the output layer is propagated backwards in order to modify the weights and
bias values.
Data at input layer is multiplied with weights to form hidden layer
h1 = (x1 * w1) + (x2 * w1)
h2 = (x1 * w2) + (x2 * w2)
h3 = (x1 * w3) + (x2 * w3)
y = fn( h1 , h2, h3 )
MLP networks are used for supervised learning format. A typical learning
algorithm for MLP networks is also called back propagation's algorithm.
Need of Back propagation ?
While designing a Neural Network, in the beginning, we initialize weights with some
random values or any variable for that fact.
It’s not necessary that whatever weight values we have selected will be correct, or it fits
our model the best.
The Back propagation algorithm looks for the minimum value of the error function in weight
space using any learning rule.
The weights that minimize the error function are then considered to be a solution to the
learning problem.
Backpropagation algorithm
44
1.Initialization:
1. Initialize the weights and biases of the neural network randomly or using a
specific initialization strategy.
2.Forward Pass:
1. Input data is fed forward through the network to generate a predicted output.
2. The forward pass involves passing the input through each layer of the network,
applying weights, biases, activation functions, and producing the final output.
45
3. Compute Loss:
1. Calculate the difference (error or loss) between the predicted output and the actual target
values using a loss function.
2. Common loss functions include Mean Squared Error (MSE) for regression tasks or Cross-
Entropy Loss for classification tasks.
4. Backward Pass (Backpropagation proper):
1. Compute the gradients of the loss with respect to the weights and biases using the chain
rule of calculus.
2. Starting from the output layer and moving backward through the network, calculate the
partial derivatives of the loss with respect to each parameter.
46
A common drawback of Supervised learning is the vast amount of data that models need
to train.
49
•An Environment, which an agent will interact with, to learn to reach a goal or perform an
action.
•A Reward if the action performed by the model is bringing us closer to the goal/is leading to
the goal. This is done to train the model in the right direction.
•A negative reward if it performs an action that will not lead to the goal to prevent it from
learning in the wrong direction.
51
Q-Learning is a Reinforcement learning policy that will find the next best action, given a current state.
It chooses this action at random and aims to maximize the reward.
52
Markov Decision Process (MDP), is used to formalize the
reinforcement learning problems.
MDP is used for decision-making problems where the outcomes are partly random and
controllable.
54
In reinforcement learning, rewards are associated with actions to provide feedback to the
learning agent about the desirability of its actions.
By associating rewards with actions, the agent can learn which actions lead to favourable
outcomes and which do not.
Discount Factor - balances immediate rewards against future rewards.
57
The discount factor is typically a value between 0 and 1, with 0 indicating that the
agent only considers immediate rewards and 1 indicating that the agent values future
rewards equally to immediate rewards.
Q-values and Q-learning
58
Q-learning is a popular algorithm that utilizes Q-values to learn an optimal policy in a model-
free manner, updating its estimates based on observed rewards and future Q-values.
Exploration and Exploitation
59
Exploration is any action that lets the agent discover new features about the environment, while
exploitation is capitalizing on knowledge already gained.
Exploitation is defined as a greedy approach in which agents try to get more rewards by using estimated
value but not the actual value. So, in this technique, agents make the best decision based on current
information.
In exploration, agents primarily focus on improving their knowledge about each action instead of getting
more rewards so that they can get long-term benefits. So, in this technique, agents work on gathering
more information to make the best overall decision.
Q-learning algorithm process includes the following
60
•Q-table initialization. The first step is to create the Q-table as a place to track each action in
each state and the associated progress.
•Observation. The agent needs to observe the current state of the environment.
•Action. The agent chooses to act in the environment. Upon completion of the action, the model
observes if the action is beneficial in the environment.
•Update. After the action has been taken, it's time to update the Q-table with the results.
•Repeat. Repeat steps 2-4 until the model reaches a termination state for a desired objective.
Convergence Conditions
61
Convergence refers to the point at which the training process reaches a stable state and
the parameters of the network (i.e., the weights and biases) have settled on values that
produce accurate predictions for the training data.
A neural network can be considered to have converged when the training error (or loss)
stops decreasing or has reached a minimum level of acceptable error.
Optimizing Process through Q learning –
find optimal strategies for decision-making in dynamic environments
62
•Energy Management: In smart grids or energy systems, Q-learning can optimize the
control of devices to manage energy consumption efficiently.
Q-learning can be applied to optimize processes in AI - Steps
64
• States, Actions, and Rewards: Clearly define the states of the system, the
possible actions an agent can take in each state, and the immediate rewards
associated with each action-state pair.
Q-learning can be applied to optimize processes in AI - Steps
65
4. Implementation of Q-learning:
• Validation Set: Use a validation set to evaluate the performance of the learned
policy during the training process.
• Test Set: Assess the final Q-values on a separate test set to ensure the
generalization of the learned policy to new situations.
Q-learning can be applied to optimize processes in AI - Steps
70
7. Scale Up to Real-World Processes:
8. Continuous Improvement:
• Reinforcement Learning Iterations: Apply the lessons learned from the initial
Q-learning implementation to iterate and improve the model continuously.
• Dynamic Environments: Adjust the Q-learning approach to handle dynamic
environments where the underlying system may change over time.
Q-learning can be applied to optimize processes in AI - Steps
72
73
1. State Representation:
Define the state of the elevator system by considering factors such as the current floor of
the elevator, the direction it is moving, the current load (number of passengers), and the
historical patterns of elevator usage.
2. Action Representation:
•Actions can be represented by the different commands the elevator can execute,
including moving up, moving down, stopping, and opening/closing doors. The control
system needs to decide which action to take based on the current state.
74
3. Q-Table Initialization:
•Initialize a Q-table with entries for each state-action pair. The Q-values represent the
expected cumulative reward for taking a particular action in a given state. Initially, these
values can be set to zero.
4. Reward Structure:
•Design a reward structure to encourage behaviours that minimize waiting times and energy
consumption. Positive rewards can be given for prompt arrivals at requested floors and
efficient use of energy, while negative rewards can be assigned for delays and excessive
energy usage.
75
5. Learning Algorithm:
•Utilize the Q-learning update rule to iteratively update the Q-values based on observed
rewards and future expected rewards. This involves adjusting the Q-values to learn optimal
strategies over time.
6. Exploration-Exploitation Strategy:
•Implement an exploration-exploitation strategy, such as epsilon-greedy, to balance between
exploring new actions and exploiting learned optimal actions. This ensures the system adapts
to changing passenger demands and traffic patterns.
76
7. Energy-Efficient Movement:
•Integrate algorithms for energy-efficient movement, such as regenerative braking and
optimizing the use of the elevator's motor. Q-learning can guide the elevator to make
decisions that minimize unnecessary energy consumption.
To-do list
The chat bot on a specific topic
Calculator
Other similar task
Thank You
79