Session 2: Fundamentals of Neural Networks

Objective: Explore the basics of neural networks, the building blocks of AI models.

Agenda:
1. Introduction (10 minutes)
 Recap of Session 1: Introduction to AI and Machine Learning
 Importance of neural networks in AI and machine learning
2. Structure and Functioning of Artificial Neurons (30 minutes)
 What is an Artificial Neuron?
 Definition: An artificial neuron, or perceptron, is a computational unit that
takes multiple inputs, processes them using weights and biases, and produces
an output using an activation function.
 Structure of an Artificial Neuron
 Input layer: Receives input features
 Weights and biases: Parameters that adjust the input values
 Activation function: Determines the output of the neuron
 Output: Transformed value sent to the next layer
 Functioning of an Artificial Neuron
 Input summation: Weighted sum of inputs and biases
 Activation: Application of an activation function to the input sum
 Output: Resulting value passed to the next layer
3. Feedforward and Backpropagation Algorithms (40 minutes)
 Feedforward Algorithm
 Definition: The feedforward algorithm is the process by which input data is
passed through the neural network to produce an output without any
feedback.
 Steps:
1. Input features are passed to the input layer
2. Inputs are weighted and summed in the hidden layers
3. Activation functions are applied to produce outputs
4. Final output is produced at the output layer
 Backpropagation Algorithm
 Definition: The backpropagation algorithm is a supervised learning algorithm
used to train neural networks by adjusting the weights and biases based on the
error between predicted and actual outputs.
 Steps:
1. Forward pass: Input data is passed through the network to produce an
output
2. Calculation of error: Difference between predicted and actual outputs
 3. Backward pass: Error is propagated backward through the network to
adjust weights and biases
4. Weight and bias updates: Adjustments are made to minimize the error
in subsequent iterations
4. Activation Functions and Their Roles (30 minutes)
 What is an Activation Function?
 Definition: An activation function introduces non-linearity to the neural
network, enabling it to learn and perform complex tasks.
 Common Activation Functions
 Sigmoid Function
1. Definition: σ(x)=1+e−x1
2. Characteristics: S-shaped curve, outputs between 0 and 1
3. Applications: Binary classification, output layer in binary classifiers
 ReLU (Rectified Linear Unit) Function
1. Definition: f(x)=max(0,x)
2. Characteristics: Piecewise linear, outputs 0 for negative inputs, linear
for positive inputs
3. Applications: Hidden layers in deep neural networks
 Tanh Function
1. Definition: tanh(x)=ex+e−xex−e−x
2. Characteristics: S-shaped curve, outputs between -1 and 1
3. Applications: Hidden layers in recurrent neural networks
 Role of Activation Functions
 Introduce non-linearity to the model
 Enable complex mapping between inputs and outputs
 Regularize the model and prevent overfitting
5. Conclusion and Q&A (10 minutes)
 Summary of key concepts covered in the session
 Open floor for questions and discussions

Recommended Reading:
 "Neural Networks and Deep Learning: A Textbook" by Charu Aggarwal
 "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
Homework Assignment:
 Implement a simple feedforward neural network from scratch using Python and NumPy.
Train the model on a synthetic dataset and analyze the impact of different activation
functions on the model's performance.

Note: This course material is a general outline for Session 2 and can be adjusted based on the
specific needs and interests of the students and the instructor.