Scribd
Upload
43 views

Neural Networks

This document provides an introduction to neural networks. It discusses the basic architecture of a single neuron, including inputs, weights, a summation function, bias, and an activation function. An example of a neural network for house pricing prediction is presented. Supervised learning and its structured vs unstructured forms are explained. Factors driving the growth of deep learning are outlined, such as availability of large datasets, increased computing power, advances in deep neural network architectures, breakthroughs in training algorithms, open source frameworks and libraries, industry adoption, transfer learning, and integration with big data and cloud computing. Basic deep learning algorithms like binary classification, logistic regression, and gradient descent are also introduced.

Uploaded by

salemamr1010
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
43 views

Neural Networks

This document provides an introduction to neural networks. It discusses the basic architecture of a single neuron, including inputs, weights, a summation function, bias, and an activation function. An example of a neural network for house pricing prediction is presented. Supervised learning and its structured vs unstructured forms are explained. Factors driving the growth of deep learning are outlined, such as availability of large datasets, increased computing power, advances in deep neural network architectures, breakthroughs in training algorithms, open source frameworks and libraries, industry adoption, transfer learning, and integration with big data and cloud computing. Basic deep learning algorithms like binary classification, logistic regression, and gradient descent are also introduced.

Uploaded by

salemamr1010
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 40

Neural Networks

Eng. Mohamed Gamal, M.Sc Candidate


, Zewail City
● Introduction to Neural Networks
● Basic Algorithms in Neural
Networks
● How to build a Neural Network?
Content ● Deep Neural Networks
● Forward and Backward
These topics will be covered in this Propagation
portion of the course ● Hyperparameters
● Code Examples

2
What is Neural Network ?
● A neural network is a computational model inspired by
the structure and functioning of the human brain.

● It is a collection of interconnected artificial neurons or


nodes that work together to process and analyze
complex data, recognize patterns, make decisions, and
perform various tasks.

● Neural networks are a fundamental component of the


field of machine learning and have gained significant
attention due to their ability to learn from data and
generalize to new situations.
3
What is Neural Network ?
● Let's discuss the basic architecture of a single neuron in a neural network, component by
component:

4
Basic Neuron
● Inputs: A neuron receives inputs
from other neurons or directly from
the input data. These inputs can be
numerical values, binary values, or
even more complex data types.
Each input is associated with a
weight that represents its
importance or contribution to the
neuron's output of a single neuron
in a neural network, component by
component:

5
Basic Neuron

● Weights: Weights are numerical


values assigned to each input. They
determine the strength and
significance of the input's
contribution to the neuron's output.
During the training process, the
network adjusts these weights to
optimize the neuron's performance.

6
Basic Neuron

● Summation Function: The neuron


computes a weighted sum of its
inputs. It multiplies each input by
its corresponding weight and sums
up these weighted inputs. This step
represents the linear combination of
inputs and weights.

7
Basic Neuron

● Bias: In addition to the weighted


inputs, a neuron typically includes a
bias term. The bias is a constant
value that acts as an offset, allowing
the neuron to adjust the output
independently of the input values. It
helps the network account for
situations where all inputs may be
zero.

8
Basic Neuron

● Activation Function: After the


summation of weighted inputs, the
neuron applies an activation function
to introduce non-linearity into the
output. The activation function
determines whether the neuron
should be "activated" and contribute
to the network's final output. It adds
flexibility and expressive power to
the neuron's computations.

9
Basic Neuron

● Activation Output: The output of


the activation function represents
the final output of the neuron. It can
be binary (0 or 1), continuous (a
real-valued number), or multi-
valued depending on the specific
problem and the chosen activation
function.

10
Example: House Pricing “Single Neuron”

11
Example: House Pricing “Neural Network”

12
Example: House Pricing “Neural Network”

13
Supervised Learning

14
Supervised Learning: Structured vs Unstructured

15
Supervised Learning

16
Why is Deep Learning Skyrocketing ?

Availability of Large- Increased Advances in DNN


Scale Datasets Computing Power Architectures

Open-Source Success in
Breakthroughs in
Frameworks and Challenging
Training Algorithms
Libraries Applications

Transfer Learning Integration with Big


Industry Adoption
and Pretrained Data and Cloud
and Applications
Models Computing

17
Availability of Large-Scale Datasets

18
Increased Computational Power

● Processor: Intel Xeon W-3175X (28 cores, 56


threads)
● RAM: 384 GB DDR5 ECC RAM
● Graphics Card: NVIDIA Quadro RTX 8000 or
NVIDIA A6000
● Storage: Multiple high-speed SSDs (e.g., NVMe
SSDs)
● Operating System: Windows 10 Pro or Linux
● Power Supply: High-wattage power supply (e.g.,
1000W or higher)
● Connectivity: Multiple USB ports, Thunderbolt
ports, Ethernet, and Wi-Fi
● Display: Multiple high-resolution monitors (e.g.,
4K or higher)

19
Advances in DNN Architectures

20
Training Algorithms

● Deep learning has seen breakthroughs in optimization algorithms and training


techniques that have made it easier to train deep neural networks.

● Techniques such as stochastic gradient descent (SGD), adaptive optimization


algorithms (e.g., Adam), and regularization methods (e.g., dropout) help in
optimizing deep networks, avoiding overfitting, and improving generalization.

● Additionally, pretraining models with large unlabeled datasets, known as pre-training


or unsupervised learning, followed by fine-tuning on labeled data, has proven
effective in transfer learning scenarios.
21
Open Source AI Softwares

22
Success in Challenging Applications

23
Industry Adoption

24
Transfer Learning

25
Big Data and Cloud Integration

26
Basic Deep Learning Algorithms

Binary Classification Logistic Regression Gradient Descent

27
Cat or Not Cat
Binary Classification

28
Binary Classification

Linear Separator Non-Linear Separator

29
Binary Classification

What does this binary classification system do ?

30
Multi-Group Classification

31
Logistic Regression

32
Logistic Regression

33
Logistic Regression - Cost Function

34
Logistic Regression - Cost Function

35
Logistic Regression - Cost Function

36
Gradient Descent

37
Gradient Descent

38
Activity I (Solved) - 2 hrs

Logistic Regression with a Neural Network mindset


● Problem Statement: You are given a dataset ("data.h5") containing: - a training set of
m_train images labeled as cat (y=1) or non-cat (y=0) - a test set of m_test images
labeled as cat or non-cat - each image is of shape (num_px, num_px, 3) where 3 is
for the 3 channels (RGB). Thus, each image is square (height = num_px) and (width
= num_px).

● You will build a simple image-recognition algorithm that can correctly classify
pictures as cat or non-cat.

39
Thank You!

40

You might also like