Neural Networks

Basic Concepts
Semester: August 2021

||Fredrick Michael Ogore||[email protected]||Cell-No:0717105568||

 What is a Neural Network?

• Artificial Neural Network ANN is an efficient computing system

whose central theme is borrowed from the analogy of biological
neural networks.
• The inventor of the first neurocomputer, Dr. Robert Hecht-
Nielsen, defines a neural network as −
"...a computing system made up of a number of simple, highly
interconnected processing elements, which process information by
their dynamic state response to external inputs.”
 What is a Neural Network?

• Neural networks reflect the behavior of the human brain,

allowing computer programs to recognize patterns and solve
common problems in the fields of AI, machine learning, and
deep learning.
• Neural networks, also known as artificial neural networks
(ANNs) or simulated neural networks (SNNs), are a subset
of machine learning and are at the heart of deep learning
algorithms.
 Basic Structure of ANNs
• The idea of ANNs is based on the belief
that working of human brain by making
the right connections, can be imitated
using silicon and wires as living neurons
and dendrites.
• ANNs are composed of multiple nodes,
which imitate biological neurons of human
brain. The neurons are connected by links
and they interact with each other. The
nodes can take input data and perform
simple operations on the data. The result
of these operations is passed to other
neurons. The output at each node is
called its activation or node value.
 What is a Neural Network?

• Their name and structure are inspired by the human brain,

mimicking the way that biological neurons signal to one
another.
• Artificial neural networks (ANNs) are comprised of a node
layers, containing an input layer, one or more hidden layers,
and an output layer.
• Each node, or artificial neuron, connects to another and has
an associated weight and threshold.
 What is a Neural Network?

• If the output of any individual node is above the specified

threshold value, that node is activated, sending data to the
next layer of the network.
• Otherwise, no data is passed along to the next layer of the
network.
 Basic Structure of ANNs
• Working of a Biological Neuron
• As shown in the above diagram, a typical
neuron consists of the following four parts with
the help of which we can explain its working −
• Dendrites − They are tree-like branches,
responsible for receiving the information from
other neurons it is connected to. In other
sense, we can say that they are like the ears of
neuron.
• Soma − It is the cell body of the neuron and is
responsible for processing of information, they
have received from dendrites.
• Axon − It is just like a cable through which
neurons send the information.
• Synapses − It is the connection between the
axon and other neuron dendrites.
 Basic Structure of ANNs

Biological Neural Network Artificial Neural Network

Dendrites Inputs
Cell nucleus Nodes
Synapse Weights
Axon Output
Artificial Neural Network primarily consists of three layers
Types of Artificial Neural Networks Cont…
 Architecture of an artificial neural network:
• Neural networks rely on training data to learn and improve their
accuracy over time.
• However, once these learning algorithms are fine-tuned for accuracy,
they are powerful tools in computer science and artificial intelligence,
allowing us to classify and cluster data at a high velocity.
• Tasks in speech recognition or image recognition can take minutes
versus hours when compared to the manual identification by human
experts. One of the most well-known neural networks is Google’s
search algorithm.
 Architecture of an artificial neural network:
• To understand the concept of the architecture of an artificial
neural network, we have to understand what a neural network
consists of.
• In order to define a neural network that consists of a large
number of artificial neurons, which are termed units arranged in
a sequence of layers.
• Lets us look at various types of layers available in an artificial
neural network.
 Architecture of an artificial neural network:
• Input Layer:
• As the name suggests, it accepts inputs in several different
formats provided by the programmer.
• Hidden Layer:
• The hidden layer presents in-between input and output
layers. It performs all the calculations to find hidden
features and patterns.
 Architecture of an artificial neural network:
• Output Layer:
• The input goes through a series of transformations using the
hidden layer, which finally results in output that is conveyed
using this layer.
• The artificial neural network takes input and computes the
weighted sum of the inputs and includes a bias. This
computation is represented in the form of a transfer function.
 Architecture of an artificial neural network:
• It determines weighted total is passed as an input to an
activation function to produce the output.
• Activation functions choose whether a node should fire or
not.
• Only those who are fired make it to the output layer.
• There are distinctive activation functions available that can
be applied upon the sort of task we are performing.
 Neural networks vs. deep learning
• Deep Learning and neural networks tend to be used
interchangeably in conversation, which can be confusing.
• As a result, it’s worth noting that the “deep” in deep learning is
just referring to the depth of layers in a neural network.
• A neural network that consists of more than three layers—which
would be inclusive of the inputs and the output—can be
considered a deep learning algorithm.
• A neural network that only has two or three layers is just a basic
neural network.
 Advantages of Artificial Neural Network (ANN)
• Parallel processing capability:
• Artificial neural networks have a numerical value that can
perform more than one task simultaneously.
• Storing data on the entire network:
• Data that is used in traditional programming is stored on the
whole network, not on a database.
• The disappearance of a couple of pieces of data in one place
doesn't prevent the network from working.
 Advantages of Artificial Neural Network (ANN)
• Capability to work with incomplete knowledge:
• After ANN training, the information may produce output
even with inadequate data.
• The loss of performance here relies upon the significance
of missing data.
 Advantages of Artificial Neural Network (ANN)
• Having a memory distribution:
• For ANN is to be able to adapt, it is important to determine the
examples and to encourage the network according to the
desired output by demonstrating these examples to the
network.
• The succession of the network is directly proportional to the
chosen instances, and if the event can't appear to the network
in all its aspects, it can produce false output.
 Advantages of Artificial Neural Network (ANN)
• Having fault tolerance:
• Extortion of one or more cells of ANN does not prohibit it
from generating output, and this feature makes the network
fault-tolerance.
Basic Structure of ANNs Cont…..
• Each link is associated
with weight. ANNs are
capable of learning,
which takes place by
altering weight values.
The following
illustration shows a
simple ANN −
 Types of Artificial Neural Networks
• There are two Artificial Neural Network
topologies −
• Feed Forward ANN-the information
flow is unidirectional.
• A unit sends information to other unit
from which it does not receive any
information. There are no feedback
loops.
• They are used in pattern
generation/recognition/classification.
They have fixed inputs and outputs.
 Types of Artificial Neural Network:
• There are various types of Artificial Neural Networks (ANN)
depending upon the human brain neuron and network
functions, an artificial neural network similarly performs tasks.
• The majority of the artificial neural networks will have some
similarities with a more complex biological partner and are
very effective at their expected tasks.
• For example, segmentation or classification.
 Types of Artificial Neural Network:
• Neural networks can be classified into different types,
which are used for different purposes.
• While this isn’t a comprehensive list of types, the below
would be representative of the most common types of
neural networks that you’ll come across for its common use
cases:
 Types of Artificial Neural Networks
• A feed-forward network is a basic neural network comprising of
an input layer, an output layer, and at least one layer of a neuron.
• Through assessment of its output by reviewing its input, the
intensity of the network can be noticed based on group behavior
of the associated neurons, and the output is decided.
• The primary advantage of this network is that it figures out how
to evaluate and recognize input patterns.
 Types of Artificial Neural Networks Cont…

• FeedBack ANN-In this

type of ANN feedback
loops are allowed.
• They are used in
content addressable
memories.
 The feedback networks.
• The feedback networks feed information back into itself
and are well suited to solve optimization issues.
• The Internal system error corrections utilize feedback
ANNs.
 The feedback networks.
• Feedforward neural networks, or multi-layer perceptrons (MLPs), are what
we’ve primarily been focusing on within this article. They are comprised of
an input layer, a hidden layer or layers, and an output layer.
• While these neural networks are also commonly referred to as MLPs, it’s
important to note that they are actually comprised of sigmoid neurons, not
perceptrons, as most real-world problems are nonlinear.
• Data usually is fed into these models to train them, and they are the
foundation for computer vision, natural language processing, and other
neural networks.
 The feedback networks.
• Convolutional neural networks (CNNs) are similar to
feedforward networks, but they’re usually utilized for
image recognition, pattern recognition, and/or computer
vision.
• These networks harness principles from linear algebra,
particularly matrix multiplication, to identify patterns
within an image.
 The feedback networks.
• Recurrent neural networks (RNNs) are identified by their
feedback loops.
• These learning algorithms are primarily leveraged when
using time-series data to make predictions about future
outcomes, such as stock market predictions or sales
forecasting.
 Working of ANNs
• Each connection has a weight, an integer number that
controls the signal between the two neurons.
• If the network generates a “good or desired” output, there is
no need to adjust the weights.
• However, if the network generates a “poor or undesired”
output or an error, then the system alters the weights in
order to improve subsequent results.
 Working of ANNs
• Think of each individual node as its own linear regression
model, composed of input data, weights, a bias (or
threshold), and an output. The formula would look
something like this:

• ∑wixi + bias = w1x1 + w2x2 + w3x3 + bias

 Working of ANNs

output = f(x) = 1 if ∑w1x1 + b>= 0; 0 if ∑w1x1 + b < 0

• Once an input layer is determined, weights are assigned. These weights
help determine the importance of any given variable, with larger ones
contributing more significantly to the output compared to other inputs.
• All inputs are then multiplied by their respective weights and then
summed.
• Afterward, the output is passed through an activation function, which
determines the output.
• If that output exceeds a given threshold, it “fires” (or activates) the
node, passing data to the next layer in the network. This results in the
output of one node becoming in the input of the next node. This
process of passing data from one layer to the next layer defines this
neural network as a feedforward network.
• Let’s break down what one single node might look like
using binary values. We can apply this concept to a
more tangible example, like whether you should go
surfing (Yes: 1, No: 0).
• The decision to go or not to go is our predicted
outcome, or y-hat. Let’s assume that there are three
factors influencing your decision-making:
• Are the waves good? (Yes: 1, No: 0)
• Is the line-up empty? (Yes: 1, No: 0)
• Has there been a recent shark attack? (Yes: 0, No: 1)

• Source Link :ibm.com/cloud/learn/neural-networks

 Bayesian Networks (BN)

• These are the graphical structures used to represent the probabilistic relationship
among a set of random variables.
• Bayesian networks are also called Belief Networks or Bayes Nets. BNs reason about
uncertain domain.
• In these networks, each node represents a random variable with specific propositions.
For example, in a medical diagnosis domain, the node Cancer represents the
proposition that a patient has cancer.
• The edges connecting the nodes represent probabilistic dependencies among those
random variables.
• If out of two nodes, one is affecting the other then they must be directly connected in
the directions of the effect.
• The strength of the relationship between variables is quantified by the probability
associated with each node.
 Building a Bayesian Network

• There are a number of steps the knowledge engineer needs to take while
building it.

Gather Relevant Information of Problem

• Is the patient a smoker? If yes, then high chances of cancer and bronchitis.
• Is the patient exposed to air pollution? If yes, what sort of air pollution?
• Take an X-Ray positive X-ray would indicate either TB or lung cancer.

Identify Interesting Variables

The knowledge engineer tries to answer the questions −
• Which nodes to represent?
• What values can they take? In which state can they be?
Common types of discrete nodes cont….

Possible nodes and values for the lung cancer

Node Name Type
example
Value Nodes Creation

Polution Binary {LOW, HIGH, MEDIUM}

Smoker Boolean {TRUE, FASLE}

Lung-Cancer Boolean {TRUE, FASLE}

X-Ray Binary {Positive, Negative}

 Adaptive Resonance Theory
The Adaptive Resonance Theory (ART) was incorporated
as a hypothesis for human cognitive data handling. The
hypothesis has prompted neural models for pattern
recognition and unsupervised learning.
ART system has been utilized to clarify different types of
cognitive and brain data.
 Associate Memory Network
An associate memory network refers to a content
addressable memory structure that associates a
relationship between the set of input patterns and
output patterns.
A content addressable memory structure is a kind of
memory structure that enables the recollection of data
based on the intensity of similarity between the input
pattern and the patterns stored in the memory.
 Hopfield Network
Hopfield network is a special kind of neural network
whose response is different from other neural networks.
It is calculated by converging iterative process.
 It has just one layer of neurons relating to the size of the
input and output, which must be the same.
When such a network recognizes, for example, digits, we
present a list of correctly rendered digits to the network.
Subsequently, the network can transform a noise input to
the relating perfect output.
 Hopfield Network
A Hopfield network is a single-layered and recurrent
network in which the neurons are entirely connected,
i.e., each neuron is associated with other neurons.
If there are two neurons i and j, then there is a
connectivity weight wij lies between them which is
symmetric wij = wji .
With zero self-connectivity, Wii =0 is given below. Here,
the given three neurons having values i = 1, 2, 3 with
values Xi=±1 have connectivity weight Wij.
Hopfield Network
 Applications of Neural Networks
 Electronics − Code sequence prediction, IC chip layout, chip failure analysis, machine vision, voice
synthesis.
 Financial − Real estate appraisal, loan advisor, mortgage screening, corporate bond rating, portfolio
trading program, corporate financial analysis, currency value prediction, document readers, credit
application evaluators.
 Industrial − Manufacturing process control, product design and analysis, quality inspection systems,
welding quality analysis, paper quality prediction, chemical product design analysis, dynamic modeling
of chemical process systems, machine maintenance analysis, project bidding, planning, and
management.
 Medical − Cancer cell analysis, EEG and ECG analysis, prosthetic design, transplant time optimizer.
 Speech − Speech recognition, speech classification, text to speech conversion.
 Telecommunications − Image and data compression, automated information services, real-time spoken
language translation.
 Transportation − Truck Brake system diagnosis, vehicle scheduling, routing systems.
 OTHER APPLICATIONS OF NEURAL NETWORKS

• Software − Pattern Recognition in facial recognition, optical character recognition, etc.

• Time Series Prediction − ANNs are used to make predictions on stocks and natural calamities.
• Signal Processing − Neural networks can be trained to process an audio signal and filter it
appropriately in the hearing aids.
• Control − ANNs are often used to make steering decisions of physical vehicles.
• Anomaly Detection − As ANNs are expert at recognizing patterns, they can also be trained to
generate an output when something unusual occurs that misfits the pattern
• Aerospace − Autopilot aircrafts, aircraft fault detection.
• Automotive − Automobile guidance systems.
• Military − Weapon orientation and steering, target tracking, object discrimination, facial
recognition, signal/image identification.
Q &A
Thank you.