UNIT 4: Artificial Neural Networks

Types of Artificial Neural Network:

4.1 Artificial Neural Networks (ANNs)
There are two Artificial Neural Network topologies.
 An ANN is the piece of a computing system designed to 1. Feedforward ANN: In this ANN, the information
simulate the way the human brain analyzes and flow is unidirectional. A unit sends information to other
processes information. unit from which it does not receive any information.
 It is the foundation of artificial intelligence (AI) and There are no feedback loops. They are used in pattern
solves problems that would prove impossible or generation/recognition/classification. They have fixed
difficult by human or statistical standards. inputs and outputs.

 ANNs have self-learning capabilities that enable them

to produce better results as more data becomes
available.
 The term "Artificial Neural Network" is derived from
Biological neural networks that develop the structure
of a human brain.
 Similar to the human brain that has neurons
interconnected to one another, artificial neural
networks also have neurons that are interconnected to
one another in various layers of the networks. These 2. Feedback ANN: Here, feedback loops are allowed.
neurons are known as nodes. They are used in content addressable memories.

Biological Neural Artificial Neural

Network Network

Dendrites Inputs

Cell nucleus Nodes

Synapse Weights

Axon Output

Architecture of an ANN:

Artificial Neural Network primarily consists of three layers: Advantages of Artificial Neural Network (ANN)
1. Input Layer: As the name suggests, it accepts inputs 1. Parallel processing: Artificial neural networks have
in several different formats provided by the a numerical value that can perform more than one task
programmer. simultaneously.
2. Hidden Layer: The hidden layer presents in-between
input and output layers. It performs all the 2. Storing data on the network: Data that is used in
calculations to find hidden features and patterns. traditional programming is stored on the whole
network, not on a database. The disappearance of a
3. Output Layer: The input goes through a series of couple of pieces of data in one place doesn't prevent
transformations using the hidden layer, which finally the network from working.
results in output that is conveyed using this layer.
3. Incomplete knowledge: After ANN training, the
information may produce output even with inadequate
data. The loss of performance here relies upon the
significance of missing data.
4. Fault tolerance: Extortion of one or more cells of
ANN does not prohibit it from generating output, and
this feature makes the network fault-tolerance.
  Software: Pattern Recognition in facial recognition,
Disadvantages of Artificial Neural Network: optical character recognition, etc.
1. Assurance of proper network structure: There is no
particular guideline for determining the structure of
artificial neural networks. The appropriate network
structure is accomplished through experience, trial,
and error.

2. Hardware dependence: Artificial neural networks

need processors with parallel processing power, as per
their structure. Therefore, the realization of the
equipment is dependent.

3. Unrecognized behaviour of the network: When

ANN produces a testing solution, it does not provide
insight concerning why and how. It decreases trust in
the network.
4. Difficulty of showing the issue to the network:
ANNS can work with numerical data. Problems must be
converted into numerical values before being
introduced to ANN. The presentation mechanism to be
resolved here will directly impact the performance of
the network. It relies on the user's abilities.

5. The duration of the network is unknown: The

network is reduced to a specific value of the error, and
this value does not give us optimum results.

Applications of Artificial Neural Networks

 Aerospace: Autopilot aircrafts, aircraft fault

detection.
 Automotive: Automobile guidance systems.
 Military: Weapon orientation and steering, target
tracking, object discrimination, facial recognition,
signal/image identification.
 Electronics: Code sequence prediction, IC chip
layout, chip failure analysis, machine vision, voice
synthesis.
Financial: Real estate appraisal, loan advisor,
mortgage screening, corporate bond rating,
portfolio trading program, value prediction,
document readers, credit application evaluators.
corporate financial analysis, currency

Medical: Cancer cell analysis, EEG and ECG analysis.

prosthetic design, transplant time optimizer.
Speech: Speech recognition, speech classification,
text to speech conversion.
 Telecommunications: Image and data compression,
automated information services, real-time spoken
language translation. Transportation - Truck Brake
system diagnosis, vehicle scheduling, routing
systems.
4.2 Deep Learning

Deep Learning is a part of machine learning, which is a

subset of Artificial Intelligence. Deep Learning is a subfield
of machine learning concerned with algorithms inspired
by the structure and function of the brain called artificial
neural networks. It is useful in processing Big Data and
can create important patterns that provide valuable
insight into important decision making.

Deep learning is a branch of machine learning which is

completely based on artificial neural networks, as neural
network is going to mimic the human brain so deep
learning is also a kind of mimic of human brain.

Working:

1. First, we need to identify the actual problem in

order to get the right solution and it should be
understood, the feasibility of the Deep Learning
should also be checked (whether it should fit Deep
Learning or not).

2. Second, we need to identify the relevant data which

should correspond to the actual problem and should
be prepared accordingly.
3. Third, Choose the Deep Learning Algorithm
appropriately.
4. Fourth, Algorithm should be used while training the
dataset.

5. Fifth, Final testing should be done on the dataset.

 The solution to these issues is the Recurrent Neural
Languages used: Network (RNN). An RNN can handle sequential data,
accepting the current input data, and previously received
R, Python, Matlab, CPP, Java, Julia, Lisp, Java Script, etc. inputs. RNNs can memorize previous inputs due to their
internal memory.
Advantages:

 Best in-class performance on problems.

 Reduces need for feature engineering.
 Eliminates unnecessary costs.
 Identifies defects easily that are difficult to detect.

Disadvantages:

× Large amount of data required.

× Computationally expensive to train.
× No strong theoretical foundation.

Applications:
1. Automatic Text Generation: Corpus of text is
learned and from this model new text is generated,
word-by-word or character-by-character.
2. Healthcare: Helps in diagnosing various diseases
and treating it.
3. Automatic Machine Translation: Certain words,
sentences or phrases in one language is
transformed into another language (Deep Learning
is achieving top results in the areas of text, images).
4. Image Recognition: Recognizes and identifies
peoples and objects in images as well as to
understand content and context. This area is
already being used in Gaming, Retail, Tourism, etc.

5. Predicting Earthquakes: Teaches a computer to

perform viscoelastic computations which are used
in predicting earthquakes.

4.3 Recurrent Neural Networks (RNNs)

Recurrent Neural Network(RNN) are a type of Neural
Network where the output from previous step are fed as
input to the current step. In traditional neural networks,
all the inputs and outputs are independent of each other,
but in cases like when it is required to predict the next
word of a sentence, the previous words are required and
hence there is a need to remember the previous words.

Why Recurrent Neural Networks?

Recurrent neural networks were created because there

were a few issues in the feed-forward neural network:

× Cannot handle sequential data

× Considers only the current input
× Cannot memorize previous inputs
Types of Recurrent Neural Networks  These convolutional layers create feature maps that
There are four types of Recurrent Neural Networks: record a region of image which is ultimately broken
1. One to One RNN: This type of neural network is into rectangles and sent out for nonlinear
known as the Vanilla Neural Network. It's used for processing.
general machine learning problems, which has a
single input and a single output.

2. One to Many RNN: This type of neural network has

a single input and multiple outputs. An example of
this is the image caption.
3. Many to One RNN: This RNN takes a sequence of
inputs and generates a single output. Sentiment
analysis is a good example of this kind of network
where a given sentence can be classified as
expressing positive or negative sentiments.

4. Many to Many RNN: This RNN takes a sequence of

inputs and generates a sequence of outputs.
Machine translation is one of the examples.

Applications of Recurrent Neural Networks

 Image Captioning: RNNs are used to caption an
image by analysing the activities present.
 Time Series Prediction: Any time series problem,
like predicting the prices of stocks in a particular
month, can be solved using an RNN.
 Next Word Prediction: Taking a sequence of words
as input, we try to predict the possibility of the next
word.

 Machine Translation: Given an input in one

language, RNNs can be used to translate the input
into different languages as output.

4.4 Convolutional Neural Networks (CNNs)

 CNNs are class of Artificial Neural Networks which
are used to process images. They can detect image
features such as bright or dark or specific color
spots edges in various orientations , patterns and
others.

So, if an image is input to a CNN, it should rightly

recognize it . So, CNN needs to be trained with all
such images using whose features it can rightly
recognize the input image.

Convolutional neural networks can operate directly on

a raw image and do not need any pre-processing.

 Convolutional neural networks (CNN) are one of the

most popular models used today.
 This neural network computational model uses a
variation of multilayer perceptrons and contains
one or more convolutional layers that can be either
entirely connected or pooled.
  Computer Vision
Comparison of Different Layers  Natural Language processing
 Financial time series
There are three types of layers in a CNN. Each of these
layers has different parameters that can be optimized and  Autonomous Cars
performs a different task on the input data.

1. Convolutional layers: are the layers where filters are

applied to the original image, or to other feature maps
in a deep CNN. This is where most of the user-specified
parameters are in the network. The most important
parameters are the number of kernels and the size of
the kernels.

2. Pooling Layers: They are used to reduce the

dimensions of the feature maps. This layer summarises
the features present in a region of the feature map
generated by a convolution layer.
3. Fully connected layers: They are placed before the
classification output of a CNN and are used to flatten
the results before classification.

What do CNN layers learn?

1. Each CNN layer learns filters of increasing

complexity.
2. The first layers learn basic feature detection filters:
edges, corners, etc
3. The middle layers learn filters that detect parts of
objects. For faces, they might learn to respond to
eyes, noses, etc
4. The last layers have higher representations: they
learn to recognize full objects, in different shapes
and positions.

Advantages

 Very High accuracy in image recognition problems.

 Automatically detects the important features
without any human supervision.
 Weight sharing.

Disadvantages

× CNN do not encode the position and orientation of

object.
× Lack of ability to be spatially invariant to the input
data.
× Lots of training data is required.

Applications of CNN

 Image and Video recognition

 Recommender systems
 Image classification
 Medical Image analysis
 18  On the other hand, they typically do not provide a
construction for the weights, but merely state that
CNN RNN
such a construction is possible.
CNN stands for RNN stands for Recurrent
Convolutional Neural Neural Network
Network
CNN considered to be more RNN includes less feature
potent compatibility
Ideal for images and video Ideal for text and speech
processing Analysis
Suitable for spatial data like RNN is used for temporal
images data
Takes fixed-size inputs and RNN can handle arbitrary
generates fixed size output input/ output lengths

4.5 Universal Approximation Theorem

 The Universal Approximation Theorem states that

a neural network with 1 hidden layer can
approximate any continuous function for inputs
within a specific range.

 Mathematically speaking, any neural network

architecture aims mathematical function y= f(x)
that can map attributes(x) to output(y).
 The accuracy of this function i.e. mapping differs
depending on the distribution of the dataset and
the architecture of the network employed.
 The function f(x) can be arbitrarily complex. The
Universal Approximation Theorem tells us that
Neural Networks has a kind of universality i.e. no
matter what f(x) is, there is a network that can
approximately approach the result and do the job.
This result holds for any number of inputs and
outputs.

In the mathematical theory of artificial neural

networks, universal approximation theorems are
results that establish the density of an
algorithmically generated class of functions within
a given function space of interest.

Most universal approximation theorems can be

parsed into two classes. The first quantifies the
approximation capabilities of neural networks with
an arbitrary number of artificial neurons
("arbitrary width" case) and the second focuses on
the case with an arbitrary at finding any number of
hidden layers, each containing a limited number of
artificial neurons ("arbitrary depth" case).

 Universal approximation theorems imply that

neural networks can represent a wide variety of
interesting functions when given appropriate
weights.
 Applications of GANs:
4.6 Generative Adversarial Networks (GANs) 1. Photo to photo translation.
2. Deep Fake
 Generative Adversarial Networks (GANs) were
3. Voice Cloning
introduced in 2014 by Ian J. Goodfellow and co-
4. Generate Cartoon Character
authors.
5. Face Aging
 GANs perform unsupervised learning tasks in
6. Super-resolution
machine learning. It consists of 2 models that
7. 3D object generation
automatically discover and learn the patterns in
input data. 8. Clothing translation
The two models are known as Generator and 9. Photograph editing

Discriminator. 10.Image enhancement

 They compete with each other to scrutinize,

capture, and replicate the variations within a
dataset. GANs can be used to generate new
examples that plausibly could have been drawn
from the original dataset.

1. Generator

A Generator in GANs is a neural network that creates

fake data to be trained on the discriminator. The main
aim of the Generator is to make the discriminator
classify its output as real.

2. Discriminator

 The Discriminator is a neural network that identifies

real data from the fake data created by the
Generator. The discriminator's training data comes
from different two sources:

 The real data instances, such as real pictures of

birds, humans, currency notes, etc., are used by the
Discriminator as positive samples during training.
 The fake data instances created by the Generator
are used as negative examples during the training
process.

How does GANs work?

1. GANs consists of two neural networks. There is a

Generator G(x) and a Discriminator D(x). Both of
them play an adversarial game.
2. The generator's aim is to fool the discriminator by
producing data that are similar to those in the
training set.

3. The discriminator will try not to be fooled by

identifying fake data from real data. Both of them
work simultaneously to learn and train complex
data like audio, video, or image files.
4. The Generator network takes a sample and
generates a fake sample of data. The Generator is
trained to increase the Discriminator network's
probability of making mistakes.
Steps for Training GAN

1. Define the problem

2. Choose the architecture of GAN
3. Train discriminator on real data
4. Generate fake inputs for the generator
5. Train discriminator on fake data
6. Train generator with the output of the discriminator