1.

Introduction
Neural network is the fusion of artificial intelligence and brain-inspired
design that reshapes modern computing. With intricate layers of
interconnected artificial neurons, these networks emulate the intricate
workings of the human brain, enabling remarkable feats in machine
learning. There are different types of neural networks, from feed
forward to recurrent and convolutional, each tailored for specific tasks.
This article covers its real-world applications across industries
like image recognition, natural language processing, and more. Read on
to know everything about neural network in machine learning!
Neural networks mimic the basic functioning of the human brain and
are inspired by how the human brain interprets information. They solve
various real-time tasks because of its ability to perform computations
quickly and its fast responses.
Artificial Neural Network has a huge number of interconnected
processing elements, also known as Nodes. These nodes are connected
with other nodes using a connection link. The connection link contains
weights, these weights contain the information about the input signal.
Each iteration and input in turn leads to updation of these weights.
After inputting all the data instances from the training data set, the final
weights of the Neural Network along with its architecture is known as
the Trained Neural Network.
2.History
In 1943, neurophysiologist Warren McCulloch and mathematician
Walter Pitts wrote a paper on how neurons might work. In order to
describe how neurons in the brain might work, they modeled a simple
neural network using electrical circuits.

In 1949, Donald Hebb wrote The Organization of Behavior, a work

which pointed out the fact that neural pathways are strengthened each
time they are used, a concept fundamentally essential to the ways in
which humans learn. If two nerves fire at the same time, he argued, the
connection between them is enhanced.

As computers became more advanced in the 1950's, it was finally

possible to simulate a hypothetical neural network. The first step
towards this was made by Nathanial Rochester from the IBM research
laboratories. Unfortunately for him, the first attempt to do so failed.

In 1959, Bernard Widrow and Marcian Hoff of Stanford developed

models called "ADALINE" and "MADALINE." In a typical display of
Stanford's love for acronymns, the names come from their use of
Multiple ADAPTIVE LINEAR Elements. ADALINE was developed to
recognize binary patterns so that if it was reading streaming bits from a
phone line, it could predict the next bit. MADALINE was the first neural
network applied to a real world problem, using an adaptive filter that
eliminates echoes on phone lines. While the system is as ancient as air
traffic control systems, like air traffic control systems, it is still in
commercial use.

In 1962, Widrow & Hoff developed a learning procedure that examines

the value before the weight adjusts it according to the rule: Weight
Change = (Pre-Weight line value) (Error / (Number of Inputs)). It is
based on the idea that while one active perceptron may have a big
error, one can adjust the weight values to distribute it across the
network, or at least to adjacent perceptrons. Applying this rule still
results in an error if the line before the weight is 0, although this will
eventually correct itself. If the error is conserved so that all of it is
distributed to all of the weights than the error is eliminated.

Despite the later success of the neural network, traditional von

Neumann architecture took over the computing scene, and neural
research was left behind. Ironically, John von Neumann himself
suggested the imitation of neural functions by using telegraph relays or
vacuum tubes.

3.Types of Neural Networks

There are many types of neural networks available or that might be in
the development stage. They can be classified depending on their
Structure, Data flow, Neurons used and their density, Layers and their
depth activation filters etc. Also, learn about the Neural network in R to
further your learning.

A. Perceptron

Perceptron model, proposed by Minsky-Papert is one of the simplest

and oldest models of Neuron. It is the smallest unit of neural network
that does certain computations to detect features or business
intelligence in the input data. It accepts weighted inputs, and apply the
activation function to obtain the output as the final result. Perceptron is
also known as TLU(threshold logic unit).

Perceptron is a supervised learning algorithm that classifies the data

into two categories, thus it is a binary classifier. A perceptron separates
the input space into two categories by a hyperplane represented by the
following equation:

B. Feed Forward Neural Networks

The simplest form of neural networks where input data travels in one
direction only, passing through artificial neural nodes and exiting
through output nodes. Where hidden layers may or may not be
present, input and output layers are present there. Based on this, they
can be further classified as a single-layered or multi-layered feed-
forward neural network.

Number of layers depends on the complexity of the function. It has uni-

directional forward propagation but no backward propagation. Weights
are static here. An activation function is fed by inputs which are
multiplied by weights. To do so, classifying activation function or step
activation function is used. For example: The neuron is activated if it is
above threshold (usually 0) and the neuron produces 1 as an output.
The neuron is not activated if it is below threshold (usually 0) which is
considered. They are fairly simple to maintain and are equipped with to
deal with data which contains a lot of noise.

C. Multilayer Perceptron

An entry point towards complex neural nets where input data travels
through various layers of artificial neurons. Every single node is
connected to all neurons in the next layer which makes it a fully
connected neural network. Input and output layers are present having
multiple hidden Layers i.e. at least three or more layers in total. It has a
bi-directional propagation i.e. forward propagation and backward
propagation.

Inputs are multiplied with weights and fed to the activation function
and in back propagation, they are modified to reduce the loss. In simple
words, weights are machine learnt values from Neural Networks. They
self-adjust depending on the difference between predicted outputs vs
training inputs. Nonlinear activation functions are used followed by
softmax as an output layer activation function.

D. Convolutional Neural Network

Convolution neural network contains a three-dimensional arrangement

of neurons instead of the standard two-dimensional array. The first
layer is called a convolutional layer. Each neuron in the convolutional
layer only processes the information from a small part of the visual
field. Input features are taken in batch-wise like a filter. The network
understands the images in parts and can compute these operations
multiple times to complete the full image processing. Processing
involves conversion of the image from RGB or HSI scale to grey-scale.

Propagation is uni-directional where CNN contains one or more

convolutional layers followed by pooling and bidirectional where the
output of convolution layer goes to a fully connected neural network
for classifying the images as shown in the above diagram. Filters are
used to extract certain parts of the image. In MLP the inputs are
multiplied with weights and fed to the activation function. Convolution
uses RELU and MLP uses nonlinear activation function followed by
softmax. Convolution neural networks show very effective results in
image and video recognition, semantic parsing and paraphrase
detection.

E. Radial Basis Function Neural Networks

Radial Basis Function Network consists of an input vector followed by a

layer of RBF neurons and an output layer with one node per category.
Classification is performed by measuring the input’s similarity to data
points from the training set where each neuron stores a prototype. This
will be one of the examples from the training set.

When a new input vector [the n-dimensional vector that you are trying
to classify needs to be classified, each neuron calculates the Euclidean
distance between the input and its prototype. For example, if we have
two classes i.e. class A and Class B, then the new input to be classified is

Each RBF neuron compares the input vector to its prototype and
outputs a value ranging which is a measure of similarity from 0 to 1. As
the input equals to the prototype, the output of that RBF neuron will be
1 and with the distance grows between the input and prototype the
response falls off exponentially towards 0. The curve generated out of
neuron’s response tends towards a typical bell curve.

F. Recurrent Neural Networks

Designed to save the output of a layer, Recurrent Neural Network is fed

back to the input to help in predicting the outcome of the layer. The
first layer is typically a feed forward neural network followed by
recurrent neural network layer where some information it had in the
previous time-step is remembered by a memory function. Forward
propagation is implemented in this case. It stores information required
for it’s future use. If the prediction is wrong, the learning rate is
employed to make small changes. Hence, making it gradually increase
towards making the right prediction during the back propagation.

4.Advantages
Ability to Learn Complex Patterns: Neural networks can learn intricate
patterns and relationships within data, making them effective for tasks
like image recognition, natural language processing, and time-series
prediction.

Adaptability: They can adapt and learn from new data, making them
suitable for dynamic environments or tasks where the underlying
patterns may change over time.
Parallel Processing: Neural networks can perform computations in
parallel, leveraging GPUs and specialized hardware to accelerate
training and inference, leading to faster processing times.
Feature Learning: They can automatically extract relevant features
from raw data, reducing the need for manual feature engineering,
which can be time-consuming and domain-specific.
Generalization: Neural networks can generalize well to unseen data,
provided they are properly trained and validated, making them robust
in handling diverse inputs.
Non-linear Relationships: They can model complex non-linear
relationships between inputs and outputs, allowing them to capture
intricate dependencies in data.

5.Disadvantages
Complexity: Neural networks can be complex and challenging to
understand, particularly deep neural networks with many layers. This
complexity can make debugging and interpreting model decisions
difficult.

Data Dependency: They require large amounts of labeled data for

training, which may not always be available or costly to obtain,
especially in specialized domains.

Computational Resources: Training deep neural networks often

demands significant computational resources, including high-
performance GPUs or TPUs, and can be time-consuming.

Over fitting: Neural networks are prone to overfitting, where the model
learns to memorize the training data rather than generalize to new
data. Techniques such as regularization and dropout are often used to
mitigate this issue.

Hyperparameter Sensitivity: They require careful tuning of

hyperparameters, such as learning rate, batch size, and network
architecture, which can significantly impact performance and require
extensive experimentation.

Black Box Nature: Neural networks are often considered "black box"
models, meaning that understanding how they arrive at a particular
decision can be challenging, leading to concerns about interpretability
and trustworthiness, especially in critical applications like healthcare or
finance.

6.Conclusion
Neural network is a vast subject. Many data scientists solely focus only
on Neural network techniques. In this session, we practiced the
introductory concepts only. Neural Networks has much more advanced
techniques. There are many algorithms other than back propagation.
Neural networks particularly work well on some particular class of
problems like image recognition. The neural network algorithms are
very calculation intensive. They require highly efficient computing
machines. Large datasets take a significant amount of runtime on R. We
need to try different types of options and packages. Currently, there is a
lot of exciting research going on, around neural networks. After gaining
sufficient knowledge in this basic session, you may want to explore
reinforced learning, deep learning.

7.Reference
1. Shao, Feng; Shen, Zheng (9 January 2022). "How can artificial
neural networks approximate the brain?". Front Psychol. 970214
^ Levitan, Irwin; Kaczmarek, Leonard (August 19, 2015).
"Intercellular communication". The Neuron: Cell and Molecular
Biology (4th ed.). New York, NY: Oxford University Press. pp. 153–
328. ISBN 978-0199773893.
2. ^ Jump up to:a b Rosenblatt, F. (1958). "The Perceptron: A
Probabilistic Model For Information Storage And Organization In
The Brain". Psychological Review.
3. ^ Bishop, Christopher M. (2006-08-17). Pattern Recognition and
Machine Learning. New York: Springer. ISBN 978-0-387-31073-2.
4. ^ Vapnik, Vladimir N.; Vapnik, Vladimir Naumovich (1998). The
nature of statistical learning theory (Corrected 2nd print. ed.).
New York Berlin Heidelberg: Springer. ISBN 978-0-387-94559-0.
5. ^ Bain (1873). Mind and Body: The Theories of Their Relation.
New York: D. Appleton and Company.
6. ^ James (1890). The Principles of Psychology. New York: H. Holt
and Company.