Neural networks

Introduction
An Artificial Neural Network (ANN) models the relationship between a set of input signals
and an output signal using a model derived from our understanding of how a biological brain
responds to stimuli from sensory inputs. Just as a brain uses a network of interconnected cells
called neurons to create a massive parallel processor, ANN uses a network of artificial
neurons or nodes to solve learning problems.
Biological motivation
Let us examine how a biological neuron function. Fig 1 gives a schematic representation of
the functioning of a biological neuron. In the cell, the incoming signals are received by the
cell’s dendrites through a biochemical process. The process allows the impulse to be
weighted according to its relative importance or frequency. As the cell body begins
accumulating the incoming signals, a threshold is reached at which the cell fires and the
output signal is transmitted via an electrochemical process down the axon. At the axon’s
terminals, the electric signal is again processed as a chemical signal to be passed to the
neighbouring neurons across a tiny gap known as a synapse.
Biological learning systems are built of very complex webs of interconnected neurons. The
human brain has an interconnected network of approximately 1011 neurons, each connected,
on an average, to 104 other neurons. Even though the neuron switching speeds are much
slower than computer switching speeds, we are able to take complex decisions relatively
quickly. Because of this, it is believed that the information processing capabilities of
biological neural systems is a consequence of the ability of such systems to carry out a huge
number of parallel processes distributed over many neurons. The developments in ANN
systems are motivated by the desire to implement this kind of highly parallel computation
using distributed representations.
Artificial neurons
Definition
An artificial neuron is a mathematical function conceived as a model of biological neurons.
Artificial neurons are elementary units in an artificial neural network. The artificial neuron
receives one or more inputs (representing excitatory postsynaptic potentials and inhibitory
postsynaptic potentials at neural dendrites) and sums them to produce an output. Each input is
separately weighted, and the sum is passed through a function known as an activation
function or transfer function.
The small circles in the schematic representation of the artificial neuron shown in Figure 9.3
are called the nodes of the neuron. The circles on the left side which receives the values of
x0, x1, . . . , xn are called the input nodes and the circle on the right side which outputs the
value of y is called output node. The squares represent the processes that are taking place
before the result is outputted.
They need not be explicitly shown in the schematic representation. In below Figure it shows a
simplified representation of an artificial neuron.
What Is a Neural Network?

To understand how an artificial neuron works, we should first understand how a biological
neuron works.

Dendrites

These receive information or signals from other neurons that get connected to it.

Cell Body

Information processing happens in a cell body. These take in all the information coming from
the different dendrites and process that information.

Axon

It sends the output signal to another neuron for the flow of information. Here, each of the
flanges connects to the dendrite or the hairs on the next one.

The image shown below depicts an ANN.

The network starts with an input layer that receives input in data form.

The lines connected to the hidden layers are called weights, and they add up on the hidden
layers. Each dot in the hidden layer processes the inputs, and it puts an output into the next
hidden layer and, lastly, into the output layer.

Looking at the above two images, you can observe how an ANN replicates a biological
neuron.

 Input to a neuron - input layer

 Neuron - hidden layer

 Output to the next neuron - output layer

A neural network is a system of hardware or software patterned after the operation of neurons
in the human brain. Neural networks, also called artificial neural networks, are a means of
achieving deep learning.

When you want to figure out how a neural network functions, you need to look at neural
network architecture.

The Architecture of a Neural Network

Have you ever asked Siri a question? The device answers accurately. Let’s take a closer look
and see how the virtual assistant accomplishes this feat of speech recognition.

There are five recognized types of neural networks.

 Single-layer feed-forward network.

 Multilayer feed-forward network.

 Single node with its own feedback.

 Single-layer recurrent network.

 Multilayer recurrent network.

Consider, for instance, the neural network shown below, an example of a single-layer
recurrent network:

There are input, hidden, and output layers on the network. But, first, the network needs to
recognize the sentence: What is the time?

Here, each word comes in as a pattern of sound. Then, the sentence gets sampled into discrete
sound waves.

Let's consider the first word: What

You can see the waveform is split based on every letter. Now we will split the sound wave
for the letter W into smaller segments.

The amplitude varies in the sound wave when we analyze the letter 'W,' as shown below.

We collect the values at intervals and form an array. Then, different amplitudes come in for
other letters, and we feed the variety of amplitudes to the input layer.
Random weights get assigned to each interconnection between the input and hidden layers.

We always start with the random key, as assigning a preset value to the weights takes a
significant amount of time when training the model.

The weights get multiplied with the inputs, and a bias is added to form the transfer function.
Weights get assigned to the interconnection between the hidden layers. The output of the
transfer function is fed as an input to the activation function. The work from one hidden layer
becomes the input to the next.

The acoustic model contains the statistical representation of each sound that makes a word.
So we start building these acoustic models, and as these layers separate them, they'll start
learning what the different models represent for other letters.

The lexicon contains the data for different pronunciations of every word. The lexicon is at the
end, where we end up with the ABCD, identifying the other letters there.

Finally, we get our output letter. Following the same process for every word and letter, the
neural network recognizes the sentence you said or your question.

Note that the terms “acoustic model” and “lexicon” are specific to the domain of
understanding speech. When dealing with other input formats, you'll have different labels, but
the process remains the same.

How do neural networks learn?

Typically, an ANN is initially trained or fed large amounts of data. Training consists of
providing input and telling the network what the output should be. For example, to build a
network that identifies the faces of actors, the initial training might be a series of pictures,
including actors, non-actors, masks, statues and animal faces. Each input is accompanied by
matching identification, such as actors' names or "not actor" or "not human" information.
Providing the answers allows the model to adjust its internal weightings to do its job better.

For example, if nodes David, Dianne and Dakota tell node Ernie the current input image is a
picture of Brad Pitt, but node Durango says it's George Clooney, and the training program
confirms it's Pitt, Ernie decreases the weight it assigns to Durango's input and increase the
weight it gives to David, Dianne and Dakota.

In defining the rules and making determinations -- the decisions of each node on what to send
to the next tier based on inputs from the previous tier -- neural networks use several
principles. These include gradient-based training, fuzzy logic, genetic algorithms and
Bayesian methods. They might be given some basic rules about object relationships in the
data being modeled.

For example, a facial recognition system might be instructed, "Eyebrows are found above
eyes," or, "Moustaches are below a nose. Moustaches are above and/or beside a mouth."
Preloading rules can make training faster and the model more powerful faster. But it also
includes assumptions about the nature of the problem, which could prove to be either
irrelevant and unhelpful or incorrect and counterproductive, making the decision about what,
if any, rules to build in important.

Further, the assumptions people make when training algorithms cause neural networks to
amplify cultural biases. Biased data sets are an ongoing challenge in training systems that
find answers on their own through pattern recognition in data. If the data feeding the
algorithm isn't neutral -- and almost no data is -- the machine propagates bias.

Types of neural networks

Neural networks are sometimes described in terms of their depth, including how many layers
they have between input and output, or the model's so-called hidden layers. This is why the
term neural network is used almost synonymously with deep learning. They can also be
described by the number of hidden nodes the model has or in terms of how many input layers
and output layers each node has. Variations on the classic neural network design enable
various forms of forward and backward propagation of information among tiers.

Specific types of artificial neural networks include the following:

Feed-forward neural networks

One of the simplest variants of neural networks, these pass information in one direction,
through various input nodes, until it makes it to the output node. The network might or might
not have hidden node layers, making their functioning more interpretable. It's prepared to
process large amounts of noise.

This type of ANN computational model is used in technologies such as facial recognition
and computer vision.

Recurrent neural networks (RNNs)

More complex in nature, RNNs save the output of processing nodes and feed the result back
into the model. This is how the model learns to predict the outcome of a layer. Each node in
the RNN model acts as a memory cell, continuing the computation and execution of
operations.

This neural network starts with the same front propagation as a feed-forward network but
then goes on to remember all processed information to reuse it in the future. If the network's
prediction is incorrect, then the system self-learns and continues working toward the correct
prediction during backpropagation.

This type of ANN is frequently used in text-to-speech conversions.

Convolutional neural networks (CNNs)

CNNs are one of the most popular models used today. This computational model uses a
variation of multilayer perceptrons and contains one or more convolutional layers that can be
either entirely connected or pooled. These convolutional layers create feature maps that
record a region of the image that's ultimately broken into rectangles and sent out for nonlinear
processing.
The CNN model is particularly popular in the realm of image recognition. It has been used in
many of the most advanced applications of AI, including facial recognition, text digitization
and NLP. Other use cases include paraphrase detection, signal processing and image
classification.

Deconvolutional neural networks

Deconvolutional neural networks use a reversed CNN model process. They try to find lost
features or signals that might have originally been considered unimportant to the CNN
system's task. This network model can be used in image synthesis and analysis.

Modular neural networks

These contain multiple neural networks working separately from one another. The networks
don't communicate or interfere with each other's activities during the computation process.
Consequently, complex or big computational processes can be performed more efficiently.

Advantages of Neural Network

ANN outputs aren't limited entirely by inputs and results given to them initially by an expert
system. This ability comes in handy for robotics and pattern recognition systems.

This neural network has the potential for high fault tolerance and can debug or diagnose a
network on its own. ANN can go through thousands of log files from a company and sort
them out. It is currently a tedious task done by administrators, but it will save a significant
amount of time, energy, and resources if it can be automated.

Nonlinear systems can find shortcuts to reach computationally expensive solutions. We see
this in the banking industry, for example, where they work on a particular Excel spreadsheet,
and as time goes by, start building codes around it. In over 20 years, they might create a
repertoire of all these functions, and the neural network rapidly comes up with the same
answers otherwise done in days, weeks, or even a month, when done by a large bank.

Applications of Neural Network

With an enormous number of applications implementations every day, now is the most
appropriate time to know about the applications of neural networks, machine learning, and
artificial intelligence. Some of them are discussed below:

Handwriting Recognition

Neural networks are used to convert handwritten characters into digital characters that a
machine can recognize.

Stock-Exchange prediction

The stock exchange is affected by many different factors, making it difficult to track and
difficult to understand. However, a neural network can examine many of these factors and
predict the prices daily, which would help stockbrokers.

Currently, this operation is still in its initial phases. However, you should know that over
three terabytes of data a day are generated from the United States stock exchange alone.
That's a lot of data to dig through, and you must sort it out before you start focusing on even a
single stock.

Traveling Issues of sales professionals

This application refers to finding an optimal path to travel between cities in a given area.
Neural networks help solve the problem of providing higher revenue at minimal costs.
However, the Logistical considerations are enormous, and we must find optimal travel paths
for sales professionals moving from town to town.

Image compression

The idea behind neural network data compression is to store, encrypt, and recreate the actual
image again. Therefore, we can optimize the size of our data using image compression neural
networks. It is the ideal application to save memory and optimize it.
Future of Neural Networks

 With the rapid pace that AI and machine learning are being adopted by companies today,
we could see more advancements in the applications of neural networks in the foreseeable
future. AI and machine learning will offer a wealth of personalized choices for users
worldwide. For example, all mobile and web applications try to give you an enhanced
customized experience based on your search history, and neural networks can make that
possible.
 Hyper-intelligent virtual assistants will make life easier. If you have ever used Google
assistant, Siri, or any other products, you can see how they're slowly evolving. They may
even predict your email responses in the future!
 Neural networks will be a lot faster in the future, and neural network tools can get
embedded in every design surface. We already have a little mini neural network that
plugs into an inexpensive processing board or even into your laptop. Instead of the
software, focusing on the hardware would make such devices even faster.
 Neural networks will also find their way into the fields of medicine, agriculture, physics,
research, and anything else you can imagine. Neural networks will also find its way into
the fields of medicine, agriculture, physics, research, and anything else you can imagine.