Artificial neural networks

Table of Contents
Introduction......................................................................................................................................2
Literature Review............................................................................................................................2
Methodology of ANNs:...............................................................................................................2
The architecture of ANNs:...........................................................................................................3
Types, Techniques and algorithms of ANNs:..............................................................................4
Learning types in ANNs:.............................................................................................................5
Classification of ANNs:...............................................................................................................5
Deep learning...............................................................................................................................6
Application and significance of ANNs:.......................................................................................6
Conclusion.......................................................................................................................................8
References........................................................................................................................................8

1
Introduction
A neural network is a mathematical model which animates a biological neuron consisting of
artificial neurons forming an inter-connected network in order to perform computation. Recently,
neural networks have become an area of exceptional opportunity for research, development, and
application in solving various real-world problems, exhibiting unique characteristics and
capabilities not found in other technologies. Since humans excel at performing tasks that involve
intelligence or pattern recognition, which are often challenging to automate, it is essential to
understand the functionality of tasks being performed by humans and imitate them in computing
systems (Mahanta 2017). Artificial neural networks (ANNs) are modeled after the structure and
function of the human brain. Just like the brain, ANNs are composed of interconnected
processing nodes or "neurons" that work together to solve problems. Each artificial neuron
receives input from other neurons or the external environment, processes that input using a set of
mathematical functions, and produces an output signal that is transmitted to other neurons
(Kukreja et al, 2016). The working of ANNs closely mimics how biological neurons in the brain
receive and transmit electrical and chemical signals. Furthermore, just as the brain learns and
adapts over time through experience, ANNs can be trained using large datasets to improve their
performance on specific tasks. Overall, the design and operation of ANNs is highly motivated by
the complex and dynamic workings of the human brain.

The purpose of this report is to discuss in detail the Techniques, algorithms, architecture,
methodology, application, and significance of artificial neural networks practiced in emerging
areas of scientific research. The report will shed light on the aforementioned areas of Artificial
neural networks by taking strong evidence from existing literature.

Literature Review
Methodology of ANNs:
Artificial neural networks (ANNs) are enormous parallel computing models that simulate human
brain function. The methodology utilized by artificial neural networks is thoroughly been
discussed by (Dongare et al, 2012) where authors have shown that ANNs are made up of several
basic processing units coupled by weighted links, which are analogous to "neurons." Each node's
output is solely determined by locally accessible stored information locally or received via
weighted connections. While a single processing node is not very powerful, the system's
capability is shown when an input is supplied to the neural network and a corresponding desired
or desired reaction is established at the output. Such type of ANN is known as supervised
training neural network. The difference between the response variable and the system output is
used to determine an error, which is then sent back to the system in order to adjust its settings
consistently which can be described as the learning rule. The process of learning is carried out
repeatedly until the results are satisfactory.
The performance of the system is strongly dependent on the data, and if there is a shortage of
data or it is noisy, neural network technology may not be the best answer. Yet, if there is enough
data and the problem is difficult to estimate, neural network architecture is an ideal solution. The
ANN does not provide a purely mathematical solution to the issue; instead, it exhibits
information processing properties that provide an approximate solution to a particular problem.

2
ANNs are widely employed in a variety of applications such as complex and nonlinear function
mapping, signal and image processing, recognizing pattern, classification, and more (Dreyfus,
2005).

The architecture of ANNs:

Artificial neural networks (ANNs) are composed of layers of interconnected nodes, also known
as neurons. The architecture of an ANN is typically organized into an input layer, one or more
hidden layers, and an output layer. The input layer receives data, such as images or numerical
values, which is processed through the hidden layers before producing an output in the output
layer. Each neuron in a layer is connected to neurons in the previous and/or subsequent layers,
with each connection weighted to determine the strength of the signal transmitted between
neurons. The weights are adjusted through a process called backpropagation, which allows the
ANN to learn from data and improve its accuracy over time. The specific number of layers and
neurons within each layer can vary based on the complexity of the problem being solved
(Dastres et al, 2021).

The basic idea behind this technique is to design novel architectures for information processing
systems, which are made up of several network consisting of units called neurons that interact to
address problems and communicate information via synaptic, or electromagnetic links. The
neurons are divided into layers, where input layer is for receiving data & the output layer is
for producing the final result. Often, one or more hidden layers are sandwiched between the two,
making it difficult to anticipate or comprehend the precise information transfer. ANNs are
structured similarly to the human brain, with interconnected neurons. Each neuron has a cell
body that processes information, including inputs and outputs (Balakrishnan et al. 2019). The
figure 1 below shows the architecture of the artificial neural network comprising of input, hidden
and output layer.

Figure 1: Architecture of Artificial neural networks

The goal of this system is to process data and information in a way that mirrors the biological
neural system, ultimately leading to the creation of knowledge. Figure 2 below shows the

3
processing being done at individual neurons. All neurons in a network are interconnected
through electromagnetic synapses, with each connection possessing a connection weight.
Neurons have a threshold value and an activation function, which determines whether a
transmitted signal is exceeding the threshold. The weighted sum of all inputs is the activation
value and this from where the output of the artificial neural network is generated. Each
connection has an associated weight, and each neuron has a predefined threshold and an
activation function. Based on the sign of the input's weight, it is estimated if each input has a
positive or negative weight. The signal intensity at a connection is affected by the weight (Liu et
al. 2018). Neurons with a threshold over which a signal is only sent if the aggregate signal is
greater than it. The Activation Value is the weighted sum of the summing units, and the signal
from this activation value is used to create the output.

Figure 2: working and processing of ANN

Artificial neural networks are capable of learning, making them highly flexible and adaptable to
new problems and equations. The learning process involves using different examples to teach the
neural network to perform specific tasks, such as pattern recognition and categorization. Neural
networks can also be used to control complex systems with unknown internal structures, such as
an engine, by learning the control function through adaptive learning. Synaptic weights change
based on the inputs given during the training process, allowing the system to produce the correct
response to new inputs (Zhang 2018).

Types, Techniques and algorithms of ANNs:

A neural network is a fragment of the human nervous system that containing numerous
interconnected neurons. The term "Neural" in ANN is for neurons while "Network" means
graphical structure. Artificial Neural Networks are graphical structures that depict human
neurons are parallel distributed processing systems," and "connectionist systems," are computing
systems whose imitation has been taken from human nervous system. To be called by these
names, a computing system must have a labeled directed graph structure where nodes perform
simple computations (Kukreja et al, 2016). (Mirjalili 2019) have discussed evolutionary
algorithms of neural networks such as particle swarm optimization, ant colony, genetic
algorithm, etc.

4
Learning types in ANNs:
Supervised learning is a type of learning in which an ANN is trained using labeled data. The
network is presented with input data, and the correct output or label is provided as the target. The
network then adjusts its weights and biases to minimize the difference between its predicted
output and the actual target. This approach is commonly used in image recognition, speech
recognition, and natural language processing tasks. Unsupervised learning, on the other hand, is
a type of learning in which the ANN is trained using unlabeled data. The network must identify
patterns and relationships within the data without any prior knowledge of the correct output. This
approach is commonly used for tasks such as clustering, dimensionality reduction, and anomaly
detection. Reinforcement learning is a type of learning in which the ANN learns through trial and
error. The network is presented with a series of states and is rewarded or penalized based on its
actions in each state. The network learns to maximize its reward over time by adjusting its
weights and biases. This approach is commonly used in robotics, game playing, and autonomous
navigation (Mahesh 2020).

Classification of ANNs:

Artificial Neural Networks have been broadly classified in two topologies i.e. Feed Forward &
Feedback networks. In Feed Forward systems, both input & output is fixed and there is no
feedback mechanism. Every unit accepts the input from its left unit and calculates the result by
multiplying the inputs with the weight of each connection. This approach is suitable for
applications such as pattern generation, identification, and classification. Feed Forward Neural
Network topologies are commonly used in commercial applications such as computer vision.
feedback neural networks, also known as recurrent neural networks (RNNs), allow information
to flow in both directions, with feedback loops connecting neurons in a recurrent manner. This
architecture enables RNNs to process sequences of input data, such as time-series or natural
language, and to capture temporal dependencies within the data. RNNs are commonly used in
speech recognition, machine translation, and other tasks that require sequential processing of
data. Feedback neural networks, also known as recurrent neural networks (RNNs), allow
information to flow in both directions, with feedback loops connecting neurons in a recurrent
manner. This architecture enables RNNs to process sequences of input data, such as time-series
or natural language, and to capture temporal dependencies within the data. RNNs are commonly
used in speech recognition, machine translation, and other tasks that require sequential
processing of data (Krogh, 2008). (Mirjalili 2019) have also classified evolutionary neural
networks including feed forward, multilayer perceptron, and deep neural networks. The study
shows that neural network evolutionary algorithms are suitable for dealing with optimization
problems as well.

5
 Figure 3: ANN learning types and algorithms (Source; Abiodun et al, 2018)

No. Type of learning Algorithms

1 Supervised learning  single-layer perceptron
 Multi-layer perceptron
 k-nearest neighbor
 Linear classifiers
 Markov Model
 Support vector machines
2 Unsupervised learning  Self organizing maps
 Fuzzy
 Clustering
3 Learning through Reinforcement
4 Deep learning
Table 1: ANN learning types and algorithms (Source; Abiodun et al, 2018)
Deep learning

Artificial neural networks (ANN) with sophisticated multilayers are referred to as deep learning.
The contrast between deep learning and neural networks, such as feedforward and feedback
neural networks, is based on their characteristics. Deep learning features more complicated
methods of linking layers, more neurons than prior networks to express complex models, more
processing power to train, and automated feature extraction (Schmidhuber, 2015).

Application and significance of ANNs:

(Abiodun et al., 2018) has done an extensive survey on the application of neural networks in
various fields and formulated the results. The study shows that computer security, medical
science, business, finance, banking, insurance, the share market, electricity production,
governance, nuclear industry, exploration and production, mineral extraction, crude oil

6
concentration quality forecasting, crop yield forecasting, and policy are among the most
promising application areas considered in artificial neural networks. According to the authors it's
worth noting that neural network data analysis improves accuracy, speed of processing, low
latency, performance, capacity, and adaptability which is quite limited as compared to other
artificial intelligence techniques. Another important application of ANN is that numerous new
and improved data handling and data analysis techniques aid with ANN management.
Developing analytics from accessible data to help in the prioritization of information and the
provision of its human business value. In turn, ANN analytics aid in overcoming obstacles and
mitigating potential dangers (Yu et al, 2005).

Neural networks have been utilized to respond to many important issues in science, health, and
engineering, and in certain situations, they give cutting-edge solutions. Nowadays, neural
networks have found numerous applications in various fields such as pattern recognition, which
involves tasks like speech recognition, image processing, line recognition, text and image
classification, and more. In addition to these, they are also used in quality management, risk
management (identifying and assessing risks), detection systems such as intrusion detection
systems, predicting loan risks, in healthcare such as detecting hepatitis and retinal failure in
diabetics etc. ANNs are also used for calculating known functions, approximating unknown
functions, pattern identification, and signal processing (Kumar and Thakur 2012).

Artificial Neural Networks are utilized for detection purposes in different applications, including
detecting potential fraud in credit card transactions. For instance, banks use this technology to
compare the signature of the client with the signature recorded in the account file. ANNs are also
used in telecommunication systems to detect and reduce noise and in optical communication
systems to increase their quality. Additionally, in marketing, ANNs are used to target more
specific audiences in internet advertising (Li et al., 2018). (Di Persio and Honchar 2016)
presented an architecture of ANNs for predicting stock price movements and developing
marketing systems. Moreover, Rezaee et al. (2018) have investigated the use of data
envelopment analysis and ANNs to predict the performance of companies in the stock exchange
system and enhance their financial performance.

Recent advancements in deep learning models have provided fresh approaches to dealing with
the problem of pattern identification and recognition. Pattern recognition is a scientific field that
focuses on determining the succession in each input. PR is a broad notion that encompasses
several subdomains, including regression, and classification. Because of industrial progress,
there are fast expanding demands for informational processing and production, which has created
new trends and problems for public relations.

In the healthcare industry the ANNs have been used in medicine and healthcare applications to
analyze and identify the indications of a pacemaker (electrocardiograph), as well as a network is
trained that can diagnose the ailment and even recommend medicines (Jain et al. 2021). A study
shows that an application using ANN systems has been devised to forecast tumor Classification
in the health care system (Nasser and Abu-Naser 2019). Medical image processing utilizing
ANN systems to tackle medical imaging challenges has also been on the rise. Researchers in a
study have introduced Artificial Neural Networks in Image Processing for Early Detection of

7
Breast Cancer in order to improve the quality of healthcare systems (Mehdy et al. 2017). ANNs
may be used in system modelling to accomplish system identification and complicated
mappings. For example, the use of Neural network models to renewable energy difficulties has
been quite successful. ANN was used to energy such as solar in the modelling of a solar steam
producing facility. The experimental outcome exhibits efficiency, workload, and fault prediction.

Conclusion
An artificial neural network (ANN) is composed of linked artificial neurons that are designed to
replicate biological neurons in the brain. Each neuron may communicate with other neurons via
links, which function similarly to synapses in the brain. Adaptive learning, where the learning
rate controls how the model adapts for mistakes in each observation, is one of the key advantages
of ANNs. A high learning rate results in faster training but lower overall accuracy, whereas a low
learning rate requires more time but may result in better accuracy. Despite ANNs' ability to
handle most tasks with adequate preparation, their biggest challenge is the time and computing
power required for complex tasks. The network architecture strongly influences an ANN's
performance in complex problems, as it helps to reach the best optimal solution for a given
problem.

References

Mahanta, J. (2017). Introduction to neural networks, advantages and applications. Towards Data
Science, 13.

Kukreja, H., Bharath, N., Siddesh, C.S. and Kuldeep, S., 2016. An introduction to artificial
neural network. Int J Adv Res Innov Ideas Educ, 1, pp.27-30.

Dongare, A.D., Kharde, R.R. and Kachare, A.D., 2012. Introduction to artificial neural network.
International Journal of Engineering and Innovative Technology (IJEIT), 2(1), pp.189-194.

Krogh, A., 2008. What are artificial neural networks?. Nature biotechnology, 26(2), pp.195-197.

Dastres, Roza & Soori, Mohsen. (2021). Artificial Neural Network Systems. International
Journal of Imaging and Robotics. 21. 13-25.

Balakrishnan, Harikrishnan Nellippallil, Aditi Kathpalia, Snehanshu Saha, and Nithin Nagaraj.
2019. "ChaosNet: A chaos based artificial neural network architecture for classification." Review
of.Chaos: An Interdisciplinary Journal of Nonlinear Science 29 (11):113125.

Mirjalili, S. (2019). Evolutionary algorithms and neural networks. In Studies in computational

intelligence (Vol. 780). Berlin/Heidelberg, Germany: Springer.

Dreyfus, G. (2005). Neural networks: methodology and applications. Springer Science &
Business Media.

8
Abiodun, O.I., Jantan, A., Omolara, A.E., Dada, K.V., Mohamed, N.A. and Arshad, H., 2018.
State-of-the-art in artificial neural network applications: A survey. Heliyon, 4(11), p.e00938.

Yu, L., Wang, S. and Lai, K.K., 2005. An integrated data preparation scheme for neural network
data analysis. IEEE Transactions on Knowledge and Data Engineering, 18(2), pp.217-230.

Mahesh, B., 2020. Machine learning algorithms-a review. International Journal of Science and
Research (IJSR).[Internet], 9, pp.381-386.

Kumar, K. and Thakur, G.S.M., 2012. Advanced applications of neural networks and artificial
intelligence: A review. International journal of information technology and computer science,
4(6), p.57.

Li, Yawen, Weifeng Jiang, Liu Yang, and Tian Wu. 2018. "On neural networks and learning
systems for business computing." Review of. Neurocomputing 275:1150-9

Di Persio, Luca, and Oleksandr Honchar. 2016. "Artificial neural networks architectures for
stock price prediction: Comparisons and applications." Review of. International journal of
circuits, systems and signal processing 10 (2016):403-13.

Schmidhuber, Jürgen. 2015. "Deep learning in neural networks: An overview." Review of.
Neural Networks 61:85-117

Zhang, Z., & Zhang, Z. (2018). Artificial neural network. Multivariate time series analysis in
climate and environmental research, 1-35.

Rezaee, Mustafa Jahangoshai, Mehrdad Jozmaleki, and Mahsa Valipour. 2018. "Integrating
dynamic fuzzy C-means, data envelopment analysis and artificial neural network to online
prediction performance of companies in stock exchange." Review of. Physica A: Statistical
Mechanics and its Applications 489:78-93.

Jain, Mahima, Apoorva Goel, Shuchi Sinha, and Sanjay Dhir. 2021. "Employability implications
of artificial intelligence in healthcare ecosystem: responding with readiness." Review of. foresigh

Nasser, Ibrahim M, and Samy S Abu-Naser. 2019. "Predicting Tumor Category Using Artificial
Neural Networks."

Mehdy, MM, PY Ng, EF Shair, NI Saleh, and Chandima Gomes. 2017. "Artificial neural
networks in image processing for early detection of breast cancer." Review of. Computational
and mathematical methods in medicine 2017.