Difference between ANN, CNN and RNN

Artificial Neural Network (ANN):

(ANN), is a group of multiple perceptrons or neurons at each layer. ANN is also known as a Feed-Forward Neural network because inputs are processed only in the forward direction. This type of neural networks are one of the simplest variants of neural networks. They pass information in one direction, through various input nodes, until it makes it to the output node. The network may or may not have hidden node layers, making their functioning more interpretable.

Advantages:

• Storing information on the entire network.
• Ability to work with incomplete knowledge.
• Having fault tolerance.
• Having a distributed memory.

Disadvantages:

• Hardware dependence.
• Unexplained behavior of the network.
• Determination of proper network structure.

Convolutional Neural Network (CNN):

(CNN) are one of the most popular models used today. This neural network computational model uses a variation of multilayer perceptrons and contains one or more convolutional layers that can be either entirely connected or pooled. These convolutional layers create feature maps that record a region of image which is ultimately broken into rectangles and sent out for nonlinear processing.

Advantages:

• Very High accuracy in image recognition problems.
• Automatically detects the important features without any human supervision.
• Weight sharing.

Disadvantages:

• CNN do not encode the position and orientation of object.
• Lack of ability to be spatially invariant to the input data.
• Lots of training data is required.

Recurrent Neural Network (RNN):

(RNN) are more complex. They save the output of processing nodes and feed the result back into the model (they did not pass the information in one direction only). This is how the model is said to learn to predict the outcome of a layer. Each node in the RNN model acts as a memory cell, continuing the computation and implementation of operations. If the network's prediction is incorrect, then the system self-learns and continues working towards the correct prediction during backpropagation.

Advantages:

• An RNN remembers each and every information through time. It is useful in time series prediction only because of the feature to remember previous inputs as well. This is called Long Short Term Memory.
• Recurrent neural network are even used with convolutional layers to extend the effective pixel neighborhood.

Disadvantages:

• Gradient vanishing and exploding problems.
• Training an RNN is a very difficult task.
• It cannot process very long sequences if using tanh or relu as an activation function.

Summation of all three networks in single table: