Scribd
Upload
22 views17 pages

2 U4-Rnn

Uploaded by

Hàmmád Khân
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
22 views17 pages

2 U4-Rnn

Uploaded by

Hàmmád Khân
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 17

RNN

General Idea of RNN


Popular Neural Networks

 Feed-Forward Neural Network: Used for general


Regression and Classification problems.
 Convolutional Neural Network: Used for object
detection and image classification.
• Recurrent Neural Network: Used for speech
recognition, voice recognition, time series prediction,
and natural language processing.
Why Recurrent Neural Networks?
• Recurrent neural networks were created because there were a
few issues in the feed-forward neural network:
 Cannot handle sequential data

 Considers only the current input

 Cannot memorize previous inputs

• The solution to these issues is the Recurrent Neural Network


(RNN). An RNN can handle sequential data, accepting the
current input data, and previously received inputs. RNNs can
memorize previous inputs due to their internal memory.
What is a Recurrent Neural Network?
• A Recurrent Neural Network works on the principle of saving the output of a particular layer and
feeding this back to the input in order to predict the output of the layer.

• Below is how you can convert a Feed-Forward Neural Network into a Recurrent Neural Network:
• The nodes in different layers of the neural network are compressed to form a single layer of
recurrent neural networks. A, B, and C are the parameters of the network.
• Here, “x” is the input layer, “h” is the hidden layer, and “y” is the output layer. A, B, and C are the
network parameters used to improve the output of the model. At any given time t, the current input is
a combination of input at x(t) and x(t-1). The output at any given time is fetched back to the network
to improve on the output.
Applications of Recurrent Neural Networks

• Image Captioning
• RNNs are used to caption an image by analysing the activities present.
Time Series Prediction
Any time series problem, like predicting the prices of stocks in a particular month, can be solved using
an RNN.

• Natural Language Processing


• Text mining and Sentiment analysis can be carried out using an RNN for Natural Language
Processing (NLP).
Machine Translation
Given an input in one language, RNNs can be used to translate the input into different languages as
output.
Types of Recurrent Neural Networks

• There are four types of Recurrent Neural Networks:


• One to One
• One to Many
• Many to One
• Many to Many
One to One RNN
This type of neural network is known as the Vanilla Neural Network. It's used for general machine
learning problems, which has a single input and a single output.
One to Many RNN
This type of neural network has a single input and multiple outputs. An example of this is the image
caption.
Many to One RNN
This RNN takes a sequence of inputs and generates a single output. Sentiment analysis is a good
example of this kind of network where a given sentence can be classified as expressing positive or
negative sentiments
Many to Many RNN
This RNN takes a sequence of inputs and generates a sequence of outputs. Machine translation is
one of the examples.
Problems with RNN
• Vanishing Gradient Problem
• Recurrent Neural Networks enable you to model time-dependent and sequential data problems,
such as stock market prediction, machine translation, and text generation. You will find, however,
that recurrent Neural Networks are hard to train because of the gradient problem.
• RNNs suffer from the problem of vanishing gradients. The gradients carry information used in the
RNN, and when the gradient becomes too small, the parameter updates become insignificant. This
makes the learning of long data sequences difficult.
• Exploding Gradient Problem
• While training a neural network, if the slope tends to grow exponentially instead of decaying, this is
called an Exploding Gradient. This problem arises when large error gradients accumulate, resulting
in very large updates to the neural network model weights during the training process.
• Long training time, poor performance, and bad accuracy are the major issues in gradient problems.

SOLUTION: LSTM(LONG SHORT TERM MEMORY) MODEL

You might also like