Training of Recurrent Neural Networks (RNN) in TensorFlow
Last Updated :
21 May, 2025
Recurrent Neural Networks (RNNs) are a type of neural network designed to handle sequential data. They maintain hidden states that capture information from previous steps. In this article we will be learning to implement RNN model using TenserFlow.
Here we will be using a clothing brands reviews as dataset and will be using RNN to analyze there reviews.
1. Importing Libraries
We will be importing Pandas, NumPy, Matplotlib, Seaborn, TensorFlow, Keras, NLTK and Scikit-learn for implemntation.
Python
import warnings
from tensorflow.keras.utils import pad_sequences
from tensorflow.keras.preprocessing.text import Tokenizer
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow import keras
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.express as px
import numpy as np
import re
import nltk
nltk.download('all')
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
from nltk.stem import WordNetLemmatizer
lemm = WordNetLemmatizer()
warnings.filterwarnings("ignore")
2. Loading the Dataset
You can download the dataset used in this article from here. We load the dataset using pd.read_csv() and display the first 7 rows with data.head(7). We remove rows where the Class Name column is null to clean the data.
Python
data = pd.read_csv("Clothing Review.csv")
data.head(7)
data = data[data['Class Name'].isnull() == False]
Output:
First five rows of the datasetWe can use data.shape to give us dimensions of the dataset.
Python
(23472, 10)
EDA helps one understand how the data is distributed. To perform EDA one must perform various visualizing techniques so that one can understand the data before building a model.
We use sns.countplot() to visualize the count of each category in the ‘Class Name’ column. The plt.xticks(rotation=90) rotates the x-axis labels for better readability.
Python
sns.countplot(data=data, x='Class Name', palette='rainbow')
plt.xticks(rotation=90)
plt.show()
Output:
CountplotWe create a figure with size 12x5 inches using plt.subplots():
- Using plt.subplot(1, 2, 1) we plot a countplot of the Rating column
- Using plt.subplot(1, 2, 2) we plot a countplot of the Recommended IND column.
Python
plt.subplots(figsize=(12, 5))
plt.subplot(1, 2, 1)
sns.countplot(data=data, x='Rating',palette="deep")
plt.subplot(1, 2, 2)
sns.countplot(data=data, x="Recommended IND", palette="deep")
plt.show()
Output:
Countplot for the Rating and Recommended IND categoryWe create a histogram using px.histogram() to show the frequency distribution of Age. The histogram includes a box plot as a marginal plot. Data is colored based on Recommended IND values using green and red colors. Number of bins is set to the range from 18 to 65. fig.update_layout() adjusts the gap between bars.
Python
fig = px.histogram(data, marginal='box',
x="Age", title="Age Group",
color="Recommended IND",
nbins=65-18,
color_discrete_sequence=['green', 'red'])
fig.update_layout(bargap=0.2)
Output:
The histogram on the bottom shows age distribution with green bars for recommended individuals and red bars for non-recommended ones. The box plots at the top display the spread and outliers of ages for each recommendation group helping to visualize differences in age distribution between the two groups.
We can visualize the distribution of the age columns data along with the Rating.
Python
fig = px.histogram(data,
x="Age",
marginal='box',
title="Age Group",
color="Rating",
nbins=65-18,
color_discrete_sequence
=['black', 'green', 'blue', 'red', 'yellow'])
fig.update_layout(bargap=0.2)
Output:
The histogram at the bottom represents the count of individuals in each age group with bars color coded by rating from 1 to 5. The boxplots at the top provide a summary of age distribution for each rating showing the median, interquartile range and outliers. It helps to analyze how ratings vary with age groups.
4. Prepare the Data to build Model
Since we are working on the NLP-based dataset it could be valid to use Text columns as the feature. So we select the features that are text and the Rating column is used for Sentiment Analysis. By the above Rating counterplot we can observe that there is too much of an imbalance between the rating. So all the rating above 3 is made as 1 and below 3 as 0.
Python
def filter_score(rating):
return int(rating > 3)
features = ['Class Name', 'Title', 'Review Text']
X = data[features]
y = data['Rating']
y = y.apply(filter_score)
5. Text Preprocessing
The text data we have comes with too much noise. This noise can be in form of repeated words or commonly used sentences. In text preprocessing we need the text in the same format so we first convert the entire text into lowercase and then perform Lemmatization to remove the superposition of the words. Since we need clean text we also remove common words such as Stopwords and punctuation.
Note: Lemmatization and stop words are some concepts used in NLP and they are apart from RNN.
Python
def toLower(data):
if isinstance(data, float):
return '<UNK>'
else:
return data.lower()
stop_words = stopwords.words("english")
def remove_stopwords(text):
no_stop = []
for word in text.split(' '):
if word not in stop_words:
no_stop.append(word)
return " ".join(no_stop)
def remove_punctuation_func(text):
return re.sub(r'[^a-zA-Z0-9]', ' ', text)
X['Title'] = X['Title'].apply(toLower)
X['Review Text'] = X['Review Text'].apply(toLower)
X['Title'] = X['Title'].apply(remove_stopwords)
X['Review Text'] = X['Review Text'].apply(remove_stopwords)
X['Title'] = X['Title'].apply(lambda x: lemm.lemmatize(x))
X['Review Text'] = X['Review Text'].apply(lambda x: lemm.lemmatize(x))
X['Title'] = X['Title'].apply(remove_punctuation_func)
X['Review Text'] = X['Review Text'].apply(remove_punctuation_func)
X['Text'] = list(X['Title']+X['Review Text']+X['Class Name'])
X_train, X_test, y_train, y_test = train_test_split(
X['Text'], y, test_size=0.25, random_state=42)
If you notice at the end of the code, we have created a new column "Text" which is of type list. The reason we did this is that we need to perform Tokenization on the entire feature taken to train the model.
6. Tokenization
In Tokenization we convert the text into Vectors. Keras API supports text pre-processing. This API consists of Tokenizer that takes in the total num_words to create the Word index. OOV stands for out of vocabulary this is triggered when new text is encountered. Also remember that we fit_on_texts only on training data and not testing.
Vectors are numeric representation of data so that machine can easily be understand and use it for training.
Python
tokenizer = Tokenizer(num_words=10000, oov_token='<OOV>')
tokenizer.fit_on_texts(X_train)
7. Padding the Text Data
Keras preprocessing helps in organizing the text. Padding helps in building models of the same size that further becomes easy to train neural network models. The padding adds extra zeros to satisfy the maximum length to feed a neural network. If the text length exceeds then it can be truncated from either the beginning or end. By default it is pre, we can set it to post or leave it as it is.
Both padding and tokenization are a part of NLP processing not specific for RNN task.
Python
train_seq = tokenizer.texts_to_sequences(X_train)
test_seq = tokenizer.texts_to_sequences(X_test)
train_pad = pad_sequences(train_seq,
maxlen=40,
truncating="post",
padding="post")
test_pad = pad_sequences(test_seq,
maxlen=40,
truncating="post",
padding="post")
8. Building a Recurrent Neural Network (RNN) in TensorFlow
Now that the data is ready, the next step is building a Simple Recurrent Neural network. Before training with SImpleRNN, the data is passed through the Embedding layer to perform the equal size of Word Vectors.
Note: We use return_sequences = True only when we need another layer to stack.
Python
from tensorflow import keras
model = keras.models.Sequential()
model.add(keras.layers.Embedding(input_dim=10000, output_dim=128, input_length=40))
model.add(keras.layers.SimpleRNN(64, return_sequences=True))
model.add(keras.layers.SimpleRNN(64))
model.add(keras.layers.Dense(128, activation="relu"))
model.add(keras.layers.Dropout(0.4))
model.add(keras.layers.Dense(1, activation="sigmoid"))
model.build(input_shape=(None, 40))
model.summary()
Output:
Summary of the architecture of the model9. Training the Model
In Tensorflow after developing a model, it needs to be compiled using the three important parameters i.e Optimizer, Loss Function and Evaluation metrics. We will be training the model on the train_pad which we preprocessed and use 5 epochs to calculate the accuracy on y_train dataset.
Python
model.compile(loss="binary_crossentropy",
optimizer="adam",
metrics=["accuracy"])
history = model.fit(train_pad,
y_train,
epochs=5)
Output:
Training the ModelHere we can see our model got a accuracy of 92.86% which is pretty good for a RNN model.
You can also make RNN model using PyTorch and for that you can refer to this article: Implementing Recurrent Neural Networks in PyTorch
Similar Reads
Types of Recurrent Neural Networks (RNN) in Tensorflow
Recurrent neural network (RNN) is more like Artificial Neural Networks (ANN) that are mostly employed in speech recognition and natural language processing (NLP). Deep learning and the construction of models that mimic the activity of neurons in the human brain uses RNN. Text, genomes, handwriting,
2 min read
Time Series Forecasting using Recurrent Neural Networks (RNN) in TensorFlow
Time series data such as stock prices are sequence that exhibits patterns such as trends and seasonality. Each data point in a time series is linked to a timestamp which shows the exact time when the data was observed or recorded. Many fields including finance, economics, weather forecasting and mac
5 min read
Training of Convolutional Neural Network (CNN) in TensorFlow
In this article, we are going to implement and train a convolutional neural network CNN using TensorFlow a massive machine learning library. Now in this article, we are going to work on a dataset called 'rock_paper_sissors' where we need to simply classify the hand signs as rock paper or scissors.
4 min read
Recurrent Neural Networks in R
Recurrent Neural Networks (RNNs) are a type of neural network that is able to process sequential data, such as time series, text, or audio. This makes them well-suited for tasks such as language translation, speech recognition, and time series prediction. In this article, we will explore how to impl
5 min read
Sentiment Analysis with an Recurrent Neural Networks (RNN)
Recurrent Neural Networks (RNNs) are used in sequence tasks such as sentiment analysis due to their ability to capture context from sequential data. In this article we will be apply RNNs to analyze the sentiment of customer reviews from Swiggy food delivery platform. The goal is to classify reviews
5 min read
Training a Neural Network using Keras API in Tensorflow
In the field of machine learning and deep learning has been significantly transformed by tools like TensorFlow and Keras. TensorFlow, developed by Google, is an open-source platform that provides a comprehensive ecosystem for machine learning. Keras, now fully integrated into TensorFlow, offers a us
3 min read
Introduction to Recurrent Neural Networks
Recurrent Neural Networks (RNNs) differ from regular neural networks in how they process information. While standard neural networks pass information in one direction i.e from input to output, RNNs feed information back into the network at each step.Imagine reading a sentence and you try to predict
10 min read
Recurrent Layers in TensorFlow
Recurrent layers are used in Recurrent Neural Networks (RNNs), which are designed to handle sequential data. Unlike traditional feedforward networks, recurrent layers maintain information across time steps, making them suitable for tasks such as speech recognition, machine translation, and time seri
2 min read
Neural Network Layers in TensorFlow
TensorFlow provides powerful tools for building and training neural networks. Neural network layers process data and learn features to make accurate predictions. A neural network consists of multiple layers, each serving a specific purpose. These layers include:Input Layer: The entry point for data.
2 min read
Training Loop in TensorFlow
Training neural networks is at the core of machine learning, and understanding how to write a training loop from scratch is fundamental for any deep learning practitioner and TensorFlow provides powerful tools for building and training neural networks efficiently. In this article, we will get into t
7 min read