Exploiting Deep Learning for Persian

Sentiment Analysis

Kia Dashtipour1(B) , Mandar Gogate1 , Ahsan Adeel1 , Cosimo Ieracitano2 ,

Hadi Larijani3 , and Amir Hussain1
1
Department of Computing Science and Mathematics, Faculty of Natural Sciences,
University of Stirling, Stirling FK9 4LA, UK
[email protected]
2
DICEAM Department, University Mediterranea of Reggio Calabria,
89124 Reggio Calabria, Italy
3
Department of Communication, Network and Electronic Engineering,
Glasgow Caledonian University, Glasgow G4 0BA, UK

Abstract. The rise of social media is enabling people to freely express

their opinions about products and services. The aim of sentiment anal-
ysis is to automatically determine subject’s sentiment (e.g., positive,
negative, or neutral) towards a particular aspect such as topic, prod-
uct, movie, news etc. Deep learning has recently emerged as a powerful
machine learning technique to tackle a growing demand of accurate senti-
ment analysis. However, limited work has been conducted to apply deep
learning algorithms to languages other than English, such as Persian. In
this work, two deep learning models (deep autoencoders and deep convo-
lutional neural networks (CNNs)) are developed and applied to a novel
Persian movie reviews dataset. The proposed deep learning models are
analyzed and compared with the state-of-the-art shallow multilayer per-
ceptron (MLP) based machine learning model. Simulation results demon-
strate the enhanced performance of deep learning over state-of-the-art
MLP.

Keywords: Persian sentiment analysis · Persian movie reviews

Deep learning

1 Introduction

In recent years, social media, forums, blogs and other forms of online communi-
cation tools have radically affected everyday life, especially how people express
their opinions and comments. The extraction of useful information (such as peo-
ple’s opinion about companies brand) from the huge amount of unstructured
data is vital for most companies and organizations [5]. The product reviews are
important for business owners as they can take business decision accordingly to
automatically classify user’s opinions towards products and services. The appli-
cation of sentiment analysis is not limited to product or movie reviews but can
c Springer Nature Switzerland AG 2018
J. Ren et al. (Eds.): BICS 2018, LNAI 10989, pp. 597–604, 2018.
https://doi.org/10.1007/978-3-030-00563-4_58
598 K. Dashtipour et al.

be applied to different fields such as news, politics, sport etc. For example, in
online political debates, the sentiment analysis can be used to identify people’s
opinions on a certain election candidate or political parties [19,20,27]. In this
context, sentiment analysis has been widely used in different languages by using
traditional and advanced machine learning techniques. However, limited research
has been conducted to develop models for the Persian language.
The sentiment analysis is a method to automatically process large amounts
of data and classify text into positive or negative sentiments) [2,8]. Sentiment
analysis can be performed at two levels: at the document level or at the sentence
level. At document level it is used to classify the sentiment expressed in the
document (positive or negative), whereas, at sentence level is used to identify
the sentiments expressed only in the sentence under analysis [6,7].
In the literature, deep learning based automated feature extraction has been
shown to outperform state-of-the-art manual feature engineering based classi-
fiers such as Support Vector Machine (SVM), Naive Bayes (NB) or Multilayer
Perceptron (MLP) etc. One of the important techniques in deep learning is the
autoencoder that generally involves reducing the number of feature dimensions
under consideration. The aim of dimensionality reduction is to obtain a set of
principal variables to improve the performance of the approach. Similarly, CNNs
have been proven to be very effective in sentiment analysis. However, little work
has been carried out to exploit deep learning based feature representation for
Persian sentiment analysis [10,16]. In this paper, we present two deep learn-
ing models (deep autoencoders and CNNs) for Persian sentiment analysis. The
obtained deep learning results are compared with MLP.
The rest of the paper is organized as follows: Sect. 2 presents related work.
Section 3 presents methodology and experimental results. Finally, Sect. 4 con-
cludes this paper.

2 Related Works
In the literature, extensive research has been carried out to model novel sen-
timent analysis models using both shallow and deep learning algorithms. For
example, the authors in [3] proposed a novel deep learning approach for polar-
ity detection in product reviews. The authors addressed two major limitations
of stacked denoising of autoencoders, high computational cost and the lack of
scalability of high dimensional features. Their experimental results showed the
effectiveness of proposed autoencoders in achieving accuracy upto 87%. Zhai et
al. [28] proposed a five layers autoencoder for learning the specific representa-
tion of textual data. The autoencoders are generalised using loss function and
derived discriminative loss function from label information. The experimental
results showed that the model outperformed bag of words, denoising autoen-
coders and other traditional methods, achieving accuracy rate up to 85%. Sun
et al. [26] proposed a novel method to extract contextual information from text
using a convolutional autoencoder architecture. The experimental results showed
that the proposed model outperformed traditional SVM and Nave Bayes models,
reporting accuracy of 83.1 %, 63.9% and 67.8% respectively.
 Exploiting Deep Learning for Persian Sentiment Analysis 599

Su et al. [24] proposed an approach for a neural generative autoencoder for

learning bilingual word embedding. The experimental results showed the effec-
tiveness of their approach on English-Chinese, English-German, English-French
and English-Spanish (75.36% accuracy). Kim et al. [14] proposed a method to
capture the non-linear structure of data using CNN classifier. The experimen-
tal results showed the effectiveness of the method on the multi-domain dataset
(movie reviews and product reviews). However, the disadvantage is only SVM
and Naive Bayes classifiers are used to evaluate the performance of the method
and deep learning classifiers are not exploited. Zhang et al. [29] proposed an app-
roach using deep learning classifiers to detect polarity in Japanese movie reviews.
The approach used denoising autoencoder and adapted to other domains such
as product reviews. The advantage of the approach is not depended on any lan-
guage and could be used for various languages by applying different datasets.
AP et al. [1] proposed a CNN based model for cross-language learning of vecto-
rial word representations that is coherent between two languages. The method
is evaluated using English and German movie reviews dataset. The experimen-
tal results showed CNN (83.45% accuracy) outperformed as compared to SVM
(65.25% accuracy).
Zhou et al. [30] proposed an autoencoder architecture constituting an LSTM-
encoder and decoder in order to capture features in the text and reduce dimen-
sionality of data. The LSTM encoder used the interactive scheme to go through
the sequence of sentences and LSTM decoder reconstructed the vector of sen-
tences. The model is evaluated using different datasets such as book reviews,
DVD reviews, and music reviews, acquiring accuracy up to 81.05%, 81.06%,
and 79.40% respectively. Mesnil et al. [17] proposed an approach using ensem-
ble classification to detect polarity in the movie reviews. The authors combined
several machine learning algorithms such as SVM, Naive Bayes and RNN to
achieve better results, where autoencoders were used to reduce the dimension-
ality of features. The experimental results showed the combination of unigram,
bigram and trigram features (91.87% accuracy) outperformed unigram (91.56%
accuracy) and bigram (88.61% accuracy).
Scheible et al. [22] trained an approach using semi-supervised recursive
autoencoder to detect polarity in movie reviews dataset, consisted of 5000 posi-
tive and 5000 negative sentiments. The experimental results demonstrated that
the proposed approach successfully detected polarity in movie reviews dataset
(83.13% accuracy) and outperformed standard SVM (68.36% accuracy) model.
Dai et al. [4] developed an autoencoder to detect polarity in the text using deep
learning classifier. The LSTM was trained on IMDB movie reviews dataset. The
experimental results showed the outperformance of their proposed approach over
SVM. In Table 1 some of the autoencoder approaches are depicted.

3 Methodology and Experimental Results

The novel dataset used in this work was collected manually and includes Persian
movie reviews from 2014 to 2016. A subset of dataset was used to train the neural
600 K. Dashtipour et al.

network (60% training dataset) and rest of the data (40%) was used to test
and validate the performance of the trained neural network (testing set (30%),
validation set (10%)). There are two types of labels in the dataset: positive or
negative. The reviews were manually annotated by three native Persian speakers
aged between 30 and 50 years old.
After data collection, the corpus was pre-processed using tokenisation, nor-
malisation and stemming techniques. The process of converting sentences into
single word or token is called tokenisation. For example, “The movie is great” is
changed to “The”, “movie”, “is”, “great” [25]. There are some words which con-
tain numbers. For example, “great” is written as “gr8” or “gooood” as written
as “good”. The normalisation is used to convert these words into normal forms
[21]. The process of converting words into their root is called stemming. For
example, going was changed to go [15]. Words were converted into vectors. The
fasttext was used to convert each word into 300-dimensions vectors. Fasttext is
a library for text classification and representation [10,13,18].
For classification, MLP, autoencoders and CNNs have been used. Figure 1
depicts the modelled MLP architectures. MLP classifer was trained for 100 iter-
ations [9]. Figure 2 depicts the modelled autoencoder architecture. Autoencoder
is a feed-forward deep neural network with unsupervised learning and it is used
for dimensionality reduction. The autoencoder consists of input, output and hid-
den layers. Autoencoder is used to compress the input into a latent-space and
then the output is reconstructed [11,12,23]. The exploited autoencoder model
is depcited in Fig. 1. The autoencoder consists of one input layer three hidden
layers (1500, 512, 1500) and an output layer. Convolutional Neural Networks
contains three layers (input, hidden and output layer). The hidden layer consists
of convolutional layers, pooling layers, fully connected layers and normalisation
layer. The hj is denotes the hidden neurons of j, with bias of bj , is a weight
sum over continuous visible nodes v which is given by:

h j = bj + vi wij (1)

The modelled CNN architecture is depicted in Fig. 3 [11,12]. For CNN mod-
elling, each utterance was represented as a concatenation vector of constituent
words. The network has total 11 layers: 4 convolution layers, 4 max pooling and
3 fully connected layers. Convolution layers have filters of size 2 and with 15
feature maps. Each convolution layer is followed by a max polling layer with
window size 2. The last max pooling layer is followed by fully connected layers
of size 5000, 500 and 4. For final layer, softmax activation is used.
To evaluate the performance of the proposed approach, precision (1), recall
(2), f-Measure (3), and prediction accuracy (4) have been used as a performance
matrices. The experimental results are shown in Table 1, where it can be seen that
autoencoders outperformed MLP and CNN outperformed autoencoders with the
highest achieved accuracy of 82.6%.
 Exploiting Deep Learning for Persian Sentiment Analysis 601

Fig. 1. Multilayer perceptron

Fig. 2. Autoencoder

TP
P recision = (2)
TP + FP
TP
Recall = (3)
TP + FN
P recision ∗ Recall
F measure = 2 ∗ (4)
P recision + Recall
TP + TN
Accuracy = (5)
TP + TN + FP + FN
where TP is denotes true positive, TN is true negative, FP is false positive,
and FN is false negative.
602 K. Dashtipour et al.

Fig. 3. Deep convolutional neural network

Table 1. Results: MLP vs. autoencoder vs. convolutional neural network

Precision Recall F-measure Accuracy (%)

MLP
Negative 0.78 0.76 0.77
Positive 0.79 0.81 0.8
AVG 0.78 0.78 0.78 78.49
MLP-autoencoder
Negative 0.78 0.81 0.79
Positive 0.82 0.8 0.81
AVG 0.8 0.8 0.8 80.08
1D-CNN
Negative 0.90 0.78 0.83
Positive 0.77 0.89 0.82
AVG 0.84 0.83 0.83 82.86

4 Conclusion

Sentiment analysis has been used extensively for a wide of range of real-world
applications, ranging from product reviews, surveys feedback, to business intelli-
gence, and operational improvements. However, the majority of research efforts
are devoted to English-language only, where information of great importance is
also available in other languages. In this work, we focus on developing sentiment
analysis models for Persian language, specifically for Persian movie reviews. Two
 Exploiting Deep Learning for Persian Sentiment Analysis 603

deep learning models (deep autoencoders and deep CNNs) are developed and
compared with the state-of-the-art shallow MLP based machine learning model.
Simulations results revealed the outperformance of our proposed CNN model
over autoencoders and MLP. In future, we intend to exploit more advanced deep
learning models such as Long Short-Term Memory (LSTM) and LSTM-CNNs
to further evaluate the performance of our developed novel Persian dataset.

Acknowledgment. Amir Hussain and Ahsan Adeel were supported by the UK Engi-
neering and Physical Sciences Research Council (EPSRC) grant No. EP/M026981/1.

References
1. AP, S.C., et al.: An autoencoder approach to learning bilingual word represen-
tations. In: Advances in Neural Information Processing Systems, pp. 1853–1861
(2014)
2. Cambria, E., Poria, S., Hazarika, D., Kwok, K.: SenticNet 5: discovering conceptual
primitives for sentiment analysis by means of context embeddings. In: AAAI (2018)
3. Chen, M., Xu, Z., Weinberger, K., Sha, F.: Marginalized denoising autoencoders
for domain adaptation. arXiv preprint arXiv:1206.4683 (2012)
4. Dai, A.M., Le, Q.V.: Semi-supervised sequence learning. In: Advances in Neural
Information Processing Systems, pp. 3079–3087 (2015)
5. Dashtipour, K., Gogate, M., Adeel, A., Hussain, A., Alqarafi, A., Durrani, T.: A
comparative study of persian sentiment analysis based on different feature combi-
nations. In: Liang, Q., Mu, J., Jia, M., Wang, W., Feng, X., Zhang, B. (eds.) CSPS
2017. LNEE, vol. 463, pp. 2288–2294. Springer, Singapore (2019). https://doi.org/
10.1007/978-981-10-6571-2 279
6. Dashtipour, K., Gogate, M., Adeel, A., Algarafi, A., Howard, N., Hussain, A.:
Persian named entity recognition. In: 2017 IEEE 16th International Conference
on Cognitive Informatics and Cognitive Computing (ICCI* CC), pp. 79–83. IEEE
(2017)
7. Dashtipour, K., Hussain, A., Zhou, Q., Gelbukh, A., Hawalah, A.Y.A., Cambria,
E.: PerSent: a freely available persian sentiment lexicon. In: Liu, C.-L., Hussain,
A., Luo, B., Tan, K.C., Zeng, Y., Zhang, Z. (eds.) BICS 2016. LNCS (LNAI), vol.
10023, pp. 310–320. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-
49685-6 28
8. Dashtipour, K., et al.: Multilingual sentiment analysis: state of the art and inde-
pendent comparison of techniques. Cogn. Comput. 8(4), 757–771 (2016)
9. Gardner, M.W., Dorling, S.: Artificial neural networks (the multilayer perceptron)
a review of applications in the atmospheric sciences. Atmos. Environ. 32(14–15),
2627–2636 (1998)
10. Gasparini, S.: Information theoretic-based interpretation of a deep neural net-
work approach in diagnosing psychogenic non-epileptic seizures. Entropy 20(2),
43 (2018)
11. Gogate, M., Adeel, A., Hussain, A.: Deep learning driven multimodal fusion for
automated deception detection. In: 2017 IEEE Symposium Series on Computa-
tional Intelligence (SSCI), pp. 1–6. IEEE (2017)
12. Gogate, M., Adeel, A., Hussain, A.: A novel brain-inspired compression-based opti-
mised multimodal fusion for emotion recognition. In: 2017 IEEE Symposium Series
on Computational Intelligence (SSCI), pp. 1–7. IEEE (2017)
604 K. Dashtipour et al.

13. Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jégou, H., Mikolov, T.: Fast-
text.zip: Compressing text classification models. arXiv preprint arXiv:1612.03651
(2016)
14. Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint
arXiv:1408.5882 (2014)
15. Korenius, T., Laurikkala, J., Järvelin, K., Juhola, M.: Stemming and lemmatization
in the clustering of finnish text documents. In: Proceedings of the thirteenth ACM
international conference on Information and knowledge management, pp. 625–633.
ACM (2004)
16. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
17. Mesnil, G., Mikolov, T., Ranzato, M., Bengio, Y.: Ensemble of generative and
discriminative techniques for sentiment analysis of movie reviews. arXiv preprint
arXiv:1412.5335 (2014)
18. Morabito, F.C., et al.: Deep convolutional neural networks for classification of mild
cognitive impaired and alzheimer’s disease patients from scalp eeg recordings. In:
2016 IEEE 2nd International Forum on Research and Technologies for Society and
Industry Leveraging a Better Tomorrow (RTSI), pp. 1–6. IEEE (2016)
19. Ren, J., Jiang, J.: Hierarchical modeling and adaptive clustering for real-time sum-
marization of rush videos. IEEE Trans. Multimed. 11(5), 906–917 (2009)
20. Ren, J., Jiang, J., Feng, Y.: Activity-driven content adaptation for effective video
summarization. J. Vis. Commun. Image Represent. 21(8), 930–938 (2010)
21. Reynolds, D.A.: Comparison of background normalization methods for text-
independent speaker verification. In: Fifth European Conference on Speech Com-
munication and Technology (1997)
22. Scheible, C., Schütze, H.: Cutting recursive autoencoder trees. arXiv preprint
arXiv:1301.2811 (2013)
23. Semeniuta, S., Severyn, A., Barth, E.: A hybrid convolutional variational autoen-
coder for text generation. arXiv preprint arXiv:1702.02390 (2017)
24. Su, J., Wu, S., Zhang, B., Wu, C., Qin, Y., Xiong, D.: A neural generative autoen-
coder for bilingual word embeddings. Inf. Sci. 424, 287–300 (2018)
25. Sumathy, K., Chidambaram, M.: Text mining: concepts, applications, tools and
issues-an overview. Int. J. Comput. Appl. 80(4) (2013)
26. Sun, X., Li, C., Ren, F.: Sentiment analysis for chinese microblog based on deep
neural networks with convolutional extension features. Neurocomputing 210, 227–
236 (2016)
27. Tan, S.-S., Na, J.-C.: Mining semantic patterns for sentiment analysis of product
reviews. In: Kamps, J., Tsakonas, G., Manolopoulos, Y., Iliadis, L., Karydis, I.
(eds.) TPDL 2017. LNCS, vol. 10450, pp. 382–393. Springer, Cham (2017). https://
doi.org/10.1007/978-3-319-67008-9 30
28. Zhai, S., Zhang, Z.M.: Semisupervised autoencoder for sentiment analysis. In:
AAAI, pp. 1394–1400 (2016)
29. Zhang, P., Komachi, M.: Japanese sentiment classification with stacked denoising
auto-encoder using distributed word representation. In: Proceedings of the 29th
Pacific Asia Conference on Language, Information and Computation, pp. 150–159
(2015)
30. Zhou, H., Chen, L., Shi, F., Huang, D.: Learning bilingual sentiment word embed-
dings for cross-language sentiment classification. In: Proceedings of the 53rd
Annual Meeting of the Association for Computational Linguistics and the 7th
International Joint Conference on Natural Language Processing (Volume 1: Long
Papers), vol. 1, pp. 430–440 (2015)