R. L. Rosa et al.

: Music Recommendation System Based on User’s Sentiments Extracted from Social Networks 359

Music Recommendation System Based on User’s

Sentiments Extracted from Social Networks
Renata L. Rosa, Demóstenes Z. Rodríguez, and Graça Bressan

Abstract —In recent years, the sentiment analysis has been information and allows to know more about a person who
explored by several Internet services to recommend contents in expresses herself or himself about an event, product or content.
accordance with human emotions, which are expressed through Sentiment analysis is a technique of natural language
informal texts posted on social networks. However, the metrics processing and text analytics, which can be applied to many
used in the sentiment analysis only classify a sentence with areas, such as e-learning, e-commerce [1], and multimedia [2]
positive, neutral or negative intensity, and do not detect sentiment among others. However, its use in recommendation systems
variations in accordance with the user’s profile. In this arena, remains a challenge; people express their feelings in different
this paper presents a music recommendation system based on a ways, making it difficult to create reliable recommendations
sentiment intensity metric, named enhanced Sentiment Metric based on sentiments [3].
(eSM) that is the association of a lexicon-based sentiment metric Sentiment analysis is starting to be explored in music
with a correction factor based on the user’s profile. This recommendation systems to suggest a specific song depending
correction factor is discovered by means of subjective tests,
on the emotional state of a person, since the song is totally
conducted in a laboratory environment. Based on the
related to the current emotion and feelings of the person.
experimental results, the correction factor is formulated and used
There is sentiment analysis research based on physiological
to adjust the final sentiment intensity. The users' sentiments are
extracted from sentences posted on social networks and the music signals [4], [5], subjective emotion assessment [6], tag-based
recommendation system is performed through a framework of extractions [7], [8], web semantic [9], [10], machine learning,
low complexity for mobile devices, which suggests songs based such as, Support Vector Machines (SVM) and its derivation
on the current user’s sentiment intensity. Also, the framework was [11], and the lexicon-based technique such as the ANEW [12].
built considering ergonomic criteria of usability. The A user of a music recommendation system can choose his or
performance of the proposed framework is evaluated with remote her emotional state manually [6] using emoticon faces or by
users using the crowdsourcing method, reaching a rating of 91% physiological sensors in a music recommendation system [5].
of user satisfaction, outperforming a randomly assigned song However, this collection of emotions makes the system
suggestion that reached 65% of user satisfaction. Furthermore, dependent on the user input and equipment. It is therefore ideal
the paper presents low perceived impacts on the analysis of to have an independent and automatic recommendation system.
energy consumption, network and latency in accordance with the Tag-based recommendations and lexicon-based techniques
processing and memory perception of the recommendation [13] aim to extract emotions in accordance with words, not
system, showing advantages for the consumer electronic world. 1 differentiating whether or not a person is more emotional.
Consequently, there is a gap in current studies in how people
Index Terms — Recommendation System, Social Network, express themselves with greater or lesser intensity of feeling.
Mobile Devices, Sentiment Analysis. People use social networks at certain times of day [14] and
knowing their habits and behaviors may be useful for several
I. INTRODUCTION applications; therefore, recording these data in logs is very
People use the Internet to express themselves and social important. By the implementation of logs, it is possible to
networks have become a popular way to share information, develop a recommendation for a customized period of time.
ideas and experiences. People use social networks to write For instance, a person usually reads financial news, e.g., from
sentences with positive, negative or neutral emotions, 9a.m. to 10a.m. on Monday and Tuesday and tends to read
expressing their feelings; in this context, studies concerning entertainment news on Saturday [14].
sentiment intensity have started to emerge. Knowledge of the A recommendation system can collect data all the time and
sentiment intensity of a sentence can help to collect useful disseminate a large amount of information to users, but that is
an inconvenience for the system and its users, in addition to
consuming more processing, bandwidth, and memory
1
R. L. Rosa, and G. Bressan are with the Computer Engineering Department
resources of a consumer electronic device. Therefore, widely
at the School of Engineering - University of São Paulo, CEP 5508-010, Brazil used applications must consume fewer resources in times of an
(e-mails: [email protected] and [email protected]). increased number of consumers' portable devices.
D. Z. Rodríguez is with the Computer Science Department at the Federal
This paper is an extended contribution to the work [2],
University of Lavras, CEP 37200-000, Brazil (e-mail:
[email protected]). which proposes a personal music recommendation system

Contributed Paper
Manuscript received 06/30/15
Current version published 09/29/15
Electronic version published 09/29/15. 0098 3063/15/$20.00 © 2015 IEEE

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply
 360 IEEE Transactions on Consumer Electronics, Vol. 61, No. 3, August 2015

using a new lexicon-based sentiment metric named eSM, These approaches are used to perform sentiment analysis.
which uses the Sentimeter-Br2 [2] metric associated with a However, notice that the use of machine learning requires a
novel correction factor based on the user’s profile, to be large amount of data to obtain a reliable sentiment result,
applied to the sentiment analysis. because an unusual sentence may cause noise in the
The Sentimeter-Br2 metric is a word dictionary with calculation of the sentiment.
respective sentiment intensity of positive or negative value, This research focuses on the lexicon-based approach using a
which considers n-grams, adverbs, removes stopwords, words word dictionary, which is used to define the sentiment
which do not add sentiment to a sentence. Also, the metric intensity metric. Many studies of sentiment analysis try to
differentiates sentiment values depending on verbal tenses, in improve the metric to find a more accurate sentiment intensity
which a verb in the past tense has a lesser sentiment value than of a text. A manual dictionary consists of words, in which
a verb in the present tense. Sentimeter-Br2 is based on the each word has a respective classification, e.g., a positive scale
Sentimeter-Br [1]. from +1 to +5 or a negative scale from -1 to -5 [12], as shown
The main goal of this paper is to demonstrate that a lexicon- in Table I. A dictionary of affective language [18] uses an
based sentiment intensity metric associated to a correction intensity scale with many emotional words. WordNet is
factor can improve the performance of a music another dictionary of words that can be used for sentiment
recommendation system, using a low complexity solution with analysis. In this study, both the SentiStrength [19] and the
advantages to the consumer electronic devices. The correction Sentimeter-Br2 metrics are used, because they both support
factor uses the characteristics of a person that can easily be the Portuguese language and a comparison of their
extracted from social networks. The new metric, eSM, performance can be made.
therefore presents a more accurate sentiment value.
TABLE I
Additionally, the proposed recommendation system
WORDS FROM A DICTIONARY WITH RESPECTIVE SENTIMENT INTENSITY
considered ergonomic factors of usability in order to improve VALUES
the user’s quality of experience.
In this context, the remainder of this paper is structured as Word Sentiment intensity value
follows. Section II presents the related work. Section III
presents the proposed sentiment intensity model used to bad -2
determine the eSM. Section IV shows the proposed like +2
recommendation system architecture based on sentiments. combat -3
Section V presents the experimental results that include the beautiful +3
proposed recommendation system with the eSM performance
evaluation. Finally, section VI presents the conclusions. Once the dictionary to be used is defined, the sentiment
intensity metric can be modeled. The basic metric to obtain
II. RELATED WORK the sentiment of a sentence is commonly obtained by an
In this section, firstly, the main studies regarding sentiment arithmetic sum of each word found in the dictionary. For
analysis are reviewed. Secondly, the recommendation systems example, in (1), Sentiment (F ) represents the total sentiment
using metrics of sentiment analysis are treated. of sentence F, and the variable dictionary .value is the
A. Sentiment Analysis sentiment value defined in the dictionary of each word Wi that
Sentiment analysis can be performed using three composing the sentence.
approaches, as shown in Fig. 1, the corpus-based approach
using machine learning [15], the lexicon-based approach using m
a word dictionary [16] and a hybrid-based method which Sentiment ( F )   dictionary.value(Wi ) (1)
combines both approaches [17]. i 1

In the sentence "I like beautiful shoes" only the words "like"
and "beautiful" have sentiment intensity values, of +2 and +3,
respectively, in a word-based dictionary. Using (1), the result
of the total sentiment of this sentence in (2) is equal to +5;
therefore, the sentence has a positive sentiment intensity
value:

m
Sentiment ( F )    2  (3)  5 (2)
i 1

In order to calculate the sentiment intensity of more

Fig. 1. Different approaches for performing the sentiments analysis. complex sentence it is necessary to identify the verb tenses,

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply
 R. L. Rosa et al.: Music Recommendation System Based on User’s Sentiments Extracted from Social Networks 361

adverbs and n-grams. This study uses a metric of sentiments user feedback to be more effective [24]. Users do not often
based on [1], because this metric considers different words, like to write manual feedback; it is therefore important to use
grammatical classes and verb tenses. an automatic technique to collect user feedback. Sentiment
Sentiment analysis is beginning to be explored in many analysis is a way of collecting users’ satisfaction feedback
fields, but there are many points to be studied, such as whether automatically.
the user’s profile really influences a sentiment metric, the The user’s profile is another important factor that must be
characteristics of which must be considered and how to included in a recommendation system, as collected by a
perform the association between the user’s profile and the questionnaire or rating [1], [24]. On social networks it is
sentiment intensity metric. For example, Thelwall et al. [20] possible to capture some user profile details, such as gender,
states that the sentiment intensity may vary depending on age, educational level and city of birth, automatically. 93.8%
gender, but the association with a metric was not defined. of people fill the gender category [25] and almost 70% of
Following this general review of sentiment analysis metrics, people fill in their country, education and age on social
it can be concluded that the lexicon-based metrics do not networks [26].
differentiate the sentiment depending on gender, age or other Studies on music recommendation systems based on
user’s characteristic. emotions [6] usually suggest songs depending on a manual
evaluation of emotion through emoticon faces, as shown in
B. Recommendation Systems Fig. 2. However, for a system to represent the person's mood
There are three recommendation systems approaches, swings over the course of a day, one would have to choose
content-based, collaborative and hybrid-based. The content- their mood manually and daily in the morning, afternoon and
based approach works with the association between the evening. [20].
description of an item and the user’s profile; the suggestion of
items is based on the user’s preferences. The collaborative-
based approach analyses the user behavior and preferences
and explores similar preferences between people [21], [22].
The hybrid approach combines both methods.
A recommendation system is allocated on a server [23]
containing a database with users’ preferences or history. The Fig. 2. Example of emoticon faces used on music recommendation
advantages of using the server allocation are that they do not systems.
overload the memory of the user’s device with data, since the
client-side application is of low complexity. There is a data It is possible to obtain the user’s emotion automatically
transmission flow between the client and server, but the through sensors [4], [5]. The equipment for measuring
current data network is not as restricted as it used to be. emotions includes electroencephalogram signals and
In the recommendation systems, the sentiment analysis electrodes for the acquiring peripheral physiological signals
began to be explored to suggest more updated contents and [5]. However, people do not generally have a sensor to
based on the person's mood and feelings. measure physical parameters to constantly transmit this
Tag-based recommendation [7], [8], [13] focuses on music information by Internet.
use suggestions to search for words and count the word Ideally, current recommendation systems calculate the
frequency; the system recommends songs containing the sentiment intensity of sentences extracted from real social
words that were searched by people. However, the networks as opposed to extracting sentences from a specific
recommendations are only based on words already searched trusted network [27].
by the user, limiting the suggestions of new words. As stated before, most of the recommendation systems send
The web semantic recommendations consider syntactic a suggestion based on both variables, based on the user history
similarity metrics to offer similar preferences. However, more and on the user’s profile, but nowadays, the recommendations
intelligent techniques are necessary to offer contents in a more are starting to explore making more suggestions based on
flexible way, related to user’s preference and semantic sentiment analysis. The possibility of recognizing if a person
associations [10]. Content-based recommendations consider is happy or sad and suggesting content in accordance to their
the relations between words and ontology to recommend current mood state help to discover when the person is more
contents and not the user’s feeling variation for recommending likely to receive suggestions; allowing the user to feel less
contents. flooded with general or past suggestions.
Recommendation systems in specific areas such as in
multimedia need to suggest contents based on the person’s III. THE PROPOSED SENTIMENT INTENSITY METRIC
current mood, emotion and feeling because the person chooses The proposed sentiment model is presented in this section.
a particular film or song depending on their current mood and The model shows that if a sentiment metric does not consider
feeling. the user’s profile, the metric is not able to generate a real
The user feedback in any recommendation system is an sentiment intensity value.
important tool to be considered. Studies which focus on digital A correction factor, which is discovered by subjective tests,
TV or music recommendations make suggestions according to needs to be applied in accordance with the user’s profile on

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply
 362 IEEE Transactions on Consumer Electronics, Vol. 61, No. 3, August 2015

social networks. The subjective tests were performed in two The assessors were monitored to ensure they collected all the
phases; firstly, in a laboratory environment to find which sentences they posted on social networks every hour over a test
parameters of the user’s profile affect the sentiment value of a period of 3 weeks. After the first day, the tests were carried out
sentence and to evaluate the performance and cognitive factors from a distance because people were not available to be at the
of the framework. In the second phase, a remote subjective laboratory for several hours at a time during the test period. The
method was used to validate the proposed model. The tests social network username was already known and the script
conducted in a laboratory served to test initial theories, and the captured the user's sentences automatically. After 3 weeks, all the
remote method served to validate these theories. captured sentences were analyzed by both the Sentimeter-Br2 and
the assessor who posted the sentences.
A. Initial Studies using Subjective Tests in a Laboratory At the end of the 3-week period, the assessors went to the
Environment laboratory again to test the performance of the framework. Over a
At that stage, initial subjective tests were performed in a 15-day period, each assessor chose a day to evaluate the
laboratory environment by Portuguese native-speaking application through tests with a standard mobile phone.
assessors. In the tests, it was possible to work with a wider In total, 19,600 sentences were extracted from the social
diversity of assessor profiles, such as region of birth (North, network and evaluated using the scale shown in Table II. Out of
Northeast, Midwest, Southeast and South of Brazil), religion, the 19,600, only 652 sentences were discarded because they were
race, educational level, among other characteristics discovered considered spam.
through the web interface questionnaire presented in Fig. 3. Access logs were collected to study the users' routines. In the
Furthermore, a question about which musical genre the experiments, the observation that each user had a custom period
assessor prefers was asked in the questionnaire, based on of time to access and post sentences on social networks was
his/her emotional state (sad, happy and calm); the person made, with a window of 5 to 20 minutes; for instance, user A
could choose one or two musical genre options. preferred to post sentences between at 10:00 pm with a window
of 15 minutes, and later, user A usually posted more sentences
between 9:45 pm to 10:15 pm.
The average value of the access time to the social network is
added to the music recommendation framework in order to
capture the sentences of each user. This information is useful for
capturing sentences in a customized period of time instead of
randomly and constantly capturing the sentences. The framework
thereby saves more processing memory and energy resources.
A sentiment correction factor based on the user profile was
obtained, through the subjective tests. The mathematical model
represents a correction factor to be applied to traditional sentiment
metrics.
B. The Proposed Correction Factor obtained through User´s
Profile
The results of the initial studies using subjective tests helped to
create the proposed mathematical model of the correction factor
(CF) that adjusts Sentimetr-Br2 giving the new eSM metric as a
result, as shown in Fig. 4.

Fig. 3. Web interface available for the assessors to answer the

questions.

The initial characteristics extracted from Fig. 3 helped to

know what person's characteristics could affect a sentiment
Fig. 4. Sentiment correction factor (CF) function used to adjust the
metric. sentiment intensity obtained by a sentiment intensity metric.
The assessors' answers were collected and the field
corresponding to age was separated into four groups. The eSM ( F1 ) S value of a sentence F1 is given by (3). The
The studies were performed by 200 assessors, consisting of
100 women and 100 men; each participant wrote his or her eSM ( F1 ) S represents the sentiment intensity value of the
musical preference and filled out a questionnaire with his or sentence F1. This value was obtained from the subjective test
her profile. Later, each person posted sentences on social results, which represent the sentiment intensity value given by
networks, which were captured by a script. These sentences the assessors. The eSM ( F1 ) is the sentiment intensity value
were then analyzed by both the same person who posted the of the sentence determined by the Sentimeter-Br2 metric.
sentences and by the Sentimeter-Br2 metric. The person
evaluated each sentence on a sentiment scale from +5 to -5 in eSM ( F1 ) S  SM ( F1 ) * C  exp( a1 * A1  a 2 * A2  ... (3)
the first phase of the tests.  a n * An  g 1 * M  g 2 * F  e1 * G  e2 *nG )

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply
 R. L. Rosa et al.: Music Recommendation System Based on User’s Sentiments Extracted from Social Networks 363

Where: C is a scale constant; a1…an are binary factors weeks the system has logged the times of day that the person
related to age ranges, if one of them is equal to one, the others commonly uses the social network. The system thereby only
are zeros; A1…An are the weight factors of each age range. captures sentences at that time of day or week, with a time
This paper considers four ranges; g1 and g2 are binary factors window of ±20 min.
related to the gender, if one of them is equal to one, the other As can be observed in Fig. 5, the framework follows four
is zero; M and F are the weight factors of gender, man or recommendation system models. Firstly, the acquisition of
woman, respectively; e1 e e2 are binary factors related to user’s profile is performed and the data is sent to the database
educational level (higher education or not); G e nG are the (1a); the monitoring of user’s routine starts(1b) and (2), in
weight factors of educational level, higher education or not, which the time and date is recorded in a log and sent to the
respectively. database (3) and to the recommendation engine (4a) and (4b);
It is worth noting that all the parameters presented in Table the user’s sentence are extracted from the social networks (5)
II were not considered in (3), because subjective test results and the sentences are sent to the extraction module engine (6);
demonstrated that they do not affect the eSM ( F1 ) S value. later, the sentences are stored in the database (7) and sent to
The linear and exponential functions were tested, and the the metrics to calculates the sentiment intensity (8) and (9).
last function presented the lowest squared error. The sentiment intensity values are calculated by the metrics
The metric Sentimeter-Br2 was considered in this paper, but and the sentiment associated to the user’s profile (11).The
any sentiment intensity metric can be used to calculate eSM is calculated using (10) and (11). The 3-sentiment metrics
(12a), (12b), (12c) and the randomly selected song (12d) are
eSM ( F1 ) in (3). sent to the Recommendation Engine, which requires (13) the
selected songs and receives them (14). Finally, these song
IV. PROPOSED RECOMMENDATION SYSTEM BASED ON recommendations are sent to the user application (15).
SENTIMENTS If no sentence is posted on social networks, then songs of
This section covers the details and methodology of the his or her preference style are recommended.
proposed music recommendation system based on the eSM A. Databases
sentiment metric - the system architecture is introduced in Fig. 240 songs are extracted from a Brazilian music portal and
5. categorized in sentiment intensity and emotions by specialists
The mathematical model obtained in (3) is applied to the in music; the songs are stored as file streams in MPEG-I Layer
proposed recommendation system to recommend music based III audio coding scheme known as MP3 in the database to be
on the user’s sentiments in a customized period of time. used in the recommendation system.
The musical categories used in the recommendation system
are based on happy, sad, angry and calm emotions [28], shown
in Fig. 6, which are based on Russell's circumplex model [29]
of affect with arousal and valence dimensions. Each song used
in the framework was classified in one or two categories, for
example, a song can be considered calm and sad, at the same
time.

Fig. 6. Plot of emotions used to categorize the musical genres: happy,

sad, angry and calm.
Fig. 5. Framework architecture of the proposed recommendation
system based on sentiment intensity metrics.
A set of records containing the name of the song, style,
The recommendation system has a user’s profile database, singer, sentiment intensity and emotion is stored in the database.
which contains the user's musical preference, the profile The majority of the songs have 3.0 MB as maximum size and
parameters as age, gender, educational level, musical the average length is in the range of 2 to 3 minutes. The
preference based on his/her emotional state and the social following types of music were considered: sad, happy and calm.
network username of the person who completed the The database also stores the user’s profile, social network
questionnaire on the system for the first time. Initially, the username, category and the user's favorite song. The data is
system captures every sentence posted by a person; after three stored in an Open Source database.

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply
 364 IEEE Transactions on Consumer Electronics, Vol. 61, No. 3, August 2015

B. Client Application For this project, the assessors were selected from a
The programming languages used in the client and in the commercial crowdsourcing portal. 300 remote Portuguese-
server-side applications are Open Source languages. The client speaking users were selected to answer the questions of Table
web application is written in the Hypertext Preprocessor II, which parameters are used in eSM ( F1 ) and in the
(PHP), JavaScript Object Notation (JSON) and HTML5. recommendation system. Also, they evaluated the
The web interface made in PHP and HTML5 is presented to performance of the proposed recommendation system.
the person with the song suggestion; the music emotion
classification, song name, singer and the song is presented on TABLE II
the interface. As shown in Fig. 7, the person reads the music PARAMETERS COLLECTED FROM THE ASSESSORS
suggestion name according to the four recommendations and Field Kind
clicks on the plus symbol to listen the recommended song. Gender man, woman

Educational level higher education or not

+0.1 until +5 (positive

sentences),
Rating scale to score the posted
-0.1 until -5 (negative sentences)
sentences
or neutral sentences

sad emotional state: sad, happy or

calm
Musical preference based on happy emotional state: sad,
his/her emotional state happy or calm
calm emotional state: sad, happy
or calm

Fig. 7. Main menu screen presented in the mobile device with four Later, the assessors answered the questionnaire; they were
recommendation models. separated into different groups taking into account age, gender
and educational level.
C. Server
The framework has the modules engine of recommendation, V. EXPERIMENTAL RESULTS
sentence extraction and database storage. The server uses the Subjective tests were performed in a laboratory
Apache Web Server and PHP modules. The PHP language is a environment proposing to determine the eSM ( F1 ) S metric,
server-side programming language and communicates with a
relational database to store the data about the user and music. the perceived resources consumed in the electronic device and
The sentences are extracted from the social network by an the usability of the mobile application. Finally, the remote
automatic script written in the PHP language and JSON. The method was used to validate the performance of
databases and the engine of the sentiment intensity metric the eSM ( F1 ) S , in which the satisfaction in accordance with
occur on the server side, so that the user's device is not the recommendation messages is measured. The results are
overloaded, and only needs a network communication to explained as follows.
transfer data.
Each time the user writes sentences on social networks, the A. Subjective Tests in a Laboratory Environment
server collects the sentences in the period of time registered in Assessors went to a laboratory and the tests were conducted
the log and sends the suggestions in another period of time. individually with no disturbing noises. At the beginning of the
For instance, user A only posts 4 sentences a day and user B tests, an explanation was given stating that each assessor
prefers to post sentences every 10 minutes throughout the day. should fill in a questionnaire with his or her user’s profile,
The songs are suggested every 20 minutes for both users, but post sentences and score each sentence. At the end of 3 weeks,
with a limit of 12 suggestions in the morning, afternoon and the assessor should evaluate the framework performance.
evening. This is because the person's emotion stabilizes for a Fig. 8 presents the weight of each parameter considered in
few hours of the day, normally changing from one period of (3). The results show that the gender and age parameters are
the day to another, for example, changing from morning to the the most influential factors in a sentiment intensity metric.
afternoon [19]. Notice that the recommendation system needs to extract
simple user’s characteristics, as age, gender, educational level
D. Evaluation Tests using Remote Assessors and song preferences based on the user's emotion. The
Tests conducted by remote assessors are valid to evaluate application does not need to extract complex user’s
the performance of framework and applications [30]-[32]. characteristics, as monthly income, religion and race because
Remote tests can be performed though commercial platforms these characteristics do not affect the sentiment analysis.
[33], in which workers perform a task and receive a monetary Therefore, the study does not violate any privacy rule and can
compensation, or solutions that use voluntary workers. be applied in any country.

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply
 R. L. Rosa et al.: Music Recommendation System Based on User’s Sentiments Extracted from Social Networks 365

TABLE IV
SPECIFICATIONS OF THE ELECTRONIC DEVICE USED IN THE TESTS

Specifications Values

Battery Capacity 2,600 mAh

Touchscreen Yes
Display Resolution 1080 x 1920 pixels
Display Size 5.0-inch touchscreen - Full HD
Colours 16M colors
Processor 1.6 GHz octa-core
Random Access Memory (RAM) 2Gb
Internal data storage 16Gb

The usability parameters tested in the proposed framework

included: (1) the prompting parameter that refers to all the
means that help users to recognize the alternatives when
Fig. 8. Weight factors obtained by subjective tests performed in several actions are presented in the system; (2) the immediate
laboratory on the parameters: gender, age and educational level. feedback that corresponds to a fast response from the system
with the information provided in the requested transaction; (3)
Assessors evaluated the perceived consumed resources the legibility, in which the information presented on the screen
from the recommendation system application running in the must facilitate the reading of this information; (4) the
electronic device and presented in Table III. The scores are concision that concerns perceptual workload for individual
based on a 5-point scale, in which a grading value of 5 inputs or outputs; (5) the user control, the system's users
means excellent quality and an imperceptible interruption should always be in control of the system; (6) the flexibility
and a grading value of 1 means bad quality or a very that reflects possible ways of achieving a given goal; (7) the
annoying interruption. user experience that refers to the means available to take into
The high average values of parameters presented in account the level of user experience in the system; (8) the
Table III show that the person perceives that the application error protection that detects and prevents errors in general; (9)
consumes low resources in the electronic device. the consistency that refers to the fact that information should
always be presented in similar procedures to access the menu
TABLE III options.
PERCEIVED VALUE OF CONSUMED RESOURCES IN THE ELECTRONIC
DEVICE AND RESPECTIVE EVALUATION IN ACCORDANCE WITH A 5- TABLE V
POINT SCALE PERCEIVED VALUE OF USABILITY PARAMETERS OF THE FRAMEWORK IN
ACCORDANCE WITH A 5-POINT SCALE
Performance Regarding the Perceived Average
Parameters Value
Usability Parameters Perceived Value
Latency of the recommendation 4.2
system in general Prompting 4.9
Immediate Feedback 4.6
Energy consumption of the 4.2 Legibility 4.8
recommendation system application Concision 4.2
User Control 4.1
Flexibility 4.4
Apparent network resource 4.5
User Experience 3.9
consumption
Error Protection 4.1
Consistency 4.7
The recommendation system application was tested using a
mobile device with a Wireless Fidelity interface (Wi-Fi) to The goal of using the usability parameters in an application
transfer the data between the client and the server. The device assessment is to show that an application must consider
characteristics are shown in Table IV. parameters of ergonomic criteria of usability without
The application was built to be performed in 1 core consuming many resources from the electronic devices. The
processor of 1.6 GHz octa-core because it is a simple code to ergonomic parameters are therefore important to present a
be applied in a recommendation system. useful and easy system platform to people.
Some usability parameters [34] that consider extrinsic and
intrinsic properties [35] were also analyzed by the assessors in B. Performance Evaluation of the Proposed
relation to the framework. The 5-point scale average value of Recommendation System
the parameters is shown in Table V. These usability properties The performance evaluation of the proposed
express the transparency of the operative interface [36] of the recommendation system was carried out by the remote
framework. assessors.

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply
 366 IEEE Transactions on Consumer Electronics, Vol. 61, No. 3, August 2015

Table VII shows the results of the users’ satisfaction level The remote subjective tests reached 91% user satisfaction
with the recommendation systems. The answer options are in regarding the eSM in contrast to 65% of a randomly assigned
accordance with a scale based on adjectives described in song suggestion that did not consider a sentiment intensity,
Likert [37], which are: very good, good, neutral, poor and very and 78% user satisfaction was reached by considering only a
poor. This scale represents a qualitative measurement and has sentiment intensity metric, the Sentimeter-Br2.
been widely applied to many studies [38], [39]. The users profile were analyzed and the results showed that
The users evaluated the recommendation system positively 78% of users preferred to listen to a musical genre similar to
using the eSM metric. The results reached 91% users their current emotional state, and only 22% preferred to listen
satisfaction with the framework using the proposed metric to a different musical genre in relation to their current
emotional state. For example, if a person has a state of mood
eSM.
of sadness than this person prefers to listen to a more
TABLE VII
melancholic song.
% OF LIKERT ANSWERS OF THE RECOMMENDATION USING METRICS OF The solution does not include complex programming
SENTIMENT AND RANDOMLY ASSIGNED SONG languages; therefore, the proposed solution consumes low
Randomly resources from current electronic devices. The assessors
assigned evaluated an imperceptible interruption regarding the
song eSM
suggestion
Sentimeter-
SentiStrengh (proposed
consumed resources in the electronic device.
Br2 Perceived parameters of the framework in relation to
(no metric)
sentiment usability present good results and show the importance of the
metric) framework to integrate all these parameters to be a complete
Very good 65% 78% 70% 91%
Good 15% 13% 16% 7%
tool.
Neutral 10% 6% 4% 1% The study presents the sentiment analysis applied to a music
Poor 8% 2% 8% 1% recommendation system; however, sentiment metrics could be
Very poor 2% 1% 2% 0% applied to many other areas.

The correction factor initially applied to Sentimeter-Br2 ACKNOWLEDGMENT

metric to obtain eSM as shown in (3), was also applied to the The authors thank both the Laboratory of Computer
SentiStrengh metric. The results are presented in Table VIII. It
Architecture and Networks (LARC) at Escola Politécnica -
is worth noting that another sentiment intensity metric can be
University of São Paulo and the Computer Science
used.
Department at the Federal University of Lavras for the
TABLE VIII motivation to research in the recommendation system and
% OF LIKERT ANSWERS OF THE RECOMMENDATION USING THE social network area in multimedia services.
SENTIMENT ESM METRIC WITH THE SENTISTRENGH AND SENTIMETER-
B R2
REFERENCES
eSM
eSM [1] R. L. Rosa, D. Z. Rodríguez, and G. Bressan, “SentiMeter-Br: a new
(using
(using SentiStrengh social web analysis metric to discover consumers' sentiment,” in Proc.
Sentimeter-Br2
metric) IEEE International Symposium on Consumer Electronics, Hsinchu,
metric)
Very Good 84% 91% Taiwan, pp. 153-154, Jun. 2013.
Good 9% 7% [2] R. L. Rosa, D. Z. Rodriguez, and G. Bressan, “Music recommendation
Neutral 5% 1% system based on user s sentiments extracted from social networks,” in
Poor 2% 1% Proc. IEEE International Conference on Consumer Electronics, Las
Very Poor 0% 0% Vegas, USA, pp. 408-409, Jan. 2015.
[3] Y. Shi, M. A. Larson, and A. Hanjalic, “Towards understanding the
challenges facing effective trust-aware recommendation,” in Proc. on
Recommender Systems and the Social Web, Barcelona, Spain, pp. 40-43,
VI. CONCLUSION Sep. 2010.
[4] J. Healey, R. Picard, and F. Dabek, "A new affect-perceiving interface
The subjective tests results highlight the importance of and its application to personalized music selection," in Proc. Workshop
considering the user’s profile in a sentiment metric. Thus, the Perceptual User Interfaces, Nara, Japan, pp. 4-6, Nov. 1998.
tests in the laboratory environment have shown what [5] S. Koelstra, C. Muhl, M. Soleymani, J. Lee, A. Yazdani, T. Ebrahimi, T.
Pun, A. Nijholt, and I. Patras, “DEAP: a database for emotion analysis
parameters may influence the final sentiment intensity of a using physiological signals,” IEEE Trans. Affect. Comput., vol. 3, no. 1,
sentence. Based on the results, a correction factor was pp. 18-31, Jan. 2012.
obtained, which depend on age, educational level and gender. [6] K. Yoon, J. Lee, and M.-U. Kim, "Music recommendation system using
The correction factor was used to obtain a more real sentiment emotion triggering low-level features," IEEE Trans. Consumer
Electron., vol. 58, no. 2, pp. 612-618, May 2012.
intensity value. [7] C.-M. Chen, M.-F. Tsai, J.-Y. Liu, and Y.-H. Yang, “Using emotional
The new sentiment intensity metric, eSM, improved the context from article for contextual music recommendation,” in Proc.
music recommendation system, showing that the sentiments ACM International Conference on Multimedia, New York, USA, pp.
can change depending on the user's profile. The tests showed 649-652, Oct. 2013.
[8] R. Cai, C. Zhang, C. Wang, L. Zhang, and W.-Y. Ma, “MusicSense:
that the weight factor used in the eSM can be applied to contextual music recommendation using emotional allocation
another sentiment metric, for instance, the SentiStrengh modeling,” in Proc. International Conference on Multimedia, Augsburg,
metric. Germany, pp. 553-556, Sep. 2007.

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply
 R. L. Rosa et al.: Music Recommendation System Based on User’s Sentiments Extracted from Social Networks 367

[9] A. G. Crespo, R. C. Palacios, J. M. G. Berbís, and F. G. Sánchez, [29] J. A. Russell, “A circumplex model of affect,” Journal of Personality and
"SOLAR: social link advanced recommendation system", Future Social Psychology, vol. 39, no. 6, pp. 1161-1178, Dec. 1980.
Generation Computer Systems, vol. 26, no. 3, pp. 374-380, Mar. 2010. [30] A. H. Kronbauer, C. A. S. Santos, and V. Vieira, "Smartphone
[10] Y. B. Fernandez, J. P. Arias, A. G. Solla, M. R. Cabrer, and M. L. applications usability evaluation: a hybrid model and its
Nores, "Providing entertainment by content-based filtering and semantic implementation," in Proc. International Conference on Human-Centered
reasoning in intelligent recommender systems," IEEE Trans. Consumer Software Engineering, Toulouse, France, pp. 146-163, Oct. 2012.
Electron., vol. 54, no. 2, pp. 727-735, May 2008. [31] X. Gu, Z. Xu, T. Wang, and Y. Fang, "Trusted service application
[11] S. Rendle, "Factorization machines with libFM," ACM Trans. Intell. framework on mobile network," in Proc. International Conference on
Syst. Technol., vol. 3, no. 3, pp. 1-22, May 2012. Ubiquitous Intelligence and Computing, Fukuoka, Japan, pp. 979-984,
[12] F. A . Nielsen, "A new ANEW: evaluation of a word list for sentiment Sep. 2012.
analysis in microblogs," in Proc. Workshop on Making Sense of [32] D. Z. Rodriguez, R. L. Rosa, E. A. Costa, J. Abrahao, and G. Bressan,
Microposts: Big Things come in Small Packages, Crete, Greece, pp. 93- “Video quality assessment in video streaming services considering user
98, May 2011. preference for video content,” IEEE Trans. Consumer Electron., vol. 60,
[13] H. H. Kim, “A semantically enhanced tag-based music recommendation no. 3, pp. 436-444, Aug. 2014.
using emotion ontology,” in Proc. Asian Conference on Intelligent [33] Q. Xu, Q. Huang, and Y. Yao, “Online crowdsourcing subjective image
Information and Database Systems, Kuala Lumpur, Malaysia, pp. 119- quality assessment,” in Proc. ACM International Conference on
128, Jan. 2013. Multimedia, Nara, Japan, pp. 359-368, Oct. 2012.
[14] A.C.M. Fong, B. Zhou, S.C. Hui, G.Y. Hong, and T. A. Do, "Web [34] J. M. C. Bastien, and D. Scapin, “Ergonomic criteria for the evaluation
content recommender system based on consumer behavior modeling," of human-computer interfaces,” Institut National de Recherche en
IEEE Trans. Consumer Electron., vol. 57, no. 2, pp. 962-969, May 2011. Informatique et en Automatique, INRIA, Rocquencourt, France,
[15] Q. Ye, R. Law, and B. Gu, “The impact of online user reviews on hotel Technical Report 156, 1993.
room sales,” International Journal of Hospitality Management, vol. 28, [35] B. Senach “Ergonomic evaluation of the human-computer interfaces: a
no. 1, pp. 180-182, Jul. 2009. review of the literature,” Research report, INRIA, Sophia Antipolis,
[16] R. Feldman, “Techniques and applications for sentiment analysis,” ACM Technical Report 1180, 1990.
Commun., vol. 56, no. 4, pp. 82-89, Apr. 2013. [36] P. Rabardel, and P. Béguin, “Instrumented mediated activity: from
[17] S. Poria, A. Gelbukh, B. Agarwal, E. Cambria, and N. Howard, "Sentic subject development to anthropocentric design", Theor. Issues Ergonon.
demo: a hybrid concept-level aspect-based sentiment analysis toolkit," in Sci., vol.7, no. 5, pp. 429-461, Feb. 2005.
Proc. European Semantic Web Conference, pp. 31-35, Crete, Greece, [37] R. Likert, “A technique for the measurement of attitudes,” Archives of
May 2014. psychology, vol. 22, no. 140, pp. 1-55, Jun. 1932.
[18] C. M. Whissell, “The Dictionary of Affect in Language,” in Emotion: [38] A. Alexandrov, “Characteristics of single-item measures in Likert scale
Theory, Research and Experience. The Measurement of Emotions, format,” Electron. Journal of Business Research Methods, vol. 8, no. 1,
R.Plutchik and H. Kellerman, eds., Academic Press, vol. 4, pp. 113-131, pp. 1-12, Sep. 2010.
1989. [39] H. F. Hoffman, and F. Lehner. “Requirements engineering as a success
[19] G. Paltoglou, and M. Thelwall, “Twitter, MySpace, Digg: unsupervised factor in software projects,” IEEE Software, vol. 18, no. 4, pp. 58-66,
sentiment analysis in social media,” ACM Trans. on Intelligent Systems Jul. 2001.
and Technology, vol. 3, no. 4, pp. 1-19, Sep. 2012.
[20] M. Thelwall, D. Wilkinson, and S. Uppal, “Data mining emotion in BIOGRAPHIES
social network communication: gender differences in MySpace,” J. Am.
Soc. Inf. Sci. Technol., vol. 61, no. 1, pp. 190-199, Sept. 2010. Renata Lopes Rosa received her M.S. degree from the
[21] D. Yang, and W. Lee, "Music emotion identification from lyrics," in University of São Paulo - USP (2009). She is a Ph.D.
Proc. IEEE International Symposium on Multimedia, San Diego, student at Escola Politécnica from the University of São
California, USA, pp. 624-629, Dec. 2009. Paulo (EPUSP). Her current research interests include
[22] G. Qiu, F. Zhang, J. Bu, and C. Chen, “Domain specific opinion computer networks, quality of experience of multimedia
retrieval,” in Proc. Asia Information Retrieval Symposium on service, social networks and recommendation systems.
Information Retrieval Technology, Sapporo, Japan, pp. 318-329, Oct.
2009.
[23] R. L. Rosa, D. Z. Rodriguez, V. A. Souza, and G. Bressan, Demóstenes Zegarra Rodríguez received his M.S. degree
“Recommendation system based on user profile extracted from an IMS (2009) and PhD degree (2013) from the University of São
network with emphasis on social network and digital TV,” in Proc. Latin Paulo - USP. He is currently an Adjunct Professor at the
America Networking Conference, Quito, Ecuador, pp. 40-47, May 2011. Department of Computer Science, Federal University of
[24] S. E. Shepstone, Z.-H Tan, and S.H Jensen, "Audio-based age and Lavras, Brazil. He has a solid knowledge in
gender identification to enhance the recommendation of TV content," Telecommunication Systems and Computer Science based
IEEE Trans. Consumer Electron., vol. 59, no. 3, pp.721-729, Aug. 2013. on 14 years of professional experience in important
[25] A. C. Lampe, N. Ellison, and C. Steinfield, “A familiar Face(book): companies. His research interest includes QoS and QoE in Multimedia
profile elements as signals in an online social network,” in Proc. services, Digital TV, recommendation systems, social networks and architect
Conference on Human Factors in Computing Systems, New York, USA, solutions in Telecommunication Systems.
pp. 435-444, May 2007.
[26] A. Yamada, M. Hara, and Y. Miyake, "Exploiting privacy policy Graça Bressan got her PhD degree (1986) in Digital
conflicts in online social networks," CMU Cylab Technical Report, Feb. Systems at Escola Politécnica from the University of São
2012. Paulo (EPUSP). At the Computer Engineering
[27] J. Kunegis, A. Lommatzsch, and C. Bauckhage, “The Slashdot Zoo: Department of EPUSP, she teaches and develops research
mining a social network with negative edges”, in Proc. International on computer network. Her research interests includes
Conference on World Wide Web, New York, USA, pp. 741-750, May computer networks and digital television, distributed
2009. systems, QoS mechanisms, middleware for Digital TV,
[28] C. Laurier, M. Sordo, J. Serrà, and P. Herrera, "Music mood interactive digital TV, recommendation systems, video-conferencing,
representations from social tags," in Proc. International Conference on modeling and performance analysis of networks, and application in distance
Music Information Retrieval, Kobe, Japan, pp. 381-386, Oct. 2009. education.

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 16,2021 at 11:38:27 UTC from IEEE Xplore. Restrictions apply