Abstract

An emotion-based smart music player utilizes technology to personalize the music experience
based on a user’s current emotional state. This project explores different techniques to achieve this,
including Facial Expression Recognition: By capturing the user’s face through a camera and employing
machine learning algorithms (often Convolutional Neural Networks), the system identifies emotions
like happiness, sadness, anger, or neutrality.

Speech Emotion Recognition: Analyzing the user’s speech patterns (independent of the meaning of the
words) allows the system to gauge emotions.

Music Emotion Classification: Music is categorized based on the emotions it typically evokes. This can
be done through audio feature extraction and machine learning.

By combining these elements, the music player can:

Automatically generate playlists: Based on the user’s detected emotion, the system selects songs
categorized to match that feeling.

Recommend music: The player suggests songs that could elevate a user’s mood or complement their
current emotional state.

This project offers a novel and potentially therapeutic approach to music listening, catering to the
user’s
emotional well-being and enhancing the overall music experience.

1
Contents
1 Introduction..................................................................................................................................5

1.1 Background...............................................................................................................................6

1.2 Motivation................................................................................................................................6

1.3 Objectives.................................................................................................................................6

1.4 Scope........................................................................................................................................6

2 System Overview...........................................................................................................................7

2.1 Capture Image...........................................................................................................................7

2.2 Face Detection Using CNN..........................................................................................................7

2.3 Feature Extraction.....................................................................................................................8

2.4 Emotion Detection....................................................................................................................8

2.5 Music Classifier.......................................................................................................................10

2.6 Music Database.......................................................................................................................10

2.7 Recommended Playlists...........................................................................................................10

2.8 Play The Songs(Output)...........................................................................................................10

3 Proposed Algorithms..................................................................................................................12

3.1 CNN Algorithm........................................................................................................................12

3.1.1 Convolutional Layer........................................................................................................12

3.1.2 Strides............................................................................................................................13

3.1.3 Padding...........................................................................................................................14

4 System Requirements..................................................................................................................15

4.1 Hardware requirements..........................................................................................................15

4.2 Software Requirements...........................................................................................................15

4.3 Language Specification............................................................................................................15

4.3.1 Python............................................................................................................................15

4.3.2 History of Python............................................................................................................15

4.3.3 Applications of Python....................................................................................................15

4.3.4 Features of Python..........................................................................................................16

4.4 Feasibility Study......................................................................................................................16

4.4.1 Economical Feasibility.....................................................................................................16

2
 4.4.2 Technical Feasibility........................................................................................................16

4.4.3 Social Feasibility.............................................................................................................16

5 System Anlysis.............................................................................................................................17

5.1 Purpose..................................................................................................................................17

5.2 Scope......................................................................................................................................17

5.3 Existing System.......................................................................................................................17

5.4 Proposed System.....................................................................................................................17

6 System Designs............................................................................................................................18

6.1 Input Design...........................................................................................................................18

6.2 Output Design.........................................................................................................................18

6.3 Data Flow diagram(DFD).........................................................................................................18

6.4 UML Diagrams.........................................................................................................................19

6.5 UseCase Diagram.....................................................................................................................20

6.6 Sequence Diagram...................................................................................................................21

6.7 Activity Diagram.....................................................................................................................21

7 Modules.......................................................................................................................................23

7.1 Data Collection Module............................................................................................................23

7.2 Emotion Extraction Module.....................................................................................................23

7.3 Audio Extraction Module.........................................................................................................23

7.4 Emotion - Audio Integration Module........................................................................................23

8 System Implementation..............................................................................................................24

8.1 System Architecture................................................................................................................24

9 System Testing.............................................................................................................................25

9.1 Test Plan.................................................................................................................................25

9.2 Verification.............................................................................................................................25

9.3 Validation...............................................................................................................................25

9.4 Basics of Software Testing.......................................................................................................25

9.4.1 Black Box Testing............................................................................................................25

9.4.2 White Box Testing...........................................................................................................25

9.5 Types of Testing.................................................................................................................25

3
 9.5.1 Unit Testing.....................................................................................................................26

9.5.2 Integration Testing..........................................................................................................26

9.5.3 Functional Testing...........................................................................................................26

9.5.4 System Testing................................................................................................................26

9.5.5 Stress Testing..................................................................................................................26

9.5.6 Performance Testing.......................................................................................................26

9.5.7 Usability Testing..............................................................................................................26

9.5.8 Acceptance Testing..........................................................................................................26

9.5.9 Regression Testinbg........................................................................................................26

9.6 Requirement Analysis..............................................................................................................26

9.7 Functional Requirements.........................................................................................................27

9.8 Usability.................................................................................................................................27

9.8.1 Robustness.....................................................................................................................27

9.8.2 Security..........................................................................................................................27

9.8.3 Reliability.......................................................................................................................27

9.8.4 Compatibility..................................................................................................................27

9.8.5 Flexibility........................................................................................................................27

9.8.6 Safety..............................................................................................................................27

9.9 Non-Functional Requirements.................................................................................................27

9.9.1 Portability.......................................................................................................................27

9.9.2 Performance...................................................................................................................28

9.9.3 Accuracy.........................................................................................................................28

9.9.4 Maintainability................................................................................................................28

10 Conclusion.................................................................................................................................29

References......................................................................................................................................30

4
CHAPTER-1
1 Introduction
The link between music and human emotions has long been acknowledged, with music having the
power to elicit various emotional states. Recently, the utilization of facial emotion recognition
technology has garnered considerable interest in music recommendation systems. A facial emotion-
driven music recommendation system is an inventive solution aiming to tailor music suggestions
according to the listener’s facial expressions. This technology can assess a user’s facial expressions in
real-time and propose music that best suits their current emotional state. This study delves into the
creation and deployment of a facial emotion-driven music recommendation system, covering its core
technology, potential advantages, and the challenges that must be tackled to enhance its efficiency. The
objective of this research is to offer a thorough insight into the capabilities of such a system, along with
its potential applications in the music industry and beyond. The growing accessibility of advanced facial
recognition technology and the expanding array of music streaming services position this technology as
a promising avenue for personalized music recommendations. This system has the potential to

Figure 1: Interface Of Starting Page.

revolutionize the way we interact with music, making it more personalized and emotionally engaging.
By integrating facial emotion recognition technology into music recommendation systems, we can
create a more personalized and enjoyable music experience for listeners. Moreover, this technology can
also have practical applications in other fields such as healthcare, education, and entertainment.
However, as with any emerging technology, there are ethical, privacy, and data security concerns that
need to be addressed. This research aims to identify these concerns and provide recommendations to
ensure the safe and ethical use of this technology. Ultimately, this paper aims to contribute to the
development of facial emotion-based music recommendation systems and provide insights for future
research in this field. To achieve the objectives of this study, we will review and analyze previous
research studies on facial emotion recognition technology and music recommendation systems. We will
also explore various machine learning and deep learning algorithms that are commonly used in
developing facial emotion recognition models. Additionally, we will conduct an empirical study by
collecting data from participants to evaluate the effectiveness of the proposed system. The findings of
this study can have significant implications for the music industry, providing music streaming services
and music marketers with new ways to personalize their offerings. This system can help to increase
listener engagement, enhance user experience, and ultimately, boost revenue. Furthermore, this
technology has practical applications in healthcare, where it can be used to monitor patients’ emotional

5
state and provide personalized therapy. Overall, this study aims to contribute to the understanding of
the potential applications of facial emotion recognition technology in music and other fields, paving the
way for further research and development in this promising area of technology.

1.1 Background
Provide a thorough explanation of the concept of facial expression-based music player systems,
including how advancements in technology, particularly in computer vision and machine learning, have
enabled the development of such systems. Discuss the importance of emotional engagement in user
experiences and how integrating facial expression recognition technology can enhance music playback
by providing a more personalized and immersive experience.

1.2 Motivation
Delve deeper into the motivations behind the project, discussing the potential benefits of facial
expression-based music systems in various domains such as entertainment, therapy, education, and
marketing. Highlight the increasing interest in emotional intelligence and its applications in technology,
emphasizing the significance of understanding and responding to human emotions in interactive
systems.

1.3 Objectives
Elaborate on the specific goals of the project, such as designing and implementing a real-time facial
expression recognition system integrated with a music player to dynamically select and play music
based on the user’s emotional state. Discuss how achieving these objectives could contribute to
advancements in human-computer interaction and user experience design.

1.4 Scope
Provide a detailed overview of the scope of the project, including the technologies, methodologies, and
applications considered. Clarify any limitations or constraints, such as hardware requirements,
available datasets, and time constraints, to set realistic expectations for the project’s outcomes.

CHAPTER-2
2 System Overview
The facial emotion-based song recommendation system consists of three main components: the face
recognition system, the emotion classification system and the song recommendation system. The facial
emotion recognition system captures a video of the listener’s face and extracts facial features such as
eye movement, eyebrow position, and mouth shape. These features are then used to determine the
listener’s emotional state using a trained machine learning model. The song recommendation system
uses a recommendation algorithm to generate song recommendations based on the listener’s emotional
state. The algorithm considers the emotional characteristics of songs, such as tempo, rhythm, and
melody, to generate recommendations that align with the listener’s emotional state. The
recommendations can be personalized based on the listener’s musical preferences and past listening
history.

2.1 Capture Image

Capturing images is a crucial step in facial emotion recognition, with the help of computer vision
libraries such as OpenCV, capturing images and processing them has become much easier and efficient.
Facial emotion recognition systems work by analyzing the various features of a person’s face and
identifying the emotions they are experiencing. To capture the image of a face, a camera is used to take a
photograph of the person. This image is then processed using OpenCV libraries to identify various facial

6
features such as the eyes, nose, and mouth. OpenCV provides a variety of functions for image
processing, including face detection, which can be used to identify the face in an image. Once the face
has been detected, it is possible to extract various features such as the position of the eyes, mouth, and
nose, as well as the shape of the face. Using this information, it is possible to determine the emotion that
the person is experiencing. For example, if the corners of the mouth are turned upward, it is likely that
the person is experiencing happiness. Similarly, if the eyebrows are furrowed and the mouth is turned
downward, the person may be experiencing sadness.

2.2 Face Detection Using CNN

CNNs are a type of deep learning algorithm that is particularly suited to image recognition tasks. The
training process involves feeding the CNN a large number of images and adjusting the weights of the
neurons in the network to minimize the difference between the predicted output and the actual output.
Once the training is complete, the CNN can be used to detect faces in new images. Face detection using
CNNs has many advantages over traditional methods such as Haar cascades. CNNs are much more
accurate and can detect faces in a wide range of lighting conditions, orientations, and poses. They can
also detect multiple faces in an image and provide a bounding box around each face.

Figure 2: Model Guessing Fearful Emotion in Face

2.3 Feature Extraction

Feature extraction involves identifying and extracting the relevant features of a face that are necessary
for emotion recognition. In facial emotion recognition, the features of a face that are most commonly
used include the position and shape of the eyes, eyebrows, mouth, and nose. Other features such as the
texture and color of the skin and the shape of the face may also be used. Haar cascades are one of the
most common techniques for feature extraction in facial emotion recognition. Haar cascades are
essentially a set of features that are used to detect objects in an image. In the case of facial emotion
recognition, these features are designed to detect the eyes, nose, mouth, and other relevant facial
features. Haar cascades work by scanning an image at different scales and sizes, looking for the
presence of the features. The features are essentially rectangular regions of the image with dark and
light regions. By combining multiple features, it is possible to detect more complex objects such as faces.
Once the features have been detected, they can be used to train a machine learning model such as CNN
to recognize emotions.

2.4 Emotion Detection

One of the most effective machine learning algorithms for emotion classification using facial features is
the convolutional neural network (CNN) algorithm. CNNs are a type of deep learning algorithm that are
particularly effective in image recognition tasks. The emotion classification module can be implemented
using a CNN algorithm that is trained on a large dataset of labeled facial expressions. CNN learns to
recognize patterns in the facial features that are associated with different emotions and can accurately
classify the listener’s current emotional state based on these features. One of the advantages of using a

7
CNN algorithm for emotion classification is that it can capture both local and global features of the facial
expression. For example, the algorithm can learn to recognize specific patterns in the position of the
eyes or mouth, as well as broader patterns in the overall shape of the face. Another advantage of CNNs is
that they can be trained using transfer learning techniques. Transfer learning involves using a pre-
trained CNN model that has already been trained on a large dataset of images and finetuning it on a
smaller dataset of facial expressions for emotion classification. This approach can reduce the amount of
data required for training and improve the accuracy of the model. Despite the effectiveness of CNNs for
emotion classification, there are still challenges and limitations to this approach. For example, CNNs
may not be as effective for classifying subtle or nuanced emotions that are expressed in more subtle
ways. Additionally, the accuracy of the model can be affected by variations in lighting conditions, facial
occlusions, and individual differences in facial expressions. Overall, the emotion classification module
using a CNN algorithm is a powerful approach for accurately classifying the listener’s current emotional
state based on facial features. With continued research and development, this approach has the
potential to significantly enhance the music listening experience for users through personalized song
recommendations based on their emotional state.

Figure 3: Model Guessing Sad Emotion in Face

Figure 4: Model Guessing Happy Emotion in Face

8
 Figure 5: Model Guessing Angry Emotion in Face

Figure 6: Model Guessing Neutral Emotion in Face

Figure 7: Model Guessing Surprise Emotion in Face

2.5 Music Classifier

A music classifier is a machine learning model that is trained to recognize different types of music based
on their audio features. These features may include tempo, rhythm, melody, and timbre. To develop a
music classifier for a facial emotion-based music recommendation system, a large dataset of labeled
music samples is needed. The music samples should be labeled with the emotions they evoke, such as
happy, sad, or angry. Once the audio features have been extracted, they can be used to train a machine
learning model such as convolutional neural network. During the training process, the model adjusts its
parameters to minimize the difference between the predicted output and the actual output. Once the

9
music classifier has been trained, it can be used to classify new music samples based on their audio
features. When a user’s facial emotion is recognized by the facial emotion recognition system, the music
classifier can recommend music that matches the user’s emotional state.

2.6 Music Database

To develop a music database for a facial emotion-based music recommendation system, a large
collection of music tracks is needed. These tracks can be sourced from various online music streaming
services, music libraries, or by recording music tracks in-house. Once the music tracks are collected,
they need to be analyzed and tagged with various metadata, including the emotional content of the
music. This can be done using machine learning techniques, where a machine learning model is trained
to recognize different emotions in the music, such as happiness, sadness, or anger. The metadata of the
music tracks, including the emotional content, can be stored in a database, which can be queried by the
facial emotion recognition system to recommend music that matches the user’s emotional state.

2.7 Recommended Playlists

The recommended playlist is a list of music tracks that are selected by the system based on the user’s
facial emotions. Once the user’s emotional state has been determined, the system can query the music
database to recommend music tracks that match the user’s emotional state. The recommended playlist
can be generated using various algorithms, such as collaborative filtering, content-based filtering, or
hybrid filtering. The recommended playlist can be further customized by incorporating additional
factors, such as the time of day, the user’s location, and the user’s activity. For example, the system can
recommend upbeat music tracks in the morning and relaxing music tracks in the evening.

2.8 Play The Songs(Output)

The output of a facial emotion-based music recommendation system is the selection and playback of
music tracks that match the user’s emotional state. This output is typically achieved through a music
player that can stream or play music tracks from the music database. The music player can be
integrated with the facial emotion recognition system to automatically select and play music tracks
based on the user’s emotional state. The music player can be designed with a simple interface that
allows the user to control playback, skip tracks, and adjust the volume. To ensure a seamless listening
experience, the music player can be optimized for different platforms, such as mobile devices, desktops,
and web browsers. The music player can also be customized with additional features, such as the ability
to create playlists, shuffle tracks, and repeat tracks. This output will enhance the user’s listening
experience and provide a personalized music experience.

10
CHAPTER-3
3 Proposed Algorithms
3.1 CNN Algorithm
Convolutional Neural Network is one of the main categories to do image classification and image
recognition in neural networks. Scene labeling, objects detections, and face recognition, etc., are some of
the areas where convolutional neural networks are widely used. CNN takes an image as input, which is
classified and process under a certain category such as dog, cat, lion, tiger, etc. The computer sees an
image as an array of pixels and depends on the resolution of the image. Based on image resolution, it
will see as h * w * d, where h= height w= width and d= dimension. For example, An RGB image is 6 * 6 *
3 array of the matrix, and the grayscale image is 4 * 4 * 1 array of the matrix. In CNN, each input image
will pass through a sequence of convolution layers along with pooling, fully connected layers, filters
(Also known as kernels). After that, we will apply the Soft-max function to classify an object with
probabilistic values 0 and 1.

11
 Figure 8: Feature Learning and Classification Works.

3.1.1 Convolutional Layer

Convolution layer is the first layer to extract features from an input image. By learning image features
using a small square of input data, the convolutional layer preserves the relationship between pixels.

It is a mathematical operation which takes two inputs such as image matrix and a kernel or filter.

Figure 9: Image matrix multiples kernel or filter matrix.

1. The dimension of the image matrix is h×w×d

2. The dimension of the filter is fh×fw×d.

3. The dimension of the output is (h-fh+1)×(w-fw+1)×1

Let’s start with consideration a 5*5 image whose pixel values are 0, 1, and filter matrix 3*3 as:

12
 Figure 10: Matrix Multiplication.

The convolution of 5*5 image matrix multiplies with 3*3 filter matrix is called ”Features Map” and
show as an output.

Figure 11: Matrix Multiplication and its Fetured Matrix.

Convolution of an image with different filters can perform an operation such as blur, sharpen, and
edge detection by applying filters.

3.1.2 Strides
Stride is the number of pixels which are shift over the input matrix. When the stride is equaled to

1. then we move the filters to 1 pixel at a time and similarly, if the stride is equaled to 1

2. then we move the filters to 2 pixels at a time. The following figure shows that the convolution
would work with a stride of 2.

13
 Figure 12: Striding .

3.1.3 Padding
Padding plays a crucial role in building the convolutional neural network. If the image will get shrink
and if we will take a neural network with 100’s of layers on it, it will give us a small image after filtered
in the end. If we take a three by three filter on top of a grayscale image and do the convolving then what
will happen?

Figure 13: Padding.

CHAPTER-4
4 System Requirements
4.1 Hardware requirements
• System : Pentium i3 Processor

• Hard Disk : 500 GB.

• Monitor : 15” LED

• Input Devices : Keyboard, Mouse

• Ram : 2 GB

4.2 Software Requirements

• Operating system : Windows 10

• Coding Language : Python

4.3 Language Specification

4.3.1 Python
Python is a general-purpose interpreted, interactive, object-oriented, and high-level programming
language. It was created by Guido van Rossum during 1985- 1990. Like Perl, Python source code is also

14
available under the GNU General Public License (GPL). This tutorial gives enough understanding on
Python programming language.

4.3.2 History of Python

Python was developed by Guido van Rossum in the late eighties and early nineties at the National
Research Institute for Mathematics and Computer Science in the Netherlands. Python is derived from
many other languages, including ABC, Modula-3, C, C++, Algol-68, SmallTalk, and Unix shell and other
scripting languages. Python is copyrighted. Like Perl, Python source code is now available under the
GNU General Public License (GPL). Python is now maintained by a core development team at the
institute, although Guido van Rossum still holds a vital role in directing its progress.

4.3.3 Applications of Python

1. Easy-to-learn Python has few keywords, simple structure, and a clearly defined syntax. This
allows the student to pick up the language quickly.

2. Easy-to-read Python code is more clearly defined and visible to the eyes.

3. Easy-to-maintain Python’s source code is fairly easy-to-maintain.

4. A broad standard library Python’s bulk of the library is very portable and cross-platform
compatible on UNIX, Windows, and Macintosh.

5. Interactive Mode Python has support for an interactive mode which allows interactive testing
and debugging of snippets of code.

6. Portable Python can run on a wide variety of hardware platforms and has the same interface on
all platforms.

7. Extendable You can add low-level modules to the Python interpreter. These modules enable
programmers to add to or customize their tools to be more efficient.

8. Databases Python provides interfaces to all major commercial databases.

9. GUI Programming Python supports GUI applications that can be created and ported to many
system calls, libraries and windows systems, such as Windows MFC, Macintosh, and the X
Window system of Unix.

10. Scalable Python provides a better structure and support for large programs than shell scripting.

4.3.4 Features of Python

It supports functional and structured programming methods as well as OOP. It can be used as a
scripting language or can be compiled to byte-code for building large applications. It provides
very high-level dynamic data types and supports dynamic type checking. It supports automatic
garbage collection. It can be easily integrated with C, C++, COM, ActiveX, CORBA, and Java.

4.4 Feasibility Study

The feasibility of the project is analyzed in this phase and business proposal is put forth with a very
general plan for the project and some cost estimates. During system analysis the feasibility study of the
proposed system is to be carried out. This is to ensure that the proposed system is not a burden to the
company. For feasibility analysis, some understanding of the major requirements for the system is
essential. The feasibility study investigates the problem and the information needs of the stakeholders.

15
It seeks to determine the resources required to provide an information systems solution, the cost and
benefits of such a solution, and the feasibility of such a solution. The goal of the feasibility study is to
consider alternative information systems solutions, evaluate their feasibility, and propose the
alternative most suitable to the organization. The feasibility of a proposed solution is evaluated in terms
of its components.

4.4.1 Economical Feasibility

This study is carried out to check the economic impact that the system will have on the organization.
The amount of fund that the company can pour into the research and development of the system is
limited. The expenditures mustjustified. Thus the developed system as well within the budget and this
was achieved because most of the technologies used are freely available. Only the customized products
had to be purchased.

4.4.2 Technical Feasibility

This study is carried out to check the technical feasibility, that is, the technical requirements of the
system. Any system developed must not have a high demand on the available technical resources. This
will lead to high demands on the available technical resources. This will lead to high demands being
placed on the client. The developed system must have a modest requirement, as only minimal or null
changes are required for implementing this system.

4.4.3 Social Feasibility

The aspect of study is to check the level of acceptance of the system by user. This includes the process of
training the user to use the system efficiently. The user must not feel threatened by the system, instead
must accept it as a necessity.

CHAPTER-5
5 System Anlysis
5.1 Purpose
The purpose of this document is real time facial expression based music recommender system using
machine learning algorithms. In detail, this document will provide a general description of our project,
including user requirements, product perspective, and overview of requirements, general constraints.
In addition, it will also provide the specific requirements and functionality needed for this project such
as interface, functional requirements and performance requirements.

5.2 Scope
The scope of this SRSdocument persists for the entire life cycle of the project. This document defines the
final state of the software requirements agreed upon by the customers and designers. Finally at the end
of the project execution all the functionalities may be traceable from the SRSto the product. The
document describes the functionality, performance, constraints, interface and reliability for the entire
life cycle of the project.

5.3 Existing System

Nikhil et al. determines the mindset of the user by using facial expression Humans often express their
feeling by their expressions, hand gestures, and by raising the voice of tone but mostly humans express
their feelings by their face. Emotion-based music player reduces the time complexity of the user.
Generally, people have a large number of songs on their playlist. Playing songs randomly does not

16
satisfy the mood of the user. This system helps user to play songs automatically according to their
mood. The image of the user is captured by the web camera, and the images are saved. The images are
first converted from RGB to binary format. This process of representing the data is called a featurepoint
detection method. This process can also be done by using Haar Cascade technology provided by Open
CV. The music player is developed by using a java program. It manages the database and plays the song
according to the mood of the user.

5.4 Proposed System

The proposed system can detect the facial expressions of the user and based on his/her facial
expressions extract the facial landmarks, which would then be classified to get a particular emotion of
the user. Once the emotion has been classified the songs matching the user’s emotions would be shown
to the user. In this proposed system, we develop a prototype in recommendation of dynamic music
recommendation system based on human emotions. Based on each human listening pattern, the songs
for each emotions are trained. Integration of feature extraction and machine learning techniques, from
the real face the emotion are detected and once the mood is derived from the input image, respective
songs for the specific mood would be played to hold the users. In this approach, the application gets
connected with human feelings thus giving a personal touch to the users. Therefore our projected
system concentrate on identifying the human feelings for developing emotion based music player using
computer vision and machine learning techniques. For experimental results, we use openCV for
emotion detection and music recommendation.

CHAPTER-6
6 System Designs
6.1 Input Design
The input design is the link between the information system and the user. It comprises the developing
specification and procedures for data preparation and those steps are necessary to put transaction data
in to a usable form for processing can be achieved by inspecting the computer to read data from a
written or printed document or it can occur by having people keying the data directly into the system.
The design of input focuses on controlling the amount of input required, controlling the errors, avoiding
delay, avoiding extra steps and keeping the process simple. The input is designed in such a way so that it
provides security and ease of use with retaining the privacy. Input Design considered the following
things:

• What data should be given as input?

• How the data should be arranged or coded?

• The dialog to guide the operating personnel in providing input.

• Methods for preparing input validations and steps to follow when error occur.

6.2 Output Design

A quality output is one, which meets the requirements of the end user and presents the information
clearly. In any system results of processing are communicated to the users and to other system through
outputs. In output design it is determined how the information is to be displaced for immediate need
and also the hard copy output. It is the most important and direct source information to the user.

17
Efficient and intelligent output design improves the system’s relationship to help user decision-making.
The output form of an information system should accomplish one or more of the following objectives •
Convey information about past activities, current status or projections of the

• Future.

• Signal important events, opportunities, problems, or warnings.

• Trigger an action.

• Confirm an action

6.3 Data Flow diagram(DFD)

1. The DFD is also called as bubble chart. It is a simple graphical formalism that can be used to
represent a system in terms of input data to the system,various processing carried out on this
data, and the output data is generated by this system.

2. The data flow diagram (DFD) is one of the most important modeling tools. It is used to model the
system components. These components are the system process, the data used by the process, an
external entity that interacts with the system and the information flows in the system.

3. DFD shows how the information moves through the system and how it is modified by a series of
transformations. It is a graphical technique that depicts information flow and the transformations
that are applied as data moves from input to output.

4. DFD is also known as bubble chart. A DFD may be used to represent a system at any level of
abstraction. DFD may be partitioned into levels that represent increasing information flow and
functional detail.

Figure 14: DFD of Emotion Based Music System

18
6.4 UML Diagrams
UML stands for Unified Modeling Language. UML is a standardized general-purpose modeling language
in the field of object-oriented software engineering. The standard is managed, and was created by, the
Object Management Group. The goal is for UML to become a common language for creating models of
object oriented computer software. In its current form UML is comprised of two major components: a
Meta-model and a notation. In the future, some form of method or process may also be added to; or
associated with, UML. The Unified Modeling Language is a standard language for specifying,
Visualization, Constructing and documenting the artifacts of software system, as well as for business
modeling and other non-software systems. The UML represents a collection of best engineering
practices that have proven successful in the modeling of large and complex systems. The UML is a very
important part of developing objects oriented software and the software development process. The
UML uses mostly graphical notations to express the design of software projects.

Figure 15: UML Diagram

19
6.5 UseCase Diagram
A use case diagram in the Unified Modeling Language (UML) is a type of behavioral diagram defined by
and created from a Use-case analysis. Its purpose is to present a graphical overview of the functionality
provided by a system in terms of actors, their goals (represented as use cases), and any dependencies
between those use cases. The main purpose of a use case diagram is to show what system functions are
performed for which actor. Roles of the actors in the system can be depicted.

Figure 16: UseCase Diagram

6.6 Sequence Diagram

A sequence diagram in Unified Modeling Language (UML) is a kind of interaction diagram that shows
how processes operate with one another and in what order. It is a construct of a Message Sequence
Chart. Sequence diagrams are sometimes called event diagrams, event scenarios, and timing diagrams

Figure 17: Sequence Diagram

6.7 Activity Diagram

Activity diagrams are graphical representations of workflows of stepwise activities and actions with
support for choice, iteration and concurrency. In the Unified Modeling Language, activity diagrams can
be used to describe the business and operational step-by-step workflows of components in a system. An
activity diagram shows the overall flow of control.

20
 Figure 18: Activity Diagram

CHAPTER-7

21
7 Modules
• Data Collection Module

• Emotion Extraction Module

• Audio Extraction Module

• Emotion - Audio Integration Module

7.1 Data Collection Module

A survey was collected from users based on 3 parameters which are, 1. What type of songs would they
want to listen to when they are happy? 2. What type of songs would they want to listen to when they are
sad? 3. What type of songs would they want to listen to when they are angry.

7.2 Emotion Extraction Module

The image of the user is captured with the help of a camera/webcam. Once the picture captured, the
frame of the captured image from webcam feed is converted to a grayscale image to improve the
performance of the classifier, which is used to identify the face present in the picture. Once the
conversion is complete, the image is sent to the classifier algorithm which, with the help of feature
extraction techniques can extract the face from the frame of the web camera feed. From the extracted
face, individual features are obtained and are sent to the trained network to detect the emotion
expressed by the user. These images will be used to train the classifier so that when a completely new
and unknown set of images is presented to the classifier, it is able to extract the position of facial
landmarks from those images based on the knowledge that it had already acquired from the training set
and return the coordinates of the new facial landmarks that it detected. The network is trained with the
help of extensive data set. This is used to identify the emotion being voiced by the user.

7.3 Audio Extraction Module

After the emotion of the user is extracted the music/audio based on the emotion voiced by the user is
displayed to the user, a list of songs based on the emotion is displayed, and the user can listen to any
song he/she would like to. Based on the regularity that the user would listen to the songs are displayed
in that order.

7.4 Emotion - Audio Integration Module

The emotions which are extracted for the songs are stored, and the songs based on the emotion are
displayed on the web page. For example, if the emotion or the facial feature is categorized under happy,
then songs from the happy database are displayed to the user.

CHAPTER-8

22
8 System Implementation
8.1 System Architecture
Describing the overall features of the software is concerned with defining the requirements and
establishing the high level of the system. During architectural design, the various web pages and their
interconnections are identified and designed. The major software components are identified and
decomposed into processing modules and conceptual data structures and the interconnections among
the modules are identified. The following modules are identified in the proposed system.

Figure 19: System Architecture Overview

CHAPTER-9

23
9 System Testing
9.1 Test Plan
Software testing is the process of evaluation a software item to detect differences between given input
and expected output. Also to assess the feature of a software item. Testing assesses the quality of the
product. Software testing is a process that should be done during the development process. In other
words software testing is a verification and validation process.

9.2 Verification
Verification is the process to make sure the product satisfies the conditions imposed at the start of the
development phase. In other words, to make sure the product behaves the way we want it to.

9.3 Validation
Validation is the process to make sure the product satisfies the specified requirements at the end of the
development phase. In other words, to make sure the product is built as per customer requirements.

9.4 Basics of Software Testing

There are two basics of software testing: black box testing and white box testing.

9.4.1 Black Box Testing

Black box testing is a testing technique that ignores the internal mechanism of the system and focuses
on the output generated against any input and execution of the system. It is also called functional
testing.

9.4.2 White Box Testing

White box testing is a testing technique that takes into account the internal mechanism of a system. It is
also called structural testing and glass box testing. Black box testing is often used for validation and
white box testing is often used for verification.

9.5 Types of Testing

There are many types of testing like

• Unit Testing

• Integration Testing

• Functional Testing

• System Testing

• Stress Testing

• Performance Testing

• Usability Testing

• Acceptance Testing

24
 • Regression Testing

• Beta Testing

9.5.1 Unit Testing

Unit testing is the testing of an individual unit or group of related units. It falls under the class of white
box testing. It is often done by the programmer to test that the unit he/she has implemented is
producing expected output against given input.

9.5.2 Integration Testing

Integration testing is testing in which a group of components are combined to produce output. Also, the
interaction between software and hardware is tested in integration testing if software and hardware
components have any relation. It may fall under both white box testing and black box testing

9.5.3 Functional Testing

Functional testing is the testing to ensure that the specified functionality required in the system
requirements works. It falls under the class of black box testing

9.5.4 System Testing

System testing is the testing to ensure that by putting the software in different environments (e.g.,
Operating Systems) it still works. System testing is done with full system implementation and
environment. It falls under the class of black box testing.

9.5.5 Stress Testing

Stress testing is the testing to evaluate how system behaves under unfavorable conditions. Testing is
conducted at beyond limits of the specifications. It falls under the class of black box testing.

9.5.6 Performance Testing

Performance testing is the testing to assess the speed and effectiveness of the system and to make sure
it is generating results within a specified time as in performance requirements. It falls under the class of
black box testing.

9.5.7 Usability Testing

Usability testing is performed to the perspective of the client, to evaluate how the GUI is user-friendly?
How easily can the client learn? After learning how to use, how proficiently can the client perform? How
pleasing is it to use its design? This falls under the class of black box testing.

9.5.8 Acceptance Testing

Acceptance testing is often done by the customer to ensure that the delivered product meets the
requirements and works as the customer expected. It falls under the class of black box testing

9.5.9 Regression Testinbg

Regression testing is the testing after modification of a system, component, or a group of related units to
ensure that the modification is working correctly and is not damaging or imposing other modules to
produce unexpected results. It falls under the class of black box testing

25
9.6 Requirement Analysis
Requirement analysis, also called requirement engineering, is the process of determining user
expectations for a new modified product. It encompasses the tasks that determine the need for
analyzing, documenting, validating and managing software or system requirements. The requirements
should be documentable, actionable, measurable, testable and traceable related to identified business
needs or opportunities and define to a level of detail, sufficient for system design.

9.7 Functional Requirements

It is a technical specification requirement for the software products. It is the first step in the
requirement analysis process which lists the requirements of particular software systems including
functional, performance and security requirements. The function of the system depends mainly on the
quality hardware used to run the software with given functionality.

9.8 Usability
It specifies how easy the system must be use. It is easy to ask queries in any format which is short or
long, porter stemming algorithm stimulates the desired response for user.

9.8.1 Robustness
It refers to a program that performs well not only under ordinary conditions but also under unusual
conditions. It is the ability of the user to cope with errors for irrelevant queries during execution.

9.8.2 Security
The state of providing protected access to resource is security. The system provides good security and
unauthorized users cannot access the system there by providing high security.

9.8.3 Reliability
It is the probability of how often the software fails. The measurement is often expressed in MTBF (Mean
Time Between Failures). The requirement is needed in order to ensure that the processes work
correctly and completely without being aborted. It can handle any load and survive and survive and
even capable of working around any failure.

9.8.4 Compatibility
It is supported by version above all web browsers. Using any web servers like localhost makes the
system real-time experience.

9.8.5 Flexibility
The flexibility of the project is provided in such a way that is has the ability to run on different
environments being executed by different users.

9.8.6 Safety
Safety is a measure taken to prevent trouble. Every query is processed in a secured manner without
letting others to know one’s personal information.

26
9.9 Non-Functional Requirements
9.9.1 Portability
It is the usability of the same software in different environments. The project can be run in any
operating system.

9.9.2 Performance
These requirements determine the resources required, time interval, throughput and everything that
deals with the performance of the system.

9.9.3 Accuracy
The result of the requesting query is very accurate and high speed of retrieving information. The degree
of security provided by the system is high and effective.

9.9.4 Maintainability
Project is simple as further updates can be easily done without affecting its stability. Maintainability
basically defines that how easy it is to maintain the system. It means that how easy it is to maintain the
system, analyse, change and test the application. Maintainability of this project is simple as further
updates can be easily done without affecting its stability

27
CHAPTER-10
10 Conclusion
• The development of the emotion-based music player represents a significant milestone in the
intersection of technology and human emotion. Throughout this project, we have delved into the
complexities of emotion recognition, music analysis, and user experience design to create a
unique platform that enhances the way we interact with music.

• By leveraging cutting-edge technologies such as machine learning and signal processing, we have
successfully engineered an intelligent system capable of interpreting users’ emotions and curating
personalized playlists accordingly. Through rigorous experimentation and iterative refinement,
we have validated the efficacy and accuracy of our emotion recognition algorithms, ensuring a
seamless and immersive user experience.

• Moreover, the integration of user feedback and usability testing has been integral to the iterative
design process, enabling us to tailor the music player to meet the diverse needs and preferences of
our users. By prioritizing user-centric design principles, we have ensured that the emotion-based
music player not only performs optimally but also resonates deeply with its users on an emotional
level.

• Looking ahead, the potential applications of this technology are vast and diverse. From enhancing
mood regulation and emotional well-being to revolutionizing the way we consume and engage
with music, the emotion-based music player opens up new avenues for exploration and
innovation. As technology continues to evolve and our understanding of human emotion deepens,
the possibilities for further advancements in this field are limitless.

• In conclusion, the development of the emotion-based music player represents a pioneering effort
to harness the power of technology to enhance our emotional experiences and enrich our lives
through music. As we continue to refine and expand upon this technology, we are poised to unlock
new realms of creativity, expression, and emotional connection in the realm of music
References:
1. Emanuel I. Andelin and Alina S. Rusu,”Investigation of facial microexpressions of emotions in
psychopathy - a case study of an individual in detention”, 2015, Published by Elsevier Ltd.

2. Paul Ekman, Wallace V Friesen, and Phoebe Ellsworth. Emotion in the human face: Guidelines for
research and an integration of findings. Elsevier 2013.

3. F. De la Torre and J. F. Cohn, “Facial expression analysis,” Vis. Anal. Hum.,pp. 377–410, 2011.

4. Bavkar, Sandeep, Rangole, Jyoti, Deshmukh,” Geometric Approach for Human Emotion
Recognition using Facial Expression”, International Journal of Computer Applications, 2015.

5. Zhang, Z. Feature-based facial expression recognition: Sensitivity analysis and experiments with
a multilayer perceptron. International Journal of Patten Recognition and Artificial Intelligence.

6. Remi Delbouys, Romain ´ Hennequin, Francesco Piccoli, Jimena RoyoLetelier, Manuel

Moussallam. “Music mood detection based on audio and lyrics with Deep Neural Net”, 19th
International Society for Music Information Retrieval Conference, Paris, France, 2018.

7. Krittrin Chankuptarat, etal, “Emotion Based Music Player”, IEEE 2019 conference.

8. Kim, Y.: Convolutional Neural Networks for Sentence Classification. In: Proceedings of the 2014
Conference on EMNLP, pp. 1746–1751 (2014).

28
9. Tripathi, S., Beigi, H.: Multi-Modal Emotion recognition on IEMOCAP Dataset using Deep
Learning. In: arXiv:1804.05788 (2018).

10. Teng et al.,”Recognition of Emotion with SVMs”, Lecture Notes in Computer Science, August 2006.

11. B.T. Nguyen, M.H. Trinh, T.V. Phan, H.D. NguyenAn efficient realtime emotion detection using
camera and facial landmarks , 2017 seventh international conference on information science and
technology (ICIST) (2017)

29