DEPARTMENT OF COMPUTER SCIENCE ENGINEERING

HUMAN COMPUTER INTERACTION BASED EYE CONTROLLED MOUSE

TEAM MEMBERS
KANCHARLA NEHA (21K91A05B9)
GAJARAMODDI MANIVARDHAN REDDY (21K91A0579)
JAKKA SANJAY (21K91A05A2)
K SAI MANISH (21K91A05B0)
 ABSTRACT

With advanced technologies in this digital era, there is always scope for development in the field of
computing. Hands free computing is in demand as of today it addresses the needs of quadriplegics.
This paper presents a Human computer interaction (HCI) system that is of great importance to
amputees and those who have issues with using their hands. The system built is an eye based
interface that acts as a computer mouse to translate eye movements such as blinking, gazing and
squinting towards the mouse cursor actions. The system in discussion makes use of a simple
webcam and its software requirements are Python(3.6), OpenCv , numpy and a few other packages
which are necessary for face recognition. The face detector can be built using the HOG (Histogram
of oriented Gradients) feature along with a linear classifier, and the sliding window technique. It is
hands free and no external hardware or sensors are required.
 INTRODUCTION

The computer mouse or moving the finger has been a very common approach to move the cursor along
the screen in the current technology. The system detects any movement in the mouse or the finger to
map it to the movement of the cursor. Some people who do not have their arms to be operational, called
as ‘amputees’ will not be able to make use of the current technology to use the mouse. Hence, if the
movement of their eyeball can be tracked and if the direction towards which the eye is looking at can be
determined, the movement of the eyeball can be mapped to the cursor and the amputee will be able to
move the cursor at will. An ‘eye tracking mouse’ will be of a lot of use to an amputee Currently, the eye
tracking mouse is not available at a large scale, and only a few companies have developed this
technology and have made it available. We intend to prepare an eye tracking mouse where most of the
functions of the mouse will be available, so that the user can move the cursor using his eye. We try to
estimate the ‘gaze’ direction of the user and move the cursor along the direction along which his eye is
trying to focus.
 LITERATURE SURVEY

• [1] Margrit Betke , James Gips, “The Camera Mouse: Visual Tracking of Body Features to Provide Computer Access
for People With Severe Disabilities ”,in IEEE transactions on neural systems and rehabilitation engineering, vol.10,
no.1, March 2002
• [2] Robert Gabriel Lupu , Florina Ungureanu, Valentin Siriteanu , “Eye Tracking Mouse for Human Computer
Interaction” , in The 4th IEEE International Conference on E-Health and Bioengineering - EHB 2013.
• [3] Prof. Prashant Salunkhe , Miss. Ashwini R. Patil , “A Device Controlled Using Eye Movement ”, in International
Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT) - 2016.
• [4] Muhammad Usman Ghani, Sarah Chaudhry, Maryam Sohail, Muhammad Nafees Geelani , “ Gaze Pointer : A Real
Time Mouse Pointer Control Implementation Based On Eye Gaze Tracking ”, in INMIC 19-20 Dec. 2013.
• [5] Alex Poole and Linden J. Ball, “Eye Tracking in Human-Computer Interaction and Usability Research: Current
Status and Future Prospects,”in Encyclopedia of Human Computer Interaction (30 December 2005) Key:
citeulike:3431568, 2006, pp. 211-219.
• [6] Vaibhav Nangare, Utkarsha Samant, Sonit Rabha , “Controlling Mouse Motions Using Eye Blinks, Head
Movements and Voice Recognition”,in International Journal of Scientific and Research Publications, Volume 6, Issue
3, March 2016.
• [7] Anamika Mali, Ritisha Chavan, Priyanka Dhanawade, “Optimal System for Manipulating Mouse Pointer through
Eyes”,in International Research Journal of Engineering and Technology (IRJET) March 2016.
• [8] Xuebai Zhang, Xiaolong Liu, Shyan-Ming Yuan,Shu-Fan Lin, “Eye Tracking Based Control System for
Natural Human-Computer Interaction” , in Hindawi Computational Intelligence and Neuroscience Volume
2017, Article ID 5739301, 9 pages.
• [9] Tereza Soukupova ´ and Jan Cˇ ech . “Real-Time Eye Blink Detection using Facial Landmarks.” In21st
Computer Vision Winter Workshop, February 2016.
• [10] Adrian Rosebrock . “Detect eyes, nose, lips, and jaw” with dlib , OpenCV, and Python.”
• [11]Adrian Rosebrock . Eye blink detection with OpenCV, Python, and dlib .
• [12] C . Sagonas , G. Tzimiropoulos , S. Zafeiriou , M. Pantic . 300 Faces in-the-Wild Challenge: The first facial
landmark localization Challenge. Proceedings of IEEE Int’l Conf. on Computer Vision (ICCV-W), 300 Faces in-
the-Wild Challenge (300-W). Sydney, Australia, December 2013
• [13] Akshay Chandra Lagandula . Mouse Cursor Control Using Facial Movements.
https://towardsdatascience.com/c16b0494a971.
 CONCLUSION
The development of a Human-Computer Interaction (HCI) based eye-controlled mouse offers a promising
solution for enhancing accessibility and usability, especially for individuals with physical limitations. This
innovative technology empowers users to interact with digital interfaces using only their eye movements,
making it a more inclusive and intuitive way to engage with computers. By focusing on continuous
improvements in accuracy, user customization, and integration into various applications, eye-controlled mouse
have the potential to revolutionize the way we interact with technology, making it more accessible and user-
friendly for all . This work can be extended to improve the speed of the system by using better trained models.
Also, the system can be made more dynamic by making the change in the position of the cursor, proportional
to the amount of rotation of the user’s head, i.e., the user can decide, at what rate he wants the position of the
cursor to change. Also, future research work can be done on making the ratio more accurate, since the range of
the values are the result of the aspect ratios, which is usually small. Hence, to make the algorithm detect the
actions more accurately, there can be some modification in the formulae for the aspect ratios used. Also, to
make the process of detection of the face more easy, some image processing techniques can be used before
the model detects the face and features of the face.
THANK YOU