Detection of visual fatigue by analyzing the blink rate

Guillermo León Ramírez Clavijo, Jonathan Osorio Patiño, Daniel Martínez León
Instituto Tecnológico Metropolitano, Facultad de Ingenierías
[email protected], [email protected],
[email protected]

Abstract 2.3 Blink rate calculation

For this application it requires knowing the number of blinks per minute and the duration
One of the most common causes of traffic accidents is the driver’s fatigue. So in this paper, a of each blink. Through conditionals at the moment of closing the eyes for the first time,
methodology for detection of eyestrain, in different lighting conditions, is presented by the program will activate a counter, which will store the number of blinks and at the end
analyzing the blink rate using computer vision techniques. An algorithm that takes face of this time a message gives out the blinks per minute frequency. When the program
recognition was used, then a technique based on edge detection and measurement of texture to detects no eyes (Fig. 5a), a timer is activated that measures the time the eyes remain
segment the eye and to count the changes in the characteristics of the eyes over time was closed. This time is compared to the average maximum, blink which is about 50 ms [21].
applied. It was found that with this algorithm a good identification of the blink rate is achieved. If this time is exceeded a drowsiness alert will be sent to the driver with a message (Fig.
5b), or sound, or may inform a center, which then communicates with said driver as a
1. Introduction precaution.
Traffic accidents caused by motor vehicles are one of the leading causes of death in modern
society. Studies in different countries found that not only driving while under the influence of
alcohol or at high speeds are the reasons why traffic accidents occur, also the momentary nodding
off, so called micro-sleep, has become one of the main causes of road accidents [1].

In this article, a simple algorithm is presented to determine drowsy drivers by analyzing the blink
rate. The here presented new method uses the LBP (Local Binary Pattern) classifier to detect the
face and to extract relevant texture characteristics and the overall pattern of the face in a binary
manner; and the cascaded Haar classifier to detect the eye area, and thus providing useful
3 . Experiment
information to determine eyestrain in a simple way and hence drowsiness. The results show that
this new approach produces high rates of blink detections, with their respective time span, and
causes alerts in cases where drowsiness is detected, for medium and high brightness. The eyestrain detector algorithm was applied to analyze the blink rate of 7 people with the
following characteristics: subjects 1, 2, 3 are fair-skinned people age 30, 42 and 51 years, subjects
2. Methodology 5, 6 have darker complexion and are 23 and 39 years old and subject seven is a fair-skinned
person wearing eyeglasses, age 55. Two lighting conditions were tested: medium and high
To avoid traffic accidents caused by drowsy drivers, a system of artificial vision is implemented illumination, since in low light conditions the algorithm does not detect the face. The data
that captures the face and the eyes to acquire the driver’s blink rate and hence identify obtained in the blink detection analysis is shown in Table 2.
drowsiness.
Facial recognition is achieved with the use of computer vision techniques that help pre-process
and extract specific features of an image or video.
For image acquisition a Night Vision USB 2.0 webcam is used with 6 LEDs (Fig. 1a), which
offers VGA video capturing quality of 640x480 at 15 fps, as shown in Fig. 1b.
5, 4, 6 and 3 events of sleepiness in the subjects 1,
4, 5 and 7, respectively were detected. Table 3
demonstrates the data detected of each subject with
drowsiness under the two lighting conditions.

Based on the data comparative percentages between

actual and experimental data for the effectiveness of
the blink and drowsiness detection (Table 4) were
2.1 Image processing obtained.
After the acquisition, the pre-processing of the image begins with a change from color space
(RGB) to grayscale (Fig. 2a) and histogram equalization follows to improve the brightness (Fig.
2b) and to obtain as many features as possible 4. Results
2.2 Detection of face and eyes An effective algorithm for the recognition of the face and the eyes was implemented, which
The face detector is loaded with the LBP operator, which is included as a classifier in OpenCV allows their location in an image in real time, managing to detect whether the eyes are either open
libraries [10]. That allows extracting facial features via texture, describing the environment of a or closed. An efficiency of 98.88% with high illumination and 90.88% with medium illumination
pixel and considering the result as a binary number [16]. LBP is mainly used for face detection, was obtained.
classification of texture, and pattern recognition and classification. Based on the above, the In order to perform the measurement of the time span of each blink and the frequency of blinks
operator lbpcascade_frontalface.xml [13] is used, which detects the face in a frontal way and per minute to determine possible drowsiness, an effectiveness of 95.83% with a high illumination
delivers the data that then stored in a vector with the coordinates of its location. With these and 87.5% with a medium illumination was obtained. With the medium lighting the algorithm
coordinates the location of the face is highlighted in the output image, as shown in Fig. 3. detected prolonged deviations as eyestrain, so three false detections occurred.
As the aim is detecting the eyes on the image of the detected face, the haarcascade_eye_
tree_eyeglasses.xml classifier, which is trained to detect eyes, is used; but to work properly a
search region must be defined, which is the section of the image where you want to detect the
5. Conclusions
eyes, so it is recommended to use the values listed in Table 1 from [19], to determine the
An algorithm performing face recognition was used, then a technique based on edge detection
corresponding region for the classifier.
and measurement of textures to segment the eye and to count changings in the characteristics of
To mark the areas containing the region where the eyes are located in the moment of detection,
the eyes over time was applied. It was found that with this algorithm a good measurement of the
two different detectors, each for one eye, are used, which detect the location of each eye (Fig. 4).
blink rate is achieved.
The proposed algorithm is able to detect the face and the eyes at medium and high illumination
and is independent of gender, age and the use of translucent glasses, but for optimal detection the
camera must be positioned as frontally as possible.
In order to counteract the effects of poor detection due to insufficient light or backlight, it is
proposed for work to implement an infrared camera, with which better results, unaffected by
changes in or lack of brightness, can be obtained [18].

6. References

[1] R, Jorge; R, Edmundo; E, Martha. “Somnolencia y cansancio durante la conducción: accidentes de tránsito en las carreteras del Perú”. Acta méd. peruana (online). 2009, vol.26, n.1, pp. 48-54 Mar. 2009.
[10] Singh, S.; Papanikolopoulos, N.P., "Monitoring driver fatigue using facial analysis techniques," Intelligent Transportation Systems, 1999. Proceedings. 1999 IEEE/IEEJ/JSAI International Conference on, vol., no., pp.314,
318, 1999.
[13] Xing Li; Guang Han; Guangteng Ma; Yanshan Chen, "A new method for detecting fatigue driving with camera based on OpenCV," Wireless Communications and Signal Processing (WCSP), 2011 International Conference
on, vol., no., pp.1, 5, 9-11 Nov. 2011.
[16] Pinjari, S.A.; Patil, N.N., "A modified approach of fragile watermarking using Local Binary Pattern (LBP)," Pervasive Computing (ICPC), 2015 International Conference on, vol., no., pp.1,4, 8-10 Jan. 2015.
[18] Dan Witzner Hansen, Riad I, Hammoud, “An improved likelihood model for eye tracking”, Computer Vision and Image Understanding, 2007, 106 (2-3):220-230.
[19] OpenCV 2.4.11.0 documentation; OpenCV Tutorials; objdetect module; Object Detection; http://docs.opencv.org/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html
[21]Qiong Wang; Huan Wang; Chunxia Zhao; Jingyu Yang, "Driver fatigue detection technology in active safety systems," Remote Sensing, Environment and Transportation Engineering (RSETE), 2011 International Conference
on , vol., no., pp.3097, 3100, 24-26 June 2011.

9-1-4673-9461-1/15/$31.00 ©2015 IEEE