TABLE OF CONTENTS

CHAPTER NO. TOPIC PAGE NO

Certificate 2

Acknowledgement 3

Summary 4

List of Figures 5

Chapter 1 Introduction 6

Chapter 2 Different Color Tones 8

Chapter 3 Algorithm 13

Chapter 4 Matlab Functions Used 17

Chapter 5 Threshold Conditions 19

Chapter 6 Matlab Code 21

Chapter 7 Experimental Results 27

Chapter 8 Conclusion 30

References 31

CERTIFICATE

1
This is to certify that the work titled “SKIN SEGMENTATION BASED FACE
DETECTION” submitted by “Anurag Gupta” in partial fulfillment for the
award of degree of B. Tech of Jaypee Institute of Information Technology
University, Noida has been carried out under my supervision. This work has not
been submitted partially or wholly to any other University or Institute for the
award of this or any other degree or diploma.

Signature of Supervisor :

Name of Supervisor : Mrs. Bhawna Gupta

Designation : Senior Lecturer

Date : 19th November 2009

2
 ACKNOWLEDGEMENT

It is indeed a privileged opportunity to express my sincere gratitude to all those

who have helped me in completing this project. In completing this work, I avail
the opportunity to express my thanks with deep sense of gratitude to my respected
supervisor Mrs. Bhawna Gupta whose technical and analytical acumen prompted
the genesis of this project. Her esteemed individual guidance and helpful
suggestions for improving the presentation of this thesis are certainly invaluable.

I am especially grateful to the Faculty of Electronics & Communication

Department for all the help provided by them and the resources that they made
available without which the project would not have reached its current stage.

I would like to thank my parents whose good wishes and silent blessings always
remained with me throughout the course of the project. I would also like to thank
all the fellow students and all those who have directly or indirectly helped me
during this project.

Name of Student : Anurag Gupta

Signature of student :

Date : 19th November 2009

3
 SUMMARY

Face Detection is finding out the location of human faces in images. This report is
the outcome of application of face detection algorithm for the above defined
purpose. A brief introduction about face detection and its importance has been
given. Various color modes of images have been described followed by detailed
explanation of the algorithm used and its subsequent outcome. A few examples
have been discussed along with the reasons for not obtaining full accuracy. A brief
description of the Matlab functions used has also been given.

Signature of Student Signature of Supervisor

Name : Anurag Gupta Name : Mrs. Bhawna Gupta

Date : 19th November 2009 Date : 19th November 2009

4
 LIST OF FIGURES

Fig 2.1 – RGB colors

Fig 2.2 – RGB Color cube
Fig 2.3 – YcbCr Cube
Fig 2.4 – HSV mode
Fig 2.5 – Double cone model of hsv mode
Fig 3.1 – Expected Outcome
Fig 7.1- Original Image
Fig 7.2 - Binary mask for YCbCr mode
Fig 7.3 - Binary mask for HSV mode
Fig 7.4 - Binary mask for RGB mode
Fig 7.5 – Original Image
Fig 7.6 – Binary mask for YCbCr mode
Fig 7.7 – Binary mask for HSV mode
Fig 7.8 – Binary mask for RGB mode
Fig 7.9 – Cb vs Cr plot

5
 CHAPTER 1

INTRODUCTION

The goal of this project is to take a color digital image and indicate the location of
faces in that image. Detection of faces in a digital image has gained much
importance in the last decade, with applications in fields such as law enforcement
and security. Although facial detection is an extremely simple task to the human
eye, automating the process to a computer requires the use of various image
processing techniques. Locating and tracking human faces is a prerequisite for
face recognition and/or facial expressions analysis, although it is often assumed
that a normalized face image is available. In order to locate a human face, the
system needs to capture an image using a camera and a frame-grabber to process
the image for locating the faces present in the image, if any.

1.1 What is face detection

Face Detection is a computer based technology to determine the location of a face

( or faces) in an image regardless of its size, color, illumination and ignoring all
other constituents of the image. Face detection can be regarded as a specific case
of object-class detection. In object-class detection, the task is to find the locations
and sizes of all objects in an image that belong to a given class.

1.2 Why face detection is important

Human face perception is currently an active research area in the computer vision
community. Face detection is an important area of image processing, being the
first step towards face recognition, video surveillance and image database
management. It has important applications in bioinformatics. The task of face
detection is extremely trivial for humans, but it is a challenge to enable computers
to carry out the same task. Given an image, a face detection algorithm will locate
all the faces in the image. There exist different approaches for face detection, the

6
main ones being feature based and image based algorithms. Feature based
algorithm uses methods like edge detection, skin color, symmetry analysis, while
the latter uses neural networks.

7
 CHAPTER 2

DIFFERENT COLOR TONES

Different color tones of an image

The study on skin color classification has gained increasing attention in recent
years due to the active research in content-based image representation. For
instance, the ability to locate image object as a face can be exploited for image
coding, editing, indexing or other user interactivity purposes. Moreover, face
localization also provides a good stepping stone in facial expression studies.

An image can be represented in various color tones. Each color tone has different
properties and parameters from which we can generate different color tones. Some
of them are RGB, HSV, YCbCr, CMYK (Cyan Magenta Yellow Key black), and
TSL (Tint Saturation Lightness). Our emphasis in this project is on RGB, YCbCr
and HSV modes.

2.1 RGB Mode (Red Green Blue)

The RGB color model is an additive color model in which red, green,
and blue light are added together in various ways to reproduce a broad array of

8
colors. The name of the model comes from the initials of the three additive
primary colors, red, green, and blue. Spectral components of these colors combine
additively to produce a resultant color. It is one of the most widely used color
spaces for processing and storing of digital image data.

Zero intensity for each component gives the darkest color (no light, considered
the black), and full intensity of each gives a white. The quality of this white
depends on the nature of the primary light sources, but if they are properly
balanced, the result is a neutral white matching the system's white point. When the
intensities are different, the result is a colorized hue, more or
less saturated depending on the difference of the strongest and weakest of the
intensities of the primary colors employed.

Fig 2.1 – RGB colors

The RGB model can be represented by a 3-dimensional cube in 3-D space with its
center at the origin and R, G, B components along the three axes. At the origin,
R=G=B=0 representing black while the point R=G=B=1 represents white. On the
color cube red is (1, 0, 0), green is (0, 1, 0) and blue is (0, 0, 1). In a 24-bit color
graphics system with 8 bits per color channel, red is (255, 0, 0), green is (0, 255,
0) and blue is (0, 0,255).

9
 Fig 2.2 – RGB Color cube

2.2 YCbCr Mode (Yellow Blue difference Green difference)

YCbCr color space has been defined in response to increasing demands for digital
algorithms in handling video information, and has since become a widely used
model in a digital video.

These color spaces separate RGB (Red-Green-Blue) into luminance and

chrominance components. Y is the luminance and Cb and Cr are the blue-
difference (Blue-luminance) and red-difference (Red-
luminance) chroma components. YCbCr is not an absolute color space. It is a way
of encoding RGB information. The actual color displayed depends on the actual
RGB colorants used to display the original image. YCbCr mode is used for image
compression work.

10
Y has an excursion of 219 and an offset of +16. This coding places black at code
16 and white at code 235. Cb and Cr have excursions of +112 and offset of +128,
producing a range from 16 to 240 inclusively. (128 represent 0).

Fig 2.3 – YcbCr Cube

2.3 HSV/HSI mode (Hue saturation Value/Intensity)

HSV stands for hue, saturation and value. Hue is one of the main properties of a
color. It represents a pure color (without any tint or shade). A hue is an element
of the color wheel.

Fig 2.4 – HSV mode

11
Saturation is the difference of a color against its own brightness. The saturation of
a color is determined by a combination of light intensity and how much it is
distributed across the spectrum of different wavelengths. The purest color is
achieved by using just one wavelength at a high intensity, such as in laser light. If
the intensity drops, so does the saturation.

Intensity is the brightness or dullness of a hue. One may lower the intensity by
adding white or black.

Many applications use the HSI color model. Machine vision uses HSI color space
in identifying the color of different objects. Image processing applications such as
histogram operations, intensity transformations and convolutions operate only on
an intensity image. These operations are performed with much ease on an image
in the HIS color space.

Fig 2.5 – Double cone model of hsv mode

12
The hue (H) is represented as the angle, varying from 0 to 360. Saturation (S)
corresponds to the radius, varying from 0 to 1. Intensity (I) varies along the z axis
with 0 being black and 1 being white. When S = 0, color is a gray value of
intensity 1. When S = 1, color is on the boundary of top cone base. The greater the
saturation, the farther the color is from white/gray/black (depending on the
intensity). Adjusting the hue will vary the color from red at 0o, through green at
120o, blue at 240o, and back to red at 360o. When I = 0, the color is black and
therefore H is undefined. When S = 0, the color is grayscale. H is also undefined
in this case. By adjusting I, a color can be made darker or lighter. By maintaining
S = 1 and adjusting I, shades of that color are created.

CHAPTER 3

ALGORITHM

For detecting face there are various algorithms including skin color based
algorithms.

There are two major algorithms for face detection.

• Human Face Detection in Cluttered Color Images Using Skin Color and
Edge Information
• Face Detection based on skin colors using neural networks.

Color is an important feature of human faces. Using skin-color as a feature for

tracking a face has several advantages. Color processing is much faster than
processing other facial features. Under certain lighting conditions, color is
orientation invariant. This property makes motion estimation much easier because
only a translation model is needed for motion estimation.

13
The algorithm used in this project is ‘Skin Segmentation Based Face Detection’.

The steps involved in the algorithm are:

1. Skin Pixel Classification: Different color spaces used in skin detection

include HSV, RGB, and YCbCr. The image is converted from RGB mode
to HSV and YcbCr modes. The values of the individual parameters in all
three color modes are compared with the threshold values of each mode
for skin pixel classification. Based on these threshold values a pixel is
classified as skin (pixel=1) or nor skin (pixel=0) in a new skin matrix.
Then the skin tone pixels from all the three modes are combined to get the
skin tone region from the image.

2. Connectivity Analysis: Using the skin detected image, we know whether

a pixel is a skin pixel or not, but cannot say anything about whether a pixel
belongs to a face or not. We have to group pixels that are connected to
each other geometrically. We group the skin pixels in the image based on a
8-connected neighborhood i.e. if a skin pixel has got another skin pixel

in any of its 8 neighboring places, then both the pixels belong to the same
region. At this stage, we have different regions and we have to classify
each of these regions as a human face or not. This is done by finding the
ratio of height and width of the region and comparing it with given
threshold values as well as the percentage of skin in the rectangular area
defined by the above parameters. For finding height, we locate the first
and the last row of the skin matrix which contains skin pixel of a
connected segment. To obtain the width, we locate the leftmost and
rightmost columns in the skin matrix which contain skin pixels of the
connected segment. We can imagine a rectangle at the boundary of each
segment of different connected components.

14
 3. Edge Information: Inside the rectangle obtained from the connectivity
analysis, we determine the percentage of skin in that region and compare it
with the threshold value.

If all the above conditions are satisfied then we mark the region as a face.

Proposed algorithm

Convert the input RGB image ( rgb(i,j) ) into HSV image ( hsv(i,j) ) and YcbCr
image(Ycbcr(i,j))

Extraction of skin tone pixels from all 3 image modes

Calculate connected components for each segment

15
Calculation of ratio of length : width for each segment

Calculation of percentage of skin in a rectangle segment

Mark region as a face

16
 Expected outcome

Fig 3.1 – Expected outcome

17
 CHAPTER 4

MATLAB FUNCTIONS USED

1. Imread:-A = IMREAD(FILENAME,FMT) reads a grayscale or color

image from the file specified by the string FILENAME. If the file is not in
the current directory, or in a directory on the MATLAB path, specify the
full pathname. The return value A is an array containing the image data. If
the file contains a grayscale image, A is an M-by-N array. If the file
contains a true color image, A is an M-by-N-by-3 array. For TIFF files
containing color images that use the CMYK color space, A is an M-by-N-
by-4 array. The class of A depends on the bits-per-sample of the image
data, rounded to the next byte boundary. For example, IMREAD returns
24-bit color data as an array of uint8 data because the sample size for each
color component is 8 bits.

2. RGB2HSV:- H = RGB2HSV(M) converts an RGB color map to an HSV

color map. Each map is a matrix with any number of rows, exactly three
columns, and elements in the interval 0 to 1. The columns of the input
matrix, M, represent intensity of red, blue and green, respectively. The
columns of the resulting output matrix, H, represent hue, saturation and
color value, respectively.

HSV = RGB2HSV(RGB) converts the RGB image RGB (3-D array) to

the equivalent HSV image HSV (3-D array).

3. RGB2YCBCR :- YCBCRMAP = RGB2YCBCR(MAP) converts the RGB

values in MAP to the YCBCR color space. MAP must be an M-by-3 array.
YCBCRMAP is an M-by-3 matrix that contains the YCBCR luminance
(Y) and chrominance (Cb and Cr) color values as columns. Each row
represents the equivalent color to the corresponding row in the RGB color
map.

18
 YCBCR = RGB2YCBCR(RGB) converts the true color image RGB to the
equivalent image in the YCBCR color space. RGB must be an M-by-N-by-
3 array.

4. BWLABEL :- L = BWLABEL(BW,N) returns a matrix L, of the same

size as BW, containing labels for the connected components in BW. N can
have a value of either 4 or 8, where 4 specifies 4-connected objects and 8
specifies 8-connected objects; if the argument is omitted, it defaults to 8.
The elements of L are integer values greater than or equal to 0. The pixels
labeled 0 are the background. The pixels labeled 1 make up one object,
the pixels labeled 2 make up a second object, and so on.

[L,NUM] = BWLABEL(BW,N) returns in NUM the number of connected

objects found in BW.

19
 CHAPTER 5

THRESHOLD CONDITIONS

5.1 RGB Mode

Threshold conditions used to extract skin region from an image in RGB color tone
are:

R > 95 and G > 40 and B > 20

max{R,G,B}−min{R,G,B} > 15
|R−G| > 15 and R > G and R > B
Where R,G,B specify the individual components of each pixel. These three
conditions have to be simultaneously satisfied to categorize the pixel as skin type
or non skin type.

5.2 YCbCr Mode

Threshold conditions used to extract skin region from an image in YCbCr mode
are:

133<Cr<177
90<Cb<130

where Cr and Cb represent red difference and blue difference components of each
pixel.

5.3 HSV mode

In hsv color mode, a pixel is classified as skin type if:

0<H<50
.23<S<.68

Where H and S are values of Hue and Saturation for each pixel.

20
5.4 Golden Ratio

To check weather a certain skin region lies within the range of ratio of length:
width, we define a term called Golden Ratio

GR = 1+sqrt (5)
2

We also define a tolerance level: T=0.65

To categorize a region as a face:

GR-T < Ratio (length: width) < GR+T

5.5 Skin Threshold

Inside the rectangular region obtained from the above four conditions, the
percentage of skin must be above a certain level to mark it as a face. This
threshold value is .56

21
 CHAPTER 6

MATLAB CODE

clc;
close all
img=imread('test01.jpg'); % img as an m by n by 3 matrix
imshow(img),title('original');
img; % look at the matrix
m = size(img,1); % number of rows
n = size(img,2); % number of columns
p = size(img,3); % 3 values(RGB) for each pixel
img1=rgb2ycbcr(img);
ycbcr_skin = zeros(m,n);
r=img(:,:,1); % r,g,b are all matrices here having dimensions m by
n and contain the respective r g and b color values for each pixel
g=img(:,:,2);
b=img(:,:,3);
y=img1(:,:,1); % y cb and cr values of each pixel after converting
image into ycbcr color tone
cb=img1(:,:,2);
cr=img1(:,:,3);
%figure;
%imshow(img1); % display the ycbcr image
cr1=137;% threshold values of cb and cr for skin region
cr2=177;
cb1=90;
cb2=130;
for i = 1:m%this will check all the pixel with the threshold
values of cb and cr and assign value 1 for that pixel
for j = 1:n
if(cr(i,j)>cr1 && cr(i,j)<cr2 && cb(i,j)>cb1 &&
cb(i,j)<cb2)
ycbcr_skin(i,j)=1;
end
end
end
%ycbcr_skin;
final_ycbcr = zeros(m,n);
for i = 1:m%making a matrix with original pixel values for skin
region and 0 for non skin tone regions
for j = 1:n
if(ycbcr_skin(i,j)==1)
final_ycbcr(i,j)=img(i,j);
end
end
end
figure;
subplot(221),imshow(final_ycbcr),title('binary mask for ycbcr
mode');

%%%%%%% for the hsv mode -------

img1=rgb2hsv(img);

m=size(img1,1);
n=size(img1,2);

22
h=zeros(m,n);
s=zeros(m,n);
v=zeros(m,n);
for i=1:m
for j=1:n
h(i,j)=360*img1(i,j,1);
end
end

s=img1(:,:,2);
v=img1(:,:,3);
max(max(s))
hsv_skin=zeros(m,n);
for i=1:m
for j=1:n
if(h(i,j)>0 && h(i,j)<50 && .23<s(i,j) && .68>s(i,j))
hsv_skin(i,j)=1;
end
end
end

for i=1:m
for j=1:n
if(hsv_skin(i,j)==1)
final_hsv(i,j)=img(i,j);
end
end
end

final_hsv;
subplot(222),imshow(hsv_skin),title('binary mask for hsv mode');

%%%%% from rgb mode

%r1=180;
%r2=249;
%g1=130;
%g2=195;
%b1=120;
%b2=215;

rgb_skin=zeros(m,n);
final_rgb=zeros(m,n);

for i=1:m
for j=1:n
if(r(i,j)>95 && g(i,j)>40 && b(i,j)>20)
if(max(max(r(i,j),g(i,j)),b(i,j))-
min(min(r(i,j),g(i,j)),b(i,j))>15)
if(abs(r(i,j)-b(i,j))>15 && r(i,j)>b(i,j) &&
r(i,j)>g(i,j))
rgb_skin(i,j)=1;
end
end
end
end
end

for i=1:m
for j=1:n

23
 if(rgb_skin(i,j)==1)
final_rgb(i,j)=img(i,j);
end
end
end
subplot(223),imshow(final_rgb),title('binary mask for rgb mode');

combo_skin=zeros(m,n);
for i=1:m
for j=1:n
combo_skin(i,j)=ycbcr_skin(i,j)+rgb_skin(i,j)
+hsv_skin(i,j);
end
end
noofskinpixels=0;
final_combo=zeros(m,n);
for i=1:m
for j=1:n
if combo_skin(i,j)~=0
final_combo(i,j)=img(i,j);
noofskinpixels= noofskinpixels+1;
end
end
end
subplot(224),imshow(final_combo),title('binary mask of all 3 color
modes');

%%%%%to calculate the number of connected components in the final

skin tone
%%%%%image obtained

L = BWLABEL(combo_skin,8);
L1 = max(max(L))
pixval('ON');

%%%% to count the number of pixels in each connected segment

count=zeros(1,max(max(L)));
for k=1:L1
for i = 1:m
for j=1:n

if(k==L(i,j))
count(k) = count(k)+1;
end
end
end
end
count
max(count);

k=61;
tempimage=zeros(m,n);
for i=1:m
for j=1:n
if(L(i,j)==k)

24
 tempimage(i,j)=1; %%%%%%this loop is to see which
portion of the image is selected at a particular value of k
end
end
end
%figure;
%imshow(tempimage);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

ratio=zeros(1,L1); %% ratio is the array to store the ratio of

length and width of each segment
x1=zeros(1,L1);
x2=zeros(1,L1);
y1=zeros(1,L1);
y2=zeros(1,L1);
length=zeros(1,L1);
width=zeros(1,L1);
for k=1:L1
temp=0;
for i=1:m
for j=1:n
if(k==L(i,j))
temp=temp+1;
if(temp==1)
x1(k)=i;
end
if(temp==count(k))
x2(k)=i;
end
end
end
end

temp=0;
for j=1:n
for i=1:m
if(k==L(i,j))
temp=temp+1;
if(temp==1)
y1(k)=j;
end
if(temp==count(k))
y2(k)=j;
end
end
end
end

length(k) = x2(k)-x1(k);
width(k) = y2(k)-y1(k);
if(length(k) ~= 0 && width(k) ~= 0 )
ratio(k)=length(k)/width(k);
end
end
x1;
x2;
y1;
y2;
ratio
temp=0;

25
skinthresh=zeros(1,L1);
for k=1:L1
if(length(k)~=0 && width(k)~=0)
skinthresh(k)=count(k)/(length(k).*width(k));
end

end

skinthresh

%%% we define a golden ratio(gr) and tolerance(t)to check wether a

given segment has
%%% dimensions of the face or not

finaloutput=zeros(m,n);
gr= (1+sqrt(5))/2
t=.65;
gr1=gr+t
gr2=gr-t
figure;
imshow(img);
hold('on')
for k=1:L1
if(gr2<=ratio(k) && ratio(k)<=gr1 && skinthresh(k)>=.56 &&
count(k)>100)

color('red');
rectangle('position',[y1(k),x1(k),width(k),length(k)]);
k
count(k)
ratio(k)
for i=1:m
for j=1:n
if(L(i,j)==k)
finaloutput(i,j)=1;
end
end
end
end
end
figure;
imshow(finaloutput);

%%%below is code for cb vs cr graph for skin region

cb1=zeros(1,noofskinpixels+1);
cr1=zeros(1,noofskinpixels+1);
y1=zeros(1,noofskinpixels+1);
temp=1;
for i=1:m
for j=1:n
if(combo_skin(i,j)==1)
cb1(temp)=cr(i,j);
cr1(temp)=cb(i,j);
y1(temp)=y(i,j);
temp=temp+1;
end
end
end

26
%figure;
%stem(cb1,'b.');title('cb');
%figure;
%stem(cr1,'b.');title('cr');
%figure;
%stem(y1,'b.');title('y');
figure
stem(cb1,y1,'r.');
%figure;
%stem(cb1,'r.');
%hold('on');
%stem(cr1,'g.');
%stem(y,'b.');

27
 CHAPTER 7

EXPERIMENTAL RESULTS

We will now discuss the results obtained from the three different color modes.

Fig 7.1- Original Image

Fig 7.2 - Binary mask for YCbCr mode

Fig 7.3 - Binary mask for HSV mode

28
 Fig 7.4 - Binary mask for RGB mode

Fig 7.5 - Original Image

Fig 7.6 - Binary Mask for YCbCr Mode

Fig 7.7 - Binary Mask for HSV Mode

29
 Fig 7.8 - Binary Mask for RGB Mode

While we see that using the combination of the three color tones, we are able to
extract almost all the skin region from an image and pick out the faces, the
efficiency of this algorithm is not 100 % due to obstructions present in the images
like presence of structural components like beards and glasses, Occlusion (face
may partially be covered by another object).

Below is a plot of Cb and Cr values of an image. The highly clustered region

represents the skin region. There is a strong correlation between Cb and Cr values
for skin pixels.

Fig 7.9 – Cb vs Cr plot

The outputs demonstrate that the system works extremely well for the images in
which faces are full, upright, and facing towards the front. The false positives
primarily occurred on large non-face body parts such as arms and legs. The false
negatives were typically due to an obstructed face or variations that caused
separations in the face such as sunglass.

30
31
 CHAPTER 8

CONCLUSION

A face detection algorithm is proposed that combines pixel from different color
tones .The algorithm is able to detect faces at different scales in a given image, as
well as slightly tilted images. However, it has a hard time breaking apart
overlapping faces if they are too near each other vertically. Future work could
include the use of rotated eigen images or the implementation of a neural network
or a linear classifier as a secondary detection scheme.

32
 REFERENCES

• V. Vezhenevets, V. Sazonov and A. Andreeva : A survey on pixel based

skin color detection techniques, Graphics Media Laboratory, Faculty of
Computational Mathematics and Cybernetics, Moscow State University,
Moscow, Russia.
• V. Nabiyev and A. Günay : Towards a biometric purpose image filter
according to skin detection, Department of Computer Engineering, KTU,
Trabzon, Turkey
• K. Sandeep and A. Rajagopalan : Human Face Detection in cluttered
images using skin color and edge information, Department of Electrical
Engineering, IIT Madras

33