ICIIBMS 2018, Track 1: Image Processing, Computer Science and Information technology, Bangkok, Thailand

Convolution based Face Recognition

using DWT and HOG
Jyothi Ravikumar1, Ramachandra A C2, K B Raja3 and Venugopal K R4
1
Dept. of Electronics and Communication Engineering, BMS College, Banglaore,India
2
Dept. of Computer Science and Engineering, NMIT, Banglaore,India
3
Dept. of Electronics and Communication Engineering, University Visvesvaraya College of Engineering,
Bangalore University, Bangalore, India
4
Vice-Chancellor, Bangalore University, Bangalore, India
[email protected] [email protected]

Abstract— Physiological face biometric trait is used to efficients to obtain oriented gradients. The final features are
identify a person for many real time applications. The obtained by convolving LL and HOG co-efficients.
convolution based feature extraction technique for face The paper organisation are as follows. Section II contains
identification using Discrete Wavelet Transform (DWT) and the Literature Survey. Section III gives the description of the
Histogram of Oriented Gradient (HOG) is proposed to recognize proposed model. In section IV performance parameters such as
human beings effectively. The four standard face databases with MTSR, OTSR, EER are discussed and also comparison of
different sizes are considered and resized to 128X128 to have proposed method with the existing methods are tabulated and
uniform size of images. The 2D-DWT (Two Dimensional Discrete conclusion is given in section V.
Wavelet Transform) is applied on resized face images and
considered only (LL) sub-band. The HOG is applied on LL sub- II. LITERATURE SURVEY
band to obtain HOG coefficients. The 2D convolution is used on
LL sub-band and HOG matrix to obtain final features. The In this section, the literature review of existing techniques
resized face image is compressed using DWT and HOG. The to recognize human being based on face images are explained.
Euclidean distance(ED) is used to compare features of database The various techniques of preprocessing, feature extraction
face images with test images to compute performance parameters. and matching techniques presented by various researchers are
The performance of the proposed method is better than the given. Bilel Ameur et al., [1] proposed GLBSIF algorithm
existing methods. which is combined method of Local Binary Pattern (LBP),
Gabor Wavelets and Binarized Statistical Image Features
Keywords—Discrete Wavelet Transform (DWT); Histogram
of Oriented Gradient (HOG); Biometrics ; Face Identification. (BSIF) for using as feature extraction. Then they used PCA for
dimensional reduction and KNN-SRC for classification. The
results shown that GLBSIF can reduced the time consuming
I. INTRODUCTION
and give a better accuracy than the traditional methods. Dong
Biometrics system used for recognizing the person based on Wang et al., [2] proposed an improved method of 2D-DLDA.
a physiological and behavioral characteristics. The need for The method deteriorates the effect that the edge classes have
higher security of identification and verification have become on the selection of the projection direction by redefining the
significant. The biometric-based solutions are capable of class scatter matrix. The 2D-VLDA algorithm can achieve a
providing authentic and confidential transactions. There are higher recognition rate than the existing algorithm. Michael F.
various biometric applications found in the military, Valstar et al., [3] proposed a method which includes Action
government and in commercial applications. Developing
Units (AU) event detection and AU intensity. The common
biometrics for personal authentication have become suitable
binary classifiers are applied to the problem include Artificial
and significantly increase in the accuracy compated with the
traditional methods. Neural Network (ANN), Boosting technique, Support Vector
Face recognition (FR) is a popular biometric techniques Machine (SVM), Adaboost and gentle boost. The BP4D
which is widely used in the field of human computer database is used to check the performance paremeters of the
interaction. Lot of research have been proposed in the field of proposed model and the results indicated that it achieved
FR to extract the feature in order to achieve the higher accuracy higher recognition accuracy. Nawal Yousef et al., [4] proposed
and to reduce the processing time. One of the famous feature the algorithm for face recognition to retrieve the suspect image
extraction algorithm is Histogram of Oriented Gradient (HOG), from the database using face descriptors. The proposed
which is a face descriptor. method discovers the automation of face recognition LFW
Contribution: in this, paper the convolution based face database and the results shown signigicant retrieval accuracy.
recognition using DWT and HOG is proposed. The DWT is
used to obtained compressed number of transform co-efficients. III. PROPOSED MODEL
The HOG is used on LL band to further compress LL co-

978-1-5386-7516-8/18/$31.00 ©2018 IEEE

327
 ICIIBMS 2018, Your Track number here, Bangkok, Thailand
In this section, the proposed model using DWT, HOG and details in horizontal direction leads to detection of horizontal
convolution is discussed to obtain better performance. The edges, details in vertical direction leads to detection of vertical
block diagram of the proposed model is as shown in Fig. 1. edges and details in both horizontal and vertical directions
leads to detection of diagonal edges.
The 2-D signals analysis uses of following two
dimensional filter functions through the multiplication of
separable scaling and wavelet functions in n1 samples in
horizontal and n2 samples in vertical directions as defined by
following equations.
For the approximate signal,
φ(n1,n2)=φ(n1)φ(n2) (1)

For the horizontal signal,

ᴪᴴ(n1,n2)=ᴪ(n1)φ(n2) (2)

For the vertical signal,

ᴪᵛ(n1,n2)=φ(n1)ᴪ(n2) (3)

For the diagonal signals,

ᴪ ᴰ(n1,n2)= ᴪ(n1) ᴪ(n2) (4)

The filtering in each direction follows sub-sampling by a

factor of 2 and each of the sub-bands corresponds to the filter
output contain one-fourth of the number of samples.The bands
φ(n1,n2 ), ᴪᴴ(n1,n2) , ᴪV(n1,n2) and ᴪᴰ(n1,n2) are referred to
as LL, LH, HL and HH respectively. The letter L represents
Fig.1. Proposed Method of Face Recognition low-pass filter and H letter represents high-pass filter, the first
A. Face Database letter represents filtering along coloumns and second letter
The performance of the proposed method is tested using represents filtering along rows. This filtering analysis is
standard face databases such as, JAFFE, L-SPACEK, YALE lossless. Thereby it is possible to have a perfect reconstruction
and ORL. of the original image by a reverse process of synthesis filtering.
1. JAFFE[5]: The database consists of 213 face images It may be noted that LL sub-band has most significant
of 10 subjects. The image has 7 different facial expressions, information content and all other sub-band has less
posed under different illuminations. The size of each image is information content. The original image of size 128×128 is
256×256. converted into sub-bands having dimension of 64×64. In the
2. L-SPACEK[6]: The database consists of 2,261 face proposed model LL sub-band features are used.
images of 119 subjects. The face images of each person has D. Feature Extraction of LL Sub-Band Using Histogram of
captured under diffenrent variations in expressions, pose Oriented Gradient
angles and illumination. The HOG face descriptor is computed to calculate the
3. YALE[7]: The face database consists of 165 face horizontal and vertical gradients. This is done by filtering the
images of 15 subjects. Each individual are having different image with the kernels dx and dy as shown in Fig. 2.
facial expression and captured with occlusion. Image
resolution of this database is 320×243.
4. ORL[8]: The face database consists of 400 face
images of 40 subjects, all the face images are captured under
various poses, facial expression and lighting conditions. The
resolution of each image is 112×92
B. Preprocessing
The RGB color image is first converted to gray scale image, Fig.2. Kernel Filters
then the image is resized to 128×128 dimension.
The magnitude and direction of gradient using the Eq.5 and 6
C. Two dimensional Discrete Wavelet Transform(DWT) are computed.
The 1-D DWT and its implementation can be easily extended Mag =√(Ix^2+Iy^2 ) (5)
to two-dimensional DWT signals through sub-band coding Angle= arctan ( Iy/Ix) (6)
for digital images. Using sub band analysis images extracted To calculate HOG of an image, the image is sub divided into
in approximate forms in both horizontal and vertical directions, 8×8 cells and a histogram of gradients is calculated. The

328
 ICIIBMS 2018, Your Track number here, Bangkok, Thailand
reason for dividing the whole image into 8×8 cells is that, it features. The steps are same as mentioned in enrolment
provides a compact representation. The representation is not section.
only more dense but calculating a histogram over a patch
G. Matching
makes this representation more robust to noise. Individual
gradients may have noise, but a histogram over 8×8 patch The final features of test images are compared with features
makes the representation much less sensitive to noise. The of database images using ED formula as given in the Eq.8.
angles are between 0 and 180 degrees instead of 0 to 360
degrees. These are called unsigned gradients because a (8)
gradient and its negative are represented by the same numbers. Where M is the dimension of feature vector, pi is the
The Fig. 3 illustrates the process of computing HOG in the database feature vector, qi is test feature vector.
magnitude and direction of the gradient for the 8X8 patch.
Based on the direction bin is selected and vote of the value is IV. PERFORMANCE ANALYSIS
selected based on the magnitude. In this section, the definitions of performance parameters,
and the performance analysis using various face databases are
given. The values of FAR, FRR, EER, MTSR and OTSR are
computed for different combinations of Person Inside Databse
(PID) and Person Outside Database (POD).

A. Definitions of Performance Parameters

1. False Accept Rate (FAR): It is the possibility that
system wrongly matches with images stored inside the
database. The FAR can be calculated using Eq.9.

(9)
2. False Rejection Rate (FRR): It is the possibility of
number of correct persons inside the database are rejected and
can be calculated using Eq.10.

Fig.3. Computation of HOG (10)

All the pixels in the 8X8 cells are combined to create 3. Equal Error Rate (EER): It is the intersection value of
9-bin histogram. For normalization 16×16 block is used which FRR and FAR.
contains four 8×8 blocks, which will give 36×1 feature vector. 4. True Success Rate (TSR): It is the number of
For the image size of 64×64, total no of (16X16) overlapping authorized person successfully matched in the database and is
blocks are 7×7, hence the total number features are 7×7×36. given by Eq.11.
That means feature vector size is 1764×1. The row vector of (11)
HOG of LL is converted into matrix. Thus 1764×1 vector is
converted into 42×42 matrix. 5. Maximum TSR (MTSR): It is the maximum value of
E. Two dimensional convolution TSR.
6. Optimum TSR (OTSR): It is the TSR value
In the proposed model, convolution of LL sub-band
corresponding to EER value.
64×64 and the feature vector 42×42 are convolved. The 2-D
Convolution block calculates the 2-D convolution of 2 input
The face databases viz., JAFFE, L-SPACEK, ORL and YALE
matrices. Consider the matrix A which has dimensions Ma,
are used to test the proposed algorithm for performance
Na and matrix B has dimensions Mb and Nb.The output
analysis.
matrix is calculated using the Eq.7,
B. Analysis Using ORL Database
C(i,j)=*B(i-m,j-n) (7) The performance parameters such as percentage FRR, FAR,
where 0≤i<Ma+Mb−1 and 0≤j<Na+Nb−1. and TSR are noted for variations in threshold values and the
graphs are obtained as shown in Fig.s 4, 5 and 6 for PID and
The dimension of the output matrix is Ma+Mb−1 , Na+Nb−1. POD combination of 10:10, 20:10, 30:10 respectively i.e.,
F. Test Section POD constant values. It is observed that the value of FRR
decreases with increase in threshold values whereas, the value
The test images are considered and resized to an
of FAR and TSR increases with threshold values. The
appropriate size for feature extraction. The HOG is applied on
percentage EER values are 20%, 15% and 20% for PID and
LL sub-band of DWT to obtain HOG coefficients. The LL
POD combination of 10:10, 20:10 and 30:10 respectively.
sub-band and HOG coefficients are convolved to obtain final

329
 ICIIBMS 2018, Your Track number here, Bangkok, Thailand

Fig 4. Variations of performance parameters for PID and POD of 10:10 Fig 7. Variations of performance parameters for PID and POD of 10:10
combination combination.

Fig 5. Variations of performance parameters for PID and POD of 20:10 Fig 8. Variations of performance parameterss for PID and POD of 10:20
combination combination.

Fig 9. Variations of performance parameters for PID and POD of 10:30

Fig 6. Variations of performance parameters for PID and POD of 30:10
combination.
combination
The percentage variations of EER, OTSR and MTSR for PID The percentage variations of EER, OTSR and MTSR
and POD combinations of 10:10, 20:10, 30:10 are tabulated in for PID and POD combinations of 10:10, 10:20, 10:30 are
table 1. It is observed that PID values increases and POD is tabulated in table 2. It is observed that for PID is kept constant
kept constant, the EER ,MTSR and OTSR varies as shown in and POD values increases, the EER ,MTSR and OTSR varies
Table 1. The MTSR values decreases with increase in PID as shown in Table 2. The performance parameters are almost
values. constant for constant PID values.
TABLE 1 TABLE 2
VARIATION OF PERFORMANCE PARAMETERS WITH PID FOR VARIATION OF PERFORMANCE PARAMETERS WITH POD FOR
ORL. ORL
%MTS
PID POD %EER %OTSR %MTSR PID POD %EER %OTSR
R
10 10 20 80 100 10 10 20 80 100
20 10 15 75 85 10 20 20 80 100
30 10 20 78 88 10 30 18 80 100

The performance parameters such as percentage FRR, C. Analysis Using JAFFE Database
FAR, and TSR are noted for variations in threshold values and The performance parameters such as percentage FRR,
the graphs are obtained as shown in Fig. 7, 8 and 9 for PID FAR, and TSR are noted for variations in threshold values and
and POD combination of 10:10, 10:20, 10:30 respectively the graphs are obtained as shown in Fig. 10, 11 and 12 for PID
with PID constant values. and POD combinations of 4:4, 5:4, 6:4 respectively. It is
observed that the value of FRR decreases with increase in

330
 ICIIBMS 2018, Your Track number here, Bangkok, Thailand
threshold values whereas, the value of FAR and TSR increases graphs are obtained as shown in Fig. 13, 14 and 15 for PID
with threshold values. The percentage EER values are and POD combinations of 4:4, 4:5 , 4:6 respectively.
decreasing for PID and POD combinations of 4:4, 5:4 and 6:4
respectively.

Fig 13. Variations of performance parameters for PID and POD of 4:4
combination.
Fig 10. Variations of performance parameters for PID and POD of 4:4
combination.

Fig 14. Variations of performances parameters for PID and POD of 4:5
combination.
Fig 11. Variations of performance parameters for PID and POD of 5:4
combination.

Fig 15. Variations of performance parameterss for PID and POD of 4:6
combination.
Fig 12. Variations of performance parameters for PID and POD of 6:4
combination.
The percentage variations of EER, OTSR and MTSR
for PID and POD combinations of 4:4, 4:5, 4:6 are tabulated
The percentage variations of EER, OTSR and MTSR for PID
in Table 4 It is observed that, for PID is kept constant and
and POD combination of 4:4, 5:4, 6:4 are tabulated in table 3.
POD values increases, the EER decreases where as OTSR and
It is observed that, as PID values increases OTSR values
MTSR remains constant.
increases and MTSR values are constant.
TABLE 4
TABLE 3
VARIATION OF PERFORMANCE PARAMETERS WITH POD FOR
VARIATION OF PERFORMANCE PARAMETERS WITH PID FOR
JAFFE
JAFFE
PID POD %EER %OTSR %MTSR
PID POD %EER %OTSR %MTSR
4 4 12 75 100
4 4 12 75 100
4 5 15 75 100
5 4 15 80 100
4 6 12 75 100
6 4 12 84 100

The performance parameters such as percentage FRR, FAR,

and TSR are noted for variations in threshold values and the

331
 ICIIBMS 2018, Your Track number here, Bangkok, Thailand
D. Analysis Using YALE Database TABLE 5
The performance parameters such as percentage FRR, VARIATION OF PERFORMANCE PARAMETERS WITH PID FOR
FAR, and TSR are noted for variations in threshold values and YALE
the graphs are obtained as shown in Fig. 16, 17 and 18 for PID PID POD %EER %OTSR %MTSR
and POD combination of 8:5, 9:5, 10:5 respectively. It is 8 5 12 88 100
observed that the value of FRR decreases with increase in 9 5 11 89 100
threshold values whereas, the value of FAR and TSR increases 10 5 10 90 100
with threshold values. The percentage EER values decreases
with increase in PID. The performance parameters such as percentage FRR,
FAR, and TSR are noted for variations in threshold values and
the graphs are obtained as shown in Fig. 19, 20 and 21 for PID
and POD combinations of 8:5, 8:6 , 8:7 respectively.

Fig 16. Variations of performance parameters for PID and POD of 8:5
combination.

Fig 19. Variations of performance parameters for PID and POD of 8:5
combination.

Fig 17. Variations of performance parameters for PID and POD of 9:5
combination.

Fig 20. Variations of performance parameters for PID and POD of 8:6
combination.

Fig 18. Variations of performance parameters for PID and POD of 10:5
combination.
Fig 21. Variations of performance parameters for PID and POD of 8:7
The percentage variations of EER, OTSR and MTSR combination.
for PID and POD combinationss of 8:5, 9:5, 10:5 are The percentage variations of EER, OTSR and MTSR
tabulated in table 5 It is observed that as PID values increases, for PID and POD combinations of 8:5, 8:6 , 8:7 are tabulated
the EER decreases and OTSR increases and MTSR remains in Table 6. It is observed that PID is kept constant and POD
constant at 100. values increases, the EER remains constant as well OTSR and
MTSR .

332
 ICIIBMS 2018, Your Track number here, Bangkok, Thailand
TABLE 6 The percentage variations of EER, OTSR and MTSR
VARIATION OF PERFORMANCE PARAMETERS WITH POD FOR for PID and POD combinations of 40:60, 50:60, 60:59 are
YALE tabulated in table 7. It is observed that as PID values increases,
PID POD %EER %OTSR %MTSR the EER remains zero and OTSR and MTSR remains constant.
8 5 12 88 100 TABLE 7
8 6 12 88 100 VARIATION OF PERFORMANCE PARAMETERS WITH PID FOR L-
8 7 12 88 100 SPACEK
PID POD %EER %OTSR %MTSR
E. Analysis with Combined L-SPACEK Database 40 60 0 98 100
The performance parameters such as percentage FRR, FAR, 50 60 0 99 100
and TSR are noted for variations in threshold values and the 60 59 0 99 100
graphs are obtained as shown in Fig. 22, 23 and 24 for PID The performance parameters such as percentage FRR,
and POD combinations of 40:60, 50:60, 60:59 respectively. It FAR, and TSR are noted for variations in threshold values and
is observed that the value of FRR decreases with increase in the graphs are obtained as shown in Fig. 25, 26, and 27 for
threshold values whereas, the values of FAR and TSR PID and POD combinations of 60:40, 60:50 , 60:60
increases with threshold values. The percentage EER values respectively.
are zero for PID and POD combinations of 40:60, 50:60 and
60:59 respectively.

Fig 25. Variations of performance parameters for PID and

Fig 22. Variations of performance parameters for PID and POD of 40:60 POD of 60:40 combination.
combination.

Fig 23. Variations of performance parameters for PID and POD of 50:60
combination.
Fig 26. Variations of performance parameters for PID and
POD of 60:50 combination.

Fig 24. Variations of performance parameters for PID and POD of 60:60
combination.

333
 ICIIBMS 2018, Your Track number here, Bangkok, Thailand
Fig 27. Variations of performance parameters for PID and face images of different sizes are converted to uniform size of
POD of 60:59 combination. 128×128. The DWT is applied on face images and considered
The percentage variations of EER, OTSR and MTSR only the compressed LL sub-band. The HOG is used on LL
for PID and POD combinations of 60:40, 60:50 , 60:60 are sub-band to obtain further compressed initial features. The
tabulated in Table 8 It is observed that as PID is kept constant final features are obtained using 2D convolution of LL sub-
and POD values increases, the EER remains constant as well band and HOG matrix. It is observed that, the performance of
MTSR. proposed algorithm is improved compared with the existing
Table 8 algorithms. The proposed method is implemented using
Variation of performance parameters with POD for L- embedded system processor or FPGA for real time
SPACEK applications in future.
PID POD %EER %OTSR %MTSR
40 60 0 98 100 REFERENCES
50 60 0 99 100 [1] Bilel Ameur, Mebarka Belahcene, Sabeur Masmoudi,
60 59 0 99 100 Amira Guidara Derbel and Ahmed Ben Hamida, “A New
GLBSIF Descriptor for Face Recognition in the
Uncontrolled Environments”, International Conference
F. Comparision of Recognition Rate of Proposed Method on Advanced Technologies for Signal and Image
with Existing Methods Processing, pp. 22-24, 2017.
The recognition rate using ORL and YALE face database [2] Dong Wang and Shunfang Wang, “A New Method of
of proposed method is compared with existing methods Two Dimensional Direct LDA and Its Applications in
presented by Erhu Zhang et al.,[9], and Shailaja and Anuradha Face Recognition”, International Conference on Digital
[10], is given in Table 9. It is observed that recognition rate is Home, pp. 58-63, 2016.
high in the case of proposed method compared to existing [3] Michel F. Valsfar, Ennique Sanchez-Lozano, Jeffery
methods. Fcohn, Laszlo A. Jeni, Jeffry M. Girard, Zheng Zhang,
Liking Yin and Make Panic," Addressing Head Pose in
TABLE 9 the Third Facial Expression Recognition and Analysis
COMPARISON OF %MTSR VALUES OF PROPOSED METHOD WITH Challenge", IEEE 12th International Conference on
EXISTING METHODS Automatic Face and Gesture Recognition, pp.839-847,
%MTSR 2017.
Methods
ORL database YALE database [4] Nawaf Yousef Almudhahka, Mark S Nixon and Jonathan
Erhu Zhang et S Hare,“Automatic Semantic Face Recognition”, IEEE
74.06 80.77 12th International Conference on Automatic Face and
al.,[9]
K shailaja and Gesture Recognition, pp. 180-185, 2017.
87 92.8 [5] Jaffe Database, http://www.kasrl.org/jaffe_download.html.
Anuradha[10]
Proposed method 91 100 [6] L-spacek databasehttp://cswww.essex.ac.uk/mv/allfaces.
[7] YaleUniversity,http://cvc.cs.yale.edu/cvc/projects/yaleface
The recognition rate using JAFFE face database of s/yalefaces.html.
proposed method is compared with existing methods [8] ORL database, http://www.camrol.co.uk
presented by Yubo Wang et al.[11], is given in Table 10. It is [9] Erhu Zhang, Yongchao Li and Faming Zhang, “A Single
observed that recognition rate is high in the case of proposed Training Sample Face Recognition Algorithm Based on
method compared to existing methods. Sample Extension”, 2013 Sixth International Conference
on Advanced Computational Intelligence, pp. 19-21, 2013.
TABLE 10 [10] K. Shailaja and B. Anuradha, “Effective Face
COMPARISON OF %MTSR VALUES OF PROPOSED METHOD WITH Recognition using Deep Learning Based Linear
EXISTING METHODS
Discriminant Classification”, IEEE International
%MTSR Conference on Computational Intelligence and
Methods Computing Research, pp. 1- 6, 2016.
ORL database
[11] Yubo Wang, Haizhou AI, Bo WU and Chang Huang ,
Yubo Wang et al.,[11] 92.4 “Real Time Facial Expression Recognition with
Adaboost”, Proceedings of the International Conference
Proposed method 100 on Pattern Recognition IEEE, pp. 926-929, 2004
V. CONCLUSION
The face recognition is used for several applications in day
to day activities. In this paper, convolution based feature
extraction technique for face identification using DWT and
HOG is proposed. The standard face databases are used to test
the proposed method for various performance parameters. The

334