2018 IEEE Recent Advances in Intelligent Computational Systems (RAICS) | December 06 - 08, 2018 | Trivandrum

Missing Child Identification System using Deep

Learning and Multiclass SVM

Pournami S. Chandran1, Byju N B2, Deepak R U3, Nishakumari K N4, Devanand P5, Sasi P M6

Centre for Development of Advanced Computing (C-DAC)

Thiruvananthapuram, India
[email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Abstract— In India a countless number of children are Mostly missing child cases are reported to the police. The
reported missing every year. Among the missing child cases a child missing from one region may be found in another
large percentage of children remain untraced. This paper region or another state, for various reasons. So even if a child
presents a novel use of deep learning methodology for is found, it is difficult to identify him/her from the reported
identifying the reported missing child from the photos of missing cases. A framework and methodology for
multitude of children available, with the help of face developing an assistive tool for tracing missing child is
recognition. The public can upload photographs of suspicious described in this paper. An idea for maintaining a virtual
child into a common portal with landmarks and remarks. The space is proposed, such that the recent photographs of
photo will be automatically compared with the registered
children given by parents at the time of reporting missing
photos of the missing child from the repository. Classification
cases is saved in a repository. The public is given provision
of the input child image is performed and photo with best
match will be selected from the database of missing children. to voluntarily take photographs of children in suspected
For this, a deep learning model is trained to correctly identify situations and uploaded in that portal. Automatic searching
the missing child from the missing child image database of this photo among the missing child case images will be
provided, using the facial image uploaded by the public. The provided in the application. This supports the police officials
Convolutional Neural Network (CNN), a highly effective deep to locate the child anywhere in India.
learning technique for image based applications is adopted When a child is found, the photograph at that time is
here for face recognition. Face descriptors are extracted from
matched against the images uploaded by the Police/guardian
the images using a pre-trained CNN model VGG-Face deep
architecture. Compared with normal deep learning
at the time of missing. Sometimes the child has been missing
applications, our algorithm uses convolution network only as a for a long time. This age gap reflects in the images since
high level feature extractor and the child recognition is done by aging affects the shape of the face and texture of the skin.
the trained SVM classifier. Choosing the best performing CNN The feature discriminator invariant to aging effects has to be
model for face recognition, VGG-Face and proper training of it derived. This is the challenge in missing child identification
results in a deep learning model invariant to noise, compared to the other face recognition systems. Also facial
illumination, contrast, occlusion, image pose and age of the appearance of child can vary due to changes in pose,
child and it outperforms earlier methods in face recognition orientation, illumination, occlusions, noise in background
based missing child identification. The classification etc. The image taken by public may not be of good quality,
performance achieved for child identification system is as some of them may be captured from a distance without the
99.41%. It was evaluated on 43 Child cases. knowledge of the child. A deep learning [1] architecture
considering all these constrain is designed here.
Keywords— Missing child identification, face recognition,
deep learning, CNN, VGG-Face, Multi class SVM. The proposed system is comparatively an easy,
inexpensive and reliable method compared to other
I. INTRODUCTION biometrics like finger print and iris recognition systems.
Children are the greatest asset of each nation. The future
II. RELATED WORKS
of any country depends upon the right upbringing of its
children. India is the second populous country in the world Earliest methods for face recognition commonly used
and children represent a significant percentage of total computer vision features such as HOG, LBP, SIFT, or SURF
population. But unfortunately a large number of children go [2-3]. However, features extracted using a CNN network for
missing every year in India due to various reasons including getting facial representations gives better performance in
abduction or kidnapping, run-away children, trafficked face recognition than handcrafted features.
children and lost children. A deeply disturbing fact about In [4], missing child identification is proposed which
India’s missing children is that while on an average 174 employees principal component analysis using Eigen vectors
children go missing every day, half of them remain untraced. is used for face recognition system.
Children who go missing may be exploited and abused for
various purposes. As per the National Crime Records Bureau FindFace is a website that lets users search for members
(NCRB) report which was cited by the Ministry of Home of the social network VK by uploading a photograph [5].
Affairs (MHA) in the Parliament (LS Q no. 3928, 20-03- FindFace employs a facial recognition neural network
2018), more than one lakh children (1,11,569 in actual algorithm developed by N-Tech Lab to match faces in the
numbers) were reported to have gone missing till 2016, and photographs uploaded by its users against faces in
55,625 of them remained untraced till the end of the year. photographs published on VK, with a reported accuracy of
Many NGOs claim that estimates of missing children are 70 percent.
much higher than reported.

978-1-5386-7336-2/18/$31.00 ©2018 IEEE 113

 2018 IEEE Recent Advances in Intelligent Computational Systems (RAICS) | December 06 - 08, 2018 | Trivandrum

The “Tuanyuan”, or “reunion” in Chinese, app developed for any matching with the database at any time using the
by Alibaba Group Holding Ltd. helped Chinese authorities proposed system.
recover hundreds of missing children [6]. The app has
In the following sections the paper details the work flow
allowed police officers to share information and work
for child matching methodology. The flow chart of the
together with public.
automatic child face identification methodology is as shown
III. WORK FLOW OF FACE RECOGNITION in Fig 2.
Here we propose a methodology for missing child
identification which combines facial feature extraction based
Input Child Images
on deep learning and matching based on support vector
machine. The proposed system utilizes face recognition for
missing child identification. This is to help authorities and
parents in missing child investigation. The architecture of the Face Images Preprocessing
proposed frame work is given below,

Rescaling into
Portal Login
224x224

Mobile
Alert Message App/Portal
Upload Photo VGG-Face Face Descriptors
(Matching (Public)
Found)
Feature
Extraction
Statistical Upload photo of
Reports suspicious
Find Matching children with
(Automatic) details
• Place
• Date and Time
• Landmarks
Get Child Multi class SVM
• Remarks
Details

Fig. 1. Architecture of proposed child identification system Recognized Child

Face
It consists of a national portal for storing details of
missing child along with the photo. Whenever a child
missing is reported, along with the FIR, the concerned officer
uploads the photo of the missing child into the portal. Public Child1 Child2 …… Child n
can search for any matching child in the database for the Fig. 2. Software Flow of face recognition system
images with them. The system will prompt the most
matching cases. Once the matching is found, the officer can Images of reported missing children are saved in a
get the details of the child. The system also generates various repository and the face area is selected for cropping to obtain
statistical reports. input face images. Learned features from a Convolutional
The public can upload photo of any suspicious child at Neural Network (CNN), a specific type of deep learning
any time into the portal with details like place, time, algorithm, are used for training a multi class SVM classifier.
landmarks and remarks. The photo uploaded by the public This machine learning approach is used to correctly label the
will be automatically compared with photos of the registered child using the name indicated in the database provided by
missing children and if a matching photo with sufficient the concerned authority.
score is found, then an alert message will be sent to the
concerned officer. The message will also be visible in the IV. CONVOLUTIONAL NEURAL NETWORKS(CNN)
message box of the concerned officer login screen. The Convolutional Neural Networks (CNNs) are essential
portal for the public can also be maintained as a mobile app, tools for deep learning methods and are more appropriate for
where he or she can upload photo of suspicious children with working with image data [7]. CNNs or ConvNets are
details. In the mobile app, location of the person updating the composed of series of interconnected layers and these layers
photo will also be automatically recorded. consist of repeated blocks of convolutional, ReLU (rectified
linear units), pooling layers and fully connected layers.
Whenever public uploads photo of a suspected child, the
system generates template vector of the facial features from Convolutional layer convolves the input face image data
the uploaded photo. If a matching is found in the repository, with different kernels to produce activation maps or feature
the system displays the most matched photo and pushes a maps representing low level features like edges or curves.
message to the concerned Officer portal or SMSs the alert This feature map is given to next convolutional layer
message of matching child. Similarly the Officer can check producing activations which represent high level features
indicating landmarks in face. The Convolutional layer

114
 2018 IEEE Recent Advances in Intelligent Computational Systems (RAICS) | December 06 - 08, 2018 | Trivandrum

basically defines a set of filter weights which are updated mean face image, computed from all the training set images,
during network training. subtracted.
ReLU followed by each convolutional layer introduces VI. EXTRACTION OF FACIAL FEATURES
nonlinearity in the system. This layer applies the function
f(x) = max(0,x) to the input data of the layer. VGG-Face is trained to recognize the 2622 identities and
other classes can’t be identified using this. But the activation
The pooling layers merge similar features into one by vectors extracted from VGG-Face architecture can be used as
down sampling with suitable size. The basic idea behind the feature representations to classify each child category.
pooling layer is that the relative position to other feature is The last classification layer is removed and extracts the 4K
more important than the exact location of a specific feature. dimensional features from the first fully connected layers.
It reduces the dimensions of feature maps and network The resulting feature vector is normalized by dividing each
parameters. component by the L2 norm of this 4096 dimensional vector.
Thus the pre-trained CNN VGG-Face is made to perform as
The final layer called fully connected layer outputs the
an automatic facial feature extractor for training the
number of classes. There are several fully-connected layers
classifier.
converting the 2D feature maps into a 1D feature vector, for
further feature representation. VII. MULTI CLASS SVM CLASSIFIER
A. VGG-Face CNN descriptor Each face image corresponds to a child and child face
A very deep CNN called VGG-Face network [8] is used recognition is considered as an image category classification
for face recognition and its architecture is given in full detail problem. The task is to classify input image uploaded by the
in Fig 3. The CNN architecture comprises 11 blocks, each public into one of the given category based on the image
containing a linear operator followed by one or more non- representation. Basically CNN architecture consists of
linearities such as ReLU and max pooling. The first eight computational layers for feature extraction and a classifier
such blocks are said to be convolutional as the linear layer at the final stage. The VGG-face CNN model
operator is a bank of linear filters (linear convolution). It uses employs the softmax activation function for labeled class
filters of size 3x3 with stride and pad of 1, throughout the prediction, suggesting the class each image belongs to. The
network. All the convolution layers are followed by a softmax in the CNN layers is replaced with a multi class
rectification layer (ReLU). Max pooling layers used only SVM trained with feature vector array from each image.
2x2 size with stride 2. The last three blocks are fully One-versus-rest linear SVM classifier is used and is trained
connected layers, they are the same as a convolutional layer, on the dataset. Extracted feature vector array is used to train
but the size of the filters matches the size of input data, such this classifier.
that each filter provides representative data from the entire
image. Output of the first two FC layers are 4096 VIII. RESULTS AND DISCUSSIONS
dimensional and the last FC layer has 2622 dimensions The face identification algorithm is implemented using
followed by L-dimensional metric embedding. Optimization MATLAB 2018a platform. The experiments are carried on
is done by stochastic gradient descent using mini-batches of Microsoft Windows 7, 64 bit Operating System with Intel
64 samples and momentum coefficient of 0.9 core i7, 3.60GHz processors having 32GB RAM. For
dealing with CNN architectures additional processing
capability is needed. Use of GPU is recommended for
training the models and Nvidia GeForce TitanX 12GB
graphics card is used.
The user defined database includes 846 child face images
with 43 unique children cases. Training and test set is
prepared by splitting the database images. 80% of images
from each child category are selected for training and 20%
for testing, resulting in 677 training set images and 169 test
set images. The training set and validation set consists of
images of each child in the earlier days and testing is done
Fig. 3. VGG-Face network architecture with images of children after an age gap to evaluate the
system in all conditions.
V. PREPROCESSING CNN implementation is based on MatConvNet package
Preprocessing input raw image in the context of face [9] with deep integration of CNN building blocks in
recognition involves acquiring the face region and MATLAB environment. Pre-trained VGG-Face CNN is also
standardizing images in a format compatible with the CNN provided by MatConvNet. For the experiments here
architecture employed. Each CNN has a different input size MatConvNet 1.0-beta25 version is downloaded and used.
requirement. The photographs of missing child acquired by a
digital camera or mobile phone are taken and categorized The training set images are preprocessed to the size
into separate cases for creating the database of face specified by the CNN architecture before passing to the CNN
recognition system. The face region in each image is model. The face region is cropped within a rectangular
identified and cropped for getting the input face images. The region from every image of the acquired input database. The
cropped face images are resized to 224x224 because VGG- images fed to VGG-Face are of fixed size by rescaling to
face network can process only RGB images in this particular 224x224. The activations to the input image produced by the
size. The input to the deep network is fixed sized image with first fully connected layer of the VGG-Face network
architecture is taken as the CNN Feature descriptor. The

115
 2018 IEEE Recent Advances in Intelligent Computational Systems (RAICS) | December 06 - 08, 2018 | Trivandrum

normalized feature vector, each having a length of 4096, is softmax of the VGG-Face model and extracting CNN image
used for training the SVM classifier for classifying the image features to train a multi class SVM, it was possible to
of face and recognizes the child. achieve superior performance. Performance of the proposed
system is tested using the photographs of children with
different lighting conditions, noises and also images at
different ages of children. The classification achieved a
higher accuracy of 99.41% which shows that the proposed
methodology of face recognition could be used for reliable
missing children identification.
REFERENCES
[1] Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning", Nature,
521(7553):436–444, 2015.
[2] O. Deniz, G. Bueno, J. Salido, and F. D. la Torre, "Face recognition
using histograms of oriented gradients", Pattern Recognition Letters,
32(12):1598–1603, 2011.
[3] C. Geng and X. Jiang, "Face recognition using sift features", IEEE
International Conference on Image Processing(ICIP), 2009.
[4] Rohit Satle, Vishnuprasad Poojary, John Abraham, Shilpa Wakode,
Fig. 4. GUI for child identification showing an input image and matched "Missing child identification using face recognition system",
output image in the database International Journal of Advanced Engineering and Innovative
Technology (IJAEIT), Volume 3 Issue 1 July - August 2016.
To assess the flexibility of face recognition deep [5] https://en.wikipedia.org/wiki/FindFace
architecture against variations in image quality, artificially [6] https://www.reuters.com/article/us-china-trafficking-apps/mobile-
degraded images are created. Images obtained by changing app-helps-china-recover-hundreds-of-missing-children-
noise level, brightness, contrast, lighting conditions, idUSKBN15J0GU
obstructions, blur, aspect ratio and face positions are used for [7] Simonyan, Karen and Andrew Zisserman, "Very deep convolutional
testing the child identification system. networks for large-scale image recognition", International Conference
on Learning Representations ( ICLR), April 2015.
Face identification accuracy is computed as the ratio of [8] O. M. Parkhi, A. Vedaldi, and A. Zisserman, "Deep Face
correctly identified face images to the total number of child Recognition," in British Machine Vision Conference, vol. 1, no. 3, pp.
face images in the test set. 1-12, 2015.
[9] A. Vedaldi, and K. Lenc, "MatConvNet: Convolutional Neural
Networks for MATLAB", ACM International Conference on
(1) Multimedia, Brisbane, October 2015.

The computed recognition accuracy of the multi class

SVM using learned features from CNN is 99.41%.

Fig. 5. Images with variations correctly classified by the system

IX. CONCLUSION
A missing child identification system is proposed, which
combines the powerful CNN based deep learning approach
for feature extraction and support vector machine classifier
for classification of different child categories. This system is
evaluated with the deep learning model which is trained with
feature representations of children faces. By discarding the

116