Face Detection & Recognition Report

Andreea Diana Dinca

TABLE OF CONTENTS
Summary ....................................................................................................................................................... 1
Introduction .................................................................................................................................................. 2
Purpose ..................................................................................................................................................... 2
Scope ......................................................................................................................................................... 2
Method ..................................................................................................................................................... 2
Face Detection In OpenCV ............................................................................................................................ 2
Problems Encountered And Their Solutions ............................................................................................. 3
Face Recognition With Microsoft’s Face API ................................................................................................ 3
API Limitations .......................................................................................................................................... 3
Problems Encountered And Their Solutions ............................................................................................. 3
SWOT Analysis............................................................................................................................................... 5
Conclusion ..................................................................................................................................................... 5
Recommendations ........................................................................................................................................ 6
Appendix I. Evaluation Matrix Scores ........................................................................................................... 6

SUMMARY
Following the project, the purpose of this report is to analyze the advantages and
disadvantages, problems encountered and solutions found, when using face detection and
recognition in an academic environment to keep track of the attendance of students.
A small introduction will outline the content of the report and it will continue with the
advantages and disadvantages of the face detection and recognition system compared to the
other biometrics authentication methods. Subsequently, detailed description of the two main
parts of the project and how they work will follow, as well as problems encountered during the
project and how they were solved. Even though it is difficult to build a flawless system,

1|Page
Ref: Face Detection & Recognition Report 08/09/2017
Andreea Diana Dinca, IT Innovation Intern
[email protected]
solutions were found for the most impactful issues and work arounds were found for the
problems caused by the limitations of the software used.

INTRODUCTION

PURPOSE
The purpose of this report is to follow up on a 10-week project on face detection and
recognition and give insight on how feasible it is to use a face recognition attendance system in
a university environment.

SCOPE
The system should be built to be used for a prolonged period of time anywhere in the university
campus where attendance would be tracked.

METHOD
Building such a system from scratch using the Scala programming language helped achieve a
better understanding of the field as well as its advantages and disadvantages compared to
other biometric authentication methods. After some research, the decision to do face detection
using an OpenCV library for Scala and face recognition using Microsoft’s Face API was
unavoidable due to not having a system that could reliably do both detection and recognition in
the project’s circumstances.

FACE DETECTION IN OPENCV

OpenCV (Open Source Computer Vision) is a library of programming functions for real-time
computer vision. The face detection part of the project was made using an OpenCV Library for
Scala. The reason was that most Face APIs are restricted to doing detection on pictures only,
whereas the project was required to have face detection done on a live video footage to speed
up the process of checking student attendance and prevent queues before lectures.
The OpenCV library proved to be flexible enough for the project as it can accurately detect a
face in real time and highlight it by drawing a rectangle around the faces of the students
passing by. This all happens in a window separate from the face recognition so the lecturer can
keep track of both students passing by while having their faces detected and the feedback from
the recognition part of the system. While faces are being detected, the application takes a
snapshot of the live footage every second and then sends it to the recognition system.

2|Page
Ref: Face Detection & Recognition Report 08/09/2017
Andreea Diana Dinca, IT Innovation Intern
[email protected]
PROBLEMS ENCOUNTERED AND THEIR SOLUTIONS
It only supports a few programming languages
Even though OpenCV offers more than 3000 optimised algorithms, it only offers them for a
small number of programming languages, namely C/C++, Python and Java for Android. However,
wrapper libraries have been developed for other languages to encourage adoption by a wider
audience.
SOLUTION: An organisation named Bytedeco, who constantly work on adapting C/C++ libraries
to Java, have made an OpenCV library for Java and was later adapted to work with Scala. This
library was used for having face detection work in real-time on a webcam feed.

FACE RECOGNITION WITH MICROSOFT’S FACE API

The recognition part of the project was made using Microsoft’s Face API. The reason is that
Microsoft’s API offers the ability to create, delete, and update a face list, which represents a
group of pictures that must have only one face in them, that can be used to compare a face
from outside the list against all the faces in the list and find a match. Therefore, these face lists
can be used to their full potential in such an environment.

API LIMITATIONS
However, there are a few restrictions when using Microsoft’s Face API: a free account can make
30.000 calls to the API per month and 20 per minute, whereas paid accounts can make 10 calls
per second; only 64 face lists are allowed in one subscription; a face list cannot have more than
1000 faces; once a face has been added to a face list, the user receives the ID that was
associated to that face, but there is no way to physically see what face/picture is represented
by that ID anymore.

PROBLEMS ENCOUNTERED AND THEIR SOLUTIONS

Not being able to retrieve a face after adding it to a face list
This issue makes it difficult to find mistakes in the system, such as a student ID being linked to
the wrong face, which can greatly reduce efficiency and accuracy, especially in an environment
where such a system would be used on a great number of students on a daily basis. It would be
impossible to know when a mistake has been made without being able to check if a student ID
matches the right face and therefore a solution is essential for the system to be used
appropriately.
SOLUTION: A work around to this issue is to keep track of all the face lists by creating a folder
for each new list. Each time a face is added to a face list the same picture is copied to the face
3|Page
Ref: Face Detection & Recognition Report 08/09/2017
Andreea Diana Dinca, IT Innovation Intern
[email protected]
list’s folder with the name of the student ID it is associated with. It can be a naïve solution if
implemented to work on a local machine since if the application would be used on a different
machine, it would not be able to reproduce all the folders and pictures that were created on
the previous machine. However, if the application would be connected to the university's
servers and the solution would create the folders on the mentioned servers, then it would be
easy to keep track of all the face lists regardless of what machine is being used.
OTHER POTENTIAL SOLUTIONS:

 A solution that avoids the use of folders would be to upload the pictures in an online
database, but the downside is that storing a massive number of pictures in a database
negatively impacts its performance and maintainability.
 A different solution is to upload the pictures on a photo-management website or on
cloud storage. The main disadvantage of this approach would be uploading confidential
information and/or pictures of the students on a website that is not guaranteed to be
secure.

Having a limited number of calls to the API

It is a minor but inconvenient issue for the prototype as the calls to the API could be reduced by
slowing down the application, however a final product would need to make more calls than the
limit for a free account. As mentioned above, a free account can make 30.000 calls in a month
and only 20 calls a minute, which means that the application cannot run for longer than a few
seconds since it can only make a few calls even before it starts the face recognition system. The
first call, to list all the existing face lists on the account, is made when the user goes to the face
list menu; creating, deleting, updating a face list, adding a face to a face list and deleting a face
from a face list would all make one call to the API; after the webcam is started, each snapshot
has to go through Microsoft’s face detection to obtain the face’s ID, which is then passed to the
face recognition API, returning the face ID of the recognised face, then makes another call to
the API to get the student ID that is matched to the face, therefore each snapshot makes in
total 3 calls. For the application to run continuously for a longer amount of time, in the best
case scenario where only one API call is made before starting the face recognition system, a
snapshot of the detected faces would be saved every 10 seconds. It is enough for testing
purposes, however such a pace in a university environment would be very slow and inefficient.
SOLUTION: The only solution for this issue would be to use a paid Microsoft account which
allows up to 10 calls per second. This would significantly improve the speed and consistency of
the entire process.

4|Page
Ref: Face Detection & Recognition Report 08/09/2017
Andreea Diana Dinca, IT Innovation Intern
[email protected]
SWOT ANALYSIS
The following SWOT Analysis outlines the advantages and disadvantages a facial detection and
recognition system can bring.

STRENGTHS OPPORTUNITIES

 It can be faster than other biometrics  It can speed up the process of checking
authentication methods. attendance at school/work
 There are a lot of APIs and SDKs online  Face recognition systems are becoming
that help build such a system. more reliable and accurate
 It is easy to detect intruders.  Such systems are easy to implement
 Footage can be recorded to be due to how advanced technology is
manually checked.  It offers protection against intruders as
 It can detect emotions and other face they can be easily detected.
details such as eye and hair colour.

WEAKNESSES THREATS
 It is not as accurate as other biometrics  Other biometrics authentication
authentication methods. methods can be more accurate.
 It depends on the distance and pose of  Intruders can abuse the system by
the face as well as the quality, wearing accessories to hide their
illumination of the image/video used. identity.
 Accessories such as glasses or hats can  It has to use a live video footage for
negatively impact the accuracy of the recognition to be faster than other
system. systems.
 It is slow if it does not use a live video
footage for recognition.

CONCLUSION
A face detection and recognition system would certainly speed up the process of checking
student attendance in comparison to other biometrics authentication methods and in the right
circumstances it would be able to match their accuracy. Nowadays there are a wide variety of
software, whether it is a Face API like Microsoft’s or a library like OpenCV, that makes face
detection and recognition accessible and reliable and is constantly improving. Each software

5|Page
Ref: Face Detection & Recognition Report 08/09/2017
Andreea Diana Dinca, IT Innovation Intern
[email protected]
imposes various restrictions, such as the limited number of calls you can make to Microsoft’s
Face API. However, using more than one software can reduce these restrictions and lead to
better results.

RECOMMENDATIONS
If this system would be implemented in a university, the following would be recommended:

 The camera that would be used for the face recognition should be placed in front of the
lecture theatre door at a distance of 3 feet and a height of 65 inches. This would assure
a better accuracy from the face recognition system.
 The pictures that are added to face lists should ideally be the same as the pictures used
for student IDs since they are guaranteed to contain only one face.
 The application should be hosted on the university’s servers to ensure consistency with
the folders corresponding to each face list created and with the pictures in each folder
corresponding to what faces each face list contains.

APPENDIX I. EVALUATION MATRIX SCORES

Scoring
Area System Score Reason
1 = Idea Some companies already have such systems in place to
Maturity 5 = Mainstream check attendance and there are a few products on the
Product 3 market that can be used for this purpose.
Technology Any type of a video camera would be compatible with a
(Adoption 1 = > 3 years facial detection and recognition system. The software
5 = < 3 months
4 would run on any operating system.
timescales)

Business The cameras and machines cost would have to be taken

Process into consideration.
1 = > 3 years
(Adoption 5 = < 3 months
4
timescales)

Adoption 1 = v long time Placing the cameras where they are needed and installing
5 = very short
4 the software on a machine is a quick process.
Overview
Existing There is little or no negative impact on existing technology.
1 = v large impact
Technology 5 = very little
4
(Impact)
Depending on the system used the requirements could
Resources 1 = v large impact
5 = very little
3 differ, such as a system could only work on machines that
Required have Java installed.

6|Page
Ref: Face Detection & Recognition Report 08/09/2017
Andreea Diana Dinca, IT Innovation Intern
[email protected]
 It could be used to check attendance in the workplace or at
1=very difficult
Scope 5=very easy
4 school and the software would run on newer machines as
well as older ones.
It is an automated system that can be straightforward to
1=very difficult
Usability 5=very easy
4 use, however staff would need to be trained in case the
system malfunctions.
It is be secure as it would detect people that are not
1 = very poor
Security 5 = excellent
4 recognised, however staff members would have to
intervene and deal with the intruder.
Innovation 1 = low innov. Provides an easy and quick way to check attendance at
5 = high innov.
4 school or work.
Value
Camera will need to be installed in each lecture room and
1=very expensive
Cost compatible software needs to be set up. There are
5=very cost 3
Effectiveness effective commercial off-the-shelf products to buy, although the in-
house development may be another possible solution.
Facial detection and recognition systems that check
Adoption <20 - not ready
attendance have been on the market for a while. It is not a
20-29 - emerging
Readiness 30-39 - Adoptable 31 new idea, but it is an efficient system even with today's
Score >39 Fully Ready technologies. Such systems are reliable in terms of cost and
time and they are easy to implement.
Note: Rows that have no highlight colour indicate the score value is not added to the adoption
readiness total. Instead, the overview score for that area is used as part of the total score.

7|Page
Ref: Face Detection & Recognition Report 08/09/2017
Andreea Diana Dinca, IT Innovation Intern
[email protected]