2020 IEEE Pune Section International Conference (PuneCon)

Vishwakarma Institute of Technology, Pune India. Dec 16-18, 2020

An Approach for e-Voting using Face and

Fingerprint Verification
Shubham Shinde Manas Shende Jeet Shah
Information Technology Information Technology Information Technology
Pune Institute of Computer Technology Pune Institute of Computer Technology Pune Institute of Computer Technology
2020 IEEE Pune Section International Conference (PuneCon) | 978-1-7281-9600-8/20/$31.00 ©2020 IEEE | DOI: 10.1109/PuneCon50868.2020.9362470

Pune, India Pune, India Pune, India

[email protected] [email protected] [email protected]

Harshdeep Shelar
Information Technology
Pune Institute of Computer Technology
Pune, India
[email protected]

Abstract—India is the world’s largest democracy with nearly and to create a reliable system through which NRI and service voters
900 million eligible voters. The election period in India spans over can cast their votes securely from anywhere.
nearly six weeks for general elections and there is no alternative We propose a client-server software architecture. According to our
system working for eligible voters who are at outstations or system, a person who is a service voter or an NRI voter needs to
willing to cast their vote but not able to do so due to location first register with his fingerprint and face image. The data related to
constraints. Moreover, there is no special provision made for NRI the face and fingerprint of the voter is stored at the server-side file-
voter / overseas elector for whom it is difficult to vote in person system. The information related to voters, candidates, constituencies,
at the polling station. Service voters have to use postal ballot and and votes are stored in the Google Firebase Real-time Database.
go through a tedious process to cast their vote and this process At the client-side, there will be a fingerprint device that will take
is also prone to human errors. the fingerprint of the voter, and a GUI which will take the login
Our system is an e-Voting system that will use fingerprint and details and capture the face image of the voter. We use the HID
face verification along with a combination of firebase-database DigitalPersona 4500 Fingerprint Reader to capture the fingerprint
and server-side file-system at its back-end. Our system is designed image of the voter and a camera to capture the face image of the voter.
especially for NRI voters and Service voters for whom it is Then, the information would be sent to the server for verification,
difficult to cast their votes through the existing system. It provides in an encrypted format. The server will process the information,
an efficient, convenient, and secure mechanism for voters to compare it with the information present in the database, and then
cast their votes. The design of this system will make the voting give the response back to the client. Once the voter is successfully
process more convenient and may, therefore, lead to improving logged in, the voter would be able to cast the vote for the constituency
the turnout. in which he has registered. Once the vote is cast successfully by the
Keywords— Face Verification, Fingerprint Verification, Convo- voter, he would be logged out and can’t log in again to avoid multiple
lutional Neural Network(CNN), Encryption, Vote Security, Voting votes from one person.
System, Multimodal Biometrics, Biometrics In the current voting system which uses the EVM, the candidate
gets an idea about which booths voted for or against them. In contrast
I. I NTRODUCTION to that in our system, the candidates won’t get any such information
In paper-based voting, people cast their votes with the help of and so they cannot misuse the information, especially in the context
ballot papers. This is mainly used for the people who are on of the widely reported ‘cash for vote’ practice. In the current voting
government duty and may not be physically present to cast their system, the NRI voters and service voters need to be physically
vote. So, the service voters can cast their votes with the help of present at the booth or cast their vote through ballot papers. In our
ballot papers. In this process, there can be an error in counting the system, they can simply cast their votes from their remote location.
votes. Sometimes votes are also manipulated. Also, the people who This, in turn, would increase the voter turnout and help in practicing
are Non-Residential Indians(NRI) who cannot cast their votes as they an impartial election. The cost and manpower required to conduct
can’t be physically present in the voting booth center. the election using the current voting system are higher as compared
India used paper ballots till the 1990s to cast votes. After that, only to our voting system. Previously, an Indian EVM could be used to
NRI voters and service voters used ballot papers to cast their vote cater to a maximum of 16 candidates. Now, since the upgrade in
and all other votes were cast using EVM machine. Electronic Voting 2013, it can be used to vote for a maximum of 384 candidates along
Machine is a simple electronic device used to record votes in place of with the ”None Of The Above” (NOTA) option. Compared to the
ballot papers and boxes which were used earlier in the conventional current voting system, our voting system is more scalable with higher
voting system [1]. In Kerala in the year 1982, EVMs were first used in capacity.
70-Parur Assembly Constituency [2]. Biometric e-Voting systems are The outline of the paper would be as follows. Section II briefly
not a phenomenon anymore, they are being actively used in countries reviews related previous research works carried out in the areas of
like Ghana and Ireland, and are spreading to many other developing biometric recognition and e-Voting systems. Section III describes the
nations [3]. In this paper, we propose a method for e-Voting which methodology adopted for Face Verification, Fingerprint Verification,
can be used parallelly, with the current Electronic Voting Machine, Storage, and Security in our voting system. Section IV presents the

978-1-7281-9600-8/20/$31.00 ©2020 IEEE 59

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 16,2021 at 19:41:33 UTC from IEEE Xplore. Restrictions apply.
results and experiments conducted on the proposed system. Finally, III. M ETHODOLOGY
Section V concludes the paper and discusses some possible future The e-Voting system is implemented using a classical client-server
directions. architecture model [9]. From the client-side, the request is sent to the
server. The request comprises the face and finger image file, let’s call
II. L ITERATURE R EVIEW this file as a test image. This test image is processed at the server and
it’s output feature vectors are generated and they are matched against
Dr. S. Ravi et al. [4] explained an approach using face, fingerprint, the feature vectors present in the file-system. After processing, the
and iris recognition for biometrics. It throws light on authentication result of the verification is sent to the client i.e. voter and then the
modalities along with their pros and cons and also enlists the voter is automatically redirected to another server, called a Voting
different limitations of biometrics unimodal systems. Technological server. Based on the voter’s information, his constituency is identified
advancements in facial, fingerprints, and iris recognition techniques and on the basis of it, the voting server fetches the information
are discussed. Subsequently, the stature of multimodal biometrics of candidates present in the concerned constituency. After voting,
systems is conveyed along with the comparative study of biometric this vote is encrypted at the client-side and is sent to the voting
furtherance. server along with the corresponding constituency information over
David Menotti et al. [5] proposed two deep representation ap- a secured channel. After reading the request, the server identifies
proaches for detecting spoofing in different biometric modalities. the constituency of the voter and routes the encrypted vote to the
These two proposed approaches recognize the spoof with help of database.
iris, face, and fingerprints. From the data, the first proposed method
approached the problem via architecture optimization, by learning
representations. The second one suggests focusing on the back-
propagation algorithm by learning the weights of the network. The
test result with the Replay-attack benchmark for face modality
achieving an accuracy of 98.75%, Italdata benchmark for fingerprint
modality with an accuracy of 97.45%.
Kai Cao et al. [6] explains the methodology for improvement
of latent fingerprints, found in a variety of criminal activities by
comparing it with a dataset of reference points. The method states an
automated algorithm for latent fingerprint recognition which uses the
Convolutional Neural Networks for ridge flow estimation and minu-
tiae descriptor extraction. The minutiae correspondence algorithm is
described in this paper along with its experimentation to find out
the final minutiae correspondences. The exploratory results for NIST
SD27 is 64.7% and WVU latent databases is 75.3%. Fig. 1. Voting System Architecture
John Soldera et al. [7] explores the challenges around facial
recognition, having related topics like face segmentation along with Voter verification is a very critical and significant step in an e-
their applications. The authors have put efforts to show the biometrics Voting system. Various CPU intensive machine learning algorithms
involved in the facial biometrics and the superior applications of are run at the server-side to verify the voter. This step provides
it which includes face spoof detection, yawning detection which reliability as well as security to the whole process. To achieve this,
could be incorporated with driver alertness systems dependent on two-level security check is implemented as follows-
Bayesian decision theory. The investigational maximum result of 1. Level 1 Security Check: Face Verification
Correct Detection Rate, for driver yawning detection with 2443 2. Level 2 Security Check: Fingerprint Verification
number of frames, is 92%. The means of voter verification are discussed in the succeeding
Chris Harris et al. [8] addresses the problem of the unconstrained topics.
3D world scenes which contain heterogeneity of objects. The authors
also gives the solution to this issue by making use of computer vision- A. Level 1 Security Check: Face Verification
based motion analysis of an image. Biometrics can successfully allow systems to authenticate and is
R. Patel et al. [3]The system proposed in this paper focuses on used for access control as well as in security systems. Our system
the fingerprint verification of an individual for the accomplishment has a two-step authentication process: face verification and fingerprint
of eVoting. The UIDAI dataset is used at the back-end which has verification [10]. The first step to login into our system is face
fingerprints to match them with the captured ones. The paper throws verification.
light on ease of casting votes in general elections as well as other This deep learning-based facial recognition algorithm has two
elections in India, accurate and fast counting and labour-intensive following features- It is highly accurate. The capability of being
approaches. executed in real-time.
The previous work done about E-voting mainly uses a single-level The whole process of face recognition is divided into 4 following
security check, either face or fingerprint verification. But, our system parts:
uses a two-level security check i.e. face and fingerprint verification 1. Detecting faces in Image
which adds an additional layer of security to the whole system. Our 2. Isolating and projecting faces
system takes voter ID as the primary information from the voter, to 3. Face Encoding
enter into the system. Then, it checks the fingerprint impression and 4. Face identification
facial characteristics of an individual which are stored previously in 1) Detecting faces in Image: It is the first step in facial
the database. A folder is dedicated to every authorized voter, which recognition where faces in the image are detected. If there are more
is used to check the biometrics of the voter in that folder only at the than two faces in the image, then the image is discarded(as the only
time of the election. This increases the speed of execution and reduces voter should in the frame). Faces in the image can be detected in
false-positive outputs. Along with the above points, our system also two ways, the first one is HOG (Histogram of Oriented Gradients)
encrypts the votes and store it in a centralized real-time database and the other one is CNN. As HOG detects faces very rapidly but
which depletes the chances of data redundancy and replication. compromises accuracy, on the other hand, CNN is a time-consuming

60

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 16,2021 at 19:41:33 UTC from IEEE Xplore. Restrictions apply.
 in the server-side file system. This stored file is used to match
the features (encoded values) generated from the test image. It is
explained in step 4.
4) Face Identification: The last step is to identify the person
from our database who has the closest measurement with the test
image provided. The classification of the image is done using KNN
(k-nearest neighbour) classifier [13]. KNN is a supervised learning
algorithm which classifies the given instance based on the majority
of K-nearest neighbor. This algorithm relies on the distance between
the feature vectors, and it also has labels associated with each image,
so that it can return the name of the person who is being classified.
The distance can be calculated with the help of Manhattan distance
or Euclidean Distance. The KNN algorithm classifies by finding the
most common class among the k-closest examples.
Let X be the point which represents the class C for all i belongs
Fig. 2. Steps of Face Verification Process. to n. Here, Xt is the point which represents the class Ct whose label
is not known. It should be noted that X , Xt ∈ X and C , Ct ∈ C ∀
 ∈ n.
process but is highly accurate. As far as our project is concerned,
accuracy should not be compromised and hence we have used CNN Algorithm 1 KNN Algorithm
for ‘face-detection’. Also, the CNN based face-detection method 1: procedure KNN
identifies faces at different angles more accurately as compared to
HOG based face-detection. The output of this step is a box containing 2: Define the distance metric -d (eg. Euclidean Distance,
the x,y coordinate of the top left corner of the box along with its width Manhattan Distance, Minkowski Distance etc)
and height. 3: Calculate d(Xt , X ) ∀  ∈ n.
For this dlib’s face detection model, 4: Sort all the distances in non-decreasing order.
’cnn face detection model v1()’ is used. This model is an 5: Define value for ‘K’.
implementation of ‘One Millisecond Face Alignment with an
Ensemble of Regression Trees’ [11]. Initially, training sets are used 6: Select the first ‘K’ points from the sorted list.
in which landmarks are manually located. Based on these landmarks, 7: Let K denotes the number of points belonging to the
the regression tree is trained which estimates the facial landmark th class among K points.
positions with respect to the pixel intensities. The decision on each 8: If K ≥ Kj ∀  = j then assign label of class ‘C ’ to
node of the regression tree is taken on the basis of the probability of ‘Xt ’.
difference between intensities of two pairs of pixels. This step gives
9: end procedure
us the location of 68 x-y coordinates that maps to facial structures
on the faces.
2) Isolating and Projecting faces: The next step is to warp In the context of this paper, X represents the embeddings (fea-
the faces including eyes, lips, etc perfectly in the frame. Aligning tures) for the individual class C in the array X of size ‘n’. The value
faces accurately during both training and testing gives a boost in of K is set to 1 so that the only one class which closely resembles the
performance [12]. For this, the face landmark estimation algorithm Ct is selected.Also, Euclidean distance formula is used as a distance
is used. This algorithm identifies major landmarks that exist on the metric which is given by:
faces including the iris of an eye, the inner edge of eyebrows, the 
top of the chin, lips, etc.  n

Various basic image transformations like rotation and scale are d(Xt , Xi ) =  (Xt − X )2 (1)
done on the image so that the image is perfectly aligned in the frame. =1
3) Face Encoding: The next step is to extract certain measure-
ments from the face such as the color and size of eyes, length of the Where:
nose, the gap between eyebrows, etc. All these put together define Xt = 128-d feature vector of the test image
the face-encoding, the information which is obtained from the image. X = 128-d feature vector of an image in the database
Algorithm figures out the most accurate measurements themselves.
The algorithm uses the Deep Learning Convolutional Neural Network The corresponding image of the person is sent to the server, the
to figure out which parts of the face are important to measure. image goes through the face detection phase, where the face in the
Here we will be providing 3 images at a time:- image is detected. Further, this image is processed to get the 128d
The first two images of the same person. The third image is the output feature vector. This output feature vector is compared with the
random image from the dataset which is not of the same person encodings (feature vectors) present in the PICKLE file. If the output
whose first two images are taken. feature vector matches with the feature vectors of a particular face
From there, the general idea is that we’ll tweak the weights of our in the database, then face verification is said to be successful.
neural network so that the 128-d measurements of the first two images
will be closer to each other and farther from the measurements of the B. Level 2 Security Check: Fingerprint Verification
third image. After repeating this step many times our neural network Biometrics, the technology which can identify the person based on
learns to reliably generate 128 measurements for each person. Thus, the physical characteristics. The fingerprints can be used to identify
the output feature vector is 128-d (i.e. a list of 128 real-valued and verify a person for plenty of applications. The level 2 check in
numbers) that is used to quantify the face. The network architecture the voter verification is fingerprint verification. The finger impression
for face recognition is based on ResNet-34 as it is highly accurate is collected from a device and various techniques are applied to an
and can be executed in real-time. image in order to remove noisy components of it. Then the critical
Similar encoded values that are recorded during the registration points from the image are compared with reference to the hamming
process are stored in the PICKLE file. This PICKLE file is stored distances.

61

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 16,2021 at 19:41:33 UTC from IEEE Xplore. Restrictions apply.
 between the points on the descriptor. By employing this technique,
a score will get generated. A certain value is set to the threshold.
The score will be compared with a threshold value which will give
results whether the two prints are the same or not.
ORB: Two methods are employed in this technique: FAST key-
point Detector and BRIEF Descriptor. Firstly the key points are
determined using the FAST [14]. The FAST is the corner detection
method used to extract the feature points. The corners are the most
important features of an image as they have a lot of information.
FAST uses the circle of 16 pixels to classify whether the point under
consideration is a corner or not [15]. This is followed by applying the
Harris Corner detector algorithm. The FAST computes the intensity
of the centroid of an area with a corner at its center. In order to find
the orientation, a vector directed from the corner to centroid is used.

C. Storage
As the proposed voting system has to handle several images and
storing them into the database leads to performance and recovery
issues as the database tends to grow rapidly in size. For this, we
have used a combination of database and the file-system at the back-
end. The former one is Google Firebase realtime database. Firebase
is Google provided API to create a database and fetch from it in real
time with only a few lines of code. Data is stored as JSON and is
accessible from all the platforms [16]. It uses data synchronization
i.e. every time data changes, any connected device to the system
receives that update within milliseconds. The database used is NoSQL
databases and as such has different optimizations and functionality
Fig. 3. Flow of Fingerprint Recognition. compared to relational databases. Due to data synchronization pro-
vided by Firebase realtime database, it can provide the necessary
security and reliability required for the voting system. As the data
The method employed to compare the impressions is Oriented is stored in an unstructured and schemaless format it is possible
FAST (Features from Accelerated Segment Test) and Rotated BRIEF to scale the database for a high number of users according to the
(ORB) which is based on the FAST keypoint detector and visual de- system requirement. It is comparatively easier to fetch data from an
scriptor BRIEF i.e. Binary Robust Independent Elementary Features. unstructured database as compared to a structured database as there
The flowchart of the procedure is depicted in Fig. 3. is no rigid schema for storing the data and it offers more flexibility
The following steps are considered in order to process and match to the overall system.
the fingerprint impressions. Data related to voters, candidates, constituencies, votes, etc is
1) Image Binarization: The process of reducing the information stored in Google Firebase realtime database in the form of JSON
contained in the image from 256 shades of grey to 2, black and white objects/nodes. The entire database is in the form of a huge JSON
is called image binarization. This allows us to get a clear image tree with multiple nodes. When the voter enters Voter-ID into the
having minimal noise and lets us differentiate between the wrinkled system, the node for that particular Voter-ID is retrieved from the
surfaces and the other lines in the impression. database. Now the concept of file-system comes into the picture
2) Image Skeletonization: In order to get the unique critical when the system has to fetch the fingerprint data and face data of
points, the skeletonization of an image is carried out which reduces a particular voter for verification purpose. This data is stored in the
the binary objects to 1-pixel wide representation. file-system at the server-side rather than storing it into the database.
In this way, we can minimize the CPU usage as well as the number of
3) Minutiae Points: The image skeletonization is followed by
input/output operations on downloading the images. This file-system
detecting the crossing points on the ridges of an impression known
is well organized and structured which makes searching for the data
as minutiae points. In order to accomplish this task, Harris corner
a lot easier. To speed up the process of searching data in the file-
detection is used. The position where the edges gather, short edges
system, the concept of metadata can be used. This metadata will help
are important in fingerprint impressions [8].
the server to locate particular voter’s data rapidly and efficiently.
Minutiae feature extraction is accomplished by making use of
latent minutiae extractor using ConvNet. Minutiae are the most
important features of fingerprint recognition. A minutiae descriptor
D. Security
has an attribute that is dependent on the image characteristics in its It is necessary to store the votes cast through the system in a
neighbor. A minutiae template has a ridge flow, minutiae set and a secured manner so any unauthorized person or an attacker cannot
minutiae descriptor. A texture template is used in a small area where misuse the data. For that purpose, we use the Advanced Encryption
the latent does not contain adequate information about minutiae Standard (AES) encryption algorithm. AES algorithm is based on a
for comparison. The fully connected layer of ConvNet having 128- “substitution-permutation” network and it is a symmetric encryption
dimensional output is taken as output feature [6]. algorithm.
4) Formal Descriptor: In order to distinguish between the AES performs the following steps for encryption of data:
critical points, a formal descriptor must be created around the point. 1. Byte Substitution (SubBytes)
ORB is one of the descriptors which are most commonly used, as 2. Shiftrows
the orientation of an image may change. Subsequently, we can get 3. MixColumns
the matrix of the descriptors which identify the impression. 4. Addroundkey
5) Comparison: Once we get the matrix of descriptors for The process of decryption using AES is similar to the encryption
impressions to compare, we can calculate the Hamming distances process but is it in the reverse order.

62

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 16,2021 at 19:41:33 UTC from IEEE Xplore. Restrictions apply.
 1. Accuracy Score: 0.920635 i.e. 92.0635%
2. Precision and Recall: The table I shows precision and recall for
fingerprint verification module.

TABLE I
P RECISION AND R ECALL - F INGERPRINT V ERIFICATION M ODULE

Precision Recall F1-score Support

0 0.25 0.33 0.29 3
1 0.97 0.95 0.96 60
Macro
0.61 0.64 0.62 63
average
Weighted
0.93 0.92 0.93 63
average

3. Confusion Matrix: Confusion matrix of Fingerprint Verification

module for a sample dataset of 63 unique individuals is shown in
Fig. 5.

Fig. 4. AES algorithm

AES algorithm is at least six times faster than the triple Data
Encryption Standard (DES) encryption algorithm. The DES algorithm Fig. 5. Confusion Matrix - Fingerprint Verification Module
makes use of key size which is too small, so the DES algorithm is
replaced with the AES algorithm. The AES is a block cipher capable
of handling 128-bit blocks, using keys sized at 128, 192, and 256 B. Face Verification
bits [17]. The number of rounds in AES varies and is dependent on
The performance of the ResNet-34 model is evaluated on the
the length of the key, unlike DES. The number of rounds depends
Georgia Tech Face Dataset of images. This dataset of images con-
on key length: 10 (128-bits), 12 (192-bits), or 14 (256-bits) [18].
sists of 540 face images of 90 unique individuals under different
The vote cast using our system is encrypted at the client-side using
lighting/expressions/backgrounds. The images are in JPEG or JPG
the AES encryption algorithm using a ‘secret key’. The vote is stored
format and in varied resolution. Each pixel has an 8bit grayscale
in the Google Firebase realtime database in the form of a 16-bit
value ranging from 0 to 255.
encrypted format as a JSON node. Then while the process of counting
Metrics for the Face Recognition module :-
the votes, the votes stored in encrypted format in the database are
1. Accuracy Score : 0.887640 i.e. 88.7640%
retrieved and decrypted using the same ‘secret key’ which was used
for encryption. 2. Confusion Matrix : Confusion matrix of Face Verification
module for a sample dataset of 30 subjects is shown in Fig. 6.
IV. R ESULT 3. Precision and Recall: The table II shows precision and recall
for face verification module.
In this section, we evaluate the effectiveness of the proposed
security checks in the application. We show experiments for Face
verification and Fingerprint verification along with their combination TABLE II
for verifying the voter. P RECISION AND R ECALL - FACE V ERIFICATION M ODULE

A. Fingerprint Verification Precision Recall F1-score Support

Micro av-
The capability of the fingerprint verification module is tested on 0.89 0.89 0.89 89
erage
FVC 2006 dataset. The said module is tested on 126 samples of 60 Macro
distinctive discrete taken from different fingerprint sensors, having 0.83 0.86 0.84 89
average
Tagged Image File Format (TIFF) along with assorted resolution. Weighted
Table I, represents the precision, recall, F1 score, support of the said 0.86 0.89 0.87 89
average
module. The value ’0’ represents the two impressions are unmatched
whereas value ’1’ represents matched impressions. As seen from the
support column of table I, only 3 cases out of 63 pairs are found
to be unmatched resulting in the precision of unmatched impression The face verification module of our system works well when tested
to 25% and recall to 33%. The precision came out to be 97% and for faces with distinct feature vectors. It gives less accuracy when
recall of 95% for matching impression i.e. value ’1’ resulting in the tested for faces with very similar feature vectors, which leads to the
F1 score of 0.96. generation of false-positive outputs. Due to this, the accuracy of our
Metrics for Fingerprint Verification Module: face recognition module is affected. On the other hand, fingerprint

63

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 16,2021 at 19:41:33 UTC from IEEE Xplore. Restrictions apply.
 ACKNOWLEDGMENT
The authors of this paper would like to extend their sincere grat-
itude towards Prof. Dr Shweta Dharmadhikari (Associate Professor,
Pune Institute of Computer Technology, Pune) and Prof. Sachin Pande
(Assistant Professor, Pune Institute of Computer Technology, Pune)
for their invaluable suggestions and guidance. The authors would also
like to thank their family for their indispensable support and patience
in the complete journey of this endeavour.

R EFERENCES
[1] S. Kumar and E. Walia, “Analysis of electronic voting system in various
countries,” International Journal on Computer Science and Engineering,
vol. 3, no. 5, pp. 1825–1830, 2011.
[2] D. Nikam, D. Shetiye, and D. Bhoite, “A critical study of electronic
voting machine evm utilization in election procedure,” International
Journal of Trend in Scientific Research and Development, vol. Special
Issue, pp. 1–3, 03 2019.
[3] R. Patel, V. Ghorpade, V. Jain, and M. Kambli, “Fingerprint based e-
voting system using aadhar database,” International Journal for Re-
search in Emerging Science Technology, vol. 2, 2015.
[4] S. Ravi and D. P. Mankame, “Multimodal biometric approach using
fingerprint, face and enhanced iris features recognition,” in 2013 Inter-
national Conference on Circuits, Power and Computing Technologies
(ICCPCT). IEEE, 2013, pp. 1143–1150.
Fig. 6. Confusion Matrix - Face Verification Module [5] D. Menotti, G. Chiachia, A. Pinto, W. R. Schwartz, H. Pedrini, A. X.
Falcao, and A. Rocha, “Deep representations for iris, face, and finger-
print spoofing detection,” IEEE Transactions on Information Forensics
and Security, vol. 10, no. 4, pp. 864–879, 2015.
verification modules give precision and recall of 0.966 and 0.95 [6] K. Cao and A. K. Jain, “Automated latent fingerprint recognition,” IEEE
respectively, the module had on average two false positives and 3 transactions on pattern analysis and machine intelligence, vol. 41, no. 4,
false negatives. pp. 788–800, 2018.
[7] J. Soldera, G. Schu, L. R. Schardosim, and E. T. Beltrao, “Facial
biometrics and applications,” IEEE Instrumentation & Measurement
Magazine, vol. 20, no. 2, pp. 4–30, 2017.
V. C ONCLUSION [8] C. G. Harris, M. Stephens et al., “A combined corner and edge detector.”
in Alvey vision conference, vol. 15, no. 50. Citeseer, 1988, pp. 10–5244.
In this paper, we have presented an e-Voting system. The system [9] S. Kumar, S. Nandury, and S. Raj, “An extended client server ar-
helps to overcome the drawbacks of the current EVM based voting chitecture in mobile environment,” International Journal of Computer
system. The system prevents malpractices such as a voter can vote Engineering and Applications, vol. 5, pp. 97–107, 02 2014.
only once and cannot log in again into his account once the vote is [10] T. Ko, “Multimodal biometric identification for large user population
cast and also checks the eligibility of voters. It is an easy process and using fingerprint, face and iris recognition,” in 34th Applied Imagery
also maintains the integrity of the vote. It helps the voter to vote from and Pattern Recognition Workshop (AIPR’05), 2005, pp. 6 pp.–223.
anywhere provided that the user is a registered voter. Our system con- [11] V. Kazemi and J. Sullivan, “One millisecond face alignment with an
ensemble of regression trees,” in Proceedings of the IEEE conference
sists of voter-id verification along with 2-step biometric verification. on computer vision and pattern recognition, 2014, pp. 1867–1874.
Thus it ensures that better accuracy, security, and performance can be [12] A. Bansal, C. Castillo, R. Ranjan, and R. Chellappa, “The do’s and
achieved by using a two-step biometric authentication process which don’ts for cnn-based face verification,” in The IEEE International
includes face and fingerprint verification. This two-step authentication Conference on Computer Vision (ICCV) Workshops, Oct 2017.
also reduces the risks of electoral frauds and ballot-rigging. The said [13] J. P. Jose, P. Poornima, and K. M. Kumar, “A novel method for color
system provides Real-Time Authentication (RTA), which collects live face recognition using knn classifier,” in 2012 International Conference
samples of the biological characteristics of the individual. Hence, it on Computing, Communication and Applications, 2012, pp. 1–3.
discourages the act of bypassing security barriers by providing stolen [14] S. A. Perera, A Comparison of SIFT , SURF and ORB., Jan.
templates of an individual’s biological characteristics. 17, 2020. [Online]. Available: https://medium.com/@shehan.a.perera/a-
comparison-of-sift-surf-and-orb-333d64bcaaea
The performance and evaluation of the face verification module [15] D. Tyagi, Introduction to FAST (Features from Ac-
and fingerprint verification module are done on the Georgia Tech celerated Segment Test),., Nov. 26, 2019. [Online].
Face dataset and FVC 2006 dataset. Our system gives a good accuracy Available: https://medium.com/data-breach/introduction-to-fast-features-
score on the tested datasets which ensures that our system is secure. from-accelerated-segment-test-4ed33dde6d65
Further improvements can be made for the face verification module [16] S. Khedkar, S. Thube, W. Estate, C. Naka et al., “Real time databases
to improve its accuracy for input faces with similar feature vectors. for applications,” International Research Journal of Engineering and
Technology (IRJET), vol. 4, no. 06, pp. 2078–2082, 2017.
This system can be extended by making use of 3D image recog- [17] L. Gupta, Java AES 256 Encryption Decryption Example., Dec. 1,
nition to firmly identify the person in the frame of a webcam to 2019. [Online]. Available: https://howtodoinjava.com/security/aes-256-
nullify the chances of fake verification of facial patterns. Also, the encryption-decryption/
authenticity of fingerprint verification can be improved by integrating [18] A. Chauhan, Difference between AES and DES ciphers., Nov. 17, 2019.
liveliness detection that inspects the finger characteristics to ensure [Online]. Available: https://www.geeksforgeeks.org/difference-between-
whether the input finger is live or artificial. Furthermore, we can add aes-and-des-ciphers/
features of voice-assistant for differently-abled people so that they
can cast their votes without any hurdle. The said system can be used
on a mass level to conduct elections by adding more features like
processing power to handle more requests from any location and to
vanish the system failures.

64

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 16,2021 at 19:41:33 UTC from IEEE Xplore. Restrictions apply.