Course Title ECE3003

Microcontrollers and its Applications

(E2 Slot)

Attendance Through Face Recognition

Team Members –

D.Sai Goutham Reddy – 17BEC0585

Ashwan Srivathsava - 17BEC0478
Table of Contents

ABSTRACT CHAPTER 1
1.1 AIM
1.2 OBJECTIVE
1.3 INTRODUCTION

CHAPTER2
2.1 RELATED WORKS OR LITERATURE SURVEY
2.2 EXISTING AND PROPOSED SYSTEM.

CHAPTER 3
3.1 PROPOSED SYSTEM DESIGN ARCHITECTURE
3.2 ARCHITECTURE EXPLANATION
3.3 ALORITHMS AND PSEUDOCODE

CHAPTER 4
4.1 RESULTS AND DISCUSSION
4.3 CONCLUSION AND FUTURE WORK

REFERNCES
Abstract:
The project involves the use of machine learning to build a
Automatic Attendance System through Face Recognition. The System will
be capable of recognizing faces and taking the attendance Automatically
and can update it to any database that we aim to send. The entire project
is developed on a raspberry pi, along with a camera module. The interface
is totally wireless as the user can turn the system on and off with the VNC
android app. The system is capable of recognizing multiple users, it
captures a video feed from the camera module and then processes the
image frames using opencv to find out any strangers. Once it has
detected any user it automatically takes writes the user’s name to a file
and can automatically update once the user wish to update the database
even from the mobile app.

Chapter 1
1.1 Aim
The aim of Automatic Attendance system is to record the
attendance without any human interference and update it directly
to any given database.
1.2 Objective
Objectives of the Project are:
1) The Faculty/ User can start recording attendance via face recognition
from any given time.
2) To make it convinent for students to get attendance hassle free
3) To make it easier for faculty as it automatically updates the attendance
whenever he wants without any manual work.
4) We can even moniter from anywhere. This helps faculty to take
attendance from anywhere even from their mobile phones via VNC app.
1.3 Introduction
We have chosen this project because it provides the faculty with total
wireless control of the Attendance System. The current attendance system
(Fingerprint System) implemented require the faculty to be in the class for
taking attendance also some students escape after giving attendance
when faculty isn’t noticing. The best part of our project is that it uses
minimal hardware because it can implemented with minimal cost .The
basic Hardware consists of only a raspberry pi and a camera module. The
installation will require minimum cost and also the System can be
triggered from anywhere in the world. The user can simply log into our
App and then start taking attendance from anywhere on earth thanks to
the VNC cloud connection. In this project we simply took a raspberry pi
and a camera module attached to it and developed a simple machine
 learning model to identify Studnts of that given class. The camera module
captures a video feed of the class which is then analyzed by the raspberry
pi using OpenCV library. As soon as a Student is detected, It automatically
recognises him with images in the database and records it in a file and
then updates when ever faculty wants to. The faculty can then open the
VNC Viewer App to trigger the process even when they are not here in the
class. To control the system we used the VNC Server and Viewer to enable
total wireless control of the system, to interface with the system and also
to control execution of the system. The best application of the system can
be implemented in Classrooms of VIT itself. With a system such as
fingerprint is implimenting everywhere, we can reduce the cost of the
hardware and increase the convinience for the students as they have to
stand in the queue for a long time to give attendance or even to go out/
Come into the campus. As this system requires minimum hardware and
along with the features implemented will give a better user experience to
the faculty and students.

Chapter -2
2.1 Related Works or Literature Survey
As the more number of devices connect to internet, security Concers rises
a lot more. The best part of the system is implimented locally on a
raaspberry pi. This can avoid the privacy concerns of what the hackers can
do with the data. This system can be compramised if this is connected to
internet but fortunately we figured out a way to make it completly offline
and can update it automatically on the database or can be set according
to our need.

Nowadays most the devices becoming smarter, We can update the way we
autheticate a user to increase convience to everyone. This can help users
to locally control the things like letting the user in/ out in very less time
accurately. This can also detect if the user’s behaviour is anomalous and
can help the security to just check that individual alone.

2.2 Existing and Proposed System

The proposed solution takes the motion detection to trigger the program
on Raspberry Pi starts capturing video from cameras for updating the list
of users detected. This makes the system more efficient than the
fingerprint system. Also as we have security conserns, So we made the
algorithm to run locally on the system. This make the system not only safe
but also easy to program for the purpose. The current systems captures
image and sends it to a cloud server to process it and recognise the faces
which might raise security conserns in users.
Chapter 3

3.1 Hardware Requirements

The hardware required to build this project includesRaspberry Pi - ₹2,599
Logitech 720p Webcam - ₹ 1,099

3.2 Software Requirements

The software Components needed to build this project wereVNC
Server- installed on the raspberry pi, it sends pixel by pixel data of
the pi desktop to the VNC Viewer app. VNC Viewer- installed on the
user’s mobile device and also for interfacing with the software. The
VNC Viewer appreceives the pixel data from the VNC server and
gives the user a medium to interface with the Raspberry pi
wirelessly.

3.3 Interface Used:

VNC Cloud Server – To interact/interface with the raspberry pi, we
used the VNC server application which was installed on the
raspberry pi. The desktop pixel information was transferred from
the VNC server to the VNC Viewer and any interactions on the VNC
Viewer were transferred back to the Raspberry pi all using cloud
servers.
Camera Module – the camera module acts an interface between the
user and the system. The camera module monitors the house and
the video feed from it is used by the system to detect suspicious
activity. The user can view the video feed of his house via the
camera module only. Although the information is transmitted by the
VNC server but still it is captured by the camera and then passed on
the user.
3.4 Internal Pin Diagram
 The GPIO pins used in the project along with the ports utilized in the
schematic given above. The GPIO Pin architecture of the Rpi model
B+-

3.5: Explanation of the project

The Project is triggered if either the User with an app in his phone
and access for that or through motion detection through PIR. This
then starts capturing the video and it starts opencv to detect the
faces in the video and if it captures the faces that we provided
previously, thrnit writes that name to a file and after a certain time
we can configure to update the names in the database specified.
This process can be intiated by vnc app also. So its convinient to all.

Project CodeThe main module running at all times-

(detector.py)
import face_recognition
import cv2
import numpy as np
import time
from datetime import datetime
now = datetime.now()
f=open("Attendance.txt",'a+')
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

video_capture = cv2.VideoCapture(0)
goutham_image = face_recognition.load_image_file("goutham.jpg")
goutham_face_encoding = face_recognition.face_encodings(goutham_image)[0]

known_face_encodings = [
goutham_face_encoding
]
known_face_names = [
"Goutham"
]

# Initialize some variables

face_locations = []
face_encodings = []
face_names = []
process_this_frame = True
flag=0
while True:
# Grab a single frame of video
 ret, frame = video_capture.read()

# Resize frame of video to 1/4 size for faster face recognition processing
small_frame = cv2.resize(frame, (0, 0), fx=0.2, fy=0.2)

# Convert the image from BGR color (which OpenCV uses) to RGB color (which
face_recognition uses)
rgb_small_frame = small_frame[:, :, ::-1]

# Only process every other frame of video to save time

if process_this_frame:
# Find all the faces and face encodings in the current frame of video
face_locations = face_recognition.face_locations(rgb_small_frame)
face_encodings = face_recognition.face_encodings(rgb_small_frame,
face_locations)

face_names = []
for face_encoding in face_encodings:
# See if the face is a match for the known face(s)
matches = face_recognition.compare_faces(known_face_encodings,
face_encoding)
name = "Unknown"

# # If a match was found in known_face_encodings, just use the first one.

# if True in matches:
# first_match_index = matches.index(True)
# name = known_face_names[first_match_index]

# Or instead, use the known face with the smallest distance to the new
face
face_distances = face_recognition.face_distance(known_face_encodings,
face_encoding)
best_match_index = np.argmin(face_distances)
if matches[best_match_index]:
name = known_face_names[best_match_index]
tim = now.strftime("%H:%M:%S")
f.write(name+tim)
f.close()
flag=1
face_names.append(name)
if flag==1:
break
process_this_frame = not process_this_frame

# Display the results

for (top, right, bottom, left), name in zip(face_locations, face_names):
# Scale back up face locations since the frame we detected in was scaled to
1/4 size
 top *= 4
right *= 4
bottom *= 4
left *= 4

# Draw a box around the face

cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

# Draw a label with a name below the face

cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255),
cv2.FILLED)
font = cv2.FONT_HERSHEY_DUPLEX
cv2.putText(frame,name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

# Display the resulting image

cv2.imshow('Video', frame)

# Hit 'q' on the keyboard to quit!

if cv2.waitKey(1) & 0xFF == ord('q'):
break

# Release handle to the webcam

video_capture.release()
cv2.destroyAllWindows()

(Model.py)

from keras.layers import Conv2D, ZeroPadding2D, Activation, Input, concatenate

from keras.layers.core import Lambda, Flatten, Dense
from keras.layers.normalization import BatchNormalization
from keras.layers.pooling import MaxPooling2D, AveragePooling2D
from keras.models import Model
from keras import backend as K

import utils
from utils import LRN2D

def create_model():
myInput = Input(shape=(96, 96, 3))

x = ZeroPadding2D(padding=(3, 3), input_shape=(96, 96, 3))(myInput)

x = Conv2D(64, (7, 7), strides=(2, 2), name='conv1')(x)
x = BatchNormalization(axis=3, epsilon=0.00001, name='bn1')(x)
x = Activation('relu')(x)
x = ZeroPadding2D(padding=(1, 1))(x)
x = MaxPooling2D(pool_size=3, strides=2)(x)
x = Lambda(LRN2D, name='lrn_1')(x)
x = Conv2D(64, (1, 1), name='conv2')(x)
 x = BatchNormalization(axis=3, epsilon=0.00001, name='bn2')(x)
x = Activation('relu')(x)
x = ZeroPadding2D(padding=(1, 1))(x)
x = Conv2D(192, (3, 3), name='conv3')(x)
x = BatchNormalization(axis=3, epsilon=0.00001, name='bn3')(x)
x = Activation('relu')(x)
x = Lambda(LRN2D, name='lrn_2')(x)
x = ZeroPadding2D(padding=(1, 1))(x)
x = MaxPooling2D(pool_size=3, strides=2)(x)

# Inception3a
inception_3a_3x3 = Conv2D(96, (1, 1), name='inception_3a_3x3_conv1')(x)
inception_3a_3x3 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3a_3x3_bn1')(inception_3a_3x3)
inception_3a_3x3 = Activation('relu')(inception_3a_3x3)
inception_3a_3x3 = ZeroPadding2D(padding=(1, 1))(inception_3a_3x3)
inception_3a_3x3 = Conv2D(128, (3, 3), name='inception_3a_3x3_conv2')
(inception_3a_3x3)
inception_3a_3x3 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3a_3x3_bn2')(inception_3a_3x3)
inception_3a_3x3 = Activation('relu')(inception_3a_3x3)

inception_3a_5x5 = Conv2D(16, (1, 1), name='inception_3a_5x5_conv1')(x)

inception_3a_5x5 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3a_5x5_bn1')(inception_3a_5x5)
inception_3a_5x5 = Activation('relu')(inception_3a_5x5)
inception_3a_5x5 = ZeroPadding2D(padding=(2, 2))(inception_3a_5x5)
inception_3a_5x5 = Conv2D(32, (5, 5), name='inception_3a_5x5_conv2')
(inception_3a_5x5)
inception_3a_5x5 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3a_5x5_bn2')(inception_3a_5x5)
inception_3a_5x5 = Activation('relu')(inception_3a_5x5)

inception_3a_pool = MaxPooling2D(pool_size=3, strides=2)(x)

inception_3a_pool = Conv2D(32, (1, 1), name='inception_3a_pool_conv')
(inception_3a_pool)
inception_3a_pool = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3a_pool_bn')(inception_3a_pool)
inception_3a_pool = Activation('relu')(inception_3a_pool)
inception_3a_pool = ZeroPadding2D(padding=((3, 4), (3, 4)))(inception_3a_pool)

inception_3a_1x1 = Conv2D(64, (1, 1), name='inception_3a_1x1_conv')(x)

inception_3a_1x1 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3a_1x1_bn')(inception_3a_1x1)
inception_3a_1x1 = Activation('relu')(inception_3a_1x1)

inception_3a = concatenate([inception_3a_3x3, inception_3a_5x5, inception_3a_pool,

inception_3a_1x1], axis=3)

# Inception3b
 inception_3b_3x3 = Conv2D(96, (1, 1), name='inception_3b_3x3_conv1')
(inception_3a)
inception_3b_3x3 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3b_3x3_bn1')(inception_3b_3x3)
inception_3b_3x3 = Activation('relu')(inception_3b_3x3)
inception_3b_3x3 = ZeroPadding2D(padding=(1, 1))(inception_3b_3x3)
inception_3b_3x3 = Conv2D(128, (3, 3), name='inception_3b_3x3_conv2')
(inception_3b_3x3)
inception_3b_3x3 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3b_3x3_bn2')(inception_3b_3x3)
inception_3b_3x3 = Activation('relu')(inception_3b_3x3)

inception_3b_5x5 = Conv2D(32, (1, 1), name='inception_3b_5x5_conv1')

(inception_3a)
inception_3b_5x5 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3b_5x5_bn1')(inception_3b_5x5)
inception_3b_5x5 = Activation('relu')(inception_3b_5x5)
inception_3b_5x5 = ZeroPadding2D(padding=(2, 2))(inception_3b_5x5)
inception_3b_5x5 = Conv2D(64, (5, 5), name='inception_3b_5x5_conv2')
(inception_3b_5x5)
inception_3b_5x5 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3b_5x5_bn2')(inception_3b_5x5)
inception_3b_5x5 = Activation('relu')(inception_3b_5x5)

inception_3b_pool = AveragePooling2D(pool_size=(3, 3), strides=(3, 3))

(inception_3a)
inception_3b_pool = Conv2D(64, (1, 1), name='inception_3b_pool_conv')
(inception_3b_pool)
inception_3b_pool = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3b_pool_bn')(inception_3b_pool)
inception_3b_pool = Activation('relu')(inception_3b_pool)
inception_3b_pool = ZeroPadding2D(padding=(4, 4))(inception_3b_pool)

inception_3b_1x1 = Conv2D(64, (1, 1), name='inception_3b_1x1_conv')(inception_3a)

inception_3b_1x1 = BatchNormalization(axis=3, epsilon=0.00001,
name='inception_3b_1x1_bn')(inception_3b_1x1)
inception_3b_1x1 = Activation('relu')(inception_3b_1x1)

inception_3b = concatenate([inception_3b_3x3, inception_3b_5x5, inception_3b_pool,

inception_3b_1x1], axis=3)

# Inception3c
inception_3c_3x3 = utils.conv2d_bn(inception_3b,
layer='inception_3c_3x3',
cv1_out=128,
cv1_filter=(1, 1),
cv2_out=256,
cv2_filter=(3, 3),
cv2_strides=(2, 2),
padding=(1, 1))
 inception_3c_5x5 = utils.conv2d_bn(inception_3b,
layer='inception_3c_5x5',
cv1_out=32,
cv1_filter=(1, 1),
cv2_out=64,
cv2_filter=(5, 5),
cv2_strides=(2, 2),
padding=(2, 2))

inception_3c_pool = MaxPooling2D(pool_size=3, strides=2)(inception_3b)

inception_3c_pool = ZeroPadding2D(padding=((0, 1), (0, 1)))(inception_3c_pool)

inception_3c = concatenate([inception_3c_3x3, inception_3c_5x5, inception_3c_pool],

axis=3)

#inception 4a
inception_4a_3x3 = utils.conv2d_bn(inception_3c,
layer='inception_4a_3x3',
cv1_out=96,
cv1_filter=(1, 1),
cv2_out=192,
cv2_filter=(3, 3),
cv2_strides=(1, 1),
padding=(1, 1))
inception_4a_5x5 = utils.conv2d_bn(inception_3c,
layer='inception_4a_5x5',
cv1_out=32,
cv1_filter=(1, 1),
cv2_out=64,
cv2_filter=(5, 5),
cv2_strides=(1, 1),
padding=(2, 2))

inception_4a_pool = AveragePooling2D(pool_size=(3, 3), strides=(3, 3))(inception_3c)

inception_4a_pool = utils.conv2d_bn(inception_4a_pool,
layer='inception_4a_pool',
cv1_out=128,
cv1_filter=(1, 1),
padding=(2, 2))
inception_4a_1x1 = utils.conv2d_bn(inception_3c,
layer='inception_4a_1x1',
cv1_out=256,
cv1_filter=(1, 1))
inception_4a = concatenate([inception_4a_3x3, inception_4a_5x5, inception_4a_pool,
inception_4a_1x1], axis=3)

#inception4e
inception_4e_3x3 = utils.conv2d_bn(inception_4a,
layer='inception_4e_3x3',
 cv1_out=160,
cv1_filter=(1, 1),
cv2_out=256,
cv2_filter=(3, 3),
cv2_strides=(2, 2),
padding=(1, 1))
inception_4e_5x5 = utils.conv2d_bn(inception_4a,
layer='inception_4e_5x5',
cv1_out=64,
cv1_filter=(1, 1),
cv2_out=128,
cv2_filter=(5, 5),
cv2_strides=(2, 2),
padding=(2, 2))
inception_4e_pool = MaxPooling2D(pool_size=3, strides=2)(inception_4a)
inception_4e_pool = ZeroPadding2D(padding=((0, 1), (0, 1)))(inception_4e_pool)

inception_4e = concatenate([inception_4e_3x3, inception_4e_5x5,

inception_4e_pool], axis=3)

#inception5a
inception_5a_3x3 = utils.conv2d_bn(inception_4e,
layer='inception_5a_3x3',
cv1_out=96,
cv1_filter=(1, 1),
cv2_out=384,
cv2_filter=(3, 3),
cv2_strides=(1, 1),
padding=(1, 1))

inception_5a_pool = AveragePooling2D(pool_size=(3, 3), strides=(3, 3))(inception_4e)

inception_5a_pool = utils.conv2d_bn(inception_5a_pool,
layer='inception_5a_pool',
cv1_out=96,
cv1_filter=(1, 1),
padding=(1, 1))
inception_5a_1x1 = utils.conv2d_bn(inception_4e,
layer='inception_5a_1x1',
cv1_out=256,
cv1_filter=(1, 1))

inception_5a = concatenate([inception_5a_3x3, inception_5a_pool,

inception_5a_1x1], axis=3)

#inception_5b
inception_5b_3x3 = utils.conv2d_bn(inception_5a,
layer='inception_5b_3x3',
cv1_out=96,
cv1_filter=(1, 1),
cv2_out=384,
 cv2_filter=(3, 3),
cv2_strides=(1, 1),
padding=(1, 1))
inception_5b_pool = MaxPooling2D(pool_size=3, strides=2)(inception_5a)
inception_5b_pool = utils.conv2d_bn(inception_5b_pool,
layer='inception_5b_pool',
cv1_out=96,
cv1_filter=(1, 1))
inception_5b_pool = ZeroPadding2D(padding=(1, 1))(inception_5b_pool)

inception_5b_1x1 = utils.conv2d_bn(inception_5a,
layer='inception_5b_1x1',
cv1_out=256,
cv1_filter=(1, 1))
inception_5b = concatenate([inception_5b_3x3, inception_5b_pool,
inception_5b_1x1], axis=3)

av_pool = AveragePooling2D(pool_size=(3, 3), strides=(1, 1))(inception_5b)

reshape_layer = Flatten()(av_pool)
dense_layer = Dense(128, name='dense_layer')(reshape_layer)
norm_layer = Lambda(lambda x: K.l2_normalize(x, axis=1), name='norm_layer')
(dense_layer)

return Model(inputs=[myInput], outputs=norm_layer)

4.1 RESULTS AND DISCUSSION

The project designed and implemented a face recognition system
based on the Raspberry Pi. The aspects of the system are: motion
detection using a PIR sensor, video capturing using a Pi Camera . It
however achieved tat the cost of less accuracy than that we aimed
to get through image processing in the Raspberry Pi because of
system constraints i.e. processor speed and Memory.
4.2 Conclusion & Future Work
We aim to work on these following things to improve this project for
it’s future develoupment:
• Improving the memory optimisation of the Algorithms.
• Trying to improve the On-Chip capabilities of the Algorithm.
• Reducing the cost of the processor required
• Improving the features like More Security and making less
vulnarable.
• Improving Speed of the face detection and recognition.
Ultimately we aim to make this system more useful to everyone
by improving the convinience of the system.
REFERENCES:
[1] P. Teh, H. Ling and S. Cheong, “NFC smartphone based access
control system using information hiding”, 2013 IEEE Conference on
OpenSystems (ICOS), 2013.
[2] K. Bromley, M. Perry, and G. Webb, “Trends in smart home systems,
connectivity and services”, www.nextwave.org.uk, 2003.
[3] Danish Chowdhry, Raman Paranjape, Paul Laforge Smart Home
Automation System for
Intrusion detection system Faculty of Engineering and Applied
ScienceUniversity of Regina
Regina, Canada{ali273,raman.paranjapepaul.laforge}@uregina.ca20 15
IEEE 14th Canadian
Workshop on Information Theory (CWIT)
[4] R. Newman, “Security and Access Control Using Biometric
Technologies: Application, Technology, and Management” (1st ed.),
Course Technology Press, Boston,MA, United States. 2009.
[5] Huu-Quoc Nguyen, Ton Thi Kim Loan, Bui DinhMao and Eui-Nam
Huh, “Low cost realtime systemmonitoring using Raspberry Pi”, 2015
Seventh InternationalConference on Ubiquitous and Future Networks,
2015.
[6] IoT Based Smart Security and Home Automationsystem Ravi Kishore
Kodali, Vishal Jain,
Suvadeep Bose and Lakshmi Boppana Department of Electronics and
Communications
Engineering National Institute of Technology, Warangal International
Conference on
Computing, Communication and Automation (ICCCA2016)