See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/384642312

Smart Surveillance Camera Using AI

Conference Paper · July 2024

DOI: 10.1109/ICAIT61638.2024.10690287

CITATION READS

1 450

6 authors, including:

Dr Manu Y M Shashikala S V
BGS INSTITUTE OF TECHNOLOGY, ADICHUNCHANAGIRI UNIVERSITY BGS Institute of Technology, Adichunchanagiri university
58 PUBLICATIONS 373 CITATIONS 34 PUBLICATIONS 91 CITATIONS

SEE PROFILE SEE PROFILE

Dheeraj a P
BGSIT
1 PUBLICATION 1 CITATION

SEE PROFILE

All content following this page was uploaded by Dr Manu Y M on 08 October 2024.

The user has requested enhancement of the downloaded file.

 2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

Smart Surveillance Camera Using AI

Manu Y M Shashikala S V Darshan N
Computer Science and Engineering Computer Science and Engineering Computer Science and Engineering
BGS Institute of Technology BGS Institute of Technology BGS Institute of Technology
BG Nagar, Karnataka BG Nagar, Karnataka BG Nagar, Karnataka
[email protected] [email protected] [email protected]
2024 Second International Conference on Advances in Information Technology (ICAIT) | 979-8-3503-8386-7/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICAIT61638.2024.10690287

Dheeraj A P Hemanth N J Nishanth Gowda B

Computer Science and Engineering Computer Science and Engineering, Computer Science and Engineering
BGS Institute of Technology BGS Institute of Technology BGS Institute of Technology
BG Nagar, Karnataka BG Nagar, Karnataka BG Nagar, Karnataka
[email protected] [email protected] [email protected]

Abstract—The Python script recognizes faces in real-time technology develops further, helping to create safer and
webcam video by using computer vision algorithms. For face more secure surroundings in a variety of settings, from
detection, facial feature encoding, and face comparison with public spaces to private facilities. The emphasis is on
known faces, it makes use of the Open CV and face quickly and automatically identifying and documenting
recognition packages. When the script detects a familiar face, unknown individuals in addition to recognized individuals.
it adds the name and the current time to a CSV file, indicating
the person's attendance. In a similar manner, it records This project aims to redefine surveillance technology
attendance for unidentified faces as "unknown." It also takes by leveraging the Dlib library, state-of-the-art face
pictures of faces you don't recognize and saves them for later recognition algorithms, and the flexibility of the Python
review or identification. The program repeatedly scans the programming language. The project's lens is the camera,
video stream for faces, identifying them and cross- which records live video feeds for accurate identification
referencing them with a database of recognized faces. After a and analyzes them in real time. The main goal of the project
brief pause, if an unknown face is identified, a second face is to smoothly incorporate AI into surveillance, bridging
recognition procedure is started to either validate the identity the gap between security and technology to develop a
or label the person as unknown. The script shows the promise system that is proactive rather than reactive. The Smart
of computer vision techniques in a variety of applications, Surveillance Camera Using AI project essentially imagines
such as security, identity systems, and attendance monitoring. a future in which security personnel's abilities areleveraged
It offers a straightforward but efficient approach for real- by an intelligent, active technology, replacing the
time face recognition and attendance tracking.
traditional practice of passive observation in surveillance.
Keywords—Facial recognition, Anomaly detection,
Facial recognition, categorization, and database
Artificial Intelligence powered surveillance. management are designed to completely reshape the
surveillance landscape and offer a reliable answer to the
I. INTRODUCTION constantly changing security concerns in our ever-
changing world.
Artificial intelligence (AI) is being incorporated into
more and more aspects of our lives in this age of rapid Artificial intelligence, especially through advanced m
technological growth. The Smart Surveillance Camera achine learning algorithms and computer vision, has creat
Using AI project is an innovative endeavor that aims to ed a new era in video surveillance. By analyzing large am
transform conventional surveillance systems. The project's ounts of video data in real time, AIpowered systems can i
main goal is to improve security by installing a dentify patterns and anomalies that may escape human ey
sophisticated camera system that can use facial recognition es.
to intelligently identify and categorize people. Theproject's
main goal is to overcome the shortcomings of traditional This project is motivated by the need for more
surveillance systems, which frequently rely on manual proactive and effective security solutions in a world that is
monitoring and are incapable of subtle detection. Our becoming more and more complicated. Real-time analysis
system aims to improve the surveillance capabilities, and quick reaction to threats are two areas where
especially in dynamic contexts; by utilizing AI.Strict data traditional surveillance systems frequently fall short. Our
access limits and anonymization are two aspects that solve goals in incorporating AI are to strengthen security
privacy issues. Smart surveillance cameras are set to usher protocols, overcome the drawbacks of traditional
in a new era of intelligent and efficient surveillance as approaches, and accelerate reaction times. Significant

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Mysuru Royal Institute of Technology. Downloaded on October 07,2024 at 06:14:25 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)
classification. Capturing of the Frames of the unknown
benefits, including automatic threat identification, a
decrease in human error, and actionable insights, come individual and storing them in the database is the final step.
from AI's capacity to swiftly and effectively evaluate Fig 1 shows the suggested model architecture.
enormous volumes of data. The goal of this project is to
use technology to develop a more intelligent and efficient
monitoring system, which will ultimately lead to safer
surroundings and improved decision-making.
The novelty to this concept is unique because it
incorporates artificial intelligence (AI) into the security
camera system to provide real-time event detection and
analysis. Advanced anomaly detection, automatic picture
creation and analysis, and improved facial recognition are
some of the innovative characteristics that are most
notable. These advancements allow the system to rapidly
record and analyze crucial times, proactively identifying
and responding to dangers. By turning traditional passive
surveillance into a more intelligent and effective security
solution, AI-driven data collecting produces actionable
insights for improved decision-making.
II. RELATED WORKS Fig. 1. Proposed Model Architecture.

Well-known researchers like R. Chellappa, C.L.Wilson The artificial intelligence (AI) smart security camera
and C. Sirohey, have made great contributions to the field uses a complex architectural framework that combines
of facial recognition technology [1]. Their research has real-time processing power, deep learning models, and
focused on a variety of topics, including algorithms, computer vision algorithms. Convolutional neural
feature extraction techniques, and matching methods, networks (CNNs) are the system's primary means of object
which has improved the accuracy and efficiency of detection and recognition. This allows the system to
recognizing people from images or video streams. The recognize a wide range of items, faces, and activities that
OpenCV library from Willow Garage, Itseez, and Intel are inside its field of vision. The CNN analyzes the video
Corporation provides a variety of functions [2], including frames that the camera continually records. The input data
facial recognition and detection, making this technology for the neural network is optimized by pre-processing
available to developers. methods like image resizing and normalization. The CNN
is able to distinguish between people, and other entities
Facial recognition jobs are made easier with Adam
because it has been trained on a wide range of datasets,
Geitgey's face recognition package, which is based on
which allows it to extract pertinent properties and classify
OpenCV and dlib and offers a high-level
persons [8]. All things considered, the smart security
interface[3].Facial recognition has been the subject of
camera makes use of AI-driven features to offer thorough
investigation for a number of researchers and developers;
monitoring, analysis, and response in a variety of security
as evidenced by the work of M. Taj-Eddin, M. Abaza, and
scenarios.
M. Daoud[4], as well as Woodrow Hartzog and Evan
Selinger, it provides automated and effective attendance A. Data Collection and Preparation:
tracking while addressing issues like real-time processing The code makes the assumption that there is a directory
and privacy concerns. Additionally, research by Chunbiao called "images Basic" with pictures of known individuals
Zhu, Jinjie Gu, and Zhijun Wang [5] has highlighted the in cases when tracking down the unknown is necessary.
advantages of these systems in terms of accuracy, Every picture should feature the face of a single person.
efficiency, and data analytic capabilities, as well as their These photos are read by the code, which then transforms
efficacy and user acceptability [6]. them into numpy arrays for analysis [9]. Fig.2 Shows
Future directions for facial recognition technology and Directory, imageBasic Stored with Known Faces.
attendance systems include deep learning-based methods,
multimodal biometrics integration, and context-aware
applications, according to experts like Alex Graves, Jürgen
Schmidhuber [7], and Geoffrey Hinton. By improving
recognition accuracy, robustness, and adaptation to various
contexts and use cases, these developments hope to pave
the path for future development and growth of face
recognition attendance systems to fulfill changing
demands and tackle new problems in the industry.
III. METHODOLOGY
An outline of the four main phases of the methodology
employed in the investigation is provided in this section,
with fig. 1. The first process involves collecting and
compiling data, while the second involves encoding faces. Fig. 2. Directory, imageBasic Stored with Known Faces.
The third phase involves Face recognition and

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Mysuru Royal Institute of Technology. Downloaded on October 07,2024 at 06:14:25 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

B. Encoding Faces:
In order to enable more reliable and precise recognition
under a variety of conditions and with variations in facial
expressions, this approach highlights the flexibility of
facial appearance modeling [10]. To simulate the
appearance of faces and account for variations in lighting,
position, and facial expression, this method makes use of
statistical techniques. The method produces pertinent
information from face photos using complex algorithms,
compares them with a database of recognized faces, and
allows for quick and accurate recognition. Fig.3 shows
scheme of face recognition

Fig. 4. Classification of Known and Unknown Faces

D. Snapshots of Unknown stored in user Database:

The camera takes a picture of the person's face when
the face recognition system recognizes someone it doesn't
know [12]. Preprocessing techniques, such as converting it
to a particular format (like JPEG), is performed before
storing the snapshot.
To store data about the snapshots, create a database
schema. Potential fields in this schema include:
Fig. 3. Face recognition scheme.
 Snapshot ID: Unique identifier for each snapshot.
C. Face Recognition and Classification:  Image Data: The actual image data or a reference
From the known face images, the find encodings to the image file.
function extracts facial encodings (128-dimensional
numerical representations) using the face recognition  Timestamp: Date and time when the snapshot was
library. Each face is represented uniquely by these captured.
encodings. Using a distance metric (e.g., Euclidean IV. ALGORITHM
distance or cosine similarity) to compare encodings [11]:

Step1: Import necessary libraries such as numpy, OpenCV

Distance=distance (Encode known , Encode unknown ) (cv2), face recognition, os , and date, time.
Step2: Set the path variable to the directory containing
images.
Score cosine (Oq, Oa) = OqxOa (1) Initialize empty lists images, classnames, and mylist.
|Oq||Oa| Read the list of files in the directory specified by path and
store it in mylist.
Step3: Iterate over each file (cl) in mylist.
Read the image using OpenCV's cv2.imread() function and
Score Euclidean (Oq, Oa) = 1 (2) store it in curimg.
1+ ||Oq-Oa||2 Append the image to the images list.
Extract the class name (filename without extension) and
If the distance is below a certain threshold, the face may be append it to the classnames list.
considered a match; otherwise [12], it's labeled as Step4: Define a function find encodings () that takes a list
'unknown'. Fig.4 tells about Classification of Known and of images as input.
Unknown faces. The process of encoding faces, both Step5: Convert the image to RGB format.
known and unknown, and comparing their encodings for Use face recognition, face encodings () to find the face
recognition is summed up in these equations. encodings.
Append the face encoding to the encodelist.
Step6: Define a function markattendance () that takes a
name as input.
Open the CSV file ('information.csv') in read-write mode.
Read existing data and store it in mydata list.

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Mysuru Royal Institute of Technology. Downloaded on October 07,2024 at 06:14:25 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

Extract existing names and store them in name list. The following formula can be used to calculate it; it shows
If the name is not present, append the current time along the harmonic mean of recall and precision [14].
with the name to the CSV file.
e i io e
Encode Known Faces: 1=2
Call find encodings() with the images list to get encodings e i io + e
for known faces.
Where,
Print 'encoding complete'.
 True Positives (TP): Correctly identified faces.
Step7: Initialize a video capture object cap using
 False Positives (FP): Incorrectly identified faces
cv2.VideoCapture (0).
(false alarms).
Step8: Continuously capture frames from the webcam.
 True Negatives (TN): Correctly rejected
Convert the frame to RGB format.
unknown faces.
Detectfaces in the frame using
 False Negatives (FN): Missed detections of
face_recognition.face_locations ().
known faces.
Encode the detected faces using face_recognition.
 Accuracy: (TP + TN) / (TP + TN + FP + FN)
face_encodings ().
Compare the face encoding with known encodings using
 e o i i e
face_recognition. compare_faces (). Precision =
e o i i e + e o i i e

If a match is found:
Get the corresponding class name.  e o i i e
Draw a rectangle around the face and display the name and Recall =
e o i i e + e eg i e
timestamp.
If no match is found: Fig. 5 (b) shows how Comparison of five facial
Mark as unknown person. landmark localization between faces Recognition and
Perform additional face recognition within a short time MTCNN on the DHA.
window (5 seconds): These measurements shed light on several facets of the
Capture frames. functionality of the face recognition system[15]. In order
Process frames to detect faces and encode them. to store pictures of unidentified people in the database, you
If a match is found, mark and display the name. would normally take the following actions:
If no match is found, mark as 'unknown'.
Save frames as images. Fig. 6 (a) demonstrates classification by smart
Display the processed frame with rectangles and text. surveillance camera using AI
Wait for a key press to exit the loop. Fig.6 (b) shows Measurements and landmarks on the
Step9: Release the webcam and close OpenCV windows. lateral and frontal views of the face.

V. RESULT AND SNAPSHOTS

A. Tranning:

Fig. 5 (a). Trained Accuracy Levels

Fig. 5 (b). Comparison of five facial landmark localization between
Certain measures, such as the recall rate and the S1 Amazon Recognition and MTCNN on the DHA.
rating, are acquired during training [13]. One statistic that's
frequently used in binary classification tasks to assess how
well memory and precision are balanced is the S1 score. When cv2.imwrite() detects an unknown face, take a
picture of it.
Fig. 5 (a) gives Trained Accuracy Levels for Training
data sets Put the snapshot in the specified database or location.
You have the option to add other metadata, such as
location or more context, or to timestamp the picture [16].

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Mysuru Royal Institute of Technology. Downloaded on October 07,2024 at 06:14:25 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

Verify that the storing technique complies with

applicable privacy standards and is safe.

Fig. 8. ROC for Unknown person detection

Fig. 6. classification by smart surveillance camera using AI

Fig. 9. shows the frame level comparison with ground
truth for face image acquired from our proposed model.
Fig 10. Gives Precision-recall curves of face detection
algorithms with accurate accuracy.

Fig.6 (b). Measurements and landmarks on the lateral and frontal views
of the face.

In order to uphold privacy and security standards, you

might also want to think about putting in place a system for
Fig. 9. Comparison for known and unknown face detection
monitoring and controlling these snapshots, such as a
recurring review and deletion process. Fig. 7gives
Snapshots of Unknown Person Stored In User Database.

Figure 10: Precision-recall curves of face detection algorithms with

accuracy.
Fig. 7. Snapshots of Unknown Person Stored In User Database.
The average of the proposed method is compared with the
existing algorithms in Table I where x-axis shows the
ROC curves calculated with the perfcurve function for Recal and y-axis shows the Precision.
a perfect classifier, a typical classifier, and a classifier that
does no better than a random guess.ROC curves are
presented in Fig 8.

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Mysuru Royal Institute of Technology. Downloaded on October 07,2024 at 06:14:25 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)
SalekoppaluVenkataramu. "An integrated multi-level feature
TABLE I. COMPARISION OF PROPOSED MTHOD WITH
EXISTING APPROACH. fusion framework for crowd behaviour prediction and analysis."
Indonesian Journal of Electrical Engineering and Computer
Science 30.3 (2023): 1369-1380.
[11] Ho-Man Tang, Michael Lyu, and Irwin King, "Face recognition
Sl. Method AUC (%) committee machine," In Proceedings of IEEE International
No. Conference on Acoustics, Speech, and Signal Processing (ICASSP
1. Lloyd [141] 94 2003), pp. 837-840, April 6-10, 2003.
[12] Rui Huang, Vladimir Pavlovic, and Dimitris N. Metaxas, "A
2. SFF [134] 96 hybrid face recognition method using Markov random fields,"
3. MDDT [138] 98 ICPR (3) , pp. 157-160, 2004.
4. Chaotic invariants [135] 99 [13] M. Y. M, R. G. K and S. S. V, "Crowd Anomaly Detection Using
5. Piswas [142] 98 Machine Learning Techniques," 2022 IEEE North Karnataka
Subsection Flagship International Conference (NKCon), Vijaypur,
6. Propose Method 99
India, 2022, pp. 1-6, doi: 10.1109/NKCon56289.2022.10127015.
[14] "Anomaly Detection in Videos Using Deep Learning Techniques
", International Journal of Emerging Technologies and Innovative
VI. CONCLUSION AND FUTURE ENHANCEMENT Research (www.jetir.org | UGC and issn Approved), ISSN:2349-
5162, Vol.8, Issue 6, page no. ppc582-c588, June-2021, Available
All things considered, the Smart Surveillance Camera at : http://www.jetir.org/papers/JETIR2106349.pdf
Using AI project represents a noteworthy breakthrough in [15] Manu, YadakereMurthygowda, GuralamataKrishnegowda
the application of intelligent technology to security Ravikumar, and SalekoppaluVenkataramu Shashikala. "Análisis y
measures. The project's main objective, which was to predicción del comportamiento de las multitudes medianteelmarco
recognize faces and classify them, has been effectively de fusión de características." (2022).
[16] J. Weng, J.S. Huang, and N. Ahuja, “Learning recognition and
accomplished. The system is capable of strong facial segmentation of 3D objects from 2D images,” Proc. IEEE Int'l
recognition thanks to the use of state-of-the-art software Conf. Computer Vision, pp. 121-128, 1993
such as Dlib and Cmake and the flexibility of Python as the [17] Y M, M., G K, R. Abnormalities detection and tracking of crowd
programming language. Its capacity to swiftly categorize using kinematics approach in a video stream. Multimed ToolsAppl
(2024). https://doi.org/10.1007/s11042-024-19146-3
unknown individuals in addition to quickly identifying
known ones is what makes this study innovative. In order
to provide a complete security approach, the system takes
a picture of an unknown individual and saves it in the
database when it is identified.
This research is a first step toward using AI for real-
world security applications as technology develops. The
Smart Surveillance Camera Using AI is positioned as a
useful tool in the field of surveillance systems due to its
straightforward design and efficient functionality. This
project essentially represents the smooth coexistence of
hardware and software, which work together to improve
security procedures in an easy-to-understand and effective
way.

REFERENCES
[1] R. Chellappa, C.L. Wilson and C. Sirohey, ”Humain and machine
recognition of faces: A survey,” Proc. IEEE, vol. 83, no. 5, pp. 705-
740, may 1995.
[2] T. Fromherz, P. Stucki, M. Bichsel, “A survey of face recognition,”
MML Technical Report, No 97.01, Dept. of Computer Science,
University of Zurich, Zurich, 1997.
[3] Nefian and M. Hayes, ” An embedded hmm-based approach for
face detection and recognition,” In Proc. IEEE International
Conference on Acoustics, Speech, and Signal Processing, vol. 6,
pp. 3553–3556, 1999.
[4] M. Y. M, R. G. K and S. S. V, "Anomaly Alert System using CCTV
surveillance," 2022 IEEE 2nd Mysore Sub Section International
Conference (MysuruCon), Mysuru, India, 2022, pp. 1-5, doi:
10.1109/MysuruCon55714.2022.9972363.
[5] Dankan Gowda, V., et al. "IOT Based Smart Health Care System
to Monitor Covid-19 Patients." International Journal of Electrical
and Electronics Research (2022): 36-40.
[6] U.S. Department of Defense, ”Facial Recognition Vendor Test,
2000,” Available:
http://www.dodcounterdrug.com/facialrecognition/FRVT2000/frv
t2000.htm.
[7] G.D. Guo, H.J. Zhang, S.Z. Li. "Pairwise face recognition". In
Proceedings of 8th IEEE International Conference on Computer
Vision. Vancouver, Canada. July 9-12, 2001.
[8] Manu, Y. M., et al. "APPROACHES FOR QUALITY-AWARE
VIDEO CONTENT ANALYTICS." (2019).
[9] O. Deniz, M. Castrillon, M. Hernandez, “Face recognition using
independent component analysis and support vector machines,”
Pattern Recognition Letters, vol. 24, pp. 2153-2157, 2003.
[10] Murthygowda, Manu Yadakere, Ravikumar
GuralamataKrishnegowda, and Shashikala

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Mysuru Royal Institute of Technology. Downloaded on October 07,2024 at 06:14:25 UTC from IEEE Xplore. Restrictions apply.

View publication stats