VISVESVARAYA TECHNOLOGICAL UNIVERSITY

Jnana Sangama, Belagavi - 590018

A PROJECT PHASE-II
REPORT ON
“APPROACH FOR LOAD MONITORING AND
ACTIVITY RECOGNITION IN SMART HOMES”
Submitted in partial fulfillment of the requirements for the VIII Semester
Bachelor of Engineering
In

Computer Science and Engineering

Submitted By
AKANKSHA N RAJ (1VK19CS004)

ARAVIND S (1VK19CS010)

DARSHAN C GOWDA (1VK19CS013)

MATHISHA A (1VK19CS023)
Under the Guidance of
Prof. Soumya George
Assistant professor, Dept. of CSE,
VKIT, Bengaluru

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

VIVEKANANDA INSTITUTE OF TECHNOLOGY
Kumbalagodu, Bengaluru-560074
2022-2023
 Janatha Education Society®
VIVEKANANDA INSTITUTE OF TECHNOLOGY
Gudimavu, Kumbalagodu Post,Kengeri Hobli,Bangaluru-560074
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

CERTIFICATE
This is to certify that the Project Work Phase II entitled “Approach for Load Monitoring
and Activity Recognition in Smart Homes” carried out by Akanksha N Raj bearing the USN:
(1VK19CS004), Aravind S bearing the USN: (1VK19CS010) , Darshan C Gowda bearing the
USN: (1VK19CS013), Mathisha A bearing the USN: (1VK19CS023) the bonafide student of
Vivekananda Institute of Technology, Bengaluru in partial fulfillment of VIII semester Project
Work Phase II (18CSP83) for the award of Bachelor of Engineering in Computer Science and
Engineering of the Visveswaraya Technological University, Belagavi during the year 2022-23.
The Project Work Phase II report has been approved as it satisfies the academic requirements in
respect of project work prescribed for the said degree.

______________________ _____________________ ______________________

Signature of the Guide Signature of the HOD Signature of the Principal
Prof. Soumya George Dr. Vidya A Dr. D V Chandrashekar
Asst. Professor, Prof. & HOD, Principal,
Dept. of CSE, Dept. of CSE, VKIT, Bengaluru
VKIT, Bengaluru VKIT, Bengaluru

External Viva-voice:

Name of Examiner Signature and date

1]

2]
 ABSTRACT

A smart home is a home automation system that integrates various technologies to create
a more efficient, comfortable, and secure living environment. In this project, we propose
the use of Raspberry Pi to build a smart home.

This concept can be taken around in any field out there to make life of humans easier and
smarter. Same can be done in our security system of houses where it will become easy to
handle the safety with easy to access Smart Door Locks. This project sheds light on
building a door locking system that can recognize the face of the owner of the house and
family members who will have access to pass through the door in the house. It can be
done by using face recognition algorithms which are gaining much importance in this era
of Artificial Intelligence. This article presents the design and end-to-end implementation
of a voice-activated smart home controller for intelligent devices, deployed in a real
environment and validated in an experimental setup of motorized blinds.

i
 ACKNOWLEDGEMENT

The completion of any work is a showcase of constant dedication and co-operation of many
people who lent their hands which went seen or unseen.

We are grateful to our Institution and Management, VIVEKANANDA INSTITUTE OF

TECHNOLOGY with its ideals and inspirations for having provided us with the facilities,
which has made this project a success.

We whole heartedly express our sincere thanks to our honorable Principal Dr. D V
Chandrashekar, Vivekananda Institute of Technology for the encouragement and support.

We express our heart full gratitude to Dr. Vidya A, Professor and Head, Department of
Computer science & Engineering, VKIT, for giving us guidance, valuable advice and support
for making this a great success.

We extend my gratitude to our guide Prof. Soumya George, Asst. Professor, Department of
Computer Science & Engineering, VKIT for her valuable advice, support and constructive
suggestions.

We would like to express our gratitude to our beloved project coordinator Prof. Trupti G,
Asst. Professor, Dept. of Computer Science and Engineering for her valuable guidance
and support.

We also thank to all our professors and the entire department of Computer Science &
Engineering for their co-operation and suggestions.

The report would be incomplete if we do not thank our parents and friends for their continuous
encouragement and moral support.

AKANKSHA N RAJ - 1VK19CS004

ARAVIND S - 1VK19CS010
DARSHAN C GOWDA - 1VK19CS013
MATHISHA A - 1VK19CS023

ii
 TABLE OF CONTENTS

Abstract i

Acknowledgement ii

Table of Contents iii

Chapter Chapter Name Page

No. No.
1 INTRODUCTION 1-4
1.1 Basic Topics of the project 1
1.2 Problem Statement 3
1.3 Scope of The Project 3
1.4 Objectives 3
1.5 Organization Of The Report 4
2 LITERATURE SURVEY 5-13
2.1 Related Work 5
3 SYSTEM REQUIREMENT SPECIFICATION 14-24
3.1 Functional Requirements 14
3.2 Non-Functional Requirements 14
3.3 Hardware Requirements 15
3.4 Software Requirements 20
3.5 Preliminary Investigation 24
3.6 System Environment 24

4 SYSTEM ANALYSIS 25-26

25
4.1 Existing System
26
4.2 Proposed System

5 SYSTEM DESIGN 27-37

28
5.1 System Architecture
29
5.2 Implementation
6 RESULTS AND DISCUSSIONS 38-40
CONCLUSION 41
REFERENCES 42-43
APPENDIX 44-47

iii
 LIST OF FIGURES

FIGURE NO. DESCRIPTION PAGE NO

Fig 1.1 Three main visions of Internet of Things 2

Fig 3.1 Specification of Raspberry-PI 16

Fig 3.2 Raspberry Pi Camera module 17

Fig 3.3 Fire Sensor 17

Fig 3.4 Gas Sensor 18

Fig 3.5 PIR and IR Sensor 18

Fig 3.6 DC Motor 19

Fig 3.7 Buzzer 19

Fig 3.8 Raspbian OS 20

Fig 3.9 OpenCV 21

Fig 5.1 Proposed System Design 28

Fig 5.2 System Architecture of Proposed System 28

iv
APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

CHAPTER 1

INTRODUCTION

The concept of a smart home is rapidly gaining popularity as homeowners

seek to create a more efficient, comfortable, and secure living environment. One of
the key components of a smart home is load monitoring and activity recognition,
which enables homeowners to optimize their energy usage and improve their
security. Load monitoring involves identifying the energy consumption patterns of
individual appliances in the home. This information can be used to estimate the
energy consumed by each appliance and to identify opportunities for energy
savings. Activity recognition involves using machine learning algorithms to analyze
the energy consumption patterns and recognize the activity of the occupants in the
home. This includes identifying when a person enters or leaves a room, turns on or
off an appliance, or performs any other activity that can be identified from the
energy consumption patterns. The use of Raspberry Pi, a low-cost and widely
available microcontroller, has made it possible to implement load monitoring and
activity recognition in smart homes in a practical and affordable way. The
Raspberry Pi can be used to collect and preprocess data from smart meters and to
control various home automation components.

1.1 Three main visions of Internet of Things

Things Oriented Vision:
This vision is supported by the fact that we can track anything using sensors
and pervasive technologies using RFID. The basic philosophy is uniquely
identifying any object using specifications of Electronic Product Code (EPC). This
technique is extended using sensors.It is important to appreciate the fact that future
vision will depend upon sensors and its capabilities to fulfill the “things” oriented
vision. We will be able to generate the data collectively with the help of sensors,
and sensor type embedded system. The summarized vision will be dependent upon
sensor based networks as well as RFID- based Sensor Networks which will take
care of the integration of technology based on RFID and sophisticated sensing and
computing devices and the global connectivity.

Dept. of CSE, VKIT 2022-23 Page 1

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Internet Oriented Vision:

The internet-oriented vision has pressed upon the need to make smart
objects which are connected. The objects need to have characteristics of IP
protocols as this is one of the major protocols being followed in the world of
Internet. The sensor based object can be converted in to an understandable format,
which can be identified uniquely and its attributes can be continuously monitored.
This makes the base for smart embedded objects which can be assumed to be a
microcomputers having computing resources.

Semantic Oriented Vision:

This vision is powered by the fact that the amount of sensors which will
be available at our disposal will be huge and the data that they will collect will be
massive in nature. Thus we will have vast amount of information, possibly
redundant, which needs to be processed meaningfully. The raw data needs to be
managed, processed and churned out in an understandable manner for better
representations and understanding.
If we are able to make the sets of data into homogeneous and heterogeneous
formats, then the interoperability issues of understanding the data will be
dependent upon the semantic technologies to process the data. It is here that needs a
generic vision of processing the raw data in to meaningful data and a marked
separation of data and their interpretation.

Fig 1.1 Three main visions of Internet of Things

Dept. of CSE, VKIT 2022-23 Page 2

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

1.2 PROBLEM STATEMENT

In the present scenario the crimes are increasing exponentially, arising a need of
security. Security can also be described as a condition so that one can develop
and progress freely and with a faith that no harm may be done. Hence we are
introducing any automatic door lock security system and home automation for the
security purpose. Camera is now enormously being used and with the
development of its content that is used in various applications. One of such is
automatic door lock security system using camera. “To implement a system that
exerts better surveillance and security by inculcating alerting system and face
recognition with appropriate notification sent to the owner using IoT.”

1.3 SCOPE OF THE PROJECT

The project's scope is to develop an approach for load monitoring and activity
recognition in smart homes using Raspberry Pi. The project aims to enable
homeowners to control various aspects of their home, such as lighting, and
security devices, remotely using a mobile or web-based interface.

The project will involve designing and building hardware modules, including
sensors and actuators, to interact with the physical components of the home. The
project's scope is limited to a single home or apartment and does not involve
controlling the energy supply to the home or integrating with other smart home
systems. The primary focus of the project is to provide homeowners with a
convenient and efficient way to control and monitor their home, enhance their
security, and reduce their energy consumption. The project will also aim to be
scalable and adaptable, allowing homeowners to expand the system's capabilities
as needed.

1.4 OBJECTIVES
• To design and implement face authenticated real time security system.

• To design and implement face authentication of captured image using camera

by OpenCV/ Python platform on Raspberry Pi.
• Interfacing of camera to capture live face images.

Dept. of CSE, VKIT 2022-23 Page 3

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

• Create a database of authorized person if they exist.

• Controlling Home Appliance through Mobile Application using

IOT Technology.
• Face Detection and Face Recognition using Haar cascade Algorithm.

• Automation of door locking using by comparing face with database.

• The captured image is compared and verified with the database, if found
matching then the access to locking device is allowed.

• In case of failure of face authentication an alerting SMS can be sent to the

owner

1.5 ORGANIZATION OF THE REPORT

Chapter 1: The chapter 1 describes, in brief, the idea of the project. It begins
with the explanation of the purpose of the project, the definitions of a few
terms used in the document, the problem definition, and the scope of the
project.

Chapter 2: This chapter describes the literature survey and the background
preparation done to understand more about this project.

Chapter 3: It describes the system requirements such as hardware and

software requirements, functional and non-functional requirements,
preliminary investigation and system environment.

Chapter 4: This chapter describe the existing system and its limitations and
here it tries to improve the existing system. It explains the proposed system
and its architecture.

Finally, this project explains the problem in hand for the system that is
being designed.

Dept. of CSE, VKIT 2022-23 Page 4

CHAPTER 2

LITERATURE SURVEY
Literature Survey is the most important step in the software development
process. Before developing the tool it is necessary to determine the time factor,
economy and company strength. Once these things are satisfied, then next step is
to determine which operating system and language can be used for developing the
tool. Once the programmer start building the tool the programmers need lot of
external support. This support can be obtained from senior programmers, from
book or from websites.

2.1 RELATED WORK

Prashanth, et al., [1] They have proposed in recent years, with the demand for
better security, computers have played a large role. Due to their precision, large
memory banks and high computing power, considerable development has been
made in the area of face recognition. Computers now surpass humans in many
face recognition tasks. A human being can remember limited number of faces.
But a computer doesn't have any limits and can hence be used where large
databases of facial records are needed. Such a facial recognition system has many
potential applications including crowd and airport surveillance, private security
and improved human-computer interaction. Such a system is perfectly suited to
fix security issues and offer flexibility to smart house control. This project is
aimed to be a complete system for face recognition: easy to build, cheap cost and
effective. Main purpose is to be set as an alert for home visitors and provide
information about the visitors in a dynamic website and phone application. It can
also be used in other fields like industries, offices and even airports for
identifying wanted people. Among the other bio-metric techniques, face
recognition method offers one great advantage which is user friendliness.

Dept. of CSE, VKIT 2022-23 Page 5

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Jiang, et al., [2] They presents an Internet of Things (IoT) based face detection
system. The main objective is to build a fully automated human face detection
system for images with complex backgrounds. This system can capture images
and run face detection algorithms on each IoT smart camera device, which is
required to write face detection program code. The raspberry Pi 3 board is used as
the gateway of IoT smart camera device with a camera on it. The final detected
faces will be sent to remote devices, e.g. cell phones and laptops. Two main steps
are used to detect faces: 1) cascade classifiers for face detection first and then 2)
spatial correlation for detection results improvement. The experimental results
showed that our smart camera system gives comparable face detection
performance. The average precision of 85.7%. Several promising directions for
future research based on IoT are also concluded.

Prayag, et al., [3] In today's world, home security is of utmost priority. IOT
(Internet of Things) being an emerging technology can be used along with facial
recognition to make our task of providing smart home security easier, simpler and
foolproof. The idea behind this paper is to develop an efficient face recognition
system. The system uses Local Binary Pattern Histograms to recognize the person
from the local database created for the family members of the house. Security,
monitoring and control to automation in real time are the key components of this
system. The hardware required to implement this system are Raspberry Pi 3
microprocessor, external web camera, speaker and stepper motor.

Suraj, et al., [4] The Internet of Things (IoT) has made it possible to set up a
smart home security through which you can decide who can enter your home
using your smartphone and web application. It's also made it simple and
relatively affordable to monitor your home anytime and anywhere. the key issue
in a traditional home security system is, it is easily breakable and quite outdated.
To tackle this problem, we propose a smart home security system, which is IoT as
well as face recognition enabled. In this system, the web camera is used which is
connected to the raspberry pi accompanied by sensors such as Passive Infrared
(Pir) and Ultrasonic sensor. On motion detection camera captures an image of the
person in front of the door then real-time face recognition is done using local
binary pattern (LBP). If person's image matches with one of the home members
then the door will unlock, else doorbell will ring. if an intruder tries to break door

Dept. of CSE, VKIT 2022-23 Page 6

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

then an alarm will be raised at the same time SMS and Email containing image of
the intruder will be sent to the homeowner. Face recognition works well in multi-
face recognition and stranger identification, which meet the requirement of home
security. This system is battery powered in case of power failure. Furthermore,
the house owner can keep track of activity happening in the house using android
and web application connected to the raspberry pi using the internet. Using
Android application or web application owner can also add new person's faces
into the databases e.g., guests.

Nashwan, et al., [5] A real-time recognition system is proposed that will equip
for handling images very quickly. The main objective of this is to protect home,
office by recognizing people. For this purpose, the PIR sensor is used to detect
movement in the specific area. Afterwards, the Raspberry Pi will capture the
images. Then, the face will be detected and recognized in the captured image.
Finally, the images and notifications will be sent to a smartphone based IoT by
using Telegram application. The proposed systems are real-time, fast and has low
computational cost. The experimental results show that the proposed face
recognition system can be used in a real time system.

Narumol, et al., [6] They have proposed the image enhancement method using
the data matching with histogram shaping technique. The experiments were
conducted using images with 1,280 × 720 pixel collected from CCTV
surveillance video system. Type of images (.jpg) is in RGB24 form and first
frame's surveillance video file. The results showed that the proposed method
using images enhancement technique can improve the quality of images from
surveillance video system using the proposed image enhancement based on
histogram shaping technique.

Amritha, et al., [7] It is important to own a reliable security system that can
secure our assets as well as to protect our privacy. The traditional security system
needs an individual to use a key, identification (ID) card or password to access an
area such as home and workplace. However, the present security system has
many weaknesses wherever it is simply cast and taken. Most doors are controlled
by persons with the employment of keys, security cards, countersign or pattern to
open the door. The aim is to assist users for improvement of the door security of

Dept. of CSE, VKIT 2022-23 Page 7

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

sensitive locations by using face detection and recognition. The proposed system
mainly consists of subsystems namely image capture, face detection and
recognition, email notification and automatic door access management. Face
Recognition supported OpenCV is brought up because it uses Eigen faces and
reduces the scale of face images without losing vital features, facial images for
many persons can be stored in the database. The door lock can also be accessed
remotely from any part of the world by using Telegram android application. The
captured image from pi camera will be sent to the authorized person through
email for safety purposes.

Mahesh, et al., [8] They have proposed designing and development of anti-theft
as well as driver surveillance embedded system that uses biometric authentication
to access the vehicle. This system contains camera which take the image of a
person trying to get access of vehicle and compare with authorized person's
image and then allowing or denying access. In The case of denial of vehicle
access or even if there is an accident occurs, camera will capture the images and
email it to the owner or authorizer. This will help to catch thieves, also allows the
surveillance of driver and also the inner part of vehicle. The recent work on
proposed embedded system is written. The system is designed and developed
using raspberry pi, high resolution camera, vibration sensor and open source
software.

Shreyak, et al., [9] The management of the attendance can be a great burden on
the teachers if it is done by hand. To resolve this problem, smart and auto
attendance management system is being utilized. But authentication is an
important issue in this system. The smart attendance system is generally executed
with the help of biometrics. Face recognition is one of the biometric methods to
improve this system. Being a prime feature of biometric verification, facial
recognition is being used enormously in several such applications, like video
monitoring and CCTV footage system, an interaction between computer &
humans and access systems present indoors and network security. By utilizing
this framework, the problem of proxies and students being marked present even
though they are not physically present can easily be solved. The main
implementation steps used in this type of system are face detection and
recognizing the detected face. This proposes a model for implementing an

Dept. of CSE, VKIT 2022-23 Page 8

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

automated attendance management system for students of a class by making use

of face recognition technique, by using Eigenface values, Principle Component
Analysis (PCA) and Convolutional Neural Network (CNN). After these, the
connection of recognized faces ought to be conceivable by comparing with the
database containing student's faces. This model will be a successful technique to
manage the attendance and records of students.

Sameer, et al., [10] The developments of the internet of things (IoT)

technologies fascinated the universe and provided great opportunities to introduce
these innovations in smart house networks. Smart home automation is highly
required these days. Smart home automation is a collection of electronic devices
connected to monitor and control in the market home appliance remotely.
However, it is still needed to design a friendly and reliable system since the
system mainly depends on the devices used and the environment of the network.
NETPI and BLYNK are IoT frameworks used for hardware-agnostic with
smartphones, websites, private clouds, system security, data mining, and deep
learning. The results confirmed that NETPI provides flexibility to deal with
several NODEMCU controllers in a single control framework. The proposed
system shows its applicability in monitoring and controlling home appliances
remotely.

Tahir, et al., [11] The design and implementation of an iot-based home

automation, energy management and smart security system has been proposed.
The components used in the development of the system unit include an esp8266
wi-fi component and the universal arduino imbedded controller. The proposed
scheme is aimed at assisting and supporting an efficient usage of power, and by
doing so, power consumption as well as the risk of fire outbreaks will be greatly
reduced to the barest minimal. the capability of remotely monitoring and sensor
and control nodes are connected. The admin can access and control all the nodes
connected to each user but a single user can control only the nodes to which the
user itself is connected. This whole system using Internet of Things (iot) will
allow mobile devices and computers to remotely control all the functions and
features of home appliances from anywhere around the world using the internet
connection.

Dept. of CSE, VKIT 2022-23 Page 9

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Meena, et al., [12] They have focused on home automation using IoT. The IoT
devices controls and monitors the electronic, electrical and the mechanical
systems used in various types of buildings. The devices connected to the cloud
server are controlled by a single admin which facilitate a number of users to
which a number of sensor and control nodes are connected. The admin can access
and control all the nodes connected to each user but a single user can control only
the nodes to which the user itself is connected. This whole system using Internet
of Things (iot) will allow mobile devices and computers to remotely control all
the functions and features of home appliances from anywhere around the world
using the internet connection. The system designed is economical and can be
expanded as it allows connection and controlling of a number of different
devices. Most of the electronic devices are manually monitored on regular basis
to ensure the optimal operation. To install new application specific devices not
only increases the cost but also replaces the old device.

Karma, et al., [13] They have presented a dual lock system using radio-frequency
identification (RFID) and fingerprint recognition. It is a secure and reliable
locking system which can be implemented in home, offices, school and in any
kind of organization. Even if the users lost the RFID card, they can unlatch the
lock with their fingerprints. Only the registered users are able to access using their
fingerprints and the RFID tags. If any unauthorized persons try to have an access,
first of all short message service (SMS) will send a text message to an owner
using global system for mobile communications (GSM), and at the same time a
TTL camera will take a snap of that person and store it in a SD card for reference
of the owner. The core of the system is the Arduino Mega 2560 which is
interfaced with a fingerprint scanner module, a RFID card reader module, the TTL
camera, an SD card module, a GSM shield, a liquid crystal display (LCD) and an
electric door strike.

Majid, et al., [14] Automation can be accomplished by using the Internet of

Things (IoT). This gives the inhabitant accesses to certain data in the house and
the ability to control some parameters remotely. They have presented the
complete design of an IoT based sensing and monitoring system for smart home
automation. The proposed design uses the EmonCMS platform for collecting and
visualizing monitored data and remote controlling of home appliances and

Dept. of CSE, VKIT 2022-23 Page10

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

devices. The selected platform is very flexible and user-friendly. The sensing of
different variables inside the house is conducted using the NodeMCU-ESP8266
microcontroller board, which allows real time data sensing, processing and
uploading/downloading to/from the EmonCMS cloud server.

Jinsung, et al., [15] They have proposed a ZigBee-based intelligent self-

adjusting sensor (ZiSAS)in order to address these concerns. They have presented
a situation-based self-adjusting scheme, an event-based self-adjusting sensor
network, and hardware and middleware implementation. They also introduce
some smart home services using the proposed system. They implemented the
system in real test bed and conducted an experiment. There experiment shows
that they reduces the system's energy consumption.

Muhammad, et al., [16] They have proposed the IoT based system the user is
transmitting his confidential data over the network which may contain passwords
and other information, which are as crucial as the secrets or valuables behind the
door. This crypto Lock not only protects our valuables behind the door but also it
protects our data which is being transmitted over the network. It gives easy
remote access, controls unauthorized access and gives a complete sense of
security.

Majid, et al., [17] Automation can be accomplished by using the Internet of

Things (IoT). This gives the inhabitant accesses to certain data in the house and
the ability to control some parameters remotely. They presents the complete
design of an IoT based sensing and monitoring system for smart home
automation. The proposed design uses the EmonCMS platform for collecting and
visualizing monitored data and remote controlling of home appliances and
devices. The selected platform is very flexible and user-friendly. The sensing of
different variables inside the house is conducted using the NodeMCU-ESP8266
microcontroller board, which allows real time data sensing, processing and
uploading/downloading to/from the Emon CMS cloud server.

K. Lova, et al., [18] They describes the overall notion of the IOT based sensing
systems and monitoring systems for implementing an automated home. The
proposed prototype uses Node MCU board with internet being remotely
controlled by Android OS smart phone. Node MCU is the heart of this system

Dept. of CSE, VKIT 2022-23 Page11

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

and it can perform as a micro web server and it acts as an interface for the wide
range of hardware modules. To control lights, fans and other home appliances
which are connected to the relay system, the system offers switching
functionalities. It is also used for environmental monitoring by sensing and
analyzing data about temperature and humidity. Another notifying feature in this
system designed is the intrusion detection which is offered by this system using
motion sensor. All these activities are controlled by using Android mobile app-
Blynk.

Nikolay, et al., [19] They have introduced one home automation solution,
describing a combination of the chosen hardware components and a possible
software solution. A major hardware element in the project presented is a
Raspberry Pi microcomputer. Which is connected to sensors for data collection.
The microcomputer runs a described software system for organizing and
processing this data, configures different home authentication.

Sajjadul, et al., [20] One of the most promising development in modern

technology is home automation system. The rapid advancement in technology and
automation system made human life much easier. An advanced smart home with
an improved security system is introduced using the internet of things (IoT). The
user can control and monitor the system using web-based interfaces like IFTTT or
by using smartphone applications like Blynk. The main purpose of this project is
to make a low-cost smart home automation and security system to ensure a better
life for us and our families.

Anubhav, et al., [21] They presents an Complete Artificial Intelligence and

Internet of Things based system which provides affordable and remote
accessibility and Monitoring system to a combination of Household Appliances.
The system controls electrical appliances like lights, fans, Smoke, gas and Fire
detectors and also provides security services by giving alerts on the user's
Android Application.

Shanthini, et al., [22] The proposed approach addresses a security aspect in

smart home technologies, namely the door lock system. The door lock system
determines the security by allowing the owner to monitor the buildings with a
Smartphone-controlled, Bluetooth-connected system using Arduino UNO. Users

Dept. of CSE, VKIT 2022-23 Page12

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

tablets, smartphones, laptops, etc. by providing the login credentials like

username and password which is verified in the database over the internet. If the
credentials are invalid, the buzzer rings and an SMS alert is sent to user.

A. Z. M, et al., [23] In this IoT based home automation system through which
home security, room air quality check, and emergency assistance can be obtained.
This system can automatically provide home safety, but the user can also control
the system manually if desired. The system uses the NodeMCU module as the
mainboard and has a mobile application for remote monitoring. In addition to air
quality detection, the system can detect CFCs used in the air conditioning system
to avoid accidents. If any unwanted incident occurs, the system will immediately
turn off the home's main power connection and send the house's status in the form
of a notification to the user's mobile app.

Harsh, et al., [24] There aim is to develop home automation system based on
IOT using Wi-Fi based microcontroller. As scope of technology is widening
every day, they are making our tech advance in mobile, robotics, Machine
Learning, then why an exception for our home. Today's houses are gradually
transferring from ordinary/human's input- based appliances to smart/IOT enabled
appliances to be controlled remotely. At Present, existing home automation
systems use technology that is limited to only that device. So, in a nutshell, they
are making there devices IOT enabled not our homes. As far as this is concerned,
NodeMCU (ESP8266) microcontroller along with Relays is used to control
electrical switches remotely from the server which is built on Node.js. User can
control switches using a Web Application after authenticating.

Waheb, et al., [25] They have proposed a low-cost Wi-Fi based automation
system for Smart Home (SH) in order to monitor and control home appliances
remotely using Android- based application. An Arduino Mega microcontroller
provided with Wi-Fi module is utilized to build the automation system. In
addition, several sensors are used to monitor the temperature, humidity and
motion in home. A relay board is exploited to connect the HAS with home under
controlled appliances. The proposed automation system, can easily and efficiently
control the electrical appliances via Wi-Fi and Virtuino mobile application.

Dept. of CSE, VKIT 2022-23 Page13

CHAPTER 3

SYSTEM REQUIREMENT SPECIFICATIONS

System requirements are the prerequisites that are often used as a guideline as
opposed to an absolute rule. Requirements can be classified as functional
requirements, non-functional requirements, software requirements and hardware
requirements. The project will also aim to be scalable and adaptable, allowing
homeowners to expand the system's capabilities as needed. “To implement a
system that exerts better surveillance and security by inculcating alerting system
and face recognition with appropriate notification sent to the owner using IoT.”

3.1 FUNCTIONAL REQUIREMENTS

In systems engineering and requirements engineering, functional

requirements are the requirements which are specified by a stakeholder or
customer that define specific behavior or functions that the system should
incorporate. Functional requirements for the proposed system are:

➢ Device should do minimal computations on its own.

➢ Device should be able to capture image and display it on the screen

meanwhile the Image should be compared.
➢ Device should be able to send and receive messages.

3.2 NON-FUNCTIONAL REQUIREMENTS

In systems engineering and requirements engineering, non-functional

requirements are those requirements that specifies criteria that can be used to
judge the operation of a system.
1) The camera is enabled during the image capture.

2) The image will be refreshed as the new image is captured.

Dept. Of CSE, VKIT 2022-23 Page14

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

3.3 HARDWARE REQUIREMENTS

Hardware requirements are those requirements that specify the hardware

platform that is required to implement a system. Hardware requirements for the
proposed system are given as follows:

• ARM11 Raspberry Pi 3 board

• Pi Camera

• Fire sensor

• Gas sensor

• IR sensor

• Relay

• DC Motor

• Power supply

• Buzzer

• Appliances – Fan, LED light

• Water Pump

• H-Bridge

ARM11 Raspberry Pi 3 board

Pi is a credit-card sized computer that connects to a computer monitor or

TV and uses input devices like keyboard and mouse. It is capable of performing
various functionalities such as surveillance system, military applications,
surfing internet, playing high definition videos, live games and to make data
bases.

Dept. Of CSE, VKIT 2022-23 Page 15

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Fig 3.1. Specification of Raspberry-PI

RASPBERRY PI CAMERA MODULE

The 5MP Raspberry Pi 3 Model B Camera Module Rev 1.3 with Cable
equips flexible cable for attaching with Raspberry Pi 3 Model B. The 5MP
camera module is perfect for small Raspberry Pi projects.
The high-definition 5MP camera delivers outstanding photos but can also shoot
video, ideal for drones or a CCTV project. The lightweight camera board allows
for it is useful in more practical roles, such as a hidden camera, even a camera for
a Pi-phone. This Raspberry Pi Camera Module is a custom designed add-on for
Raspberry Pi. It attaches to Raspberry Pi by way of one of the two small sockets
on the board upper surface. This interface uses the dedicated CSI interface,
therefore it is designed especially for interfacing to cameras. The CSI bus is
capable of extremely high data rates, and it exclusively carries pixel data.

Dept. Of CSE, VKIT 2022-23 Page 16

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Fig 3.2. Raspberry Pi Camera module

FIRE SENSOR
A fire-sensor is one kind of detector which is mainly designed for
detecting as well as responding to the occurrence of a fire or flame. This sensor
module is used in fire or flame alarms.

Fig 3.3. Fire Sensor

GAS SENSOR
• The sensing circuit consists of an MQ5 sensor which senses LPG leakage of
the order 200ppm to 1000ppm (capable of causing explosions) and is
connected to the microcontroller to be supplied with 5V DC.

Dept. Of CSE, VKIT 2022-23 Page17

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

• MQ 5 gas sensor was chosen over other MQ because it is Sensitive for

detecting LPG and natural gas.

Fig 3.4. Gas Sensor

IR SENSOR
More advanced security systems include passive infrared (PIR) motion
detectors. The "motion sensing" feature on most lights (and security systems) is a
passive system that detects infrared energy. These sensors are therefore known as
PIR (passive infrared) detectors or pyro electric sensors. These sensors "see" the
infrared energy emitted by an intruder's body heat. When an intruder walks into
the field of view of the detector, the sensor detects a sharp increase in infrared
energy.

Fig 3.5. PIR and IR Sensor

Dept. Of CSE, VKIT 2022-23 Page18

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

DC Motor

Electric motors are everywhere! In your house, almost every mechanical

movement that you see around you is caused by an AC (alternating current) or
DC (direct current) electric motor. Let's start by looking at the overall plan of a
simple two-pole DC electric motor. A simple motor has six parts, as shown in the
diagram below:
• Armature or rotor
• Commutator
• Brushes
• Axle
• Field magnet

Fig 3.6. DC Motor

BUZZER

A buzzer is an audio signaling device, which may be mechanical,

electromechanical. The buzzers are used for alarm which alert the users.

Fig 3.7. Buzzer

Dept. Of CSE, VKIT 2022-23 Page19

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

3.4 SOFTWARE REQUIREMENTS

Software requirements are those requirements that specify the software
platform that is required to implement a system. Software requirements for the
proposed system are given as follows:

➢ Raspbian OS

➢ Open CV

➢ Python Language

RASPBIAN OS

Although the Raspberry Pi’s operating system is closer to the Mac than
Windows, it’s the latter that the desktop most closely resembles It might seem a
little alien at first glance, but using Raspbian is hardly any different to using
Windows (barring Windows 8 of course). There’s a menu bar, a web browser, a
file manager and no shortage of desktop shortcuts of pre-installed applications.
Raspbian is an unofficial port of Debian Wheezy armhf with compilation settings
adjusted to produce optimized "hard float" code that will run on the Raspberry Pi.
This provides significantly faster performance for applications that make heavy
use of floating point arithmetic operations. All other applications will also gain
some performance through the use of advanced instructions of the ARMv6 CPU
in Raspberry Pi. Although Raspbian is primarily the efforts of Mike Thompson
(MP Thompson) and Peter Green (plug wash), it has also benefited greatly from
the enthusiastic support of Raspberry Pi community members who wish to get the
maximum performance from their device.

Fig 3.8. Raspbian OS

Dept. Of CSE, VKIT 2022-23 Page 20

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

OpenCV
OpenCV is a library of programming functions mainly aimed at real-time
computer vision. It is developed by Intel research center and subsequently
supported by1Willow Garage and now maintained by itseez. It is written in C++
and its primary interface is also in C++. Its binding is in Python, Java, Matlab.
OpenCV runs on a variety of platform i.e. Windows, Linux, MacOS, OpenBSD in
desktop and Android, IOS and Blackberry in mobile. It is used in diverse purpose
for facial recognition, gesture recognition, object identification, mobile robotics,
segmentation etc. It is a combination of OpenCV C++ API and Python language.
In our project we are using OpenCV version 2 OpenCV is used to gesture control
to open a camera and capture the image. It is also used in the image to text and
voice conversion technique.

It has a modular structure, which means that the package includes several shared
or static libraries. We are using image processing module that includes linear and
non-linear image filtering, geometrical image transformations (resize, affine and
perspective warping, and generic table-based remapping), color space conversion,
histograms, and so on. Our project includes libraries such as Viola-Jones or Haar
classifier, LBPH (Lower Binary Pattern histogram) face recognizer, Histogram of
oriented gradients (HOG).

Fig 3.9. OpenCV

Python Introduction
Python is a general-purpose interpreted, interactive, object-oriented, and high-
level programming language. It was created by Guido van Rossum during 1985-
1990. Like Perl, Python source code is also available under the GNU General
Public License (GPL). This tutorial gives enough understanding on Python

Dept. Of CSE, VKIT 2022-23 Page 21

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

programming language. Python is a high-level, interpreted, interactive and object-

oriented scripting language. Python is designed to be highly readable. It uses
English keywords frequently where as other languages use punctuation, and it has
fewer syntactical constructions than other languages. Python is a MUST for
students and working professionals to become a great Software Engineer
specially when they are working in Web Development Domain. I will list down
some of the key advantages of learning Python:
• Python is Interpreted − Python is processed at runtime by the interpreter.
You do not need to compile your program before executing it. This is
similar to PERL and PHP.
• Python is Interactive − You can actually sit at a Python prompt and
interact with the interpreter directly to write your programs.
• Python is Object-Oriented − Python supports Object-Oriented style or
technique of programming that encapsulates code within objects.
• Python is a Beginner's Language − Python is a great language for the
beginner-level programmers and supports the development of a wide
range of applications from simple text processing to WWW browsers to
games.
Following are important characteristics of Python Programming

• It supports functional and structured programming methods as well as

OOP.
• It can be used as a scripting language or can be compiled to bytecode for
building large applications. It provides very high-level dynamic data
types and supports dynamic type checking.
• It supports automatic garbage collection.
• It can be easily integrated with C, C++, COM, ActiveX, CORBA, and
Java.
Applications of Python:

➢ Easy-to-learn − Python has few keywords, simple structure, and a clearly

defined syntax. This allows the student to pick up the language quickly.
➢ Easy-to-read − Python code is more clearly defined and visible to the
eyes.

Dept. Of CSE, VKIT 2022-23 Page 22

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

➢ Easy-to-maintain − Python's source code is fairly easy-to-maintain.

➢ A broad standard library − Python's bulk of the library is very portable
and cross-platform compatible on UNIX, Windows, and Macintosh.
➢ Interactive Mode − Python has support for an interactive mode which
allows interactive testing and debugging of snippets of code.
➢ Portable − Python can run on a wide variety of hardware platforms and
has the same interface on all platforms.
➢ Extendable − You can add low-level modules to the Python interpreter.
These modules enable programmers to add to or customize their tools to
be more efficient.
➢ Databases − Python provides interfaces to all major commercial
databases.
➢ GUI Programming − Python supports GUI applications that can be
created and ported to many system calls, libraries and windows systems,
such as Windows MFC, Macintosh, and the X Window system of Unix.
➢ Scalable − Python provides a better structure and support for large
programs than shell scripting.

3.5 PRELIMINARY INVESTIGATION

The preliminary investigation for the approach of load monitoring and

activity recognition in smart homes using Raspberry Pi involves several crucial
steps. First, understanding the problem requires researching existing solutions,
identifying challenges, and potential benefits. The next step involves researching
the available technology options such as non-intrusive load monitoring
techniques, machine learning algorithms, and Raspberry Pi. Data collection
methods such as smart meters, energy monitoring devices, and sensors are then
identified to collect energy consumption data from appliances. The next step is to
identify suitable software and hardware requirements, including machine learning
algorithms, sensors, microcontrollers, and programming languages for software
development. Finally, a feasibility analysis is conducted to assess the system's
scalability, adaptability, cost-effectiveness, and ease of use.
The preliminary investigation provides valuable insights into the problem,

Dept. Of CSE, VKIT 2022-23 Page 23

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

technology options, data collection methods, software and hardware

requirements, and feasibility of the proposed solution, guiding the development
of the approach for load monitoring and activity recognition in smart homes using
Raspberry Pi.

3.6 SYSTEM ENVIRONMENT

The system environment for load monitoring and activity recognition in smart
homes using Raspberry Pi consists of several components. First, the sensors are
used to collect data on the energy consumption of the appliances. The data is then
transmitted to the Raspberry Pi, which serves as the central processing unit for
the system. The Raspberry Pi runs the machine learning algorithms to analyze the
data and identify the activities being performed by the appliances. The results of
the analysis are then sent to the user interface, which displays the activity being
performed by the appliance, the energy consumed, and other relevant
information. The user interface can be accessed through a smartphone application
or a web-based interface. The system environment also includes the
communication network, which is used to transmit data between the sensors,
Raspberry Pi, and the user interface. Overall, the system environment is designed
to be efficient, reliable, and user-friendly, providing the user with the ability to
monitor and control the energy consumption of their appliances easily.

Dept. Of CSE, VKIT 2022-23 Page 24

CHAPTER 4
SYSTEM ANALYSIS
System Analysis as “the process of studying a procedure or business in order to
identify its goals and purpose, create system and procedures that will achieve
them in an efficient way”. Another view sees system analysis as a problem
solving technique that breaks down a system into its component pieces for the
purpose of the studying how well those component parts work and interact to
accomplish their purpose. The project will also aim to be scalable and adaptable,
allowing homeowners to expand the system's capabilities as needed. “To
implement a system that exerts better surveillance and security by inculcating
alerting system and face recognition with appropriate notification sent to the
owner using IoT.”

4.1 EXISTING SYSTEM

• People usually are outside the home for many reasons like travelling and
they want to control some devices from a far distance.
• For example they want to turn off the light and operate some electronic
devices at room and get feedback about the temperature and total control on
their home.
• No automatic motion detected device control.
• There is no environmental condition based device controlling and
monitoring.
• No Device status update to the family members.

4.1.1 Drawbacks of the Existing System

• Time taken for response is very high.

• The system proposed is very cost expensive to install for relatively small
places.
• A mobile security with video access has not been implemented.
• Manual detection is required.
• Smoke sensors are not implemented.

Dept. Of CSE, VKIT 2022-23 Page 25

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

4.2 PROPOSED SYSTEM

• The main objectives of this research is to design and implement a home

automation and security by using system using Raspberry Pi and IoT.
• Capable of controlling and automating most of the house appliances through
an easy manageable interface.
• The proposed system has a great feature of face recognition for security.
• This will decrease the deployment cost and will increase the ability of
upgrading, and system reconfiguration.

4.2.1 Advantages of Proposed System

• Time management is one of the advantages that is taken into consideration

in this system.
• Memory management is the key factor which is utilized in an efficient way.
• Even though the owner is away from his office or home, he Will be capable
of tracking and will be aware of security of his home or office.
• In case of emergency situations like gas leakages, fire accidents immediate
actions can be taken.
• People may forget to switch off the lights in home or office in their absence
which leads wastage of power; this can be easily avoided with the help of
IOT.
• This is the best solution for elders and even for physically disabled person to
operate equipment (electrical).

Dept. Of CSE, VKIT 2022-23 Page 26

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

CHAPTER 5

SYSTEM DESIGN

System design is the process of defining the architecture, components,

modules, interfaces and data for a system to satisfy specified requirements. System
design could see it as the application of systems theory to product development.
Theory is some overlap with the disciplines of system analysis, systems architecture
and systemsengineering.

If the broader topic development “blends the perspective of marketing,

design, and manufacturing into a single approach to product development,” then
design the act of talking the marketing information and creating the design of the
product to be manufactured. Systems design is therefore the process of defining and
developing systems to satisfy specified requirements of the user.

Until the 1990s systems design had crucial and respected role in the data
processing industry. In the 1990s standardization of hardware and software resulted
in the ability to build modular systems. The increasing importance of software
running on generic platforms has enhanced the discipline of software engineering.
Object-oriented analysis and design methods are becoming the most widely used
methods for computer systems design. The UML has become the standard language
in object-oriented analysis and design. It is widely used for modelling software
systems and is increasingly used for high designing non-software systems and
organizations.

System design is one of the most important phases of software development

process. The purpose of the design is to plan the solution of a problem specified by
the requirement documentation. In other words, the first step in solution is the
design of theproject.
The design of the system is perhaps the most critical factor affecting the
quality of the software. The objective of the design phase is to produce overall
design of the software. It aims to figure out the modules that should be in the system
to fulfil all the system requirements in efficient manner.

Dept. of CSE, VKIT 2022-23 Page 27

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

The design will contain the specification of all the modules, their
interaction with other modules and the desired output from each module. The
output of the design process is a description of the software architecture.

Fig 5.1. Proposed System Design

5.1 SYSTEM ARCHITECTURE

A system architecture is a conceptual model that defines the structure,
behavior, and more views of a system. An architecture description is a formal
description and representation of a system, organized in a way that supports
reasoning about the structures and behaviors of the system.

The figure 4.1 shows a general block diagram describing the activities
performedby this project.

Fig 5.2. System Architecture of Proposed System.

Dept. of CSE, VKIT 2022-23 Page 28

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

5.2 IMPLEMENTATION

Implementation is the realization of an application, or execution of a plan, idea,

model, design, specification, standard, algorithm, or policy. In other words, an
implementation is a realization of a technical specification or algorithm as a
program, software component, or other computer system through programming and
deployment. Many implementations may exist for a given specification or standard.

Implementation is one of the most important phases of the Software Development

Life Cycle (SDLC). It encompasses all the processes involved in getting new
software or hardware operating properly in its environment, including installation,
configuration, running, testing, and making necessary changes. Specifically, it
involves coding the system using a particular programming language and transferring
the design into an actual workingsystem.

This phase of the system is conducted with the idea that whatever is designed
should be implemented; keeping in mind that it fulfills user requirements, objective
and scope of the system. The implementation phase produces the solution to the
user problem.

5.2.1 OVERVIEW OF SYSTEM IMPLEMENTATION

In this system we have proposed webcam, sensors and Raspberry Pi board and the
cubeactivator along with its driver circulatory are connected together.

Camera:

Camera is pi camera interfacing to the raspberry pi module. The Raspberry Pi

Camera Board is a custom designed add-on module for Raspberry Pi hardware. It
attaches to Raspberry Pi hardware through a custom CSI interface. The sensor has
5 megapixel native resolution in still capture mode. In video mode it supports
capture resolutions up to 1080p at 30 frames per second. It is used to capture
images and send the clicked images to the raspberry pi module. Camera contains
LEDs and flashes to handle that light condition that is not explicitly supplied by
the environment and these light conditions are known as ambient light conditions.

Dept of CSE, VKIT 2022-23 Page 29

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Raspberry PI :

Raspberry pi 3 is a small computer board. When an image is taken by raspberry pi it

is compared with database image. For the first time when we capture an image to
Create a database raspberry pi module captures many images to create a database in
the system and this database is compared with the live captured images. After
comparing the two images, based on whether the output is positive or negative it
gives commands to GSM module.

MODULES DESCRIPTION
The proposed system contains 4 different modules:
• Face Detection
• Face Recognition
• OTP
• Door Lock.

STEP - 1: Face Detection

• The main component of the Face Detection system is a Web camera.
• Once person come in front of camera face will be captured.
STEP -2: Face Recognition
• The proposed face recognition system overcomes certain limitations of the
existingface recognition system.
• It is based on extracting the dominating features of a set of human faces stored in
the database and performing mathematical operations on the values
corresponding to them.
• Hence when a new image is fed into the system for recognition the main features
are extracted and computed to find the distance between the input image and the
stored images.
• Thus, some variations in the new face image to be recognized can be tolerated.
When the new image of a person differs from the images of that person stored in
the database, the system will be able to recognize the new face and identify who
the person is.

Dept of CSE, VKIT 2022-23 Page 30

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

The total system is divided into 3 modules- Database creation, Training the dataset,
Testing, sending alert messages as an extension.

1. Database Creation

a) Initialize the camera and set an alert message to grab the attention of the
students.

b) Get user id as input

c) convert the image into gray scale, detect the face and

d) Store it in database by using given input as label up to 20 frames.

2. Training
a) Initialize LBPH face recognizer.

b) Get faces and Id’s from database folder to train the LBPH face recognizer.

c) Save the trained data as xml or yml file

3. Testing

Load Haar classifier, LBPH face recognizer and trained data from xml or yml
file.

a) Capture the image from camera,

b) Convert it into gray scale,

c) Detect the face in it and

d) Predict the face using the above recognizer.

This system uses Viola Jones algorithm for face detection which uses modified
Haar Cascades for detection. Raspberry Pi is the main component in the project. We
will beusing USB webcam to capture photos. We can access Raspberry Pi’s console
either by using SSH in laptop or by using Keyboard and mouse with the display
device like TV connected to Pi. Firstly, the algorithm needs a lot of positive images
and negative images to train the Haar cascades classifier. Positive images are images
with clear faceswhere negative images are those without any faces.

Dept of CSE, VKIT 2022-23 Page 31

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

i. Haar Cascades:

Each feature is represented as a single value obtained from the difference of the
sums of pixels in white rectangle from the sum of all pixels in the black rectangle.
All different possible sizes and locations of classifier is used for calculating of
plenty of features. As the number of classifiers increase the arithmetic computations
seems to take a long time. To avoid this, we use the concept of Integral Image. In
Image Processing Integral image is a data structure which is summed area table and
algorithm for quickly and efficiently generating sum of values in a rectangular grid
subset. Integral image is derived by using the formula.

Integral Image: To solve the complexity of the number of classifiers applied for
calculation we use Ad boost machine learning algorithm, which is inbuilt in
OpenCV library that is cascade classifier, to eliminate the redundancy of the
classifiers. Any classifier which has a probability of 50% of more in detection is
treated as weak classifier. The Sum of all weak classifier gives a strong classifier
which makes the decision about detection. Although it is very vague to classify with
one strong classifier we use the cascade of classifiers. Classification takes place in
stages, if the selected region fails in the first stage, we discard it. We don’t use the
classifiers on that region which is discarded. The region which passes all the stages
i.e. all strong classifiers is treated as the detected face. Detected Faces are passed to
the Face recognition phase.
In this phase we use Local Binary Patterns algorithm for face recognition. Local
binary patterns are simple at the same time very efficient texture operator which
assigns the pixels of the image by comparing with the adjacent pixels as threshold
and which results in a binary result. The detected integral image is subjected to this
Local binary pattern which results in decimals are represented as histogram for
every integral image. Face recognition is extremely vulnerable to the environment
changes like brightness, facial expressions and position. Face pre-processing is the
module which reduces the problems that makes the picture unclear to recognize the
face such as less brightness and contrast problems and noise in the image and make
sure the facial features always be in a constant position. times, firstly for the whole
face and the other two for side faces.

Dept of CSE, VKIT 2022-23 Page 32

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

ii. Histogram of oriented gradients (HOG)

Histogram of oriented gradients (HOG) is a feature descriptor used to detect objects

in computer vision and image processing. The HOG descriptor technique counts
occurrences of gradient orientation in localized portions of an image - detection
window, or region of interest (ROI).
Implementation of the HOG descriptor algorithm is as follows:
1. Divide the image into small connected regions called cells, and for each cell
compute a histogram of gradient directions or edge orientations for the pixels
within the cell.

2. Discretize each cell into angular bins according to the gradient orientation.

3. Each cell's pixel contributes weighted gradient to its corresponding angular bin.

4. Groups of adjacent cells are considered as spatial regions called blocks. The
grouping of cells into a block is the basis for grouping and normalization of
histograms.

5. Normalized group of histograms represents the block histogram. The set of these
blockhistograms represent the descriptor.

STEP - 3: OTP

If Face Didn’t Match with Database, Face Intimation will be Sent to Owner
of theFace with SMS or E mail alert.

Dept of CSE, VKIT 2022-23 Page 33

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

PYTHON CODE
Face Recognition With Raspberry Pi and OpenCV
from imutils.video import VideoStream
from imutils.video import FPS
import face_recognition
import imutils
import pickle
import time
import cv2

currentname = "unknown"
train_model.py
encodingsP = "encodings.pickle"
print("[INFO] loading encodings + face detector...")
data = pickle.loads(open(encodingsP, "rb").read())
vs = VideoStream(usePiCamera=True).start()
time.sleep(2.0)
fps = FPS().start()
while True:
frame = vs.read()
frame = imutils.resize(frame, width=500)
boxes = face_recognition.face_locations(frame)
encodings = face_recognition.face_encodings(frame, boxes)
names = []
for encoding in encodings:
matches = face_recognition.compare_faces(data["encodings"],
encoding)
name = "Unknown" #if face is not recognized, then print Unknown
if True in matches:
matchedIdxs = [i for (i, b) in enumerate(matches) if b]
counts = {}
for i in matchedIdxs:
name = data["names"][i]

Dept. of CSE, VKIT 2022-23 Page 34

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

counts[name] = counts.get(name, 0) + 1
name = max(counts, key=counts.get)
if currentname != name:
currentname = name
print(currenttname)
names.append(name)
for ((top, right, bottom, left), name) in zip(boxes, names):
cv2.rectangle(frame, (left, top), (right, bottom),
(0, 255, 225), 2)
y = top - 15 if top - 15 > 15 else top + 15
cv2.putText(frame, name, (left, y), cv2.FONT_HERSHEY_SIMPLEX,
.8, (0, 255, 255), 2)
cv2.imshow("Facial Recognition is Running", frame)
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
break
fps.update()
fps.stop()
print("[INFO] elasped time: {:.2f}".format(fps.elapsed()))
print("[INFO] approx. FPS: {:.2f}".format(fps.fps()))
cv2.destroyAllWindows()
vs.stop()

Raspberry Pi Fire and Gas Detector

from twilio.rest import Client
import PCF8591 as ADC
import LCD1602 as LCD
import RPi.GPIO as GPIO
import time
import math
GPIO.setmode(GPIO.BCM)
gas = 17
buzzer = 18
fire = 27

Dept. of CSE, VKIT 2022-23 Page 35

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

def setup():
ADC.setup(0x48)
LCD.init(0x27, 1)
GPIO.setup(fire, GPIO.IN)
GPIO.setup(gas, GPIO.IN)
GPIO.setup(buzzer, GPIO.OUT)
global Buzz
Buzz = GPIO.PWM(buzzer, 440)
def loop():
while True:
firetmp = ADC.read(0)
gaslvl = ADC.read(1)
lvls = "Gas: " +str(gaslvl)+", Fire: "+str(firetmp)
digFire = GPIO.input(fire)
digGas = GPIO.input(gas)
if digFire == 0 or digGas == 0:
warning = 'DOH! Something is not okay!!\nLevels: '+lvls
print(warning)
time_end = time.time() + 30
while time.time() < time_end:
clearLCD()
LCD.write(0, 0, 'DOH! Something ')
LCD.write(0, 1, 'is not Okay! ')
time.sleep(3)
clearLCD()
LCD.write(0, 0, '*****Danger*****')
LCD.write(0, 1, 'Fire:'+str(firetmp)+' Gas:'+str(gaslvl))
time.sleep(3)
else:
print (lvls) # display fire and gas levels in command line
LCD.write(0, 0, 'Gas: '+str(gaslvl))
LCD.write(0, 1, 'Fire: '+str(firetmp))
time.sleep(3)
LCD.write(0, 0, 'Everything Is Ok')

Dept. of CSE, VKIT 2022-23 Page 36

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

LCD.write(0, 1, '**')
Buzz.start(50)
time.sleep(2)
Buzz.stop()
time.sleep(1)
clearLCD()
def clearLCD():
LCD.write(0, 0, ' ')
LCD.write(0, 1, ' ')
time.sleep(.1)
if __name__ == '__main__':
try:
setup()
loop()
except KeyboardInterrupt:
pass

Dept. of CSE, VKIT 2022-23 Page 37

CHAPTER 6

RESULTS AND DISCUSSIONS

TESTING
Testing is finding out how well something works. In terms of human
beings, testing tells what level of knowledge or skill has been acquired. In computer
hardware and software development, testing is used at key checkpoints in the
overall process to determine whether objectives are being met. For example, in
software development, product objectives are sometimes tested by product user
representatives. When the design is complete, coding follows and the finished code
is then tested at the unit or module level by each programmer, at the component
level by the group of programmers involved and at the system level when all
components are combined together.

System testing is the process of evaluating system’s functionality with its

requirements. These testing process includes three types of testing. The first is the
unit testing where in each module is tested for its functionality. the second is
integration testing, where in the test is made on the modules integrated together
performs the required task. The third is the System testing, where the system as a
whole meets the requirements.

There are several types of testing and few of them are listed below:
• Unit Testing

• Integration Testing

• Functional Testing

• Acceptance Testing

Unit Testing
Unit testing is a level of software testing where individual units/
components of software are tested. The purpose is to validate that each unit of the
software performs as designed. A unit is the smallest testable part of any software.

Dept of CSE, VKIT 2022-23 Page 38

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

It usually has one or a few inputs and usually a single output. In procedural
programming, a unit may be an individual program, function, procedure, etc. In
object-oriented programming, the smallest unit is a method, which may belong to
a base/ super class, abstract class or derived/ child class.The main device we have
used is Raspberry pi which is used to build and run the respective functions. We
are using Raspberry pi because of the compatibility to the ARM processor. The
below image is the testing of the components with Raspberry pi. We get the
indication of working through.

Integration Testing
Integration testing (sometimes called integration and testing, abbreviated
I&T) is the phase in software testing in which individual software modules are
combined and tested as a group. Integration testing is conducted to evaluate
the compliance of a system or component with specified functional
requirements. It occurs after unit testing and before validation testing.
Integration testing takes as its input modules that have been unit tested, groups
them in larger aggregates, applies tests defined in an integration test plan to those
aggregates, and delivers as its output the integrated system ready for system
testing. In proposed system we are integrating camera, headphones and SD card
with power supply. Some of the USB ports are used if necessary.

Functional Testing
A functional test is a form of integration test in which the application is
run "literally". You would have to make sure that an email was actually sent in a
functional test, because it tests your code end to end. It is often considered best
practice to write each type of tests for any given codebase. Unit testing often
provides the opportunity to obtain better "coverage": it's usually possible to
supply a unit under test with arguments and/or an environment which causes all of
its potential code paths to be executed. This is usually not as easy to do with a set
of integration or functional tests, but integration and functional testing provides a
measure of assurance that your "units" work together, as they will be expected to
when your application is run in production.Functional tests are typically written
using the WebTest package, which provides APIs for invoking HTTP(S) requests
to your application. We also use py.test and pytest- cov to provide simple testing
and coverage reports.

Dept. of CSE, VKIT 2022-23 Page 39

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Acceptance Testing
Acceptance Testing is often the final step before rolling out the
application. Usually the end users who will be using the applications test the
application before ‘accepting’ the application. This type of testing gives the end
users the confidence that the application being delivered to them meets their
requirements.

Test Cases
A test case is a set of conditions or variables under which a tester will
determine whether a system under test satisfies requirements or works correctly.
The process of developing test cases can also help find problems in the
requirements or design of an application.

• Writing good test cases includes.

• As far as possible, write test cases in such a way that you test only one thing
at a time.
• Do not overlap or complicate test cases. Attempt to make your test cases
‘atomic’.

• Ensure that all positive scenarios and negative scenarios are covered.

Language:

• Write in simple and easy to understand language.

• Use active voice: Do this, do that.

• Use exact and consistent names (of forms, fields, etc).

Characteristics of a good test case:

• Accurate: Exacts the purpose.

• Economical: No unnecessary steps or words.

• Traceable: Capable of being traced to requirements.

• Repeatable: Can be used to perform the test over and over.

• Reusable: Can be reused if necessary.

Dept. of CSE, VKIT 2022-23 Page 40

 CONCLUSION

In this project we have implemented a face recognition door lock system.

Recognizing of faces is done by using cascade classifiers, which gets a high accuracy
and will store in the database. For this testing, we have used live images only.
Computer vision is used in the IOT. For security purpose, we have implemented real
time face detection by Haar classifier. Time management is one of the advantage that
is taken into consideration in this system. Smart homes aim to facilitate the operation
and management of household appliances so that it can be operated automatically and
optimally. With the identification of appliance usage, a series of smart grid
applications could be carried out such as demand response and load planning. Thus
this system can useful for senior citizens living alone and for immobilized people.
Smart homes aim to facilitate the operation and management of household
appliances so that it can be operated automatically and optimally. With the
identification of appliance usage, a series of smart grid applications could be carried
out such as demand response and load planning. This work presented a framework for
an IoT approach able to support distributed sensing. Hence the proposed system is
practically easy to construct and easy to track the path.

Dept. of CSE, VKIT 2022-23 Page 41

 APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

REFERENCES

[1] Prashanth Balraj Balla, K.T.Jadhao, 2018 International Conference on Smart City and
Emerging Technology (ICSCET) “IoT Based Facial Recognition Security System” 5-5 Jan
2018

[2] Jiang Lu, 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced &
Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing,
Internet of People and Smart City Innovation, “A smart system for face detection with spatial
correlation improvement in IoT environment”

[3] Suraj Pawar, 2018 Fourth International Conference on Computing Communication

Control and Automation (ICCUBEA) “Smart Home Security Using IoT and Face
Recognition”, 25 April 2019

[4] Nashwan Adnan, 2018 6th International Istanbul Smart Grids and Cities Congress and
Fair (ICSG), “A face recognition method in the Internet of Things for security applications in
smart homes and cities”

[5] Narumol Chumuang, 2018 International Conference on Digital Arts, Media and
Technology (ICDAMT), “CCTV based surveillance system for railway station security”, 11
June 2018

[6] Amritha Nag, 2018 3rd International Conference for Convergence in Technology (I2CT)
“IOT Based Door Access Control Using Face Recognition”, 12 November 2018

[7] Shreyak Sawhney, 2019 9th International Conference on Cloud Computing, Data
Science & Engineering (Confluence) “Real-Time Smart Attendance System using Face
Recognition Techniques”, 29 July 2019

[8] Sameer Alani, Sarmad Nozad Mahmood 2021 “IoT based implemented comparison
analysis of two well-known network platforms for smart home automation” IEEE Xplore: 29
July 2021

[9]Junyoung NohSchool, “Internet of Things-based home automation, energy management

and smart security system”, October 9 2019

[10] Meena Kasbekar, “Home Automation Using IoT”, 21 April 2021

Dept. of CSE, VKIT 2022-23 Page 42

 APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

[11] Karma Tshomo, 2019 5th International Conference for Convergence in Technology
“Dual Door Lock System Using Radio-Frequency Identification and Fingerprint
Recognition”, June 06,2020
[12] Majid Al-Kuwari “Smart-home automation using IoT-based sensing and
monitoringplatform”, 07 June 2018
[13] Jinsung Byun, “An intelligent self-adjusting sensor for smart home services
based onZigBee communications”, october 9 2019
[14] Muhammad AhtshamNanjing, “IoT Based Door Lock Surveillance System Using
Cryptographic Algorithms”, May 9 2019
[15] Majid Al-Kuwari, “Smart-home automation using IoT-based sensing and monitoring
platform”, 12 April 2018

[16] K. Lova RajuVignan, 2019 International Conference on Vision Towards Emerging

Trends in Communication and Networking, “Home Automation and Security System with
Node MCU using Internet of Things”, 14 November 2019
[17] Nikolay Valov, 2020 7th International Conference on Energy Efficiency and
Agricultural Engineering (EE&AE) ,“Home Automation System with Raspberry Pi”, 17
December 2020
[18] Sajjadul Islam Nader, 2022 International Conference for Advancement in
Technology, “Cost-efficient Smart Home Automation and Security System based on IoT
and GSM” 10 March 2022
[19] Anubhav Ranjan Dasgupta ,2021 4th International Conference on Recent Trends in
Computer Science and Technology (ICRTCST), “Smart-Home Automation using AI
Assistant and IoT”, 27 May 2022
[20] Plabon Kumar Saha, “IoT Enhanced Smart Door Locking System”, 31 January 2021

[21] A. Z. M. Tahmidul Kabir, 2021 International Conference on Electronics,

Information, and Communication (ICEIC), “IoT based smart homes automation security
system using mobile application with assistant robot for developing countries”, 03
February 2021
[22] Md. Sadad Mahamud , 2019 International Conference on Vision Towards
Emerging,“Domicile -an IoT based, smart home automation system”, 17 November 2020

[23] Harsh Kumar Singh, 2019 Twelfth International Conference on Contemporary

Computing (IC3), “A step towards Home Automation using IOT”, 19 September 2019
[24] Waheb A. Jabbar ,2018 International Symposium on Networks, Computers and
Communications (ISNCC), “Design and Implementation of IoT-Based Automation
Systemfor Smart Home”, 11 November 2018

[25] Hayyan Alsibai, “IOT based facial recognition system for home security using LBPH
algorithm”, 20 September 2019

Dept. of CSE, VKIT 2022-23 Page 43

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

APPENDIX

Dept. of CSE, VKIT 2022-23 Page 44

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

SNAPSHOT

Fig. Hardware setup

Fig. Face Recognition

When face is detected from pi camera door will open and message will go to
owner.

Dept. of CSE, VKIT 2022-23 Page 45

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Fig. Face Recognition

When unknown person is detected alert message will go to owner.

Fig. Fire Detection

When fire is detected the buzzer will ring and alert message will go to owner.

Dept. of CSE, VKIT 2022-23 Page 46

APPROACH FOR LOAD MONITORING AND ACTIVITY RECOGNITION IN SMART HOMES

Fig. Gas Detection

When gas is detected the buzzer will ring and alert message will go to owner.

Fig. Movement Detection

When person enters the home the IR Sensor will detect the movement, fan
and light will on automatically.

Dept. of CSE, VKIT 2022-23 Page 47