A
PROJECT DESIGN REPORT
ON
Android application for real time object detection
using deep learning
For the subject Lab1 Project Phase 1
Submitted in partial fulfillment of the requirement for the award of
Bachelor of Engineering
In
Computer Science and Engineering
Punyashlok Ahilyadevi Holkar Solapur University
By
Name Roll. No. Exam Seat No.
Mr. Tushar Hulle 24 1823590
Mr. Prasanna Khadake 25 1724433
Mr. Faizan Makandar 26 1724661
Mr. Shahid Shaikh 27 1724491
Mr. Sourabh Nare 70 1525348
Under Guidance Of
Prof. V.D.Chavan
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
WALCHAND INSTITUE OF TECHNOLOGY
SOLAPUR - 413006
(2020-21)
CERTIFICATE
This is to certify that the Project entitled
Android application for real time object detection using deep
learning
is
Submitted by
Name Roll. No. Exam Seat No.
Mr. Tushar Hulle 24 1823590
Mr. Prasanna Khadake 25 1724433
Mr. Faizan Makandar 26 1724661
Mr. Shahid Shaikh 27 1724491
Mr. Sourabh Nare 70 1525348
as a part of Project Design Report.
Studying in BE CSE for the subject Lab1 Project Phase 1
(Prof. V.D.Chavan) (Prof. A.R.Kulkarni)
Project Guide Head
Dept of Computer Science & Engg
(Dr. S .A. Halkude)
Principal
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
WALCHAND INSTITUE OF TECHNOLOGY
SOLAPUR
(2020-21)
INDEX
Sr No. Topic Page No.
1 Abstract 4
2 Introduction 5
3 Background 6
4 Technology Required 7
5 Objectives 7
6 Proposed work 8
7 UML Diagrams 9
8 Work planned for next semester 10
9 Conclusion/Summary 10
10 References 11
Abstract
As there is a growing culture of technology in the world. Our project is to
develop an android application for the blind peoples. Our application will guide
and notify them about the things around them. When user go outside for his/her
purpose he can use our app. When user starts our app it will starts the camera of
mobile and start capturing video. Then this video will send to the server. Server
will convert that video frames into an textual data by using object detection
algorithm. And then from textual to audio. Then the audio data will send back to
the user. Then user can able to recognize the things nearer to them. This app is
very useful for them when they walk on the road. There is one app from Google
named as Lookout . But this app will support only on the Google pixel ,Lg and
some Samsung devices. We are trying to develop the android app for all devices
which work faster and take less memory of the mobile devices. We are trying to
make the server online because then app will run on the less ram devices as well.
Introduction
We are focusing on the need of visually impaired users of mobile devices.
The rapid development of information technology has made access to mobile
applications for visually impaired people became difficult. Development of
mobile applications, especially designing the user interface accessible for a blind
person, is a very serious issue and it has become one of the new field of research.
Most of mobile applications available in the Google Play or App Store have
difficult graphical user interface that is impossible to use for visually impaired
people. So we are developing an android application for the visually impaired
peoples. This app will help and guide them to recognize the things nearer to them.
This app has very simple and easy to use graphical interface. When user starts
the app it starts the camera of the device and starts recognizing the things in front
of the them.
Background
It has been observed that a handicapped person does not like the fact that a
helper is is just showing his empathy on his incomplete physique. Hence these are
the instances where technology should abridge this empathy gap by making the
handicapped person self dependent in whatever way possible. This abridging might
help relationships to grow in a healthy way because there is not much dependence
on each other on physical level. In order to bring this thought into reality we had to
analiys the problems faced by visually impaired people when they are outdoors.
The major concern was how to make people trust this technology, as any
malfunction in system can even result to users death.
Basically it’s a audio guide for visually impaired people which would verbally
instruct them of objects coming his way. Blind people mainly have fear of vehicles
which are running on roads, this system is self sufficient to inform about all the
movables in the vicinity of the user. System’s interaction speed has increased
without much effort as less attention is given to visual soothing of the system.
Technologies Required
Dataset:-
COCO dataset
Frontend :-
➢ Android Studio IDE with Java and SDK tools.
➢ The main file or implementation of a program or source file is on android
studio using Java as a source language.
➢ Sdk toolkit for device wise editing purpose.
Backend :-
➢ Python for building deep learning model .
➢ Jupyter Notebook , Pycharm IDE’s for the development purpose.
➢ Cloud for deployment of the model
Objectives
Our work is based on a motive to ease the lives of visually impaired people
who face difficulty in their day to day life. The system is designed to verbally guide
the person by instructing him/her about the objects which comes in his way while
he is outdoor.
The system is optimized to be fast and verbally interactive. Less care has
been taken to improve the visual interface, as the user is blind. Due to system’s less
visual appeal, the system manages to interact with the person at a rate at which he
is walking as the servers won’t have the load of sending highly pixeled web pages.
Proposed Work
Fig.-1
➢ Installation of IDE’s.
➢ Installing all the modules and packages required for training the model.
➢ Collecting COCO dataset.
➢ Studying different object detection algorithms based on their accuracy.
➢ Research on speech synthesizer algorithm.
UML diagrams
1. Activity diagram:-
Fig.-2
2. Use case diagram:-
Fig.-3
Work Planned for Next Semester
➢ Completion of training of the object detection model.
➢ Storing the result of detected object in text format.
➢ Research regarding speech synthesizer through different accuracy
rate.
➢ Training the speech synthesizer model using text data.
➢ Final deployment on cloud and will display the result.
Conclusion/Summary
The system is a verbally interactive arrangement to guide the blind to find
its way in the outdoors.Efforts had been taken to keep the system as interactive as
possible. Time is the main constraint kept in mind while developing this app as
everything depends on how quickly the app makes the person aware of the object
coming its way.
References
Idea : https://brailleworks.com/5-top-mobile-apps-for-the-blind/
https://www.youtube.com/watch?v=gc2_Kva-HeE&t=1s
Object detection : https://machinelearningmastery.com/object-recognition-with-
deep-learning/
https://towardsdatascience.com/object-detection-simplified-
e07aa3830954
Speech synthesizer :
https://journals.indexcopernicus.com/search/article?articleId=1756224
