INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME 9, ISSUE 02, FEBRUARY 2020 ISSN 2277-8616

Iot Based Real Time Potholes Detection System

Using Image Processing Techniques
Kulwant Singh, Sristy Hazra, Chandra Mukherjee, Sushanth.G, Sanjith Gowda

Abstract: Accidents owing to potholes has become an alarming problem in today‘s life. The first step to solve this problem requires, designing a device
embedded on the vehicle which can continuously scan the road surface for identifying potholes, alerting the driver in time and enable the driver to avoid
the pothole. The second step is to introduce a technique to enable the device to locate the position of the pothole via GPS (Global Positioning System).
The GPS data can be uploaded via a GPRS (General Packet Radio Service) module or Bluetooth module onto a data base which is stored locally. This
database can then be transferred to the cloud using WiFi or 4G technology by connecting the system. The third aspect is to link the database to a
network system incorporating mapping software such as Google Maps or Open-Street Map. The data in the system can be made available to the general
public as well as municipalities and road maintenance agencies. Awareness of the location of potholes will help drivers to avoid those roads and being
more careful while driving on the same roads. This paper focuses on the pothole detection task based on image processing algorithms and the data
captured from ultrasonic sensor placed on the vehicle. The later steps were implemented through Bluetooth interface available in smartphones.

Index Terms: Bluetooth communication, GPS, Image Processing, IoT, Real time pothole detection, Sensors, Smartphones, WiFi.
——————————  ——————————

1. INTRODUCTION be used by municipality and road maintenance agencies for

Number of vehicle on the roads has increased enormously in immediate maintenance or repairing of potholes.
the last decade. India being a developing country, construction
and development of infrastructure is still in progress. This ever Designing of the prototype model involved different
growing number of vehicles combined with the lack of electronic components. Those are explained below-
infrastructure makes the road difficult to drive upon eventually A. Required Components
leading to accidents. According to the road accidents in India Processing unit The IoT based real time pothole detection
2017 report [1] of Ministry of Road Transport and Highways system will give prior indication to the driver and store the
(MORTH), the number of accidents that occurred in 2017 location of the potholes. This is accomplished using
alone due to the potholes is a staggering 9423 accidents microcontroller. Sensors The microcontroller is interfaced with
which led to the death of 3597 and injuries to 8792 people only the ultrasonic sensor, LED (for warning) and the Bluetooth
in India. This number takes care of only the reported case and module (for sending the GPS location of the pothole to the
if the unreported cases are also considered then the same database). The proposed system is compatible with both
number may go up to tens of thousands which is an alarming Arduino and Wiznet board. Wireless communication Module
factor. Above stated factors enabled the need of designing The Bluetooth module is used to share the GPS location of the
intelligent vehicles which is able to detect potholes and humps pothole. Bluetooth module is used instead of GPS module to
on road]. Identification of pavement distress such as potholes make it cost efficient as GPS module is thrice the price of a
and humps not only helps drivers to avoid accidents or vehicle Bluetooth module. The microcontroller is also interfaced with a
damages, but also helps authorities to maintain roads. Several WiFi esp8266 module which is used to send the data from the
methods to detect humps and potholes are reported in database to the cloud. The cloud storage can be done either in
published literatures. [2-5] Authors have done mentioned four a private domain or a dedicated server may be used to store
major detection techniques namely vibration based, image the same.
processing based, 3 D image reconstruction based and vision B. Application
based. [6-8] Author proposes use of kinect sensor and smart  Detect the potholes using image processing and alert
phone sensor for the detection of potholes. [9] Illustrates the driver through the blinking of a LED
implication of SVM classification technique for pothole  Store the GPS information about the detected
detection. [10-12]Explain different techniques for detection of potholes in the local database using Bluetooth
potholes and generating an early warning for driver. [13-15]  Send the database to the cloud and share with the
Authors have illustrated methodologies to maintain pavement‘s common public so that municipality agencies or
health data for longer period of time which can later be used related government bodies can track and repair the
for analysis and improvement of road infrastructure by the same
government authorities. This paper proposes a pothole  It can be used by other vehicles also if we can link the
detection system which aims at prior warning to the drivers database to mapping software such as Google Maps
about the uneven roads and potholes on the way. This or Open Street Map
information gets updated in cloud immediately which can also

_______________
 Chandra Mukherjee is working as an Assistant Professor in the
Department of ECE, CHRIST (Deemed to be University
E-mail: [email protected]
 Sushanth.G is working as an Assistant Professor in the Department
of ECE, CHRIST (Deemed to be University
E-mail: [email protected]

785
IJSTR©2020
www.ijstr.org
 INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME 9, ISSUE 02, FEBRUARY 2020 ISSN 2277-8616

Fig 2: Overall flowchart of the process

 Use of ultrasonic sensor will be the other way to

detect the pothole. This method is shown in Fig2. The
Fig 1: Overall Block diagram of Pothole detection system ultrasonic sensor will detect the irregularity in the
road, and after confirming that it is a pothole, will send
2 METHODOLOGY an alarm to the driver through the blinking of a LED.
2. Once the pothole is detected, its location will be stored in
The methods followed in this work are as explained the database locally. GPS coordinates of the pothole will
below: be obtained from the driver‘s phone (which will be
1. The system comprises of a camera which can take interfaced with the sensor using a Bluetooth module.
snapshots in real time and send the captured images to 3. These coordinates will then be updated to the server
MATLAB which is stored in a remote server using Wi-Fi or which will have a an entire database of potholes. This
4G/5G. There are 2 ways in which the pothole will be information can be used by the municipal organization or
detected: government agencies to repair the same.
 Photos of the road will be captured using a camera. 4. The database will store the location of the potholes
These will undergo image processing using MATLAB, detected on the way. This collected data can be used for
where the detection and verification of the presence various applications.
of pothole will be done. If pothole is present, the driver 5. The database stores the location of the potholes detected
will be notified /warned with the blinking of LED. on the way and alerts the other driver travelling on same
the way i.e it will be shown in GPS. This information can
also be used by municipality people for immediate
maintenance or repairing

3 EXPERIMENTAL SETUP AND RESULTS

This work is implemented using two different approaches. the
details of that approaches are explained below in detail

A. Pothole detection using MATLAB image processing

A camera is mounted in the vehicle which captures the
images. The image captured in real time is sent to a MATLAB
code which is running on a remote server through Wi-Fi or
4G/5G. In MATLAB, the received image is converted to its
binary form. Then a database of different images considering
the depth and width of the Pothole is made and stored in
binary form. If the binary value of captured image match with
the binary values of the images stored in the database, then it
is concluded as a pothole and after this, a notification in the
form of an alert message will be provided to the driver by
blinking of a LED

786
IJSTR©2020
www.ijstr.org
 INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME 9, ISSUE 02, FEBRUARY 2020 ISSN 2277-8616

Fig 3: Original image considered for detection of Pothole

Fig 6: Threshold detection for pothole identification

Fig 4: Gradient Magnitude computation of original image

Fig 3 shows the original image considered for pothole

detection. This image is then taken to MATLAB and the
gradient magnitude of the image is computed which is as
shown in Fig4. In MATLAB, the function imgradient() returns
the gradient magnitude of an image. This is then further
processed to compute the regional maxima. The function
imregionalmax() returns this regional maximum values of an Fig 7: Pothole identification by considering shape and
image and this requires the greyscale component of the dimension
original image. Fig 5 shows the regional maxima computed for
the image shown in fig 3. This work proposes an image processing based pothole
identification mechanism by considering the shape and
dimension of holes present in captured image. The algorithm
firsts assigns a fixed threshold for segmenting captured
images into two class namely- a. portion with potential
pothole‘s presence b. non defective portion. Next probable
pothole sections are exploited and experimented with different
geometrical approximation techniques to extract the
geometrical property of the region. This algorithm takes
dimension and shape into consideration for detecting the
pothole. This algorithm is implemented through MATLAB
image processing toolbox and images were acquired from the
camera fixed on the front side of the vehicle. Fig. 7 shows the
identified pothole using proposed algorithm.
Fig 5: Computation of Regional Maxima

The image of which regional maxima is computed is then

processed further by opening by morphological reconstruction.
This processing method involves processing an image
according to the arrangement element where the image is
contracted by deleting the objects which are smaller than the
required shape. In opening by reconstruction, the image
contracting is then followed by an image expansion where the
image then regrows the objects which are left out after deletion
of the other objects to the same shape as required. This
processed image is as shown below in Fig 6.

787
IJSTR©2020
www.ijstr.org
 INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME 9, ISSUE 02, FEBRUARY 2020 ISSN 2277-8616

Fig 9: Experimental setup using Ultrasonic sensor

Fig 8: Pothole identification message on MATLAB screen

Fig 8 presents the screenshot of MATLAB screen

demonstrating the output of above mentioned image
processing algorithm. Developed algorithm incorporates driver
notifying and alerting system as well. It notifies the driver
through audio and visual message in order to avoid accidents
due to pothole.

B. Hardware setup for pothole detection

This research work demonstrates real time implementation of
image processing based pothole detection algorithm through a
simple and low cost hardware setup. Hardware setup included
ultrasonic sensor, camera, GPS module, Bluetooth module, Fig 9: Hardware setup
camera and Arduino microcontroller board. A digital camera is
fixed at the front side of the vehicle which captures image in
real time and sends to Arduino microcontroller for further 5 CONCLUSION
processing through Bluetooth module. The hardware setup The proposed system basically serves two purposes, one it is
has two parts namely- i. Sensor ii. Controller Unit. Sensing unit able to identify the potholes and humps on the road and sends
has camera and ultrasonic sensors fixed on front side of the the information regarding this to the vehicle drivers. This will
vehicle to acquire inputs required for identification of pothole. help to avoid accidents and will save valuable lives. Second, the
Ultrasonic sensors send pulsed ultrasonic waves of a certain location of the pothole is saved in the database. This is an
frequency and determine the objects distance from the effective solution for detection of humps and potholes. This
duration of the ultrasound that it reflects. This sensor consists system is effective even in rainy season when roads are flooded
of a transmitter and a receiver unit. Depending on the distance with rain-water as well as in winter during low visibility, as the
we can identify the potholes and humps on the road. The alerts are sent from the stored information in the
distance should be more for potholes and less for humps server/database and helps us to avoid dreadful potholes and
compared to the normal distance between the road and the humps and hence to avoid any tragic accidents due to bad road
sensor. The sensor can be interfaced with Arduino or Wiznet conditions. The information can also be used by the
board. Images captured by the camera are acquired using Government authorities for the maintenance of the roads. The
Arduino and is sent to MATLAB using serial communication for proposed system will give a voice alerts such as ‗Bad road
detection of potholes using image processing. A smart phone ahead‘ along with led blinking in order to help the driver be more
application is also made along with the hardware design which alert while driving/riding on such roads. The database could be
fetches the exact location of the potholes from GPS unit and linked to a network system incorporating mapping software such
updates to cloud server. as Google Maps or Open-Street-Map in future.

REFERENCES
[1] Ministry of Road Transport and Highways, Government of India,
―ROAD ACCIDENTS IN INDIA – 2017‖. [Online]. Available:
http://www.indiaenvironmentportal.org.in/files/file/road%20accide
nts%20in%20India%202017.pdf. [Accessed: 03-Dec.-2019].
[2] Rajeshwari Madli, Santosh Hebbar, Praveenraj Pattar, and
Varaprasad Golla, ―Automatic Detection and Notification of
Potholes and Humps on Roads to Aid Drivers‖, IEEE Sensors
J., Vol. 15, No. 8, August 2015.
[3] Taehyeong Kim, Seung-Ki Ryu, ―Review and Analysis of
Pothole Detection Methods‖ vol. 5, no. 8, pp.603-608, Aug.
2014.
[4] H. Youquan, W. Jian, Q. Hanxing, Z. Wei, and X. Jianfang, ―A
research of pavement potholes detection based on three-
788
IJSTR©2020
www.ijstr.org
 INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME 9, ISSUE 02, FEBRUARY 2020 ISSN 2277-8616

dimensional projection transformation,‖ in Proc. 4th Int. Congr.

Image Signal Process (CISP), Oct. 2011, pp. 1805–1808.
[5] Seung-Ki Ryu, Taehyeong Kim, Young-Ro Kim, ―Image-Based
Pothole Detection System for ITS Service and Road
Management System‖, Mathematical Problems in Engineering,
vol.. 2015, 2015.
[6] I. Moazzam, K. Kamal, S. Mathavan, S. Usman, and M.
Rahman, ―Metrology and visualization of potholes using the
microsoft Kinect sensor,‖ in Proc. 16th Int. IEEE Conf. Intell.
Transp. Syst., Oct. 2013, pp. 1284–1291.
[7] Gunjan Chugh, Divya Bansal and Sanjeev Sofat, ―Road
Condition Detection Using Smartphone Sensor: A Survey‖,
International Journal of Electronic and Electrical Engineering,
Vol. 7, No. 6, 2014.
[8] A. Mednis, G. Strazdins, R. Zviedris, G. Kanonirs, and L. Selavo,
―Real time pothole detection using Android smartphones with
accelerometers,‖ in Proceedings of the International Conference
on Distributed Computing in Sensor Systems and Workshops
(DCOSS '11), pp. 1–6, IEEE, Barcelona, Spain, June 2011.
[9] J. Lin and Y. Liu, ―Potholes detection based on SVM in the
pavement distress image,‖ in Proc. 9th Int. Symp. Distrib.
Comput. Appl. Bus. Eng. Sci., Aug. 2010, pp. 544– 547.
[10] Madhumathy P, Saurabh Singh, Shivam Shukla, Unni Krishnan,
―Detection of humps and potholes on roads and notifying the
same to the drivers,‖ International Journal of Management and
Applied Science, vol. 3, no. 1, 2017.
[11] Kwok Yu Mak, Valentin Siderskiy, Lamia Iftekhar, ―Detachable
pothole detection and warning system‖, ME thesis, NYU Tandon
School of Engineering Polytechnic University, Sanghai, 2008.
[12] R. Sundar, S. Hebbar, V. Golla, "Implementing intelligent traffic
control system for congestion control ambulance clearance and
stolen vehicle detection", IEEE Sensors J., vol. 15, no. 2, pp.
1109-1113, Feb. 2015.
[13] K. C. P. Wang, ―Challenges and feasibility for comprehensive
automated survey of pavement conditions,‖ in Proceedings of
the 8th International Conference on Applications of Advanced
Technologies in Transportaion Engineering, pp. 531–536, May
2004.
[14] J. S. Miller and W. Y. Bellinger, ―Distress identification manual for
the long-term pavement performance program,‖ FHWA RD-03-
031, Federal Highway Administration, Washington, DC, USA,
2003.
[15] S. Vijay, Low cost—FPGA based system for pothole detection
on Indian roads [M.S. thesis of Technology], Kanwal Rekhi
School of Information Technology, Indian Institute of Technology,
Mumbai, India, 2006.

789
IJSTR©2020
www.ijstr.org