2016 International Conference on FrDTT1.

4
Unmanned Aircraft Systems (ICUAS)
June 7-10, 2016. Arlington, VA USA

Implementation of Detection and Tracking Mechanism

For Small UAS
Sai Ram Ganti Yoohwan Kim
Department of Computer Science Department of Computer Science
University of Nevada, Las Vegas, NV, USA University of Nevada, Las Vegas, NV, USA
Email: [email protected] Email: [email protected]

Abstract - Unmanned Aircraft Systems (UAS) are being used situations. It can reliably perform functions such as, motion
commonly for video surveillance, providing valuable video detection with static background, motion tracking
data and reducing the risks associated with human operators. mechanism which will draw a trajectory along the path of
Thanks to its benefits, the UAS traffic is nearly doubling every the drone, feature detection and audio detection that can
year. However, the risks associated with the UAS are also
identify the presence of the drone nearby with a frequency
growing. According to the FAA, the volume of air traffic will
grow steadily, doubling in the next 20 years. Paired with the analysis.
exponential growth of the UAS traffic, the risk of collision is The static background motion detection method [2]
also growing as well as privacy concerns. An effective UAS employs a fixed camera and a single board computer that
detection and/or tracking method is critically needed for air
traffic safety.
processes the frames captured by the camera. The pan and
This research is aimed at developing a system that can tilt mechanism allows keeping track of the flying drone
identify/detect a UAS, which will subsequently enable counter within the frame. The tracking mechanism can be combined
measures against UAS. The proposed system will identify a with any counter measure equipment such as a high energy
UAS through various methods including image processing and RF (Radio Frequency) gun [3].
mechanical tracking. Once a UAS is detected, a
countermeasure can be employed along with the tracking Detection of drones is a challenging task as there are
system. In this research, we describe the design, algorithms, many objects in the air such as birds, clouds, or commercial
and implementation details of the system as well as some airplanes. To detect drones, variety of methods have been
performance aspects. The proposed system will help keep the employs as mentioned below:
malicious or harmful UAS away from the restricted or
residential areas. a) Audio Detection
b) Video Detection
Keywords - UAS (Unmanned Aircraft Systems); Air Traffic c) Thermal Detection
Safety; Detection; Tracking; Automatic Target Recognition;
d) RADAR Detection
Image Processing
e) Radio Frequency Detection
I. INTRODUCTION Each of these methods has its own advantages and
Small Unmanned Aircraft Systems (UASs) or drones drawbacks with varying accuracy. They are described
have many advantages but also bring up privacy concerns. below.
They are very affordable nowadays with a low cost and easy A. Audio Detection
to operate without lengthy training. The drone traffic is
doubling every year, which exponentially increases the risks This technique uses an array of microphones in multiple
and hazards caused by them. UASs flying nearby can be directions and captures the ambient sound. Most of the
disturbing in many ways which include capturing private microphones can pick up sound from 25ft – 30ft. The sound
images or transmitting live video without permission [1]. In waves are then processed and filtered, and the target
the coming years it will be essential to have a technology or frequency is analyzed. Most of the drones are equipped with
a system which can monitor, identify, and keep away the brushless direct current motors which generate a hissing
drones from sensitive areas. However, there has been rather high frequency sound around 40 KHz. This sound is unique
limited researches and developments in this area. for most of the drones. Using digital signal processing, the
specific frequency can be identified and the presence of a
This paper proposes a system for identifying UAS or drone nearby can be determined. This method works in
drones within a limited distance and track them while quieter environments and will fail in urban areas or noisy
providing a platform to mount a 3rd party counter measure environments due to higher noise ratio.
device. Our design is affordable and applicable to many

978-1-4673-9333-1/16/$31.00 ©2016 IEEE 1254

Authorized licensed use limited to: Birla Institute of Technology & Science. Downloaded on June 22,2022 at 13:14:01 UTC from IEEE Xplore. Restrictions apply.
 Fig. 1. Threats from Drones: (a) Private property surveillance, (b) Attack on airlines, (c) Transportation of illegal goods,
(d) Attack on people, (e) Hindrance for emergency operations, (f) Swarm attacks

B. Video Detection D. RADAR Detection

Video detection [4] is a useful tool, but with some RADAR is useful in detecting large aircrafts but fails
limitations. Cameras can see out to about 350ft with a when the size is small like a quad-copter. Radars have a
usable resolution, which is very economical. However, they hard time picking up these small, plastic, electric-powered
have a great difficulty in distinguishing birds from drones. drones because that is not what they were designed for [6].
Basically, anything flying in the air is a “drone” as far as a If this equipment is modified to detect small objects too,
camera is concerned. Even by utilizing computer algorithms then the amount of noise will increase as birds and all other
that look at flight patterns, the prevailing thought is that a flying objects will surface above the threshold. This
bird will fly a more random pattern than a drone would. drawback limits the usage of RADAR for identifying small
Unfortunately, as we have discovered, this notion fails in a drones.
place where birds glide. An excellent example of this is
E. Radio Frequency Detection
seagulls that ride wind currents and stay at a steady level,
which fools video systems. Motion detection combined with This is one of the effective ways for long-range
SURF (Speed Up Robust Features) algorithm can properly detection of drones. Drones communicate with the ground
detect drones while successfully ignoring other flying control station and transmits video image over RF. Such RF
objects. Motion detection can be also useful for drawing the signal can be detected from a long distance, and the
path along the motion of the drone. effective range of 1,400ft can be achieved. It is difficult to
design a drone which can escape RF detection [7]. UAVs
C. Thermal Detection with fixed wing propulsion engines can fly at very high
altitude and they are an exception for this technique. The
Thermal detection works much better on drones that
detection success rate depends on the power of the
have a propulsion engine, mostly on fixed wing drones. It
transmitter and sensitivity of the receiver.
works up to a distance of 350ft offering an economical
solution. [5] Propulsion engines like the turbo-fan or the
II. THREATS AND EXISTING SOLUTIONS
turbo-jet engines generate hot gases from the exhaust which
makes it easy to detect. This method fails, however, when Quad-copters or drones have no physical boundaries
drones contain more amount of plastic or radiate less heat, and can fly to anywhere through open airspaces at high
like most plastic quad-copters with electric motors. In such altitudes. These flying drones possess severe threat in the
cases this mechanism is more likely to consider a bird as a air, as many of the octo-copters are capable of lifting as
drone as it carries more heat. Due to such an unreliability, heavy as 20 lbs payload [8]. They are capable of carrying
this mechanism can be used as an add-on with other weapons and trigger them at the command of the operator.
detection mechanisms rather than standalone. Overall, the This seriousness of the issue increases as the drone traffic
implementation costs for this method are very high, the rate increases. They cause a serious privacy issue as they can
of detection is very low, and the distance is limited. carry powerful cameras that can sneak into every place.
Drones are also a big threat for the commercial airliners.

1255

Authorized licensed use limited to: Birla Institute of Technology & Science. Downloaded on June 22,2022 at 13:14:01 UTC from IEEE Xplore. Restrictions apply.
Single drone entering the engine of an airplane can cause an The increase in the drone traffic every year will also
engine failure and can bring an airplane down in certain increase the rate of aforementioned attacks. There are
conditions. Figure 1 shows possible threats and risks limited number of solutions to prevent the drones from
associated with drones, which are elaborated below.. doing dangerous and /or illegal activities. Companies like
Blighter, Drone Shield, Dedrone have available products but
A. Private Property Surveillance
they are considered not applicable for personal uses in a
Drones equipped with high quality cameras are small scale in residential areas due to their high costs [9].
available at low prices and are accessible to everyone. This Table 1 compares the features of the existing counter-UAS
is a major issue as a drone can easily break through the geo- solutions brielfy. The products and equipment developed by
boundaries. [10] This will require people to use additional these vendors vary in cost. These existing solutions do not
devices to secure their private places. Especially it poses a yield a 100% result, but are capable of detecting a drone to
great threat to military where a drone can spy on their some extent and in a specific environment.
installations. The area owners may declare their land as no Table 1. Comparison of Anti-Drone solutions by various vendors [1]
drone zone. Recently some vendors started using
rudimentary geo-fencing features around airports, but this
will be effective only if all drones comply with it.
B. Attack on Commercial Airlines
This type of attack can be disastrous and can be
responsible for lives of many. A fixed wing drone can reach
heights as much as 15,000ft. It is reported that small quad-
copters such as DJI Phantom 3 can reach an altitude of up to
1,600ft and a distance of 16,000ft [11]. This is a potential
risk for commercial airliners during take off and landing
land. Smaller drones can hover at a height of 1,000ft near III. PROPOSED METHOD
the airports and can cause an engine failure by entering into
the propulsion system of the airplane. A low cost system with a reliable detection rate, video
and audio detection capabilities, and a provision to mount an
C. Transportation of Illegal Goods RF gun will be ideal for personal use in residential areas in a
small scale. We propose a UAS detection and tracking
Depending on the propeller count, a drone’s thrust
system that are simple and cost-effective for residential
capacity is determined. An octo-copter is capable of lifting
areas specifically. The tracking mechanism may
be
20lbs [12]. This is very convenient for criminals to transport
augmented with a 3rd party counter-UAS system such as an
smaller goods or packages in a short distance without being
RF gun to disable its communication.
noticed by anyone.
The proposed design consists of a camera which captures
D. Attacks on Civilians
frames at the resolution of 640 x 480 with a frame rate of
As mentioned, a drone can lift up to 20lbs, thus a 30fps. A single board computer is used to process those
weapon can be mounted underneath and can be triggered at frames to perform image processing operations, where
the controller’s call [13]. This can be lethal in urban areas motion detection is done using the absolute difference by
where people roam around on streets and a drone with a taking two consecutive frames [15]. This difference is then
weapon is just unavoidable. Although there is no such enhanced and tracked continuously over the loop. (Fig. 2)
incident yet, these types of attacks are expected to happen in This tracking mechanism uses a set of pan and tilt servo
the coming decades. FAA is developing policies and mechanism, camera module, and a microcontroller..
regulations for all the drones that will be in the skies, but
due to different types of drones, managing the rules and
regulations can be difficult.
E. Swarm Attacks
Military is one domain which has both advantages and
disadvantages with these types of attacks. Multiple drones
can coordinate and form a net to fight the enemy
combatants. Drones in a pattern can attack an area more
effectively without loss of life. Swarm attacks will help the
military to save the lives of many soldiers who are actually
involved in life-threatening combats [14].. Fig. 2. Design of an UAV tracker [3]

1256

Authorized licensed use limited to: Birla Institute of Technology & Science. Downloaded on June 22,2022 at 13:14:01 UTC from IEEE Xplore. Restrictions apply.
 This system also uses a microphone to capture audio Series of erosion and dilation operations are applied to
signal that is analyzed by a digital signal processor (DSP) the resulting output frame, which helps in filling the gaps
[16]. The analyzed sound undergoes a template matching and forming a bigger blob. This makes it easy to identify an
technique where it is compared with sounds with frequency object as a whole. The erosion and dilation take care of the
around 40 KHz. Drones usually produce sound at this noise which moves along with the main object like the
frequency when the propellers are spinning at a high RPM. shadows or an object smaller than a drone flying nearby.
After the final image is acquired, the center coordinates of
Every frame of the camera image is individually
the blob are taken and fed into variables X and Y. These are
processed using OpenCV library [17], which is open source
the coordinates of the object in 640x480 frames. These
image processing software. They undergo a set of functions
coordinates are sent over the serial port from which the
that reduce the output to be more specific towards the
hardware takes the hand-off. A circle/cross-hair is drawn at
motion detection. The microcontroller is connected to the
the position where the detected object probably should be
main system over a serial port and the camera is also
present. In Fig. 4, the movement of a drone is tracked
connected through a serial port. An application is designed
perfectly and a green color circle is seen on the approximate
in Visual Studio in C++ which handles the entire image
center. Fig 5 shows the values of X and Y being sent over
processing functions and links those to the OpenCV library.
the serial.
The application handles the serial port of the microcontroller
and sends the tracking data of the object in motion. The
coordinates in the frame of 640 x 480 image are sent to the
Arduino board (Microcontroller) one set at a time. It is a two
tuple data of X coordinate and Y coordinate.
There are two status indicators to indicate the presence
of the drone around the equipment. One is a light indicator
that shows green when a drone is hovering within the frame
and red when it is not or it is very far. The second status
indicator is an alarm, which sounds during the presence of a
drone (Fig. 3). This system continuously monitors the video
input for any motion . The status indicators may be disabled
by the users if needed.
Fig. 4. Motion detection with a Static Background

Fig. 3. Status LED changes in the presence of a drone or motion detected

IV. IMPLEMENTATION
This design has software and hardware modules which Fig. 5. Serial values being sent to Arduino
communicate with each other through serial communication. Fig. 6 shows three windows in it. The threshold image
The software part consists of a C++ executable that includes and the difference image are shown above after calibration.
and binds the OpenCV libraries along the execution. This The first one is the final binary output where the white
application reads two consecutive frames from the camera colored blob represents the object in motion. The second
feed. Each frame is converted into Hue, Saturation and image is the absolute difference between two consecutive
Value (HSV) format, and the second frame is subtracted frames. Third image shows the drone itself in motion being
from the first frame. Then the absolute difference is taken detected.
and the leftover pixels after subtraction are represented in
binary. The resulting image is a binary image where the The threshold image is the output of multiple erosion and
white part represents the detected motion or the areas where dilation iterations, which will first remove smaller objects
the pixel values were greatly changed. and then collate larger objects which are close to each other.
The center of the detected white blob is transferred into X

1257

Authorized licensed use limited to: Birla Institute of Technology & Science. Downloaded on June 22,2022 at 13:14:01 UTC from IEEE Xplore. Restrictions apply.
and Y, which immediately is sent over the serial. Arduino
reads the serial buffer continuously and places the variables
by converting the string to integer into X and Y. This
process happens very fast and a check code is also
implemented because, over a period of time, the serial
buffer gets full and the X and Y values are sometimes
swapped, and the movement of the motor is affected. This
check code ensures that the Y value is written to the vertical
axis motor and X value is written to the horizontal axis
motor all times.

Fig. 8. Algorithm for motion tracking and serial communication

The tracking precision is dependent on the distance of the

object. The servo has a precision of 1 degree. This correlates
to a distance from 10ft to100ft at a greater distance. Hence
Fig. 6. Difference Image, Threshold Image, and Final Output using of higher quality gears will enhance the precision [18].
A stepper motor with a step count of approximately 400
Fig. 7 illustrates the implementation of SURF (Speed Up steps per rotation will enhance the precision even at longer
Robust Features) algorithm to identify drones and separate distances. The tracking mechanism and the jitter stay in
them from other objects. SURF can be triggered when control. Applying filters will smooth out the jitters and jerky
motion is detected. Further tracking is resumed only after movement of the servo motor, as the coordinate data is
SURF algorithm identifies the object as a drone. pushed at a greater speed.
This system was built with commercial-off-the-shelf
devices and open source software. As a result the cost of the
system is very low as shown in Table 2.
Table 2. Parts and respective costs

Parts Cost
Web Camera $30
Pan and Tilt Servo $25
Arduino Uno $25
Single board computer $35
TOTAL $115
Fig 7 SURF algorithm being used to identify the flying object
V. PERFORMANCE ANALYSIS
This design performs motion detection primarily on a
cropped image sensor. If a motion is detected, then SURF Our device is capable of identifying drones using
algorithm is applied on the area of detection to distinguish motion, color and audio detection. For motion detection, it
whether the object is drone or any other irrelevant thing. uses a static background as a reference. Motions were
Fig. 8 describes the algorithm. successfully detected, provided the UAS occupies above the
threshold amount of pixels. The number of pixels occupied
by an object in motion is proportional to the size of the

1258

Authorized licensed use limited to: Birla Institute of Technology & Science. Downloaded on June 22,2022 at 13:14:01 UTC from IEEE Xplore. Restrictions apply.
object and inversely proportional to the distance from the
camera [19].Assuming a fixed orientation of the UAS, we
analyzed the number of pixels vs. the distance of the UAS as
shown in Fig. 9. Using this system, any quadcopters flying
in a radius of 200ft can be identified and tracked reliably.

Fig. 11. Image showing the real-time testing of tracking commercial

airplanes

The tracking speed is limited to 33 milliseconds per

frame. Anything that passes the frame within the interval of
33ms will not be captured and cannot be detected. This can
be fixed by increasing the hardware specs and bump the
frame rate to the required target speed accordingly.
Currently we have limited quadcopter images for SURF
[Speed Up Robust Features] algorithm. It can be extended
with variety of drone images. Machine learning techniques
Fig. 9. Graph between distances (Meters.) and Number of Pixels.
can be also used to learn the image patterns of a quadcopter
A 3D-trajectory plotter is also designed for motion or a drone, to speed up the process of detecting, and to
tracking. Based on the angular coordinates of the target reduce the false detection rate. UAS’s path and speed
UAS and the number of pixels, its location within the 3D information can be used in conjunction with machine
space is determined and plotted along the path of the drone’s learning to distinguish a drone from birds and other flying
movement. Fig. 10 shows the drone movement and tracking objects. Audio detection along with video will further
results of sample trajectories [21]. improve the rate of true detection and will ignore the
accidental motion triggers.

VII. CONCLUSION AND FUTURE WORKS

Increased UAS traffic is raising concerns on the air
traffic safety and privacy. To avoid risks associated with
UAS, an effective detection and tracking tool is needed. We
have designed a low-cost system for detecting and tracking
small UASs. The proposed system uses low-cost
commercial-off-the-shelf devices and open source software,
and thus it is cost-effective for personal use in small scale. It
uses image-processing algorithms to detect moving objects
against a static background. The SURF algorithm can
Fig 10 Trajectory of the object which is in motion
reliably differentiate UAS from other objects. The system
can be easily augmented with a 3rd party counter-UAS
Fig. 11 demonstrates a real-world tracking capability of
systems such as an RF gun to jam its communication
the system with a commercial airplane at a distance of
channel. In this paper, we have described the system
~5,000ft approximately. This testing shows the reliability of
architecture, hardware components and software design, and
the system for varying sizes and distances of the drones.
its operating principles. The system has been tested with
quadcopters flying indoors and outdoors as well as
V. LIMITATIONS AND FUTURE WORKS
commercial aircrafts in open air. In all cases, the system has
The proposed system has some shortcomings. The detected the target reliably.
motion detection algorithm needs a static background in
order to work properly. If the whole frame moves, it will be
virtually impossible to extract the moving object. However, REFERENCES
the tracking device is expected to observe the sky in a fixed
[1] Recktenwald, F, Mallot, H.A. “Visual steering of UAV in unknown
position without moving, so it may not pose a problem in environments” Intelligent Robots and Systems, 2009 IROS 2009.
real usage. IEEE, RSJ International Conference.

1259

Authorized licensed use limited to: Birla Institute of Technology & Science. Downloaded on June 22,2022 at 13:14:01 UTC from IEEE Xplore. Restrictions apply.
[2] Monari, E, Pasqual, C, “Fusion of background estimation approaches [20] Hong Liu, Yueliang Qian “A Fast and Robust Pedestrian Detection
for motion detection in non-static backgrounds” Advanced Video and Framework Based on Static and Dynamic Information” Multimedia
Signal Based Surveillance, 2007 IEEE Conference. and Expo (ICME), 2012 IEEE International Conference.
[3] Haimson, J, Korbly, S, Mastovsky, I., Mecklenburg, B. “Initial testing [21] De Laet, De Schutter, “Comparison of rigid body motion
of a field symmetrized dual feed 2 MeV 17 GHz RF gun” Particle trajectory descriptors for motion representation and recognition”
Accelerator Conference, 2003. Robotics and Automation (ICRA), 2015 IEEE International
Conference.
[4] Mathe, K, Tamas, L. Busoniu, L. “Railway track following with the
AR.Drone using vanishing point detection” Automation, Quality and
Testing, Robotics, 2014 IEEE International Conference.
[5] Rees, D. Talha, M. Florescu, I. “Bayesian fusion of thermal and visible
spectra camera data for region based tracking with rapid background
adaptation” Multisensor Fusion and Integration for Intelligent Systems
(MFI), 2012 IEEE Conference.
[6] Hao Ling, “ Synthetic aperture radar imaging using a small consumer
drone” Antennas and Propagation & USNC/URSI National Radio
Science Meeting, 2015 IEEE International Symposium.
[7] Ergun, A.S, Khuri-Yakub, “Improved modeling and design of
microphones using radio frequency detection with capacitive
micromachined ultrasonic transducers” Ultrasonics Symposium, 2001
IEEE.
[8] Numata, T, Iwatani, “Indoor hovering flight of a nano-scale helicopter
by markerless visual servo control” Robotics and Biomimetics
(ROBIO), 2014 IEEE International Conference.
[9] Zelnio, A.M. Rigling, “Low-Cost Acoustic Array for Small UAV
Detection and Tracking” Aerospace and Electronics Conference, 2008.
[10] Korshunov, P. Ramponi, G. Ebrahimi, T, “Privacy in mini-drone based
video surveillance” Automatic Face and Gesture Recognition (FG),
2015 11th IEEE International Conference.
[11] Afzal, U, “Using predictive analytics to forecast drone attacks in
Pakistan” Information & Communication Technologies (ICICT), 2013
5th International Conference.
[12] Guerin, F. Guinand, F “UAV-UGV cooperation for objects
transportation in an industrial area” Industrial Technology (ICIT),
2015 IEEE International Conference.
[13] Gupta, A, Carlsony, S. “MAVR: Code Reuse Stealthy Attacks and
Mitigation on Unmanned Aerial Vehicles” Distributed Computing
Systems (ICDCS), 2015 IEEE 35th International Conference.
[14] Chunming Ye, Zihao Chen “A bee evolutionary particle swarm
optimization algorithm for vehicle routing problem” Information
Technology and Artificial Intelligence Conference (ITAIC), 2011 6th
IEEE Joint International.
[15] Guishan Xiang, “Real-Time Follow-Up Tracking Fast Moving Object
with an Active Camera”, Image and Signal Processing, 2009 CISP '09
2nd International Congress.
[16] Narayanan, S, Kuo, “Environmental Sound Recognition With Time–
Frequency Audio Features” Audio, Speech, and Language Processing,
IEEE Transactions.
[17] Abram, D.Pribanic, T. “A brief introduction to OpenCV” MIPRO,
2012 Proceedings of the 35th International Convention.
[18] Speagle, R.C, “Adaptive tracking control of a permanent magnet
stepper motor driving a mechanical load” Southeastcon '93,
Proceedings., IEEE.
[19] Yuanjie Wang, “A new pedestrian detection algorithm used for
Advanced Driver-Assistance System with one cheap camera”
Mechatronic Sciences, Electric Engineering and Computer (MEC),
Proceedings 2013 International Conference.

1260

Authorized licensed use limited to: Birla Institute of Technology & Science. Downloaded on June 22,2022 at 13:14:01 UTC from IEEE Xplore. Restrictions apply.