6. Autonomous_UAV_Flight_Control_for_GPS-Based_Navigation_2
6. Autonomous_UAV_Flight_Control_for_GPS-Based_Navigation_2
ABSTRACT This paper proposes an unmanned aerial vehicle (UAV) flight control method where a graph-
based path is generated after the collected UAV flight data by a pilot are analyzed. UAV flights are planned
by using hierarchical A∗ search algorithms based on graph-based generated flight paths to take images at
multiple surveillance points. Generating a graph-based path makes it possible for UAVs to fly autonomously
along paths shorter than that of the pilot collecting UAV flight data given that the shorter paths can be derived
by connecting partially flied paths. A∗ search algorithms can be applied hierarchically to a graph-based path
that contains circulation paths. The proposed method was experimentally verified through an analysis of
the collected UAV flight data to generate graph-based and planned paths. The pilot flew the UAV six times
and obtained 8115 UAV flight data points. The generated graph-based path included 17 monitoring points
for taking surveillance images and 90 intermediate flight points. The length of the flight paths collected by
six time flights was 1364.32 m, and the length of the flight paths by the proposed method was 764.27 m.
Given that 8115 flight points were collected and 109 flight points were selected by the proposed method,
the complexity of the generated graph-based path consisted of flight points was reduced to 1.34% by
hierarchical A∗ search algorithms.
2169-3536 2018 IEEE. Translations and content mining are permitted for academic research only.
VOLUME 6, 2018 Personal use is also permitted, but republication/redistribution requires IEEE permission. 37947
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
J. Kwak, Y. Sung: Autonomous UAV Flight Control for GPS-Based Navigation
flight path between two imaging points is derived by using the vehicles [32]. During the path planning process, if an obstacle
A∗ search algorithm. The term ‘‘image’’ refers to an image is encountered, the wrong path is eliminated, and another path
that captures a surveillance point. Finally, the derived flight is searched for. Another method to planning the UAV path
paths are combined by considering all imaging points, and the uses spline-RRT∗ in a 3D environment [33]. The problem
A∗ search algorithm is used to plan the optimal flight path of is solved by using the cubic Bezier curve method to apply
the UAV. the spline method in a 3D environment. However, a 3D flight
This paper is organized as follows. Section 2 introduces environment should be constructed.
related work. Section 3 presents the proposed method to col- One path planning method is based on the Laguerre
lect flight-related data with a UAV and use the data to gen- graph [34]. The obstacle environment is planned with the
erate flight paths. It also presents a proposed method to Laguerre graph. Preplanned paths are used to generate paths
plan flight paths that pass through multiple imaging points. to the target. The Laguerre graph can be used to optimize
Section 4 validates the proposed method to plan multiple the time to find a path. However, it is difficult to use if
flight paths for monitoring the ground. Finally, Section 5 con- there is no obstacle point information. The path planning time
cludes this paper. can also be optimized [35] by adjusting the heuristic value
according to the size of the map to be investigated. However,
II. RELATED WORKS adjusting the heuristic value can increase the cost of the flight
There are three approaches to planning flight paths for path.
UAVs: waypoint-based, simulation-based, and using pilot In order to plan flight paths through a simulation, pilots
flight records. Research on each approach is presented below. should be able to build the simulation environment easily.
A method is needed for planning flight paths by using infor-
A. FLIGHT PATH PLANNING BASED ON WAYPOINTS mation of the built environment.
Pilots sequentially input the waypoints that contains GPS
locations and the heights of the UAV flight [11], [12]. The
UAV flight paths are planned by setting the waypoints one C. FLIGHT PATH PLANNING BASED ON PILOT
by one. The UI is usually complicated so that pilots can plan FLIGHT RECORDS
detailed settings such as imaging points and directions where Flight paths are planned with the flight information recorded
the UAV should monitor. by a pilot flying the UAV [22]. The captured images are
The flight paths are planned by setting the commands analyzed to plan the flight paths. However, incorrect flight
that the UAV should perform at specific markers [14]. It is paths caused by pilot error during the recording process can
possible for a UAV to set flight paths and fly without any be planned.
equipment for measuring its location. However, the set com- Methods for making a UAV fly based on a single flight
mands are not executed if the markers are not recognized. record by a pilot include DJI [36], Agras MG-1S, and
Unattended ground sensors (UGSs) are deployed in Matrice 210 provided by DJI. The UAV follows the points
advance in the flight area and used to plan the flight paths of of the flight record. However, if a flight is based on a single
the UAV [15]. The UAV flies according to the UGS sequence flight record that contains pilot errors, the UAV can follow
planned by pilots. The UAV flight path can be planned with- the wrong path not equivalent to the expected path. Therefore,
out measuring the UAV’s location by arranging the UGSs in the optimal path should be drawn after multiple flight paths
a sequence. However, it is difficult to specify the flight path are collected.
or set the flight height for each UGS. DJI creates a 3D map based on images taken during flight
In order to plan a flight path based on waypoints, a method that can easily be used to simulate a UAV flight without
is needed that accounts for the surveillance points to be shot considering the cost. However, given the difference between
without directly considering the UAV environment. real and 3D environments, the simulation result should be
enhanced before utilization.
B. FLIGHT PATH PLANNING BY SIMULATION Flight paths to shoot surveillance points are generated
In a simulation, the genetic algorithm is repeatedly executed by analyzing the flight records of a UAV controlled by a
to generate flight paths [17], [18]. An optimal flight path is pilot [25]. However, an approach to derive an optimum flight
planned, while the flight height is adjusted so that the UAV path for imaging multiple surveillance points is needed.
can shoot objects. However, building a simulation environ- Path planning methods are required without the need to
ment can be costly, and the flight paths generated by the directly specify the imaging points to be monitored by a
simulation may not be optimal in a real environment. pilot to reduce the time of the path generation, without the
Research on planning an optimum flight path has consid- construction of a 3D environment to reduce the cost of path
ered setting the shooting positions of the UAV and environ- generation, and with multiple flight paths rather than a single
mental factors such as obstacles [19]. UAVs can fly without flight path to reflect various dynamic situations. The collected
collision. One path-planning approach combines the rapid- multiple flights are analyzed to create a flight path along the
exploring random trees (RRTs) algorithm [30], [31] with the imaging points is created. The flight path is planned without
B-spline algorithm to create a smooth path for automatic land specifying the order of imaging points.
III. UAV GRAPH GENERATION AND optimal flight path. First, the flight data generated from
PLANNING APPROACHES takeoff point B where the pilot starts flying the UAV to
Pilots can intuitively train possible UAV flight paths by landing point L where the pilot finishes flying are recorded
directly controlling the UAV. This paper introduces a method in the record phase. In particular, the pilot specifies multiple
for analyzing big data to represent possible flight paths and a surveillance points to be shot and monitored by the UAV
method for generating and planning UAV paths. during the flight. An imaging point is where a pilot captures
images containing a surveillance point while controlling a
A. AUTONOMOUS UAV CONTROL PROCESS UAV and where GPS and other data are collected. A non-
Figure 1 shows the process by which a pilot trains multiple imaging point is where only GPS and other data are collected.
possible flight paths with a single UAV and derives an optimal The process of collecting a single flight record is shown
flight path. First, the flight data generated from takeoff point in Figure 2. Pilots set imaging and non-imaging points to
B, where the pilot starts flying the UAV, to landing point L, monitor surveillance points while flying the UAV from take-
where the pilot finishes flying, are recorded in the record off point B to landing point L. Flight data are recorded at
phase. In particular, the pilot specifies multiple surveillance each point, including the GPS, flight height, and flight angle
points to be shot and monitored by the UAV during the flight. of the UAV. The jth imaging point Ii,j of the ith flight is
The imaging point is where a pilot captures images containing [i, si,j , Pi,j , Ai,j ].
a surveillance point while controlling a UAV and where GPS
and other data are collected. A non-imaging point is where
only GPS and other data are collected.
FIGURE 1. UAV control process. The UAV paths are generated and planned
in the record phase, graph generation phase, and path planning phase.
FIGURE 2. Pseudocode for collecting image and non-imaging points to
Next, the graph generation phase generates graph G as generate one flight record.
multiple paths from analysis of the big data collected during
the record phase. The non-imaging points that UAV passes The flight point sequence number si,j includes imaging
through during the flight are clustered with the K-means and non-imaging points from takeoff point B. Point Pi,j is
algorithm [29]. The imaging points where the UAV shoots the flight position [x, y, z]. Angle Ai,j is the angle [φ, θ , ϕ]
surveillance points are clustered with the bounded K-means of the camera mounted on the UAV during the flight. The
algorithm [24]. Nodes corresponding to non-imaging points k th non-imaging point Ni,k collected during the ith flight is
and imaging points are generated. The edges connecting the [i, si,k , Pi,k ].
nodes are generated by deciding the flight between corre- The four flight records collected during the record process
sponding points. Finally, flight paths based on the nodes are are shown in Figure 3. During the flight from the takeoff
generated with graph G. point B to the landing point L, four surveillance points are
In the path planning phase, the optimal flight path is monitored.
planned from takeoff point B of graph G through the nodes
corresponding to imaging points to landing point L. The C. GRAPH GENERATION PHASE
shortest distance between each imaging point node is found Based on the collected flight-related big data, graph G is
with the A∗ search algorithm. Thus, the optimal flight path derived through an analysis of the possible flight paths of the
to fly through all imaging points from takeoff point B to UAV. The number of clustering of the bounded K-means K I ;
landing point L is found by repeatedly applying the A∗ search number of clustering of the K-means K N ; point weights ωx ,
algorithm. ωy , and ωz ; angle weights ωφ , ωθ , and ωϕ ; and permissible
error range δ are set. Table I presents the maximum and
B. RECORD PHASE minimum x, y, and z values of points Pi,j and Pi,k . Table II
Figure 1 shows the process by which a pilot trains multi- presents the maximum and minimum φ, θ , and ϕ values of
ple possible flight paths with a single UAV and derives an angle Ai,j .
IV. EXPERIMENTS
In an experiment, a UAV flight path was planned to monitor
intruders on the grounds of Keimyung University. Inspire
made by DJI was utilized. The data of Inspire was collected
by a smartphone connected to the controller. In the record
phase, a pilot collected flight data by imaging the university
facilities and fences around the grounds. In the graph gen-
eration phase, the flight-related big data were analyzed to
derive a possible UAV flight path. In the path planning phase,
the flight path was planned for the UAV to monitor intruders
crossing the grounds.
FIGURE 8. Collected flight data monitoring intruders crossing the ground
fence. (a) Three of the six collected flight records. (b) Three of the six
A. RECORD PHASE collected flight records.
Seventeen surveillance points were selected to monitor
intruders crossing the grounds of Keimyung University. Sev-
enteen surveillance points were set as shown in Figure 8:
two baskets in the basketball court, two goals in the soccer
field, five grandstand areas, five clubroom entrances, and
three fences. The pilot flew the UAV six times and obtained
8,115 UAV flight data points. Figure 8(a) shows the collected
flight records for shooting the basketball court and fences,
clubroom, and seats. Figure 8(b) shows the collected flight
records for shooting the basketball court and soccer field;
clubroom and seats; and basketball court, soccer field, and
clubroom. The length of the flight path collected on six time
flights was 1,364.32m.
V. CONCLUSIONS
A flight path is derived by analyzing flight-related big data to
shoot the surveillance points. This paper proposes a method
to plan a flight path that shoots surveillance points. Flight-
related data are collected to shoot the surveillance points.
Imaging points are clustered from the flight-related data
by using the bounded K-means algorithm and generated
imaging nodes. Non-imaging points are clustered by using
the K-means algorithm and generated non-imaging nodes.
In order to connect the takeoff node, landing node, imaging
nodes, and non-imaging nodes, edges are generated by ana-
lyzing the continuity of the flight-related big data. Nodes and
edges are connected to generate a flight path. The flight path
FIGURE 12. Flight path connected with any one imaging node. to each node is planned by using the A∗ search algorithm.
The flight path for shooting the surveillance points from
the takeoff node to the landing node is additionally planned
The collected 8,115 flight points were reduced to by using the A∗ search algorithm based on the flight path
109 nodes: a takeoff node, a landing node, and imaging and between the planned nodes.
non-imaging nodes. In the experiment, six flight paths from takeoff point B
to landing point L were collected to monitor the facilities
C. PATH PLANNING PHASE and intruders on the grounds of Keimyung University. Sev-
A flight path was planned to shoot the clubroom and seats, enteen surveillance points were set during the process of
soccer field, basketball court, and fence during the flight from collecting six flight paths, and seventeen imaging nodes were
set to shoot each point. The six flight paths had 109 nodes [17] L. Geng, Y. F. Zhang, J. J. Wang, J. Y. H. Fuh, and S. H. Teo, ‘‘Mission
representing image nodes, non-image nodes, takeoff nodes, planning of autonomous UAVs for urban surveillance with evolutionary
algorithms,’’ in Proc. 10th IEEE ICCA, Hangzhou, China, Jun. 2013,
and landing nodes. The generated graph included 17 imag- pp. 828–833.
ing nodes, 90 non-imaging nodes, the takeoff node, and the [18] L. Geng, Y. F. Zhang, P. F. Wang, J. J. Wang, J. Y. H. Fuh, and S. H. Teo,
landing node. The flight paths connecting imaging nodes, the ‘‘UAV surveillance mission planning with gimbaled sensors,’’ in Proc. 11th
IEEE ICCA, Taichung, Taiwan, Jun. 2014, pp. 320–325.
takeoff node, and the landing node were planned with the A∗ [19] F. Yan, Y. Zhuang, and J. Xiao, ‘‘3D PRM based real-time path planning
search algorithm. During the flight from the takeoff node to for UAV in complex environment,’’ in Proc. IEEE Int. Conf. ROBIO,
the landing node, the flight path through the imaging nodes Guangzhou, China, Dec. 2012, pp. 1135–1140.
[20] S. Nurmaini and A. Zarkasi, ‘‘Simple pyramid RAM-based neural network
was planned with the A∗ search algorithm. For the final flight architecture for localization of swarm robots,’’ J. Inf. Process. Syst., vol. 11,
path, the destination was reached after 100 imaging and non- no. 3, pp. 370–388, 2015, doi: 10.3745/JIPS.01.0008.
imaging nodes were passed. The results were verified with [21] Y. Luo, O. Hoeber, and Y. Chen, ‘‘Enhancing Wi-Fi fingerprinting for
indoor positioning using human-centric collaborative feedback,’’ Hum.-
the A∗ search algorithm. Centric Comput. Inf. Sci., vol. 3, p. 2, Dec. 2013, doi: 10.1186/2192-1962-
3-2.
[22] A. Pfrunder, A. P. Schoellig, and T. D. D. Barfoot, ‘‘A proof-of-concept
REFERENCES demonstration of visual teach and repeat on a quadrocopter using an alti-
[1] P. Fahlstrom and T. Gleason, Introduction to UAV Systems, 4th ed. tude sensor and a monocular camera,’’ in Proc. Can. Conf. CRV, Montreal,
Hoboken, NJ, USA: Wiley, 2012. QC, Canada, 2014, pp. 238–245.
[2] S. Yahyanejad, D. Wischounig-Strucl, M. Quaritsch, and [23] M. N. Nicolescu and M. J. Mataric, ‘‘Natural methods for robot task
B. Rinner, ‘‘Incremental mosaicking of images from autonomous, learning: Instructive demonstrations, generalization and practice,’’ in Proc.
small-scale UAVs,’’ in Proc. 7th IEEE Int. Conf. AVSS, Boston, MA, USA, 2nd Int. Joint Conf. AAMS, Melbourne, VIC, Australia, Aug. 2003,
Aug./Sep. 2010, pp. 329–336. pp. 241–248.
[3] W. Shin, E. Kwon, K. Cho, H. Lee, and Y. Lee, ‘‘Implementation of [24] Y. Sung, J. Kwak, and J. H. Park, ‘‘Graph-based motor primitive
unmanned aerial vehicle sensor-based smart eye technology for local forest generation framework: UAV motor primitives by demonstration-based
fire monitoring and situational response,’’ in Proc. Spring Conf. KIPS, learning,’’ Hum.-Centric Comput. Inf. Sci., vol. 5, p. 35, Dec. 2015,
Seoul, South Korea, 2016, pp. 704–705. doi: 10.1186/s13673-015-0051-0.
[4] S. Kim, W. Lee, Y.-S. Park, H.-W. Lee, and Y.-T. Lee, ‘‘Forest fire mon- [25] J. Kwak and Y. Sung, ‘‘Structure design of surveillance location-based
itoring system based on aerial image,’’ in Proc. 3rd Int. Conf. ICT-DM, UAV motor primitives,’’ Korea Inf. Process. Soc. Trans. Softw. Data Eng.,
Vienna, Austria, 2016, pp. 1–6. vol. 5, no. 4, pp. 181–186, 2016, doi: 10.3745/KTSDE.2016.5.4.181.
[5] L. Merino, F. Caballero, J. R. Martínez-de-Dios, I. Maza, and A. Ollero, [26] J. Kwak, Y. Sung, and J. H. Park, ‘‘Unmanned aerial vehicle flight point
‘‘An unmanned aircraft system for automatic forest fire monitoring and classification algorithm based on symmetric big data,’’ Symmetry, vol. 9,
measurement,’’ J. Intell. Robot. Syst., vol. 65, no. 1, pp. 533–548, 2012, no. 1, pp. 1–19, 2017, doi: 10.3390/sym9010001.
doi: 10.1007/s10846-011-9560-x. [27] J. Kwak and Y. Sung, ‘‘Flight path generation method based on UAV flight
[6] C. S. Cho, J. S. Yoon, and P. G. Park, ‘‘Development of unmanned fire record,’’ in Proc. Fall Conf. KIPS, Busan, South Korea, 2016, pp. 644–647.
detection system using AR.Drone,’’ in Proc. Fall Conf. KICS, Seoul, South [28] P. E. Hart, N. J. Nilsson, and B. Raphael, ‘‘A formal basis for the
Korea, 2015, pp. 417–418. heuristic determination of minimum cost paths,’’ IEEE Trans. Syst. Sci.
[7] S. Choi, K. Ban, K. Park, and E. Kim, ‘‘Design of structural safety Cybern., vol. SSC-4, no. 2, pp. 100–107, Jul. 1968, doi: 10.1109/TSSC.
inspection system using micro-UAV,’’ in Proc. Fall Conf. KIICE, Seoul, 1968.300136.
South Korea, 2013, pp. 466–469. [29] J. MacQueen, ‘‘Some methods for classification and analysis of multi-
[8] M. A. Ma’Sum et al., ‘‘Simulation of intelligent unmanned aerial vehi- variate observations,’’ in Proc. 5th Berkeley Symp. Math. Stat. Probab.,
cle (UAV) For military surveillance,’’ in Proc. Int. Conf. ICACSIS, Bali, Berkeley, CA, USA, 1966, pp. 281–297.
Indonesia, 2013, pp. 161–166. [30] D. Lee, H. Song, and D. H. Shim, ‘‘Optimal path planning based on
[9] F. Rafi, S. Khan, K. Shafiq, and M. Shah, ‘‘Autonomous target following by spline-RRT∗ for fixed-wing UAVs operating in three-dimensional environ-
unmanned aerial vehicles,’’ Proc. SPIE, vol. 6230, p. 623010, May 2006, ments,’’ in Proc. 14th Int. Conf. ICCAS, Gyeonggi, South Korea, 2014,
doi: 10.1117/12.667356. pp. 835–839.
[10] U. Hashmi, F. Afshan, and M. Rafiq, ‘‘Performance analysis of different [31] Y. Lin and S. Saripalli, ‘‘Sampling-based path planning for UAV col-
optimal path planning bug algorithms on a client server based mobile lision avoidance,’’ IEEE Trans. Intell. Transp. Syst., vol. 18, no. 11,
surveillance UGV,’’ in Proc. 4th Int. Conf. ISMS, Bangkok, Thailand, 2013, pp. 3179–3192, Nov. 2017, doi: 10.1109/TITS.2017.2673778.
pp. 30–36. [32] E. Shan, B. Dai, J. Song, and Z. Sun, ‘‘A dynamic RRT path planning
[11] L. V. Santana, A. S. Brandão, and M. Sarcinelli-Filho, ‘‘Outdoor waypoint algorithm based on B-spline,’’ in Proc. 2nd Int. Symp. ISCID, Hunan,
navigation with the AR.Drone quadrotor,’’ in Proc. Int. Conf. ICUAS, China, 2009, pp. 25–29.
Denver, CO, USA, 2015, pp. 303–311. [33] K. Lee, H. Kim, and H. J. Kim, ‘‘Planning and control for collision-free
[12] L. V. Santana, A. S. Brandão, and M. Sarcinelli-Filho, ‘‘An automatic flight cooperative aerial transportation,’’ IEEE Trans. Autom. Sci. Eng., vol. 15,
control system for the AR.Drone quadrotor in outdoor environments,’’ in no. 1, pp. 189–201, Jan. 2018, doi: 10.1109/TASE.2016.2605707.
Proc. Workshop RED-UAS, Cancun, Mexico, 2015, pp. 401–410. [34] Z. Xu, R. Wei, K. Zhou, Q. Zhang, and R. He, ‘‘Laguerre graph self-
[13] D. Kim and S. K. Hong, ‘‘Target pointing and circling of a region of interest optimize path planning algorithm for UAVs in irregular obstacle envi-
with quadcopter,’’ Int. J. Appl. Eng. Res., vol. 11, no. 2, pp. 1082–1088, ronment,’’ in Proc. IEEE Conf. CGNCC, Nanjing, China, Aug. 2016,
2016. pp. 1076–1079.
[14] M. Saska, T. Krajnik, and L. Pfeucil, ‘‘Cooperative µUAV-UGV [35] P. Sudhakara and V. Ganapathy, ‘‘Trajectory planning of a mobile robot
autonomous indoor surveillance,’’ in Proc. 9th Int. Multi-Conf. SSD, using enhanced A-star algorithm,’’ Indian J. Sci. Technol., vol. 9, no. 41,
Chemnitz, Germany, 2012, pp. 1–6. pp. 1–10, 2016, doi: 10.17485/ijst/2016/v9i41/93816.
[15] K. Barton and D. Kingston, ‘‘Systematic surveillance for UAVs: A feed- [36] I. Chaari, A. Koubaa, H. Bennaceur, A. Ammar, M. Alajlan, and
forward iterative learning control approach,’’ in Proc. Conf. ACC, H. Youssef, ‘‘Design and performance analysis of global path plan-
Washington, DC, USA, 2013, pp. 5917–5922. ning techniques for autonomous mobile robots in grid environments,’’
[16] J. Kim and J. L. Crassidis, ‘‘UAV path planning for maximum visibility of Int. J. Adv. Robot. Syst., vol. 14, no. 2, pp. 1–15, 2017, doi: 10.1177/
ground targets in an urban area,’’ in Proc. 13th Conf. FUSION, Edinburgh, 1729881416663663.
U.K., 2010, pp. 1–7. [37] DJI. Accessed: Nov. 15, 2017. [Online]. Available: www.dji.com
JEONGHOON KWAK received the B.S. degree YUNSICK SUNG received the B.S. degree in
in game mobile contents and the M.S. degree in electrical and computer engineering from Pusan
computer engineering from Keimyung University, National University, Busan, South Korea, in 2004,
Daegu, South Korea, in 2015 and 2017, respec- and the M.S. degree in computer engineering and
tively, and the Ph.D. degree from the Department the Ph.D. degree in game engineering from Dong-
of Multimedia Engineering, Dongguk University, guk University, Seoul, South Korea, in 2006 and
Seoul, South Korea. His research interests are 2012, respectively. He was employed as a member
focused on the areas of unmanned aerial vehicle, of the Researcher at Samsung Electronics, South
ground control station, and demonstration-based Korea, from 2006 to 2009. He was the Plural Pro-
learning. fessor at the Shinheung College, Uijeongbu, South
Korea, in 2009, and Dongguk University in 2010. He was also the Post-
Doctoral Fellow with the University of Florida, Gainesville, FL, USA, from
2012 to 2013. He is currently an Assistant Professor with the Department
of Multimedia Engineering, Dongguk University. His research interests are
focused on the areas of games, pervasive computing, and robotics.