SPECIAL SECTION ON AI-DRIVEN BIG DATA PROCESSING: THEORY,

METHODOLOGY, AND APPLICATIONS

Received November 6, 2018, accepted November 25, 2018, date of publication November 28, 2018,
date of current version December 27, 2018.
Digital Object Identifier 10.1109/ACCESS.2018.2883702

Deep Learning-Based Unmanned Surveillance

Systems for Observing Water Levels
JINQIU PAN1 , (Student Member, IEEE), YUE YIN1 , (Student Member, IEEE),
JIAN XIONG 1,2 , (Member, IEEE), WANG LUO3 , (Member, IEEE),
GUAN GUI 1,2 , (Senior Member, IEEE), AND HIKMET SARI1,2 , (Fellow, IEEE)
1 Key Laboratory of Broadband Wireless Communication and Sensor Network Technology, Nanjing University of Posts and Telecommunications, Ministry of
Education, Nanjing 210003, China
2 National Engineering Research Center for Communication and Network Technology, Nanjing University of Posts and Telecommunications, Nanjing 210003,

China
3 NARI Group Corporations/State Grid Electric Power Research Institute, Nanjing 210000, China

Corresponding authors: Jian Xiong ([email protected]) and Guan Gui ([email protected])

This work was supported in part by the Priority Academic Program Development of Jiangsu Higher Education Institutions, National
Natural Science Foundation of China, under Grant 61701258 and Grant 61501223, in part by the Jiangsu Specially Appointed Professor
Program under Grant RK002STP16001, in part by the Program for Jiangsu Six Top Talent under Grant XYDXX-010, in part by the
Program for High-Level Entrepreneurial and Innovative Talents Introduction under Grant CZ0010617002, in part by the Natural Science
Foundation of Jiangsu Province under Grant BK20170906, in part by the Natural Science Foundation of Jiangsu Higher Education
Institutions under Grant 17KJB510044, in part by NUPTSF under Grant NY215026 and Grant NY215030, in part by the Science and
Technology Research Project of Chongqing Municipal Education Commission under Grant KJ1601408, and in part by the 1311 Talent
Plan of Nanjing University of Posts and Telecommunications.

ABSTRACT Traditional surveillance systems for observing water levels are often complex, costly, and
time-consuming. In this paper, we developed a low-cost unmanned surveillance system consisting of remote
measuring stations and a monitoring center. The system uses a map-based Web service, as well as video
cameras, water level analyzers, and wireless communication routers necessary to display real-time water
level measurements of rivers and reservoirs on a Web platform. With the aid of a wireless communication
router, the water level information is transmitted to a server connected to the Internet via a cellular network.
By combining complex water level information of different river basins, the proposed system can be used
to forecast and prevent flood disasters. In order to evaluate the proposed system, we conduct experiments
using three feasible methods, including the difference method, dictionary learning, and deep learning. The
experimental results show that the deep learning-based method performs best in terms of accuracy and
stability.

INDEX TERMS Water level surveillance, internet of things (IoT), dictionary learning, deep learning,
convolutional neural network (CNN).

I. INTRODUCTION the world’s per capita water resources, ranking 121st in the
Water links local areas to the region and is intricately related world. In addition, the distribution of water resources across
to global issues of public health, security, urbanization, and the country is also highly uneven. The southern part of the
energy. Without good planning and management, water- Yangtze River watershed, which accounts for one-third of the
related issues can represent a real and growing threat and country’s total area, has four-fifths of the country’s water,
hundreds of millions of people may be at risk of hunger, whereas the vast northern region has less than one-fifth of
disease, and energy shortages during flood disasters. The use the water. The northwestern inland water resources account
and management of water resources are central to setting the for only 4.6 percent of the country’s total water resources.
world on a more sustainable and equitable path but population The uneven distribution in water resources causes floods
growth, climate change, and other problems represent addi- in the south and droughts in the northwest every year and
tional stresses [1]. results in a lot of secondary disasters such as mudflows, land-
There are abundant water resources in China, but the per slides, desertification, and water shortages and in immense
capita water resources are only equivalent to one-fourth of property losses to people and overall economic losses to the

2169-3536 2018 IEEE. Translations and content mining are permitted for academic research only.
VOLUME 6, 2018 Personal use is also permitted, but republication/redistribution requires IEEE permission. 73561
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

country [2], [3]. Therefore, comprehensive and dependable but not least, water levels can also be used to estimate
water resource management is crucial for water security and the water storage capacity of rivers and lakes. During the
sustainable development. observations of flow, sediment, water temperature, ice con-
Water resource management is affected by population ditions, and water quality, the water level should be observed
growth, climate change [4], and other problems [5] and simultaneously.
many researchers have concentrated on this field for several Owing to the wide-ranging role of water level information
years. With the rapid development of the Internet of Things in various domains, many measurement methods have been
(IoT) [6], a variety of techniques for water resource manage- developed using different technologies. The most straight-
ment have been developed and many water resource manage- forward method for measuring the water level is the use of
ment applications for different tasks have been proposed in gauges that are visited and read periodically by employees.
the last few decades. In order to forecast floods, which fre- Although this method is simple, it has many disadvantages.
quently occur in areas with ample water, Ref. [7] designed a For example, employees are required to work 8 hours a day
system called Snowmelt Flood Forecasting Enterprise Infor- due to the law of China so that at least 3 employees are needed
mation System (SFFEIS), which integrates remote sensing to work in turn to monitor the water level in 24 hours. Because
(RS), Geographical Information Systems (GIS), and Global wages increase steadily, this conventional method becomes
Positioning Systems (GPS) [8]; the system links the Weather laborious, is expensive, and has low efficiency. Surprisingly,
Research and Forecasting (WRF) modeling system and the this method is still the most common method to measure
Distributed Hydrology Soil Vegetation Model (DHSVM) to water levels in China.
forecast snowmelt runoff, report dynamic snowfall informa- Automatic water level measurement has attracted increas-
tion, and provide snowmelt flood alerts. In addition, consid- ing attention at home and abroad during the last several
erable progress has been made in developing flood control years [14], [15]. Many studies have used different kinds
systems and operational medium-range flood forecasting sys- of sensors for the water level measurements. In Ref. [16],
tems have been increasingly moving towards the adoption of a novel Doppler radar technique was presented for the accu-
ensembles of numerical weather predictions (NWP), known rate tracking of the large-scale displacement of several carrier
as ensemble prediction systems (EPS) [9]. For regions vul- wavelengths to gauge the relative water level variations of
nerable to water shortages, Ref. [10] proposed a Satellite 3.5 wavelengths. In addition to radar, ultrasonic sensors were
Irrigation Management Support (SIMS) project that com- used with a custom algorithm for obtaining the real height of
bines NASA’s Terrestrial Observation and Prediction System the water by using the difference in the distance; this method
(TOPS), Landsat and MODIS satellite imagery, and surface achieved considerable accuracy [17], [18]. Though widely
sensor networks to map indicators of crop irrigation demand used, these methods usually have certain disadvantages in
and develop information products to support irrigation man- common such as their sensitivity to the surrounding envi-
agement and other water use decisions. The satellite-derived ronment. Thus, several methods based on computer vision
estimates of key hydrologic fluxes help agricultural producers and image processing using video cameras with image sen-
and water managers to maximize the benefits of the available sors have been proposed. Not only the water level but also
water supplies. information on the surrounding water body can be moni-
Water resource management is constrained by water avail- tored in real time. A two-step histogram-based method that
ability, quantity, and quality [11]. Whereas the availability is used a single image was presented in [19]. In [20], a differ-
related to the distance from the water source to urban cities ence image-based JPEG communication scheme and water
and the quantity refers to the volume of water, the water level measurement scheme using sparsely sampled images in
quality refers to biochemical parameters such as pH, turbid- the time domain was proposed. In our previous study [21],
ity, microorganisms, and solute concentration. Among many we proposed a novel image-based method using dictionary
parameters, the water level is the most crucial and the man- learning to measure water levels. However, the dictionary
agement of water levels is of great importance. is not capable of adapting well to different scenes. In order
First, the water level is an important indicator reflecting to obtain the capability of generalization, we thought of
changes in water bodies. Water level observations can be deep learning (DL) which has already been applied suc-
directly used for water level forecasting, providing timely cessfully in many areas. Ref. [22] trains a five-layer deep
water information for the construction, operation, and man- neural network (DNN) to spoof detection classifier. Ref. [23]
agement of flood control and for drought relief, irrigation, proposes an image-text dual neural network to improve the
shipping, and water conservancy projects [12]. Long-term classification performance on small-sample datasets. In [24],
water level data are essential for the construction, plan- a DL-aided non-orthogonal multiple access (NOMA) system
ning, and design of water conservancy and hydropower was proposed. In [25], DL-based method can achieve bet-
projects, bridges, waterways, ports, and urban water sup- ter performance in terms of the direction-of-arrival (DOA)
ply and drainage. In addition, in some hydrological tests, estimation and the channel estimation. Ref. [26] proposed an
continuous observation of water level records is used and end-to-end short utterances-based speech language identifi-
the flow rate and its changes are determined using the rela- cation (SLI) approach, which is especially suitable for the
tionship between the water level and flow rate [13]. Last short utterance based language identification. As a result,

73562 VOLUME 6, 2018

J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

in this study, we propose the use of a convolutional neural internet, the surveillance video and the water level data
network (CNN) that is suitable for generalization. are transmitted to the server in real time.
In this study, we propose and install a water resource man- • Monitoring Center: The monitoring center consists of a
agement system that determines the water level using video server and a web service. Through the internet, the server
surveillance cameras. During the rain seasons, this system can receives the surveillance video of the water data. The
be used to monitor the water level and compute the stream server is connected to multiple remote measuring sta-
flow to prevent accidents and provide flood alerts. During tions to monitor different scenes simultaneously. The
times of low flow, the system can be utilized to monitor the web platform processes the surveillance video and water
environment and the water quality. level data and different users can access the data in real
time and can obtain historical surveillance videos and
II. PROPOSED ARCHITECTURE water level data.
A. SYSTEM OVERVIEW The logical structure of the water level surveillance system
The system is based on the IoT [6]. It uses a web platform is consists of the data acquisition layer, the transmission layer,
to monitor water conditions. A remote measuring station is and the application layer as shown in Fig. 2.
installed at the water’s edge to collect surveillance video
and water level data. The system is based on the advanced
techniques of CNN, GIS, computer vision, embedded tech-
nology, and wireless communication and provides real-time
data, surveillance video display, historical data queries, and
data storage. All users can watch the surveillance videos of
the water and obtain real-time and historical data of water
levels through a web platform. With these functions, this
system can be used to determine water flows, obtain flood
alerts, etc.

B. SYSTEM ARCHITECTURE
Figure 1 shows the architecture of our IoT water level surveil-
lance system. It consists of two components: the remote
measuring stations and the monitoring center.
• Remote Measuring Stations: The remote measuring sta- FIGURE 2. Logical structure of the proposed system.
tions consist of several devices including a measuring
ruler, video camera, water analyzer, 4G router, distri- The data acquisition layer consists of a video camera and
bution box, and solar panel. The stations are installed a water analyzer and the transmission layer utilizes the 4G
near the banks of the streams and record the water router to connect the data acquisition layer with the server,
conditions and compute the water level. Through the thereby achieving real-time information surveillance. The
transmission layer uses a Wide Area Network (WAN) and
Local Area Network (LAN) to provide basic network trans-
mission for the surveillance system. The application layer is
a unified service window for users and includes web pages
and mobile apps.

C. MODULES OF THE SYSTEM

In this section, we provide details on the components of the
remote measuring station, i.e., the measuring ruler, video
camera, water analyzer, 4G router, solar panel and distribu-
tion box.

1) MEASURING RULER
The measuring ruler is an ordinary simple ruler that needs
to be installed vertically in the water. In most cases, where
the water level is read manually, this type of ruler is used.
In our proposed method, the measuring ruler does not have to
have increments. The water level is calculated using a linear
transformation of the water level by counting the pixels in the
FIGURE 1. Architecture of the proposed system. image.

VOLUME 6, 2018 73563

 J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

2) VIDEO CAMERA shown on the web platform. Users can obtain not only the
The video camera is a Network HD Camera. It focuses on the real-time water level but also the historical water level from
measuring ruler to achieve real-time monitoring. The video a few days ago. Users can also get access to photos saved
is compressed by MPEG4/H.264 encoding and is transmitted from the video periodically. Moreover, the parameters of the
over the network using a TCP/IP protocol. The camera has a camera are shown on the web platform and can be modified
built-in 10M/100M Ethernet RJ45 interface and allows users by the user.
to use a standard browser to perform various operations from
the monitoring center. In addition, the camera has a built-in D. SYSTEM WORKFLOW
SD card. It supports remote viewing and stores the video on The workflow of the proposed system is shown in Fig. 3.
the SD card, which cannot be easily damaged.

3) WATER ANALYZER
The water analyzer can be configured using two methods.
First, the water analyzer can be initialized by a built-in con-
figuration file. Second, the water analyzer can be initialized
using the parameters set by the users on the web platform
from the server. The water analyzer has several functions such
as water level calculation and data output. The water level
calculation algorithm is embedded in. In order to transmit the
water level data, the water analyzer has an additional network
communication module.

4) 4G ROUTER
The 4G router provides the function of wireless long-distance
data transmission utilizing a public long-term evolution
(LTE) network. This router does not need a network cable
but a 4G traffic card to provide WiFi and network access to
the internet and can be used wherever a 4G signal is available.
FIGURE 3. Workflow of the proposed system.
5) SOLAR PANEL
There are four main interfaces in the system, namely the
The solar panel is used as the power supply for the remote sta-
interface between the video cameras and water analyzer,
tion. The remote station converts light energy into electrical
the interface between the water analyzer and server, the inter-
energy and is more flexible without being restricted by power
face between the server and web platform, and the interface
access. Moreover, solar energy is cleaner and it costs less in
between the video cameras and web platform.
the long term.
First, the user sets the parameters of the video camera
and the water analyzer on the web platform. Because the
6) DISTRIBUTION BOX video camera is a network camera that combines a traditional
The distribution box has the advantages of small size, simple camera and network technology, these parameters can be
installation, no site limitation, stable and reliable operation, directly passed to the video camera via the interface between
good space utilization, and protection of the other devices. the web platform and the video camera. Vice versa, the real-
The 4G router, water analyzer, and wires are all placed in the time video produced by the video camera can be sent back to
distribution box for protection. the web platform through the same interface. The parameters
of the water analyzer are also set up on the web platform
7) SERVER and they are passed to the server and the water analyzer
The server is used to constantly update the water level and then fetches these parameters from the server to initialize the
parameters of the water analyzer, as well as storing the videos settings. The video camera collects real-time video of the
uploaded by the video cameras. It works as a transfer station measuring ruler and sends the real-time streaming protocol
or a storage center between the water analyzer and the web (RTSP) video stream to the water analyzer. Because the algo-
platform. Most of the data exchange in the system has to go rithm is embedded in the water analyzer, the water level is
through the server. computed and sent to the web platform step by step.

8) WEB PLATFORM III. METHODS OF WATER LEVEL DETECTION

The web platform provides an operational interface for the In order to obtain the best performance in terms of accuracy
user. Several cameras at different spots are optional and when and stability, we have designed several methods to detect the
the user chooses one, a real-time monitoring video will be water level.

73564 VOLUME 6, 2018

J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

A. DIFFERENCE METHOD to the largest inter-class variance. As a result, the optimal

The difference method is based on traditional image pro- threshold separates the difference image of the ROI into the
cessing algorithms. Because the water level changes and the water and the ruler so that the water level is obtained.
measuring ruler does not change, the difference in the water
level can be determined by the differences in two consecutive B. METHOD BASED ON DICTIONARY LEARNING
video frames. In this method, the water information image is classified
into ruler region and water region by using dictionary learn-
ing method. By searching the boundary between these two
classes, the water level can be easily calculated.
Dictionary learning is presented by formula (3),
m
X m
X
min kyi − Dxi k22 + β kxi k1 (3)
D,xi
i=1 i=1

where D is the dictionary in Ra×b , yi is the ith a-dimensional

vector and xi is a b-dimensional representative vector of yi . β
is a scalar factor.
Every yi can be represented by its xi and the same dictio-
nary D. It is said that D has a property to represent different
vectors. So we use this property of dictionary to preserve the
features of water and ruler so that they can be classified.
FIGURE 4. Water information image and difference image of the ROI of Briefly, the method based on dictionary learning can be
one scene. divided into three steps. First, the training images are con-
verted into a training matrix of Y, where each column
Two consecutive video frames are shown in Fig. 4, represents a training sample with a specific label. Second,
the lower one is the current frame and the upper one is the the training matrix is input into the dictionary learning model
previous frame. The area in the red box is considered as the to output a learned dictionary D. Finally, with the discrimina-
region of interest (ROI). By subtracting these two consecutive tive and compact D, we classify the water information image
frames, the subtracted ROI is obtained in the right of Fig. 4. into ruler and water in real time to obtain the water level.
Most of the subtracted ROI is black, indicating that there These steps are shown in Fig. 5.
are few changes, whereas the white area indicates the area In step I, an ROI of the measuring ruler is chosen to avoid
of water. With this process, the area of ruler and water are the negative influence of additional information. Although
separated. the ROI can be roughly classified into two classes as ruler and
In order to obtain the precise water level, additional oper- water, different parts of the ruler are not exactly the same and
ations are conducted on the difference image of the ROI. are divided into subclasses based on the texture. Likewise,
Because the variance and average of the water area are much the water is divided into subclasses based on texture for
larger than those of the ruler area, these two parameters of further distinction. In order to preserve the correspondence
each row in the difference image of the ROI are multiplied to between the rows, a sliding window of m × n pixels is used
magnify the differences between the water area and ruler area. to acquire the grayscale values in each subclass. We slide
To obtain the threshold for classifying the water and ruler, the window by a step of s pixels and transform each m × n
Otsu’s method is used [27]. matrix into a column vector; several training samples in the
Otsu’s method is an adaptive threshold determination form of column vectors are collected. Finally, we combine
method that classifies the image into two parts; it provides all the training samples extracted from each subclass of each
the segmentation threshold for the two parts. Part I and part training image and all the training  images are converted into
II represent ω0 and ω1 of the image. The average values of a training matrix Y defined as Y 1 , Y 2 , . . . , Y C , in which

the two parts are u0 and u1 and the average value of the entire every sub-matrix Yc is collected from the cth class of the
image is u. The inter-class variance of the image is defined training samples. Each block of the training matrix is related
as: to a subclass label.
g = ω0 × ω1 × (u0 − u1 )2 (1) In step II, the dictionary is received by inputting the train-
ing matrix into a dictionary learning model [28] along with
The equation can also be presented in the following for- its subclass labels.
mula: In step III, we use the features extracted using the sliding
ω0 window, which was used on the training images in step II
g= × (u0 − u)2 (2)
1 − ω0 and we transform the query image into a matrix of query
A larger inter-class variance indicates a larger difference samples. Based on the classifier in the dictionary learning
between the two parts and the optimal threshold corresponds model, the labels of each query sample in the matrix of query

VOLUME 6, 2018 73565

 J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

samples can be assigned. By searching the boundary of the

two classes of ruler and water, the water level in the image is
determined.

C. METHOD BASED ON CNN

CNNs have resulted in good performance in image classifica-
tion. The rapid evolution of deep learning has resulted in an
increase in the classification accuracy of many different net-
works, such as AlexNet [29], VGGNet [30], GoogLeNet [31],
ResNet [32], and Inception [33].
In the dictionary learning method, the problem of water
level calculation has been transformed into a classification
problem. The features of water and ruler are preserved in
a learned dictionary D. In CNN, convolutional kernels are
used to extract features from water and ruler. The formula
of convolution is presented as following,
k X
X k
ai,j = ωm,n · xi+m,j+n + ωb (4)
m=0 n=0

where xi+m,j+n represents the value of input image at

(i + m, j + n), ai,j represents the value of output image at FIGURE 5. Steps of the method based on dictionary learning.
(i, j), ωm,n represents convolutional kernel, k represents size
of the kernel and ωb represents bias.
The CNN method is similar to the dictionary learning
method; therefore, we replace the dictionary learning model The dictionary learning method has much higher stability
in Fig. 5with CNN. However, there are several differences and very good accuracy. It can be used in most scenes and can
between these two methods. The procedures of the CNN be embedded in most development boards as the Hi3531 we
method are shown in Fig. 6. used. However, the generalization ability of the dictionary
The CNN method consists of the training procedure and not very good so that the two classes of ruler and water
querying procedure. It is well-known that deep learning can need to be separated into subclasses. For the same reason,
be used for three types of data: image, sound, and text. we cannot input the training images of different scenes into
In this task, we feed the CNN with images. In the training one dictionary learning model. A dictionary learning model
procedure, we preprocess the training images for the input. has to be trained for each scene, otherwise, the accuracy is
Similar to the dictionary learning method, the ROI of the low.
water information image is classified into ruler and water However, the CNN has a very strong generalization ability
classes. Using a sliding window of m×n pixels in each class for separating the training images into the two classes without
of every training image, many training samples are collected. requiring subclasses. Additionally, all of the training samples
We input these training samples into the CNN to obtain a from different scenes can be input into the same CNN and a
CNN with optimized parameters. In the querying procedure, higher accuracy is obtained than for the dictionary learning
the query image is processed using the sliding window of method. The CNN requires complex calculations; therefore,
m × n pixels to obtain several query samples. We input these we choose the development board of Jetson TX2.
query samples into the trained CNN to obtain the labels of
the query samples. Finally, the water level is determined by E. WATER LEVEL CONVERSION
searching the boundary line of the labels. In real world, water level is counted in meters but the water
level calculated by the methods from water information
D. COMPARISON OF THE METHODS image is counted in pixels. So a linear conversion is required
The difference method is very simple and can be embedded to transform water level counted in pixels into water level
on any development board. It has good accuracy in some counted in meters.
scenes, especially when the video camera is stable and the As shown in Fig. 7, R represents water level in real world
water level fluctuates slowly. However, in most of the scenes, counted in meters, and lr represents water level in water
the video camera shakes slightly due to the wind. In such information image counted in pixels. For the conversion,
conditions, this method is unstable because the area of the some parameters needed to be observed in advance. L pixels
ruler changes markedly and makes it difficult to distinguish in water information image is referred to 1 meter in real
it from the area of water. Moreover, the ROI of this method is world. Water level of lw pixel in image is referred to W meter
very small, resulting in instability. in real world. Thus, the formula of this linear conversion is

73566 VOLUME 6, 2018

J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

FIGURE 8. Water levels derived from different window heights.

FIGURE 6. Procedures of the CNN method.

FIGURE 7. Sketch map of water level conversion.

FIGURE 9. Water levels derived from different window heights.

defined as: are very similar, which illustrates that the CNN learns similar
features from the training samples.
lw − lr
R=W+ (5)
L B. DIFFERENT METHODS
We conducted a comparison of the three methods using sev-
IV. PERFORMANCE EVALUATION AND DISCUSSION
eral experiments. The results are shown in Fig. 10 and Table 2.
A. DIFFERENT WINDOW HEIGHTS
It is not surprising that the CNN method achieves the best
It is apparent that in the CNN method, the size of the sliding performance. The data in Fig. 10 and Table 2 indicate that the
window affects the performance, especially the height of the difference method is not stable, as was mentioned in the last
window. Therefore, different window heights were chosen. section.
As shown in Fig. 8, the horizontal axis represents the con-
secutive images obtained from a video at equal intervals. The
C. DIFFERENT SCENES
vertical axis represents the water level in the real world. It can
Since the CNN method had best performance, we tested it on
be seen that the method is very unstable when the window
different scenes. Six different scenes are shown in Fig. 11 and
height is 1 pixel. This occurs because the window is more like
a vector rather than an image and much internal information
TABLE 1. Average Error and Variance of error of different window heights.
is discarded. Therefore, this window height of 1 pixel is not
used. Figure 8 is a magnification of Fig. 9. The average error
and the variance of the error are shown in Table 1.
The window height with the best performance is 20 pixels.
If the window height is too small, much internal information
may be discarded and if the window height is too large,
the error is large because the water level is only 1 pixel wide.
It is interesting to note that the curves of 10 and 40 pixels

VOLUME 6, 2018 73567

 J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

FIGURE 10. Water levels derived from different methods.

FIGURE 12. Water levels derived for different scenes.

TABLE 3. Average error and variance of error of different Scenes.

FIGURE 11. Application of the CCN method to different scenes.

the performance of the method on these different scenes are

shown in Fig. 12 and Table 3. It can be seen that the CNN
method exhibits good performance in all scenes.

D. DIFFERENT WINDOW OFFSETS

In order to evaluate the stability of the CNN method, we offset
the sliding window in the horizontal axis; the results are FIGURE 13. Water level derived for different window offsets.
shown in Fig. 13 and Table 4. It can be seen that the CNN

method still has good performance when the sliding window

TABLE 2. Average Error and Variance of error of different methods. offset is 15%; therefore, the method shows good stability.

E. WEB PLATFORM
The web platform is shown in Fig. 14. As mentioned above,
users can see the video, water level, and historical data on the
web. In addition, users can switch the sidebars to change the
configuration of the video or water analyzer.

73568 VOLUME 6, 2018

J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

V. CONCLUSIONS
In this paper, we proposed a low-cost water level surveillance
system that can be integrated into the IoT. A surveillance sys-
tem is designed for convenient data acquisition. An automatic
method to detect water levels is proposed and is designed
to replace manual measurements. The method is based on a
CNN and results in high accuracy and stability. In the future,
we plan on adding more practical functions to the water
analyzer such as water quality testing and flotation detection.

REFERENCES
FIGURE 14. Web platform. [1] I. Winz, G. Brierley, and S. Trowsdale, ‘‘The use of system dynamics sim-
ulation in water resources management,’’ Water Resour. Manage., vol. 23,
no. 7, pp. 1301–1323, May 2009.
TABLE 4. Average Error and Variance of error for different window
[2] Z. Jingyi and M. J. Hall, ‘‘Regional flood frequency analysis for the Gan-
OFFSETS.
Ming River Basin in China,’’ J. Hydrol., vol. 296, nos. 1–4, pp. 98–117,
Aug. 2004.
[3] X.-P. Deng, L. Shan, H. Zhang, and N. C. Turner, ‘‘Improving agricultural
water use efficiency in arid and semiarid areas of China,’’ Agricult. Water
Manage., vol. 80, nos. 1–3, pp. 23–40, Feb. 2006.
[4] S. Fang, H. Pei, Z. Liu, K. Beven, and Z. Wei, ‘‘Water resources assessment
and regional virtual water potential in the Turpan Basin, China,’’ Water
Resour. Manag., vol. 24, no. 13, pp. 3321–3332, Oct. 2010.
[5] W. Medema, B. S. McIntosh, and P. J. Jeffrey, ‘‘From premise to practice: A
critical assessment of integrated water resources management and adaptive
management approaches in the water sector,’’ Ecology Soc., vol. 13, no. 2,
pp. 1–18, Dec. 2008.
[6] L. Da Xu, W. He, and S. Li, ‘‘Internet of Things in industries: A survey,’’
IEEE Trans. Ind. Informat., vol. 10, no. 4, pp. 2233–2243, Nov. 2014.
[7] S. Fang et al., ‘‘An integrated approach to snowmelt flood forecasting in
water resource management,’’ IEEE Trans Ind. Informat., vol. 10, no. 1,
pp. 548–558, Feb. 2014.
[8] Q. Zhao, Z. Liu, B. Ye, Y. Qin, Z. Wei, and S. Fang, ‘‘A snowmelt runoff
forecasting model coupling WRF and DHSVM,’’ Hydrol. Earth Syst. Sci.,
vol. 13, no. 10, pp. 1897–1906, Oct. 2009.
[9] H. L. Cloke and F. Pappenberger, ‘‘Ensemble flood forecasting: A review,’’
J. Hydrol., vol. 375, nos. 3–4, pp. 613–626, Sep. 2009.
[10] F. S. Melton et al., ‘‘Satellite irrigation management support with the ter-
restrial observation and prediction system: A framework for integration of
satellite and surface observations to support improvements in agricultural
water resource management,’’ IEEE J. Sel. Topics Appl. Earth Observat.
Remote Sens., vol. 5, no. 6, pp. 1709–1721, Dec. 2012.
[11] A. K. Biswas, ‘‘Integrated water resources management: A reassessment,’’
Water Int., vol. 29, no. 2, pp. 248–256, Jun. 2004.
[12] F. Pappenberger, J. Bartholmes, J. Thielen, H. L. Cloke, R. Buizza, and
A. de Roo, ‘‘New dimensions in early ood warning across the globe using
FIGURE 15. Cost of manual and automatic methods. grand-ensemble weather predictions,’’ Geophys. Res. Lett., vol. 35, no. 10,
pp. 1956–1964, May 2008.
[13] L. S. Nguyen et al., ‘‘Vision-based system for the control and measurement
of wastewater flow rate in sewer systems,’’ Water Sci. Technol., vol. 60,
F. LOW COST no. 9, pp. 2281–2289, Nov. 2009.
We have determined that the salary of one employee in the [14] C. Yu, ‘‘Liquid level measurement by using an image method,’’ in Proc.
12th Int. Conf. Signal Process. (ICSP), Oct. 2014, pp. 2320–2323.
water resources bureau is about 4000 yuan per month and at
[15] M. Iwahashi, S. Udomsiri, Y. Imai, and S. Muramatsu, ‘‘Water level
least two employees should be hired. The price of the Jetson detection for functionally layered video coding,’’ in Proc. IEEE Int. Conf.
TX2 is about RMB 5,500 Yuan, the server is about RMB Image Process. (ICIP), vol. 2, Sep./Oct. 2007, pp. II-321–II-324.
20,000 Yuan, the solar panel is about RMB 300 Yuan, and the [16] C. Gu et al., ‘‘Noncontact large-scale displacement tracking: Doppler radar
for water level gauging,’’ IEEE Microw. Wireless Compon. Lett., vol. 24,
4G router is about RMB 200 Yuan. Because most reservoirs no. 12, pp. 899–901, Dec. 2014.
have measuring rulers and video cameras, we did not take [17] G. Qiu, Z. Luo, and G. Zhou, ‘‘Discussion and design of dynamic liquid
into account the cost of these devices. The maintenance costs level intelligent monitoring system,’’ Int. J. Smart Home, vol. 9, no. 11,
pp. 211–224, Nov. 2015.
are predicted to be about RMB 100 Yuan per month and [18] S. Viswanath, M. Belcastro, J. Barton, B. O’Flynn, N. Holmes, and
increase every month. Figure 15 shows the cost of using P. Dixon, ‘‘Low-power wireless liquid monitoring system using ultrasonic
the manual and automatic methods. The cost of the manual sensors,’’ Int. J. Smart Sens. Intell. Syst., vol. 8, no. 1, pp. 26–44, Mar. 2015.
method surpasses that of the automatic method in the fourth [19] H. Sakaino, ‘‘Camera-vision-based water level estimation,’’ IEEE Sensors
J., vol. 16, no. 21, pp. 7564–7565, Nov. 2016.
month, which indicates that the automatic method has lower [20] J. Yu and H. Hahn, ‘‘Remote detection and monitoring of a water level
cost in the long term. using narrow band channel,’’ J. Inf. Sci. Eng., vol. 26, pp. 71–82, Jan. 2010.

VOLUME 6, 2018 73569

 J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

[21] J. Pan, Y. Fan, H. Dong, S. Fan, J. Xiong, and G. Gui, ‘‘Image-based JIAN XIONG (M’17) received the B.Sc. degree in
detecting the level of water using dictionary learning,’’ in Proc. Int. Conf. computer science and technology from the Anhui
Commun., Signal Process., Syst. (CSPS), Jul. 2018, pp. 1–10. University of Finance and Economics, Bengbu,
[22] H. Yu, Z.-H. Tan, Z. Ma, R. Martin, and J. Guo, ‘‘Spoofing detection in China, in 2007, the M.Sc degree in computer
automatic speaker verification systems using DNN classifiers and dynamic application technology from Xihua University,
acoustic features,’’ IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 10, Chengdu, China, in 2010, and the Ph.D. degree in
pp. 4633–4644, Oct. 2018. single and information processing from the Uni-
[23] F. Zhu et al., ‘‘Image-text dual neural network with decision strategy
versity of Electronic Science and Technology of
for small-sample image classification,’’ Neurocomputing, to be published,
China, Chengdu, China, in 2015. From 2014 to
doi: 10.1016/j.neucom.2018.02.099.
[24] G. Gui, H. Huang, Y. Song, and H. Sari, ‘‘Deep learning for an effec- 2014, he was a Research Assistant with the Image
tive nonorthogonal multiple access scheme,’’ IEEE Trans. Veh. Technol., and Video Processing Laboratory, The Chinese University of Hong Kong,
vol. 67, no. 9, pp. 8440–8450, Sep. 2018. Hong Kong. Since 2015, he has been a Lecturer with the College of Telecom-
[25] H. Huang, J. Yang, Y. Song, H. Huang, and G. Gui, ‘‘Deep learning for munications and Information Engineering, Nanjing University of Posts and
super-resolution channel estimation and DOA estimation based massive Telecommunications, Nanjing, China. His current research interests include
MIMO system,’’ IEEE Trans. Veh. Technol., vol. 67, no. 9, pp. 8549–8560, image and video coding, pattern recognition, and machine learning.
Sep. 2018.
[26] Z. Ma, H. Yu, W. Chen, and J. Guo, ‘‘Short utterance based speech
language identication in intelligent vehicles with time-scale modications
and deep bottleneck features,’’ IEEE Trans. Veh. Technol., vol. 68, no. 1,
pp. 1–8, 2019, doi: 10.1109/TVT.2018.2879361.
[27] N. Otsu, ‘‘A threshold selection method from gray-level histograms,’’ IEEE
Trans. Syst., Man, Cybern., vol. SMC-9, no. 1, pp. 62–66, Jan. 1979.
[28] X. Wang and Y. Gu, ‘‘Cross-label suppression: A discriminative and fast
dictionary learning with group regularization,’’ IEEE Trans. Image Pro-
cess., vol. 26, no. 8, pp. 3859–3873, Aug. 2017.
WANG LUO (M’12) received the Dr.Eng. degree
[29] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ‘‘ImageNet classification
in information and communication engineering
with deep convolutional neural networks,’’ in Proc. Adv. Neural Inf. Pro-
cessing Syst. (NIPS), 2012, pp. 1097–1105. from the University of Electronic Science and
[30] K. Simonyan and A. Zisserman, ‘‘Very deep convolutional networks Technology of China, Chengdu, China, in 2012.
for large-scale image recognition,’’ in Proc. Int. Conf. Learn. Repre- His is current Senior Researcher with the NARI
sent. (ICLR), Sep. 2014, pp. 1–14. Group Corporations/State Grid Electric Power
[31] C. Szegedy et al., ‘‘Going deeper with convolutions,’’ in Proc. IEEE Conf. Research Institute. He is also the Adjunct Pro-
Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1–9. fessor with the Nanjing University of Posts and
[32] K. He, X. Zhang, S. Ren, and J. Sun, ‘‘Deep residual learning for Telecommunications, Nanjing, China. He is cur-
image recognition,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., rently engaged in research of deep learning-based
Jun. 2016, pp. 770–778. image/video processing.
[33] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network
training by reducing internal covariate shift,’’ in Proc. Int. Conf. Mach.
Learn., Feb. 2015, pp. 448–456.

JINQIU PAN (S’18) is currently pursuing the mas-

ter’s degree in communication engineering with GUAN GUI (M’11–SM’17) received the Dr.Eng.
the Nanjing University of Posts and Telecommu- degree in information and communication engi-
nications, Nanjing China. His research interest neering from the University of Electronic Sci-
is dictionary learning, deep learning, and convex ence and Technology of China, Chengdu, China,
optimization. in 2012. From 2009 to 2012, he was with the
Wireless Signal Processing and Network Labo-
ratory (Prof. Adachi laboratory), Department of
Communications Engineering, Graduate School of
Engineering, Tohoku University, as a Research
Assistant and a Post-Doctoral Research Fellow, for
which he received supported from the China Scholarship Council (CSC)
and the Global Center of Education (ECOE) of Tohoku University. From
2012 to 2014, he was supported by the Japan Society for the Promotion of
Science (JSPS) Fellowship as a Post-Doctoral Research Fellow.
In 2015, he was an Assistant Professor with the Department of Electronics
and Information System, Akita Prefectural University. Since 2015, he has
YUE YIN (S’18) is currently pursuing the mas- been a Professor with the Nanjing University of Posts and Telecommu-
ter’s degree in communication engineering with nications, Nanjing, China. He is currently engaged in research of deep
the Nanjing University of Posts and Telecommu- learning, compressive sensing, and advanced wireless techniques. He was
nications, Nanjing China. His research interest an Editor of Security and Communication Networks from 2012 to 2016.
is dictionary learning, deep learning, and convex He has been an Editor of the IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
optimization. since 2017 and the KSII Transactions on Internet and Information System
since 2017. He received several best paper awards, such as CSPS2018,
ICNC2018, ICC2017, ICC2014, and VTC2014. He was also selected as a
Jiangsu Specially Appointed Professor, Jiangsu High-level Innovation and
Entrepreneurial Talent and the Nanjing Youth Award.

73570 VOLUME 6, 2018

J. Pan et al.: Deep Learning-Based Unmanned Surveillance Systems for Observing Water Levels

HIKMET SARI (F’95) received the Engineering He served as a Distinguished Lecturer for the IEEE Communications Society
and Ph.D. degrees from ENST, Paris, France, from 2001 to 2006, a member of the IEEE Fellow Evaluation Commit-
and the Habilitation degree from the University tee from 2002 to 2007, and a member of the Awards Committee from
of Paris-Sud, Orsay. He held various research 2005 to 2007.
and management positions in industry, includ- Dr. Sari was the Chair of the Communication Theory Symposium of
ing Philips Research Laboratories, SAT, Alcatel, ICC 2002 (New York), the Technical Program Chair of ICC 2004 (Paris),
Pacific Broadband Communications, and Juniper the Vice General Chair of ICC 2006 (Istanbul), the General Chair of PIMRC
Networks. He is currently a Professor with the 2010 (Istanbul), the General Chair of WCNC 2012 (Paris), the Executive
Nanjing University of Posts and Telecommunica- Chair of WCNC 2014 (Istanbul), the General Chair of ICUWB 2014 (Paris),
tions (NUPT) and also a Chief Scientist of Sequans the General Co-Chair of the IEEE BlackSeaCom 2015 (Constanta, Roma-
Communications. His distinctions include the André Blondel Medal in 1995, nia), the Technical Program Chair of EuCNC 2015 (Paris), and the Executive
the Edwin H. Armstrong Achievement Award in 2003, and the Harold Sobol Co-Chair of ICC 2016 (Kuala Lumpur). He also chaired the Globecom
Award in 2012, and election to Academia Europaea (the Academy of Europe) and ICC Technical Content Committee from 2010 to 2011. He was a Vice
and the Science Academy of Turkey in 2012. President for Conferences of the IEEE Communications Society during
Dr. Sari has served as an Editor for the IEEE TRANSACTIONS ON 2014 from 2015. He is currently serving as the General Co-Chair for ATC
COMMUNICATIONS from 1987 to 1981, a Guest Editor for the European Trans- 2016 (Hanoi, Vietnam), the Executive Chair of ICC 2017 (Paris), and the
actions on Telecommunications in 1993 and the IEEE JSAC in 1999, and an General Chair of PIMRC 2018 (Istanbul).
Associate Editor for the IEEE COMMUNICATIONS LETTERS from 1999 to 2002.

VOLUME 6, 2018 73571