Journal of Big Data Analytics in Transportation

https://doi.org/10.1007/s42421-020-00020-1

ORIGINAL PAPER

Applications of Deep Learning in Intelligent Transportation Systems

Arya Ketabchi Haghighat1 · Varsha Ravichandra‑Mouli1 · Pranamesh Chakraborty1 · Yasaman Esfandiari1 ·
Saeed Arabi1 · Anuj Sharma1

Received: 19 November 2019 / Revised: 19 November 2019 / Accepted: 17 July 2020

© Springer Nature Singapore Pte Ltd. 2020

Abstract
In recent years, Intelligent Transportation Systems (ITS) have seen efficient and faster development by implementing deep
learning techniques in problem domains which were previously addressed using analytical or statistical solutions and also
in some areas that were untouched. These improvements have facilitated traffic management and traffic planning, increased
safety and security in transit roads, decreased costs of maintenance, optimized public transportation and ride-sharing com-
pany’s performance, and advanced driver-less vehicle development to a new stage. This papers primary objective was to
provide a review and comprehensive insight into the applications of deep learning models on intelligent transportation
systems accompanied by presenting the progress of ITS research due to deep learning. First, different techniques of deep
learning and their state-of-the-art are discussed, followed by an in-depth analysis and explanation of the current applications
of these techniques in transportation systems. This enumeration of deep learning on ITS highlights its significance in the
domain. The applications are furthermore categorized based on the gap they are trying to address. Finally, different embed-
ded systems for deployment of these techniques are investigated and their advantages and weaknesses over each other are
discussed. Based on this systematic review, credible benefits of deep learning models on ITS are demonstrated and directions
for future research are discussed.

Keywords Deep learning · ITS · Survey · Transportation systems

Introduction of these neural network techniques. Traffic signal control for

better traffic management, increasing the security of trans-
The emergence of machine learning and its substitution for portation via surveillance sensors, traffic rerouting systems,
several statistical models have led to better problem-solving, health monitoring of transportation infrastructure, and sev-
which in turn has led various fields of study to turn their eral other problems now have a strong new approach, and for
research paths to take advantage of this new method. Trans- several challenging problems in transportation engineering,
portation systems have been influenced by the growth of new solutions have been created.
machine learning, particularly in intelligent transportation There have been several surveys of the literature on the
systems (ITS).With the proliferation of data and advance- application and enhancement of ITS using DL techniques.
ments in computational techniques such as graphical pro- However, most of these have tended to focus on a specific
cessing units (GPUs), a specific class of machine learning aspect of DL or a specific aspect of ITS. For instance, Zhu
known as deep learning (DL) has gained popularity. The et al. (2018a) conducted survey of big data analytics in ITS.
capability of DL models to address large amounts of data A review of computer vision playing a key role in roadway
and extract knowledge from complex systems has made transportation systems was discussed in Loce et al. (2013).
them a powerful and viable solution in the domain of ITS. While (Nguyen et al. 2018) reviews DL models across the
A variety of networks in DL have helped researchers to for- transportation domain, it is not a comprehensive survey that
mulate their problems in a way that can be solved with one encompasses all current research publications on the ITS
domain and DL. One dedicated review on enhancing trans-
portation systems via DL was done in Wang et al. (2018a)
* Arya Ketabchi Haghighat
[email protected]
where substantial research was included, but it focused pri-
marily on traffic state prediction and traffic sign recognition
1
Iowa State University, Ames, IA, USA

13
Vol.:(0123456789)
 Journal of Big Data Analytics in Transportation

tasks. The ITS domain includes other tasks, such as public included “deep learning”, “convolutional”. These search
transportation, ride-sharing, vehicle re-identification, and terms were sought in the title, abstract and notes. Then the
traffic incident prediction and inference tasks, which are all references of the papers identified were examined to trace
represented in this paper to make its extent more comprehen- other trusted journals and papers. Also, online searches on
sive and holistic. The transportation and research commu- various databases such as Scopus, Science Direct, IEEE, and
nity has always taken notice of pivotal research directions, ArXiv were done. All papers obtained were included in this
with the earliest review of neural nets applied to transporta- review if they met the following criteria:
tion (Dougherty 1995), where the critical review spanned the
classes of problems, neural nets applied and the challenges • Describe solutions to ITS problems using DL, as iden-
in addressing various problems. It is this that motivates of tified by methodology sections, that include DL-based
the question we address in this paper: How effective and effi- model development
cient are the current DL research applications for the domain • Published between January 2015 and October 2019 (dur-
of ITS? To the best of the authors’ knowledge, the literature ing which period the majority of research so far using DL
in this field has suffered from the lack of a holistic survey in ITS has been conducted)
that takes a broader perspective of ITS as a whole and its • Not a book, book chapter, dissertation, thesis or technical
enhancement using DL models. report
The purpose of this paper was, therefore, to present the • Not a general introduction to ITS
systematic review we have conducted on the existing state • Not in the domain of autonomous vehicles
of the research on ITS and its foray into DL. In “Research
Approach and Methodology”, we discuss our approach taken Though DL boom was spawned by the ImageNet project
to identify relevant literature. In “Background on Techniques in 2012 (Russakovsky et al. 2015) and applications of DL
in Deep Learning”, we talk about different methods of DL on ITS first appeared in 2013, substantial growth in ITS
network systems and breakthrough research on those meth- research by means of DL methodologies did not start until
ods. In “Applications in Transportation”, we talk about dif- 2015. This is illustrated in Fig. 1. Since then, there has been
ferent applications of DL methods in transportation engi- a steady growth in the prominence of DL-based ITS studies
neering, specifically six major application categories in ITS. across journals and conferences. In the year 2019, up until
In “Discussion and Conclusion”, we investigate different October, 43 papers have been published across various ITS
available embedded systems, or devices that can facilitate applications. In light of the marked increasing importance of
the running of neural network experiments. Finally, in “Ref- DL as an ITS research method, in the following section, we
erences”, we provide a summary and an outlook for future will discuss and review the various DL structures and then
research. their key applications in the ITS domain.
The research methodology which is followed in this
paper is PRISMA (Preferred Reporting Items for Systematic
Reviews and Meta-Analyses) (Moher et al. 2009). Following Background on Techniques in Deep Learning
this method, we first produced a questionnaire and in each
paper we reviewed, we looked for answers to these ques- Deep Neural Networks (DNN)
tions. The focus of these questions is about the gap which
each paper tries to address, their proposed solutions, and Deep learning (DL) is a specific subcategory of machine
finally the performance of these solutions for their datasets. learning where several layers of stacked parameters are used

Research Approach and Methodology

This paper performs a detailed analysis of existing studies

on intelligent transportation systems (ITS) and deep learning
(DL). Articles were searched in multiple databases using the
search strategy described below. The collected articles were
then reviewed and organized. The scope of this review was
restricted to conference proceedings and journal articles,
including existing literature reviews.
Relevant articles were primarily obtained by query-
ing the TRID TRB database (Home—transport research
international documentation 2017), where the search terms Fig. 1  Year-wise publication growth in ITS domains

13
Journal of Big Data Analytics in Transportation

for the learning process (Ketkar 2017). These parameters are (1962) suggested that instead of using fully connected layers
component representations of different aspects which can of neural networks, it is possible to use a single kernel with
affect the result of the network. Each layer contains several shared weights to wisp the entire image and extract the local
perceptrons (known also as neurons or hidden units) which features. The proposed method enhanced the detection effec-
carry weights for the parameter. The input of each layer is tiveness both in terms of accuracy and memory requirement
multiplied by these parameters and, therefore, the output when compared with traditional methods, which required
is a representation of the impact of each parameter on the handcrafted feature extractions (LeCun et al. 1998).
input. Usually after each layer or several layers of neurons, CNN is a detection architecture that automatically learns
a nonlinearity function such as the tanh, sigmoid, or recti- spatial hierarchical features using back-propagation through
fied linear function (ReLU) (Glorot et al. 2011) is used to the network. A schematic figure of this architecture is pre-
generate the output layer. All these layers combine to form sented in Fig. 2a. These networks usually contain three types
a deep neural network (DNN) (Schmidhuber 2015). There of layers: convolution, pooling, and fully connected, where
are two major challenges in building a DNN: first, designing the first two are used to extract the features and the last one
the structure of the network, which includes the number of used as a classifier (Bengio et al. 2015).
layers, number of neurons in each layer, and nonlinearity The convolution layer consists of a combination of a con-
function type,and second, adjusting the weight of the param- volution kernel, which counts as a linear part of the layer and
eters to train the network on how it should perceive the input a nonlinear activation function. The main advantage of using
data and calculate the output. For the first challenge, what a kernel that shares weights in operation, is extracting the
is usually most helpful is simply trial and error and overall local features and learning the spatial hierarchies of features
experience. For the second challenge, the back-propagation efficiently by reducing the required parameters. Then the
method is the most popular method to train the weight of nonlinear activation function maps the results onto the fea-
parameters in a supervised manner. More details about this ture map. In order to reduce the number of parameters, usu-
method can be found in Schmidhuber (2015). Although all ally one pooling layer comes after a few convolutional layers
the techniques which will be discussed in the rest of this in order to downsample the data, by taking the maximum
paper can be classified as a subcategory of DNN, here in unit (max pooling) or the average (average pooling) of a col-
this paper, DNN is defined as the simplest structure of a lection of units and substituting it as a representative of these
network, in other words, fully connected layers. In this fully collections. After extracting features and downsampling the
connected model, there is a connection between all the neu- data by the convolution and pooling layers, they are mapped
rons of one layer to all the neurons in another layer, and for onto the final output by fully connected layers. The output of
each connection, there is a weight which should be deter- these layers usually is the same size as the number of classes
mined through back-propagation method. and each output indicates the probability of it belonging to
that class. Finally, this string maps onto the final result by an
Convolutional Neural Networks (CNN) activation function. This activation function can be sigmoid
for binary/multiclass classification, softmax for single/mul-
One of the major applications of neural networks was com- ticlass classification or to identity continuous values in case
puter-aided detection (CAD) that aimed to increase clas- of regression (Yamashita et al. 2018).
sification accuracy and inferencing time. A revolutionary Based on the fact that in order to train a deep model
method was proposed in LeCun et al. (1989) called convolu- a large amount of data are needed, CNN and other mod-
tional neural networks (CNN). Inspired by the vision system els’ popularity only began to rise when a large quantity of
of cats which are locally sensitive and orientation-selective, labeled data were provided for the ImageNet challenge (Rus-
as presented in LeCun et al. (1989) and Hubel and Wiesel sakovsky et al. 2015). Afterward, lots of architectures have

Fig. 2  Figures depicting CNN and RNN schematic

13
 Journal of Big Data Analytics in Transportation

been proposed which use these CNN blocks to enhance the dependencies cases, the former information related to these
efficiency of CAD. Some of these methods are AlexNet, dependencies will end up lost.
Inception, VGGNet 16/19, Resnet, etc. However, in order Long Short Term Memory (LSTM) architecture has
to increase the accuracy of detection, other concepts have been suggested by Hochreiter and Schmidhuber (1997) to
been used in the process. Some of these concepts are transfer solve both these problems together. The primary idea of
learning, which uses the knowledge of the network from this method is using a memory cell with only two gates of
retraining on a large dataset in order to train the network on input and output. The input gate decides when to keep the
a smaller dataset (Yamashita et al. 2018). The other method information in the cell and the output gate decides when to
is training with an equal prior instead of a biased prior in access the memory cell or prevent its effect on other units.
those cases where the dataset has a bias towards one of the In recent years, several corrections and improvements have
classes (imbalanced dataset). In this case, different sampling been made on LSTM architecture.
or resampling rates are applied to the dataset to balance it. As described above, LSTM contains a memory cell that
The effect of these different methods of changing the archi- holds its state over time, and based on its regulation, controls
tecture, using transfer learning and balancing the dataset for how this cell affects the network. The most common type of
various datasets are investigated in Shin et al. (2016). LSTM cell has been suggested by Graves and Schmidhu-
ber (2005). Several gates and components which are added
to this cell are different from the basic suggested LSTM
Recurrent Neural Networks (RNN) by Hochreiter and Schmidhuber (1997). A logistic sigmoid
function is usually used as the gate activation, though due
Recurrent neural networks (RNNs), another class of super- to the state-of-the-art design of Graves and Schmidhuber
vised DL models, are typically used to capture dynamic (2005), a tanh function is usually used as the block input
sequences of data. RNNs can successfully store the rep- activation and block output activation. The forget gate and
resentation of recent inputs and capture the data sequence peephole connections were first suggested by Gers and
by introducing a feedback connection to interpret the data. Schmidhuber (2001) that enables the cell to reset by forget-
This ability can play the role of memory to pass informa- ting its current state and passing the current state data from
tion selectively across sequence steps to process data at a the internal state to all gates without passing them through
certain time. Thus, each state depends on both the current an activation function.
input and the state of the network at a previous time. In other Finally, it is notable that Cho et al. (2014) has proposed
words, there is a similarity between a traditional, simple a gated recurrent unit (GRU) inspired by the LSTM block,
RNN and Markov models (Lipton et al. 2015). In 1982, the where they have eliminated the peephole connections and
first algorithm for recurrent networks was used by Hopfield output activation function. They have also coupled the input
(1982) in order to do pattern recognition. In 1990, Elman gate and forget gate into one gate called the update gate and
(1990) introduced his architecture, which is known as the what passes through their output gate is only recurrent con-
most basic RNN. A schematic figure of this architecture is nections to the block input. This architecture is much sim-
presented in Fig. 2b. In this architecture, associated with pler than LSTM and based on what it eliminates, it avoids a
each hidden unit, there is a context unit which takes the significant reduction in performance, which makes it more
exact state of the corresponding unit at the previous time as popular to use.
an input and re-feeds it with the learned weight to the same
unit in the next step. Autoencoders (AE)
Although training RNN networks seems to be straight-
forward, vanishing or exploding gradient problems remain One of the most important task in DL is access to a large
the two main difficulties. These problems can happen during amount of data to train the model. Usually, such a dataset
learning from previous states when the chain of dependen- is not readily available and producing a rich dataset would
cies gets prolonged and, in this case, it is difficult to choose be expensive. In this situation, unsupervised methods show
which information should be learned from past states. In their value. Instead of training models using labeled data,
order to solve the problem of an exploding gradient in unsupervised methods extract the features of unlabeled data
recurrent networks, which can result in oscillating weights, and use these extracted features to train the model. Autoen-
Williams and Zipser (1989) has suggested Truncated Back- coders (AEs) are one such method which aims to reconstruct
Propagation Through Time (TBPTT), which sets a certain the input data and in this manner is similar to principal com-
number of time steps as a propagation limit. In this case, ponent analysis. AEs are composed of two networks that are
to prevent exploding the gradient, a small portion of previ- concatenated to each other. The first network extracts and
ously analyzed data is collected to use during the training encodes the input data into its main features and the second
phase. However, this means that in the case of long-range network usess these features to reshape arbitrary random

13
Journal of Big Data Analytics in Transportation

data to reconstruct something similar to the input data. The return (accumulation of rewards for the episode) is calcu-
schematic figure of this architecture is presented in Fig. 3a. lated. In non-episodic MDP, there is no end of the episode
Although the concept of AEs has been used previously as and using a discount factor is vital to prevent an explosion
a denoiser (Vincent et al. 2008) and data constructor (Tan of return values (Arulkumaran et al. 2017).
and Eswaran 2008), it found a new application as variational There are two functions usually used in RL: the state-
AEs (Kingma and Welling 2013). To minimize the differ- value function, also known as the value function, is the
ence from input and output, Kingma and Welling (2013) expected return if the agent starts at a given state (no action
have used the variational inference method. They introduced limitation), whereas the action-value function, also known
a lower bound on the marginal likelihood and tried to max- as the quality function (Q-function) is the expected return
imize it to minimize the error between input and output. of starting at a given state and taking a particular action.
Doersch (2016) and Le (2015) have explained exactly how Usually, one of two methods is implemented to solve an RL
a variational AE can be built. problem. In the first approach, the Q-function is predicted
Usually, an AE’s hidden layer is smaller than its input using different methods of temporal difference controls
layer, although the opposite situation can happen as well. such as state–action–reward–state–action (SARSA), which
Also, the horizontal orientation of AEs is defined as combin- improves the estimation of Q. The second approach is Q
ing two or more AEs horizontally, and this can have differ- learning, which directly approximates the optimal Q. Both
ent motivations such as different learning algorithms (e.g., of these methods use bootstrapping and learn from incom-
RBM, neural network, or Boolean) or different initialization plete episodes.
and learning rates. In addition to details about these situa- Deep reinforcement learning (DRL) is an approach to
tions, linear and nonlinear AEs have been studied by Baldi solving the RL problem using a DNN. Although the history
(2012). It has been shown that a Boolean AE as a nonlinear of DRL began in the 1990s when Tesauro (1995) developed
type has the ability to cluster data and an AE layer on top a neural network that reached an expert level in backgam-
can be used as a pretrainer for a supervised regression or mon, its rebirth can be considered as Mnih et al. (2015)
classification task. who introduced Deep Q-Networks (DQN) as DNNs that can
approximate Q instead of reading its value from a Q table
Deep Reinforcement Learning (DRL) that indicates for each state what the Q value would be for
taking each action. In this new method, complex and high
Reinforcement learning (RL) attempts to train a machine dimensional problems have potential to be addressed easily
to act as an agent who can interact with the environment (Mnih et al. 2015). The model used by Mnih et al. (2015)
and learn to optimize these interactions by learning from extracted images from the Atari games and used a combi-
responses (Arulkumaran et al. 2017). In RL, the agent nation of a CNN model and a fully connected layer on the
observes the environment and gets a state signal and chooses data extracted from the images to obtain an estimate of the
an action that impacts the environment to produce a new Q value.
state. In the next step, a reward from the environment and However, because of the complexity of DRL, it can be
the new state is fed to the agent to help it decide more intel- unstable. Therefore, much research has been focused on
ligently in the next step. The goal of an agent in this setup solutions able to defeat this instability. Experience replay
is gaining the maximum reward over the long term by fol- (Lin 1992) and target networks (Mnih et al. 2015) are the
lowing an optimal policy. The algorithm of RL is usually two most used techniques to make RL stable. Other tech-
based on the Markov Decision Process (MDP) (Silver 2015). niques include Double-Q learning (Hasselt 2010) and
The problems that can be solved by RL algorithms can be dueling DQN (Wang et al. 2015), which have also been pro-
divided into episodic and non-episodic MDP. In episodic posed to make DRL more robust and stable. In Double-Q
MDP, the state will reset at the end of the episode and the learning, the second estimator is used for estimating an extra

Fig. 3  Figures depicting AE and GAN schematic

13
 Journal of Big Data Analytics in Transportation

assumptive Q′ to approximate the Q value more precisely. this architecture. A DCGAN allows the model to understand
On the other hand, dueling DQN (Wang et al. 2015) uses operations in latent space meaningfully and respond to these
a baseline instead of an accurate calculation of Q value to operations by acting on the semantic attributes of the input
learn relatives. (Goodfellow 2016).
The other improvisation on the GAN architecture has
Generative Adversarial Networks (GAN) been conditional GAN (Mirza and Osindero 2014), where
both networks are class conditional, which means the gen-
Generative adversarial networks (GANs) are a specific class erator tries to generate image samples for a specific class
of deep learning networks that learn how to extract the sta- and the discriminator network is trained to distinguish real
tistical distribution of training data to synthesize new data data from fake data, conditional on the particular class. The
similar to real-world data. These synthetic data can be used advantage of this architecture is better performance in mul-
for several applications such as producing high-resolution timodal data generation (Creswell et al. 2018).
images (Ledig et al. 2017), denoising low-quality images, In the next section, we discuss and review the applica-
and image-to-image translation (Isola et al. 2017). Most of tions of deep learning models to transportation.
the generative models use the maximum likelihood concept
to create a model that can estimate the probability distribu-
tion of the training data and synthesize a dataset that maxi- Applications in Transportation
mizes the likelihood of the training data (Dougherty 1995).
Although calculating maximum likelihood can directly result Performance Evaluation
in the best action of the model, sometimes these calculations
are so difficult that it is more beneficial to implicitly estimate Before reviewing papers that have already used DL meth-
this amount. In the case of explicit density calculation, three ods to investigate ITS applications, it is necessary to make
main types of models are popular: clear the model evaluation criteria used. The classification
metrics are accuracy (AC), precision (PR), recall (RL), top
• Fully visible belief networks 1 accuracy, and top 5 accuracy, while the regression met-
• Variational AEs rics are mean average precision (mAP), mean absolute error
• Markov chain approximations (MAE), mean absolute percentage error (MAPE), and root
mean squared error (RMSE):
All of these models, however, suffer from the problems
TP + TN
of low speed, low quality, and early stoppage (Goodfel- AC = (1)
TP + FP + TN + FN
low 2016). To overcome these problems, Goodfellow et al.
(2014a) has suggested a method that does not require explicit
definition of the density function. This model can generate TP
PR = (2)
samples in parallel, no Markov chain is needed to train the TP + FP
model and no variational bound is needed to make it asymp-
totically consistent. TP
RL = (3)
This method has two models: the generative model which TP + FN
is responsible to pass random noise through a multilayer net- where TP = true positive, TN = true negative, FP = false posi-
work to synthesize samples, and the discriminative model, tive, FN = false negative.
which is responsible to pass real data and artificial data Top 1 accuracy means the model’s top answer must match
through a multilayer network to detect whether the input the expected answer.
is fake or real. A schematic figure of this architecture is Top 5 is when at least one of the model’s five highest
presented in Fig. 3b. Both models use back-propagation and probability answers must match the expected answer.
dropout algorithms: the generative model to create more mAP is the mean of the average precision (AP) scores for
realistic data and the discriminative model to achieve better every query, where AP is the area under the PR vs RL curve
distinction between real and fake data. IoU is the ratio between area of overlap and area of union,
When GANs were first proposed in both their generative between the predicted and the ground truth bounding boxes:
and discriminative models, fully connected networks were
n
used. However, later in 2015, Radford et al. (2015) suggested 1∑ −
a new architecture named deep convolution GAN (DCGAN),
MAE = |yi − yi | (4)
n i=1
which uses batch normalization in all layers of both models,
except the last layer of the generator and first layer of the
discriminator. Also, no pooling or unpooling layer is used in

13
Journal of Big Data Analytics in Transportation

n | −| and Khajeh Hosseini and Talebpour (2019) have imple-

1 ∑ || yi − yi ||
MAPE = (5) mented DBNs for traffic flow prediction. Siripanpornchana
n i=1 || yi ||
| | et al. (2016) and Hou and Edara (2018) have used the same
concept for predicting travel time and traffic speed. Along
√
√ n with traffic data, weather data have been fed into DBNs
√1 ∑
RMSE = √
− 2
(yi − yi ) (6) using data fusion techniques to predict traffic flow more
n i=1 accurately (Koesdwiady et al. 2016).
However, due to the nature of the above mentioned traf-
( )2 fic features and their dependency on past traffic conditions,
−
several studies have been done to discover correlations
n
1∑ yi − yi
MSRE = (7) using RNN to predict traffic characteristics. For instance,
n i=1 yi
Zhang and Kabuka (2018) have used a gated RNN unit to
where yi is the actual value of observed travel time, yi is predict traffic flow with respect to the weather conditions,
the predicted value of travel time, and n is the number of where Jia et al. (2016) have used LSTM to overcome the
observations. same challenge. Liu et al. (2017) and Tian and Pan (2015)
We now discuss different applications of deep learn- have used LSTM to predict travel time as well as traffic
ing in ITS. The included topics have been selected based flow, while also taking into account weather conditions.
on the functional areas in ITS as mentioned in Sussman Finally, Ma et al. (2015) have implemented a combination
(2008) and have been studied substantially over the period of deep RBM and RNN to predict congestion in transpor-
of 2012–2019. tation network links.
Polson and Sokolov (2017) have tried to increase the AC
of traffic flow prediction especially for nonrecurrent traffic
Traffic Characteristics Prediction congestion, such as a special event or harsh weather, by pay-
ing more attention to the spatiotemporal feature of traffic.
One of the most considered applications of DL in transpor- This feature is grounded in the assumption that to predict
tation is related to traffic characteristics prediction. Traffic any traffic characteristic, we need both the historical data on
characteristics information can help drivers to choose their that particular location and current traffic in the neighboring
routes more wisely and traffic management agencies to man- areas. To accomplish this, Wang et al. (2016a) have tried
age traffic more efficiently. The main characteristics of inter- to combine an RNN with a CNN to pay attention to both
est are traffic flow, traffic speed, and travel time. Since these the temporal and spatial aspects of traffic. Fouladgar et al.
characteristics are not mutually exclusive, methods that are (2017), Du et al. (2017) and Goudarzi et al. (2018) have
used to predict one of them also can be used to predict the combined the power of LSTM + CNN to understand both
value for the remaining features. Due to this, methods used temporal and local dependencies to predict different traffic
to make these predictions are discussed together as follows: characteristics. Yao et al. (2018a) have considered two chal-
Based on the duration of prediction for each traffic char- lenges, the first being the dynamic dependency of traffic on
acteristic, a forecast value is usually classified as short-term temporal features, that is, in different hours of the day, this
(S) for predictions within less than 30 min, medium-term dependency may differ from one direction of traffic flow to
(M) for a prediction window between 30 and 60 min, and another direction. The second challenge has been the proba-
long-term (L) within more than 60 min (Yu et al. 2017a). bility of shifting time periods in relation to traffic density. In
Since driving behavior and traffic characteristics can vary other words, a periodic temporal dependency may shift from
across locations, results from one dataset are difficult to one time to another (e.g., on different days of the week). As
apply to other datasets (Wang et al. 2018a). Previously, traf- a result Yao et al. (2018a) designed a network consisting of
fic feature prediction has predominantly used parametric and a flow-gated local CNN network to capture the dynamic of
statistical methods, such as autoregressive integrated mov- the spatial dependencies and an LSTM network as a periodi-
ing average (ARIMA) modeling, but most of the time these cally shifted attention mechanism for handling the periodic
methods have been incapable of predicting irregular traffic dependencies. One other approach to accounting for both
flows (Wang et al. 2018a). However, through the emergence types of dependencies was taken by Ma et al. (2017). They
of machine learning and furthermore DL methods, nonpara- converted data into images representing the two dimensions
metric methods are now being used in traffic characteristics of time and space. By converting their data matrices into
prediction to achieve higher accuracy. images, they were able to use a CNN model to extract image
One of the first attempts to predict traffic characteristics features and predict the network-wide traffic speed. Yu et al.
has used deep belief networks (DBN) as an unsupervised (2019) improved this approach later by adding a temporal
feature learner. Chen et al. (2017a), Huang et al. (2014) gated convolution layer to extract temporal features.

13
 Journal of Big Data Analytics in Transportation

To extract both spatial and temporal features, Cui et al. incident is more challenging than the prediction of incident
(2018a) have used a deep model called the stacked bidirec- risk since data for the former are usually heterogeneous (i.e.,
tional and unidirectional LSTM (SBU-LSTM) model where traffic incidents happen rarely, compared to the amount of
the bidirectional LSTM considers both the backward and data for the cases where there is no incident). To overcome
forward dependencies in time-series data. Since traffic con- this issue, Yuan et al. (2017) in each step changed only one
ditions have periodicity, by analyzing both backward and feature of the data (hour, day, or location) and then checked
forward features, the AC can be increased. if the resulting data point was negative or not. In negative
One of the other models able to consider the spatiotem- cases, it was added to the pool of data to be considered.
poral property of traffic has been AE, which was proposed To measure the traffic incident risk based on surveil-
first by Lv et al. (2014) and improved by Duan et al. (2016) lance camera data, different approaches have been used. For
using denoising Stacked AE (dSAE) and Yu et al. (2017a) example, Chen et al. (2016) have used a stack denoising AE
by combining LSTM and AE to predict traffic conditions at (SDAE) to learn the hierarchical features of human mobil-
peak hours and in post-accident situations. To predict post- ity and their correlation with a traffic incident. In contrast,
accident situations, they extracted a latent representation 7 Ren et al. (2017) and (Bao et al. 2019) have implemented an
of the static features that are common in all accidents from LSTM model to evaluate risk, but Ren et al. (2017) achieved
stacks of AE and combined this with a temporal correlation better performance due to learning from more features.
to traffic flow that came from stacks of LSTM, using a linear To predict traffic incidents in a macroscopic manner,
regression (LR) layer. Yuan et al. (2017) and Pan et al. (2017) have tried imple-
Table 1 summarizes all these papers, with the columns menting DNN models, Yuan et al. (2017) by considering
from left to right describing for each study the traffic charac- the curvature of the road as well as the number of inter-
teristics investigated and its DL model, dataset, experiment sections and density of the area in order to overcome the
results (best results achieved), baseline model, and the base- spatial heterogeneity problem. For the same concern, Dong
line model’s best results, prediction window length, hyper- et al. (2018) have used AE by considering both continuous
link to the given paper and its year of publication. and categorical variables, and Yuan et al. (2018) have used
To the best of the authors’ knowledge, all studies match- a Conv-LSTM that breaks regions into smaller regions in
ing the meta-analysis criteria described in “Research order to overcome spatial heterogeneity.
Approach and Methodology” of the current paper related to If, following Yuan et al. (2018), we consider the macro-
travel time, traffic speed, traffic flow, traffic conditions, and scopic prediction of traffic incidents as not focused on any
traffic density have been tabulated here. For traffic condi- single vehicle, but instead as predicting the probability of an
tions, the goal was to predict if the road is congested or not. accident between any pair of vehicles in the wider region,
Results performed on multiple datasets are also represented microscopic incident prediction studies can also be intro-
in Table 1. To have uniformity, the best results are those duced that—by getting data about the location, speed, and
achieved when the window length is ‘S’ (short-term). This direction of each vehicle in the surrounding area—predict
table structure is followed across all tables in this paper. the probability of an incident in the near future between any
certain pair of vehicles. In this regard, Chen et al. (2018b)
Traffic Incident Inference and Theofilatos et al. (2019) have trained a DNN to predict
likely collisions. Theofilatos et al. (2019) have used a simple
The goals of predicting traffic incident risk for a given loca- NN with four layers, which, though it does not compare well
tion as well as incident detection based on traffic features are with the baseline results of machine learning (ML) tech-
to help traffic management agencies to reduce incident risk niques, is still preferred, as the ML techniques have poor
in a hazardous area and traffic jams in incident locations. sensitivities.
Although there are parameters such as drivers’ behavior, that Suzuki et al. (2018) have annotated their large dataset
are not very predictable, there are several key features that of near-miss traffic accidents to train a quasi-RNN model.
can help predict traffic incidents. The innovation of their work was introducing an adaptive
Human mobility (Chen et al. 2016), traffic flow, geo- loss function for early anticipation (AdaLEA), which gives
graphical position, weather, time period, and day of the their model the ability to predict a collision 3.65 s before it
week [97] are some of these features that can be investi- happens.
gated as indicators of a traffic incident. However, a single Another challenge in traffic incident inferencing is detect-
model cannot generally be used in different places because ing an accident by processing only raw data. To address this,
accident factors in metropolitan areas, where the population Hatri and Boumhidi (2018) and Singh and Mohan (2018)
and vehicles are generally dense, are completely different have used a stacked AE (SAE) to extract the features of
from accident factors in a small town with a scattered popu- traffic patterns in the context of an accident. Also, Hatri
lation (Yuan et al. 2017). The prediction and detection of an and Boumhidi (2018) have used a fuzzy DNN to control

13
 Table 1  Overview of papers using deep learning techniques for traffic characteristic prediction
Characteristic Model Dataset Experiment results Baseline model results Length References Year
MAPE% Others MAPE% Others Model

Travel time AE Simulated data 5.79 M Gang et al. (2015) 2015

DBN PeMS 3 S,M,L Siripanpornchana et al. 2017
(2016)
LSTM + DNN PeMS 0.961 1.006 Ridge Reg S Liu et al. (2017) 2018
LSTM + DNN St.Louis transport net 7.09 9.8 Instantanious travel Length Hou and Edara (2018) 2018
time
Traffic speed DBN Beijing Arterial 5.809 5.968 BP-NN S,M Jia et al. (2016) 2016
eRCNN Beijing Ring 0.19 0.21 CNN S Wang et al. (2016a) 2017
Journal of Big Data Analytics in Transportation

CNN Beijing Ring AC:93.21% AC:91.7% OLS S Ma et al. (2017) 2017

SBU + LSTM INRIX 5.674 6.3 Random Forest S Cui et al. (2018a) 2018
DLSTM G42toll data 10.8 S Ding et al. (2019) 2019
Bi-LSTM AVI,Xuancheng MSE:7.32 MSE:10.51 LSTM NN S,M Wang et al. (2019) 2019
CNN + LSTM Rozelle road segment RMSE:4.46(km/h) RMSE:4.48(km/h) LSTM S,M Nguyen et al. (2019) 2019
CNN Traffic Centre, Seoul RMSE:2.48(km/h) RMSE:2.75(km/h) LSTM S,M,L Jo et al. (2018) 2019
LSTM + GRU​ RTMS, Beijing 5.85 8.61 LSTM L Gu et al. (2019) 2019
GCN Beijing Ring 9.57 10.25 LSTM S Zhang et al. (2019) 2019
Traffic flow DBN PeMS EESH AC:90% AC:86% NN S,M,L Huang et al. (2014) 2014
SAE PeMS 6.75 7.4 RBF NN S,M,L Lv et al. (2014) 2015
LSTM PeMS 6.49 7.63 SAE S,M,L Tian and Pan (2015) 2015
DBN PeMS RMSE:0.06(v/15 min) RMSE:0.063 ANN L Koesdwiady et al. 2016
(2016)
DSAE PeMS 23.2 23.7 NN L Duan et al. (2016) 2016
LSTM + DBN Beijing Ring 11.69 12.84 LSTM S,M Jia et al. (2017) 2017
LSTM + SAE PeMS 5 10 Random Walk S,M,L Yu et al. (2017a) 2017
CNN PeMS RMSE:0.0061 L Fouladgar et al. (2017) 2017
LSTM Beijing Arterial 6.05 6.32 SAE S,M,L Zhao et al. (2017) 2017
RNN + CNN PeMS RMSE:0.028 (v/5 min) RMSE:0.030 LSTM L Du et al. (2017) 2018
DBN Visum Simulator 8.75 9.39 ARIMA-PSO S,L Goudarzi et al. (2018) 2018
ConvLSTM Beijing Traffic RMSE:6.95(km/h) RMSE:7.359 LSTM L Yang et al. (2018a) 2018
DNN PeMS RMSE:27.91(V/5 min) RMSE:29.84 DeepST M Wu et al. (2018a) 2018
CNN Washington Interstate RMSE:5.5(V/5 min) RMSE:15 DSAE L Zhuang et al. (2018) 2018
DBN PeMS 3.19 5.32 SAE S,M,L Arif et al. (2018) 2018

13
 Table 1  (continued)
Characteristic Model Dataset Experiment results Baseline model results Length References Year

13
MAPE% Others MAPE% Others Model

CNN BJER4 + PeMS 9.11 9.31 Graph GRU​ S,M Yu et al. (2017) 2018
LSTM + CNN NY Bike/Ca 16.3 17.36 DMVST-Net M Yao et al. (2018a) 2018
LSTM + CNN INRIX data Seattle 3.28 3.74 SGC-LSTM L Cui et al. (2018b) 2018
LSTM AIMSUN intersection RMSE:7.931(V.sec) RMSE:8.586 Kalman Filter L Lee et al. (2018) 2018
CNN NGSIM 3.84 8.56 SVR S Khajeh Hosseini and 2019
Talebpour (2019)
CNN PeMS 0.105 0.124 GCNN S,M Dai et al. (2019) 2019
DNN Alberta Intersections R-squared:0.89 Tawfeek and El-Basy- 2019
ouny (2019)
GAN PeMS 36.91 37.63 SAE S Lin et al. (2018a) 2019
SAE Beijing Roads RMSE:18.30 RMSE:20.8 LSTM S Li and Wang (2017) 2019
SAE + DNN Beijing Car 12 15 BPNN S Zhao et al. (2019) 2019
Traffic conditions RBM + RNN Ningbo, China AC:88.2% AC:71% SVM L Ma et al. (2015) 2015
CNN VPRs Jinan, China RMSE:0.241 RMSE:0.285 LSTM S,M,L Chen et al. (2018a) 2018
DLN + CNN Motorway dataset,UK AC:89% AC:70.5% AE Pamula 2018) 2018
YOLO IOWA DOT CCTV AC:91.4% AC:85.7% SVM Chakraborty et al. 2018
2018)
DBN + SVR PEK airport data RMSE:12.65 (min) RMSE:16.01 KNN Yu et al. (2019) 2019
CNN + RNN Taxi-Schenzan data AC:97.59% AC:77.92% GRU​ S Guo et al. (2019) 2019
Traffic density CNN Seoul Intersection RMSE:4.15 (carcount) RMSE:7.109 Crowd CNN Chung and Sohn (2017) 2018
CNN Traffic Simulator 3.68 9.34 ARIMA S Khajeh Hosseini and 2019
Talebpour (2019)
Journal of Big Data Analytics in Transportation
Journal of Big Data Analytics in Transportation

the learning of traffic-incident-related parameters. Zhang and saving other features of each car besides the scheme of
et al. (2018a) have trained their DBN model on a dataset the license plate to do Re-ID. Also, Yu et al. (2017b) have
that includes tweets related to traffic accidents, showing that used faster RCNN to detect vehicles in images. In addition, a
non-traffic features can be used along with traffic feature data modified version of the Single Shot Detection (SSD) method
to validate traffic incident detection. to localize and classify the different types of construction
Incident severity prediction based on recorded incident equipment by employing MobileNet as the feature extrac-
features have been studied in Wang et al. (2016a), Sameen tion network has been done by Arabi et al. (2020). Wu et al.
and Pradhan (2017) and Alkheder et al. (2017). The artificial (2018b) has worked on the same idea but trained their model
neural network (ANN) trained in Alkheder et al. (2017) has based more on spatiotemporal data, pruning their results
shown an improvement in baseline performance as com- with the fact that (1) a vehicle cannot be in two places at
pared to the LSTM model with fully connected layers in one time and (2) a vehicle that has already passed a section
Sameen and Pradhan (2017). is unlikely to pass it again. However, their model could not
Table 2 summarizes all these papers, shows their model, compete with the model defined in Tang et al. (2018) that
the dataset which their model was trained on, evaluation of proposed a Markov chain random fields to prepare several
their model for their testing dataset as well as comparison queries based on a visual spatiotemporal path and then used
of their model’s performance to that of their baseline model. a combined Siamese-CNN and path-LSTM model.
In the first section of this table, different studies regarding Table 3 summarizes all these papers, shows their mod-
parameters effective in predicting increased incident risk and els, the dataset which model is trained on, and their per-
the manner in which incident risk is affected are listed. In the formances on those dataset and comparison to the baseline
next section, macroscopic studies on incident prediction are model.
categorized as “traffic incident prediction,” whereas micro-
scopic studies are categorized as “collision prediction.”
In the incident detection 9 section, all studies focused on Traffic Signal Timing
detecting incidents by analyzing raw traffic data have been
gathered and, finally, in the last section, investigations pre- One of the main tasks of ITS management based on multiple
dicting the severity of the incident are listed. types of data is controlling traffic via traffic signal lights.
For several years, research on optimizing signal light timing
Vehicle Identification to have the best performance has been one of the greatest
challenges in the transportation field. The results of studies
Applications of re-identification (Re-ID) vary from calculat- in this area have endowed traffic agencies with analytical
ing travel time to automatic ticketing. Since license plates models that use mathematical methods to address this opti-
are unique to each vehicle, the first task in Re-ID is recog- mization problem. However, through emerging DL studies,
nizing them. modeling the dynamics of traffic to achieve the best perfor-
Zang et al. (2015) and Abedin et al. (2017) have imple- mance has taken a new path. This is because the nature of
mented DL models to recognize license plates by using a RL has facilitated its application in different studies to find
visual attention model that first generates a feature map the best traffic signal timing.
using a combination of the most commonly used colors in Li et al. (2016) has used DRL to tackle traffic light tim-
license plates, extracts data from plates using a CNN model, ing. In DRL, a DL model is usually used to implement the
and ultimately runs an SVM on the extracted data. How- Q-function in a complex system to capture the dynamics
ever, bad lighting, blurriness due to vehicle movement, low of traffic flow. A dSAE network is used to take the state
camera quality, and even traffic occlusion where the plate as input and give the Q-function for any possible action
is covered behind other cars can make reading license plate as the output of the network. Li et al. (2016) has shown
characters impossible. To overcome this, Liu et al. (2016) a 14% reduction in cumulative delay in the case of using
have proposed a CNN layer to extract conspicuous features an SAE to predict the Q-function instead of conventional
such as the color and model of the vehicle and have used a prediction.
Siamese neural network to distinguish similar plates. (This Gao et al. 2017) has suggested an alternative novel idea
network has been used before in signature verification tasks). for choosing RL states. They argue that instead of taking
Note that for some feature extractions, such as vehicle color raw data as the state, it could be more effective if the CNN
recognition, solutions like what Hu et al. (2015) did using a extracts important features from the raw data—e.g., the posi-
combination of CNN for feature extraction and SVM for cat- tion of the cars and their speeds—and feeds it to a DRL net-
egorizing are also available. Tang et al. (2018) have similarly work with a fully connected network to predict the Q-value
used a histogram-based adaptive appearance model like what for each of four states of green, yellow, red, and protected
Zheng et al. (2017) did for target re-identification, detecting left turn light, considering cumulative staying time as the

13
 Journal of Big Data Analytics in Transportation

Table 2  Overview of papers using deep learning techniques for traffic incident inference
Characteristic Model Dataset Experiment results Baseline results References Year
RMSE Others RMSE Others Model

Incident risk SdAE 7 month Heteroge- 1 1.41 Logistic Regr Chen et al. (2016) 2016
neous data
LSTM Accident records 0.63 0.75 SdAE Ren et al. (2017) 2017
(Beijing)
LSTM NYPD 9.44 10.46 CNN Bao et al. (2019) 2019
DNN Accident records AC:85% AC:69% Decision Tree(DT) Ali et al. (2019) 2019
(VTTI)
DBN Civil Aviation Data MSE:0.2 MSE:0.05 SVR Ni et al. (2019) 2019
Traffic incident DNN Accident records AC:95.12% AC:89.58% RF Yuan et al. (2017) 1207
prediction (IOWA)
DBN Highways dataset 1.48 1.6 Bayesian ANN Pan et al. (2017) 2017
AE TRIMS and PMS MAE:0.150 MAE:0.660 SVM Dong et al. (2018) 2018
LSTM Accident records 0.078 0.121 7 year Avg Yuan et al. (2018) 2018
(IOWA)
Collision predic- DNN Internet of vehicles Chen et al. (2018b) 2018
tion DBN Collision Data, 15.24 16.51 Bayesian NN Pan et al. (2018) 2018
Ontario
LSTM Annotated data mAP:62.1% mAP:57.8% RNN Suzuki et al. 2018
(2018)
DNN Attica Tollway AC:68.95% AC:72.15% DT Theofilatos et al. 2019
(2019)
Incident detection SAE SUMO MSE:0.13 MSE:0.18 DNN Hatri and Boum- 2018
hidi (2018)
DBN NYC + NOVA AC:85% AC:79% SVM Zhang et al. 2018
accidents (2018a)
Incident severity LSTM Accident records AC:71.77% AC:70.30% Bayesian LR Sameen and Prad- 2017
prediction (Malaysia) han (2017)
DNN Accident records AC:74.6% AC:59.5% Ordered Probit Alkheder et al. 2017
(AbuDhabi) (2017)
CNN Accident records 0.231 Das et al. (2018) 2018
(Louisiana)

reward. They have also used the experience replay and target The results of this study showed that using high-resolution
network techniques to stabilize the algorithm and converge data is not substantially effective and conventional data are
it to the optimal policy as suggested in Tan and Eswaran good enough for their model. However, one of the reasons
(2008). that may have contributed to this conclusion is that they used
Liang et al. (2018) have also used CNN to map states. They a simple fully connected model that could not extract deep
use several state-of-the-art techniques such as the target network, features from more precise states very well.
experience replay, double Q-learning network, and dueling net- Finally, Wei et al. (2018) have tested their model on real-
work methods to increase the performance of the network and world traffic data to see how effective its results could be. They
make it stable. Their results have shown a great reduction in suggest that instead of only studying the reward, we need to
waiting time (more than 30%) for a fixed-time scenario. consider different policies that may result in the same reward
Genders and Razavi (2018) have investigated the impor- and then take the most feasible one. The final results of this
tance of choosing delay time states. The main goal of this study have shown great performance in reducing queue length,
study was investigating whether the data from conventional delay time, and duration compared with other methods.
sensors, such as occupancy and average speed, are satisfac- Table 4 summarizes all these papers, shows their model,
tory or more precise data are needed, such as vehicle density the dataset which their model was trained on, and the per-
and queue length, or even data with the highest resolution, formance of their model for the testing dataset as well as
such as discretizing each incoming lane into cells and con- comparison of their model’s performance to that of the base-
sidering the presence of a vehicle in each cell separately. line model.

13
Journal of Big Data Analytics in Transportation

Table 3  Overview of papers using deep learning techniques for vehicle id tasks
Charac- Model Dataset Experiment results Baseline rResults Refer- Year
teristic ences
AC% Others AC% Others Model

License SIFT + SVM Chinese PR:98.6 PR:98.4 CNN + SVM Zang 2015
plate license et al.
recog- plate (2015)
nition CNN On road data 99 Bulan 2017
et al.
(2017)
CNN Bangla 92 Abedin 2018
license et al.
plate (2017)
Kernel-CNN Chinese 96.38 93.35 SVM-RBF Yang 2018
license data Kernel et al.
(2017)
CNN Thailand 96.94 Puarun- 2018
license groj and
plate Boon-
siri-
sumpun
(2018)
CNN AOLP PR:99.5 PR:90.7 Single shot Xie et al. 2018
detector (2018)
Vehicle CNN Generated 99.07 Huang 2015
type dataset et al.
classi- (2015)
fication
CNN BIT-vehicle 96.1 93.7 SVM Dong 2015
dataset et al.
(2015)
Faster RCNN Vehicle 89 Yu et al. 2017
dataset (2017b)
CNN VEDAI/ 54.6/73.7 32/53.9 Fast RCNN Zhong 2017
Munich et al.
dataset (2017)
CNN ILSVRC2012 98.29 83.78 Ensemble Fang et al. 2017
classifiers (2016)
CNN Iowa CCTV PR:95 Adu- 2017
data Gyamfi
et al.
(2017)
Deep CNN CarFlag/ 98.67/99.3 90.34/93.58 Hu et al. 2017
CompCars (2017a)
Deep CNN XMUPlus 99.1 99.07 Pre-trained Soon 2018
CNN et al.
(2018a)
Deep CNN MIT-CBCL/ 94.12/95.04 93.71/94.27 RCNN Li et al. 2018
Caltech (2018a)
CNN COSMO- 97.66 95.48 CNN Wang 2018
SkyMed et al.
(2018b)
CNN CompCars 54.56 42 Pre-trained Wang 2018
CNN et al.
(2017a)

13
 Journal of Big Data Analytics in Transportation

Table 3  (continued)
Charac- Model Dataset Experiment results Baseline rResults Refer- Year
teristic ences
AC% Others AC% Others Model

Faster RCNN + RPN ILS- mAP:89.93% mAP:89.12% Faster RCNN Xiang 2018
VRC-2012 et al.
(2018)
LSTM Fleetmatics 85 Simoncini 2018
data, US et al.
(2018)
D-CNN CompCars Top 5:0.922 Top 5:0.917 CNN Yan et al. 2018
(2017)
Fast RCNN + RPN MIT/ 84.4 84 Fast RCNN Suhao 2018
CALTECH et al.
dataset (2018)
Deep CNN + AE Chengdu 97.62 95.18 CNN Chang 2018
express- et al.
ways (2018)
CNN LabelMe/BIT 98.95/95.12 Hussain 2019
datasets et al.
(2018)
PCA-CNN PLUS Malay- 99.51 98.65 Ensemble Soon 2019
sia NSE Classifier et al.
(2018b)
RE ID CNN + SNN VeRi-776 mAP:27.77% mAP:18.49% CNN Liu et al. 2016
(2016)
MRF + SNN + LSTM VeRi-776 mAP:58.27% mAP:46.25% LSTM + CNN Shen et al. 2017
(2017)
CNN VOT2016 54 54 CNN Tapu et al. 2017
(2017)
CNN AI City Chal- PR:99.25 Tang et al. 2018
lenge (2018)
CNN + AFL VeRi AI City mAP:57.43% mAP:58.27% CNN Wu et al. 2018
Challenge (2018b)
CNN AI City Chal- PR:99.25 Maŕın- 2018
lenge Reyes
et al.
(2018)
RNN Brisbane PR:37.5 Choi et al. 2018
vehicle data (2018)
Faster RCNN Korea High- MAPE:3.4% MAPE:4.9% SSD Kim et al. 2019
ways (2019a)
Vehicle CNN SVM Vehicle Color 93.78 91.89 SVM Hu et al. 2015
color Dataset (2015)
recog-
nition

Ride Sharing and Public Transportation has endowed companies with increasingly optimal routing
maps that take into account data such as passenger demand
Public transportation systems (including bus or metro for a given mode of 11 travel at particular places and times.
systems, taxis, etc.) are one of the main means of moving DL has been adopted to make predictions even more accu-
passengers within cities. To increase city planning perfor- rate compared to existing ML techniques.
mance and also passenger satisfaction, the nature of DNN

13
Journal of Big Data Analytics in Transportation

Saadi et al. (2017) have investigated the performance of choose at each time point. Also, “passenger flow” is defined
several ML techniques and a fully connected DL model with as the number of passengers flowing in or out of a given
only two hidden layers and have shown that their very simple location at a certain time point).
DL model outperforms almost all other techniques except a
boosted decision tree. Besides the simple DNN models in Visual Recognition Tasks
Dominguez-Sanchez et al. (2017), Jung and Sohn (2017),
Wan et al. (2018) and Zhu et al. (2018b), a hybrid model One of the most significant applications of DL is the use
containing a stacked AE and a DNN has been implemented of nonintrusive recognition and detection systems, such as
by Liu and Chen (2017) to predict hourly passenger flow. camera-image-based systems. These applications can vary
To capture all related features such as the spatial, tempo- from providing a suitable roadway infrastructure for driving
ral, and exogenous features impacting passenger demand, vehicles to endowing the autonomous vehicles with a safe
a fusion convolutional LSTM network (FCL-Net (Ke et al. and reliable driving strategy.
2017) has been proposed. This network includes stacked One of the first visual recognition challenges tackled has
Conv-LSTM layers to analyze spatiotemporal variables, such been obstacle detection via exploiting vehicle sensors. To
as historical demand intensity and travel time, and LSTM do this, a variety of networks with unique architectures have
layers to evaluate nonspatial time-series variables, such as been implemented. Kim and Ghosh (2016) have merged data
weather, day of the week, and time of the day. With the from an RGB camera and LIDAR sensors to increase obsta-
same idea, Zhang et al. (2017) has proposed a spatiotem- cle detection performance in different illumination condi-
poral Resnet (ST-Resnet) which includes several convolu- tions. Dairi et al. (2018a, b), on the other hand, have con-
tional layers. Liao et al. (2018) has implemented both of fronted obstacle detection as an anomaly detection problem.
these techniques on a New York City taxi record dataset and They have used a hybrid encoder model to extract features of
their comparison has shown that better performance with a Deep Boltzmann Machine (DBM) and then an autoencoder
faster training time can be achieved using ST-Resnet. The to reduce the dimensionality and obtain vertical disparity
authors suggest two reasons for this. First, LSTM captures (V-disparity) map coordinate system data from images. The
fine temporal dependencies which are not as fundamental as key feature of V-disparity data is that these data are mostly
the coarse-grained dependencies from the convolutional lay- stable with small variations from noise and they change
ers. Their second explanation is that spatial features may be drastically only if an obstacle appears in an image.
more important than temporal ones and since the ST-Resnet Wang et al. (2016b) and Cai et al. (2016) have used data
focuses more on spatial features, it outperforms the FCL- from far-infrared sensors to improve vehicle detection at
Net. Zheng et al. (2017) and Lin et al. (2018b) work directly night. While the former used only far infrared data, the lat-
on graphs structures to leverage structural information by ter, in order to decrease the false positive percentage used
considering the nodes as stations and the edges as depend- both camera and far-infrared data. Wang et al. (2016c) have
encies among stations. Finally, Yao et al. (2018b) and Ma tried to address requirements in regard to vehicle following,
et al. (2018) have proposed a deep multiview spatiotemporal which include detecting brake lights. They used the Histo-
network to capture all dependencies separately. gram of Oriented Gradient (HOG) approach implemented
Another research area related to public transportation with LIDAR and camera data. To decrease the false positive
deals with travel mode selection. Nam et al. (2017) has rate and speed up the process, they also used the vanishing
implemented a simple fully connected DNN on Swiss Metro point technique. Next, they used AlexNet to detect if the rear
data to reveal demand based on mode. Another issue for middle brake light was on or off.
transportation network companies is route scheduling for Another important task in navigating safely is traffic
their drivers to pick up passengers in order to minimize pas- sign detection. These signs obligate, prohibit or alert driv-
senger waiting time as well as cost for the driver and com- ers. One of the most common DL models to detect traffic
pany. Shi et al. (2018) has suggested a DRL model aiming signs are CNNs. Qian et al. (2015), Yang et al. (2015), Lin
to give drivers the best route. This paper considers different et al. (2016, 2019), Lim et al. (2017), Zeng et al. (2016),
factors such as the current location of vehicles, time of day, Hu et al. (2017b), Yuan et al. (2016), Arcos-Garcia et al.
and competition between drivers, resulting in a significantly (2018), Natarajan et al. (2018), Lee and Kim (2018), Li et al.
shorter search time and more long-term revenue for drivers. (2018b) and You et al. (2018) have all used CNN as their
Table 5 summarizes all these papers, shows their model, main feature extractor, each trying to tune their model to get
the dataset which their model was trained on, and evaluation the best results. Qian et al. (2015) have used RCNN to derive
of their model for their testing dataset as well as comparison regions of interest from RGB images. Lim et al. (2017) have
of their model’s performance to that of their baseline model. focused on low-illumination images. They used a classifier
(In this table, “travel mode” refers to studies which tried to to detect regions of interest and an SVM to verify if any traf-
predict the mode of transportation that passengers would fic signs were present inside the region or not. Then, a CNN

13
 Journal of Big Data Analytics in Transportation

Table 4  Overview of papers using deep learning techniques for traffic signal timing
References Year Model Dataset State Reward Actions

Li et al. (2016) 2016 SAE + DRL PARAMICS Queue length —Queue length difference— 2
Pol and Oliehoek (2016) 2016 DQN SUMO Position, speed Number of stop switch and 2
delay
Gao et al. (2017) 2017 CNN + DRL SUMO F (position, speed) (Cumulative staying time) 4
Liang et al. (2018) 2017 CNN + DRL SUMO Position, speed (Cumulative waiting time) 8
Mousavi et al. (2017) 2017 DQN SUMO Snapshot of the current state Difference between the total 2
of a graphical view of the cumulative delays oftwo
intersection consecutive actions
Genders and Razavi (2018) 2018 DRL SUMO Occupancy and speed/vehicle Change in cumulative delay 4
density and queue length/pres-
ence of vehicles in each lane
Wei et al. (2018) 2018 DRL SUMO Queue length, number of vehi- F(queue length, delay, updated 2
cles, updated waiting time, waiting time, light switches
current phase, next phase and indicator, number of vehicles
an image of the intersection pass the intersection, travel
analyzed by CNN time)
Wan and Hwang (2018) 2018 DQN VISSIM Current phase, green and red System delay 8
duration, remaining carsand
left turn bay occupation
Muresan et al. (2019) 2018 DRL VISSIM Queue length, signal state, and Discharged vehicle 2
time of day
Liang et al. (2019) 2019 3DQN SUMO The position and speed of Change of the cumulative 8
vehicles waiting time between two
neighboring cycles
Gong et al. (2019) 2019 3DQN Simulatedtraffic Current traffic state and current Difference between the current 4
signal phase and previous waiting times of
all vehicles
Huang et al. (2019) 2019 DQN SUMO Number of input and output Summation of que length in 4
vehicles of adjucents intersec- multiple intersections
tions

model using the Byte-MCT technique classified the traffic image-processing techniques such as image drizzling and
sign. Experiments have shown that this method is robust in gray-scale normalization to reduce noise.
deficient lighting, outperforming other methods in cases of Weber et al. (2016), Behrendt et al. (2017) and Kim
low illumination. et al. (2018a) have focused more on traffic light detection
Zeng et al. (2016) have suggested that the RGB space and classification. This has a very significant role in man-
cannot provide as much useful data as the perceptual lab aging traffic, and correct detection has a high correlation
color space. Therefore, after space changing, they extracted to reduced risk. Weber et al. (2016) have proposed their
the deep perceptual features using a CNN and fed these fea- deep traffic light recognition (DeepTLR) model that first
tures to a kernel-based ELM classifier to identify the traffic classifies each fine-grained pixel of the input data, calcu-
sign. This classifier used the radial basis function to map the lating the probability for each class. Then, for the regions
features in a higher dimension space in order to disconnect with higher probability toward the presence of a traffic
features to get the best outcome. light, a CNN was used to classify the status of the traf-
Arcos-Garcia et al. (2018) have tried different optimi- fic light. (In this model, temporal data were not used and
zation methods on a CNN model containing several con- each frame was analyzed separately). However, Behrendt
volutional layers and spatial transformer networks (STN) et al. (2017) have used traffic speed information as well
that make the CNN spatially independent, resulting in no as stereovision data to track detected traffic lights. Lin
need for supervised training, data augmentation or even nor- et al. (2016) have used a combination of region-of-interest
malization. In contrast, Li and Yang (2016), instead of using (ROI) performance, CNN feature extraction and an SVM
a CNN, have used a DBM that is boosted with canonical as a classifier to detect arrow signs on the roadway and
correlation analysis for feature extraction and then an SVM classify their direction. Gurghian et al. (2016) have used
for classification. Also, they have used certain conventional a CNN to detect lane position in the road.

13
Journal of Big Data Analytics in Transportation

Finally, the monitoring of civil infrastructure has always several advantages, such as efficient and fast intelligent deci-
been a focus for engineers and researchers. Various moni- sion-making as well as decreased data transfer cost. Emerg-
toring techniques have been used for infrastructure perfor- ing technologies such as DL have significantly increased the
mance evaluation, ranging from conventional short-term importance of edge computing devices. Though discussing
(Arabi et al. 2018) and long-term (Arabi et al. 2019, 2017; edge computing devices in detail goes beyond the scope of
Constantinescu et al. 2018) sensor-based monitoring to non- this paper, we briefly overview and compare the edge com-
destructive and noncontact techniques (Moll et al. 2018). puting devices popularly used for DL. Figure 4 illustrates the
Among the applications of nondestructive damage detec- various edge computing platforms discussed in this section.
tion, pavement crack detection, in particular, has received Also, Table 7 summarizes the technical specifications of the
attention, due to its importance in civil infrastructure man- covered hardware.
agement. For instance, Hosseini et al. (2020) and Hosseini The Jetson Xavier is the high-end system-on-a-chip (SoC)
and Smadi (2020) developed pavement prediction models computing unit in the Jetson family, which exploits the Volta
that can help agencies to come up with more accurate main- GPU. An integrated GPU with Tensor Cores and dual Deep
tenance and rehabilitation activities. Zhang et al. (2018c) Learning Accelerators (DLAs) make this module ideal to
have proposed a unified pavement crack detection approach deploy computationally extensive DL based solutions.
that can distinguish between cracks, sealed cracks, and back- NVIDIA Jetson Xavier is capable of providing 32 TeraOPS
ground regions. Through their approach, they have been of computing performance with a configurable power con-
able to effectively separate different cracks having similar sumption of 10, 15 or 30 W.
intensity and width. Moreover, Bang et al. (2019) have pro- Another widely used embedded SoC is NVIDIA Jet-
posed pixel-level pavement crack detection in black-box son TX2 which takes advantage of NVIDIA Pascal GPU.
images using an encoder-decoder network and found that Although it delivers less computing performance than
ResNet-152 with transfer learning outperformed other net- NVIDIA Xavier, it can be a reliable edge computing
works. Additionally, CrackNet, which performs pixel-level device for certain applications. The module can provide
pavement crack detection on laser-based 3D asphalt images, more than 1TFLOPS of FP16 computing performance
was introduced by Zhang et al. (2018d). In a separate using less than 7.5 W of power consumption. The Jetson
study, Zhang et al. (2018d) extended their previous study Nano, which utilizes the Maxwell GPU, is newest product
to CrackNet-R, which utilizes RNN with a gated recurrent from the Jetson family introduced by NVIDIA. It is suit-
multilayer perceptron (GRMLP) to update the memory of able for deploying computer vision and other DL models
the network, showing their model outperforms other models and can deliver 472 GFLOPS of FP16 computing perfor-
based on LSTM and GRU. Also, Nhat-Duc et al. (2018) have mance with 5–10 W of power consumption.
investigated pavement crack detection performance using Another family of edge computing devices is the Rasp-
metaheuristic-optimized Canny and Sobel edge detection berry Pi family, which introduces affordable SoCs capable
algorithms, comparing these algorithms with their proposed of high performance in basic computer tasks. The Rasp-
CNN and confirming the superior performance of DL over berry Pi3 Model B + is the latest version of the Raspberry
conventional edge detection models. Pi which uses a 1.4-GHz 64-bit quad-core processor and
Table 6 summarizes all these papers, shows their model, can be used alongside deep learning accelerators to achieve
the dataset which their model was trained on, and evaluation high performance in computationally expensive tasks.
of their model for their testing dataset as well as comparison Finally, the Intel Neural Computing Stick 2 (NCS 2) is a
of their model’s performance to that of the baseline model. USB-sized fanless unit, which utilizes the Myriad X Vision
Processing Unit (VPU) that is capable of accelerating com-
putationally intensive inference on the edge. Very low power
Discussion and Conclusion consumption along with supporting popular DL frameworks
such as Tensorflow and Caffe have made the NCS 2 ideal to
Hardware use with resource-restricted platforms such as Raspberry Pi3
B + . There have been limited studies investigating the infer-
Generally, there are two types of intelligent decision-mak- ence speed of these hardware, though Arabi et al. (2020) has
ing, namely cloud-computing-based and edge-computing- compared the inference speed of an SSD-MobileNet model
based. While computing services are delivered over the of the abovementioned embedded devices on a construction
internet via the cloud computing approach, they are per- vehicle dataset. Utilizing the Jetson TX2, they achieved 47
formed at the edge of the network via the edge-computing FPS, and utilizing a Raspberry Pi and NCS combination,
approach. The edge-computing approach has introduced they achieved 8 FPS.

13
 Journal of Big Data Analytics in Transportation

Table 5  Overview of papers using deep learning techniques for ride sharing and public transportation
Charac- Model Dataset Experiment Results Baseline Results References Year
teristic
RMSE Others RMSE Others Model

Travel DNN Swiss Metro AC:66.1% AC:65.57% ANN Nam et al. 2017
mode dataset (2017)
CNN GPS—GeoLife AC:84.8% AC:78.1% RF Dabiri and 2018
project Heaslip
(2018)
Route DRL Didi Chuxing Shi et al. 2019
sched- (2018)
uling
Passenger SAE + DNN Xiamen bus 50.4 51.4 SVM Liu and 2017
flow station Chen
(2017)
CNN Passenger data AC:96% Dominguez- 2017
Sanchez
et al.
(2017)
CNN AFC, Seoul AC:60.10% AC:54.83% Statistics Jung and 2017
Sohn
(2017)
DNN Unity-3D envi- Wan et al. 2018
ronment (2018)
DNN Shanghai rail MSRE:0.00000125 MSRE:0.00178 Lin. Regr Zhu et al. 2018
transit (2018b)
CNN California HTS AC:93.59% AC:69.76% RF Cui et al. 2018
(2018c)
LSTM Nanjing Metro 8.19 11.54 ARIMA Liu et al. 2019
System (2019)
AE + LSTM Singapores 20.37 24.82 LSTM Hao et al. 2019
Metro System (2019)
Passenger ConveLSTM Didi Chuxing 0.016 0.0175 CNN Ke et al. 2017
demand (2017)
predic- CNN Beijing taxi/NY 16.69/6.33 18.18/7.43 DNN Zhang et al. 2017
tion bike (2017)
DNN Didi Chuxing 20.09 16.41 DT Saadi et al. 2017
(2017)
CNN + LSTM Didi Chuxing 9.642 10.012 XGBoost Yao et al. 2018
(2018b)
GCNN-DDGF Citi Bike data, 2.12 2.43 XGBoost Lin et al. 2018
NY (2018b)
LSTM TAZ MAPE:46.49% MAPE:65.128% XGBoost Xu et al. 2018
Nanjing,China (2018)
CNN + RNN Porto Taxi Tra- AC:78.80% AC:75.62% CNN Zhang et al. 2018
jectory (2018b)
DQN London travel Waiting time:158.2 Wen et al. 2018
data (2017)
CNN Citi Bike Sys- 18.995 19.784 NN Yang et al. 2018
tem, NY (2018b)
DNN NYC taxi data 11.13 16.05 LSBoost Liao et al. 2018
(2018)
CNN + LSTM Beijing metro 7.5 8.89 LSTM Ma et al. 2019
(2018)
GCN Seouls Bike data 2.26 2.45 LSTM Kim et al. 2019
(2019b)

13
Journal of Big Data Analytics in Transportation

Summary AlexNet and ResNet architectures being the most popular

to build on. This can be attributed to the fact that visual
Below, we provide a summary of the studies cited in the cur- recognition tasks are not limited to ITS, so research done
rent paper. We have classified these studies according to our in other domains can be utilized to accomplish ITS-
six ITS application categories in relation to the DL models related visual recognition tasks.
they use (see Fig. 5). The following are our observations:
Based on all the studies reviewed in the current paper,
• Traffic characteristics: CNN, RNN, and CNN-RNN deep learning as an approach for addressing intelligent trans-
hybrid models are most frequently used. The main rea- portation problems has undeniably achieved better results as
son is undoubtedly related to the nature of traffic that has compared to existing techniques. The major growth has been
two main dependencies: spatial and temporal. Because seen in the past 3 years, constituting more than 70% of all
various datasets and performance evaluation metrics ITS-related DL research performed so far.
have been used, it is hard to compare different studies
related to traffic characteristics, but in traffic flow stud- Future Work and Challenges
ies, the PeMS dataset has been widely used. The major-
ity of research has used hybrid CNN and RNN models, In recent years, DL methods have been able to achieve state-
which can identify both long temporal dependencies and of-the-art results in different visual recognition and traffic
local trend features. Although most papers have defined state prediction tasks. The majority of the visual recogni-
their own CNN model rather than using an existing archi- tion work such as vehicle and pedestrian detection, traffic
tecture, CNN has generally shown better performance sign recognition, etc. have focused on autonomous driving
across papers when compared to RNN, which shows or in-vehicle cameras. However, there have also been a sig-
lower computation/training time. nificant number of overhead cameras installed by city traffic
• Traffic incidents: the most widely used model is RNN, agencies and state Departments of Transportation that are
since the result of an incident shows itself at a specific mostly used for human-evaluated surveillance purposes. To
time that requires a powerful network model to identify. date, there have been only a few studies that have focused
Autoencoders are also popular models, since they can on using these cameras for determining traffic volumes on
learn traffic patterns and then detect and isolate acci- freeways and arterials, traffic speed, and also for surveil-
dent conditions from regular conditions. lance purposes such as automatically detecting anomalies
• Vehicle ID: CNN is the most widely used model, given or traffic incidents (particularly at a large-scale, citywide
its power in inferencing from images, as detection and level). Currently, the majority of traffic intersections rely on
tracking is the main task in license plate and vehicle using loop detectors for vehicle counting and for developing
type/color identification. Existing CNN architectures actuated traffic signals. However, installation of these loop
that have been popularly utilized are AlexNet and VGG detectors is intrusive, in that road closures are required for
models that have been pretrained on ImageNet. installing such sensors. Cameras, on the other hand, can be
• Traffic signal timing: RL has been the most commonly used as a cheap, nonintrusive detection sensor technology
used model, given the control strategy nature of the traf- for counting traffic volume in all directions as well as turn-
fic signal timing task. Hybrids of CNN and SAE have ing movements, the presence of pedestrians, etc., thereby
been used to approximate or learn Q-values to improve facilitating smart traffic signal control strategies. However,
DRL performance. two main challenges need to be considered for developing
• Ride-sharing and public transportation: CNN, RNN, DL techniques able to handle the use of cameras as sensors.
and DNN have been the most frequently used models in First, such methods need to be able to handle the large vol-
the domain. Most researchers have built their own DL ume of data collected from hundreds or thousands of cam-
architecture to accomplish tasks in this category. Public eras installed at a citywide or statewide level. Efficiently
transportation demand and traffic flow prediction tasks providing real-time or near-real-time inferencing from this
have generally been done by either CNN or hybrid CNN large volume of data is currently one of the primary chal-
models. lenges of using cameras as sensors. Second, the methods
• Visual recognition tasks: CNN has been the most com- developed need to be able to perform with minimal or no
monly used DL model for visual recognition tasks, again calibration such that they are feasible to apply and main-
because detection and tracking are efficient via CNN. tain at a large-scale level. Also, the ITS community needs
Especially in traffic sign recognition tasks, the GTSRB to focus on creating more benchmark datasets for different
dataset has been one of the most frequently used bench- research tasks related to DL applications. Although PeMS
marks. Existing architecture such as ResNet, AlexNet, has been used as a popular dataset for traffic state prediction
VGG, and YOLO have been used extensively, with the as shown in 1, the absence of any comparable benchmark

13
 Journal of Big Data Analytics in Transportation

Table 6  Overview of papers using deep learning techniques for visual recognition tasks
Characteristic Model Dataset Experiment Results Baseline Results References Year
AC% Others AC% Others Model

Obstacle Detec- Fast RCNN KITTI PR:88.99 PR:88.01 CaffeNet Kim and Ghosh 2016
tion (2016)
AE CNN CCD Stereo 98.15 96.14 RCNN Nguyen et al. 2016
data (2016)
CNN Caltech Pedes- Missrate:54% Missrate:69 HOG + SVM He et al. (2017) 2017
trian
SdAE + KNN Bahnhof data 91 81 DBN
Dairi et al. 2018
(2018a)
AE + SVM Malaga, Daim- 93.08 89.53 SVM Dairi et al. 2018
ler data (2018b)
CNN Video data PR:95 PR:90 CNN Li et al. 2018
(2018c)
CNN Video data 96.8 Zhang et al. 2018
(2018e)
CNN GMVRT/UCF- 99.71 92.36 HOG + SVM Oliveira and 2018
ARG​ Wehrmeister
(2018)
CNN Railway video mAP:89.53% mAP:88.61% SSD Ye et al. 2018
data (2018a)
CNN Caltech data Missrate:42.27 Missrate:60.95 MS-CNN Zhang et al. 2018
(2018f)
CNN FCTD Camera PR:90.81 PR:70.61 SSD Zhou et al. 2019
(2019)
DNN Video data 98 Rahman et al. 2019
(2019)
Vehicle detec- DBN Far Infrared RL:93.9 RL:91.4 SVM Wang et al. 2016
tion images (2016b)
DBN Far Infrared RL:92.3 RL:91.8 DBN Cai et al. 2016
images (2016)
CNN Built from Recognition- Yao et al. 2017
videos rate:94.68 (2016)
HRPN + Boost Munich vehicle PR:89.2 PR:86.2 HRPN Tang et al. 2017
Classifiers dataset (2017)
Deep CNN Recorded vehi- Top 5:97.51% Luo et al. 2017
cle data (2017)
DNN LISA 2010 PR:81.10 PR:77.09 Faster RCNN Zhou et al. 2018
2018)
DBN RNN KITTI 95.36 92.82 Encoded SVM Wang et al. 2018
(2018c)
Scale Insensitive KITTI 89.6 89.02 MS-CNN Hu et al. (2018) 2018
CNN
CNN Video data 90.7 90.4 CNN(Resnet) Nezafat et al. 2019
(2019)
Traffic sign DBN GTSRB 96.68 95.16 HOG Li and Yang 2016
recognition (2016)
HOG + DBM GTSDB 96.68 95.16 HOG Yang et al. 2016
(2015)
CNN + SVM Built from 71.87 Lin et al. 2016
videos (2016)
CNN + SVM Korea daylight PR:99.03 PR:73.49 CNN Lim et al. 2017
(2017)

13
Journal of Big Data Analytics in Transportation

Table 6  (continued)
Characteristic Model Dataset Experiment Results Baseline Results References Year
AC% Others AC% Others Model

CNN + KELM GTSRB 99.54 99.65 Ensemble Zeng et al. 2017

CNN (2016)
CNN GTSDB 99.4 77.3 HOG Shustanov and 2017
Yakimov
(2017)
Fast BCNN GTSRB 99.01 99.12 BCNN Hu et al. 2017
(2017b)
CNN MASTIF 97.78 98.97 R-LSTM Yuan et al. 2017
(2016)
CNN + STN GTSRB 99.71 99.65 Ensemble Arcos-Garcia 2018
CNN et al. (2018)
CNN GTSRB 99.75 99.67 CNN Natarajan et al. 2018
(2018)
CNN SDTS PR:89.4 Lee and Kim 2018
(2018)
CNN GTSDB PR:90.7 PR:84.20 HOG + SVM Li et al. 2019
(2018b)
CNN HDR PR:94.24 PR:89.33 Guassian You et al. 2019
Mixture (2018)
CNN GTSRB mAP:83.3% mAP:80.8% CNN Lin et al. 2019
(2019)
Traffic light CNN LaRA data PR:96.9 PR:61.22% Image Proc Weber et al. 2016
recognition (2016)
CNN Bosch Traffic 95.1 Behrendt et al. 2017
Lights (2017)
Faster RCNN Bosch Traffic mAP:20.40% Kim et al. 2018
Lights (2018a)
Lane Detection DNN Generated data Top 5:98.55% Gurghian et al. 2016
(2016)
CNN Caltech 99.35 97.21 Image Proc Ye et al. 2018
(2018b)
GBNN NGSIM 97.7 96.6 CNN Dou et al. 2019
(2018)
CNN 98.37 Zhang et al. 2019
(2018g)
Vehicle signal HOG CNN Built from 99 Wang et al. 2016
detection videos (2016c)
FRCN + RPN + F SYSU data 95.58 94.61 FRCN + RPN Chen et al. 2017
(2017b)
Road surface RNN + LSTM Built from 94.6 Park et al. 2018
detection videos (2018)
Deep CNN Cambridge 100 82.6 Faster RCNN Hoang et al. 2019
(2019)
Street scene S-CNN Camvid 53.2 47.4 FCN Wang et al. 2018
labelling (2017b)
Traffic scene AE 78.8 76.4 SegNet Li et al. (2017) 2018
segmentation CNN + IAL Cityscape IoU:74.8 IoU:71.3 CNN Chen et al. 2019
(2018c)
CNN + MFI Built from 91.7 81.1 CNN Cai et al. 2019
videos (2018)

13
 Journal of Big Data Analytics in Transportation

Table 6  (continued)
Characteristic Model Dataset Experiment Results Baseline Results References Year
AC% Others AC% Others Model
Crack detection CNN 3D pavement 94.29 Wang et al. 2017
data (2017c)
CNN PaveVison3D PR:90.20 PR:90.13 CrackNet Zhang et al. 2018
data (2018c)
RCNN Cifar-10 data Kim et al. 2018
(2018b)
Deep CNN Da Nang, 92.08 81 DFP-Sobel Bang et al. 2018
Vietnam (2019)
Deep CNN Captured Data PR:84.7 PR:51.5 RF Hosseini et al. 2018
(2020)
Deep CNN Railway data 97.8 Daneshgaran 2019
et al. (2019)
Deep CNN Generated data, PR:77.68 PR:25.14 SegNet Hosseini and 2019
Seoul Smadi (2020)
Deep RNN Pavement Data 70.1 44 Resgression Hosseini 2020
(2020)

Fig. 4  Hardware (left to right): NVIDIA Jetson Xavier (Jetson AGX 2020), Raspberry (2020), Intel NCS 2 (Intel® Neural Compute Stick
Xavier Developer Kit 2020), NVIDIA Jetson TX2 (Jetson TX2 - 2 Product Specifications 2020)
Elinux.Org 2020), NVIDIA Jetson Nano (Jetson Nano Developer Kit

dataset for traffic incident inference and ride-sharing stud- (Adversarial attacks in this domain are, in most of the cases,
ies has resulted in most of these studies using an original small changes in the input which are imperceptible to the
dataset. This has created difficulties in comparing different human eye but make the classifier classify incorrectly.) For
algorithms to determine the state-of-the-art model. Indeed, example, self-driving cars use DL algorithms to recognize
one of the reasons these research areas have still not been traffic signs (Cireşan et al. 2012), other vehicles, and related
significantly explored using DL models is likely attribut- objects for navigation purposes. However, if DL models fail
able to their lack of a recognized benchmark dataset. While to detect a stop sign due to slight modification in a couple
this study has shown that DL models have been successfully pixels, this can create serious impedance to the adoption
applied to traffic state prediction, vehicle ID and visual rec- of self-driving cars. Adversarial attacks, are, therefore, an
ognition tasks, significant improvements need to be made increasing area of focus in different DL application research
in the use of DL models for other research topics such as topics such as natural language processing, computer vision,
traffic incident inference, traffic signal timing, ride sharing, speech recognition, and malware detection (Najafabadi et al.
and other public transportation concerns. These topics have 2015; Collobert and Weston 2008; LeCun et al. 2010; Deng
still not been fully explored using DL models and hence et al. 2013; Hardy et al. 2016; Tan et al. 2020).
there remains significant scope for improving detection and Biggio et al. (2013) has called into question the advisabil-
prediction accuracy in these areas. ity of using neural networks and SVMs in security-sensitive
While DL models are becoming increasingly popu- applications, demonstrating the legitimacy of their concern
lar among researchers as the most effective classification by attacking some arbitrary PDF files and the MNIST data-
method in visual recognition tasks in the ITS domain, pri- set using the gradient descent evasion attack algorithm that
vacy and security are extremely important. Therefore, the they proposed. Their suggested solution is employing regu-
potential for adversarial attacks and thus the need for robus- larization terms in classifiers. In the same vein Szegedy et al.
tifying DL models have been receiving greater attention. (2013) has shown that accuracy for perturbed input due to

13
Journal of Big Data Analytics in Transportation

Table 7  Detailed specifications of the popular edge-computing devices used for DL

Jetson Xavier Jetson TX2 Jetson Nano Raspberry Pi 3 B + Intel NCS 2

GPU 512-core Volta GPU @ NVIDIA Pascal, 256 128-core Maxwell Broadcom VideoCore IntelR©MovidiusTM
with 64 Tensor Cores CUDAcores IV MyriadTMX VPU
CPU Octal-core NVIDIA HMP Dual Denver Quad-core ARM A57 4*ARMCortex- N.A
Carmel ARMv8.2 CPU 2/2 MB L2 + Quad @1.43 GHz A53,1.2 GHz
@ 2.26 GHz ARMR©A57/2 MB L2
Memory 16 GB 256 bit 8 GB 128 bit LPDDR4 4 GB 64-bit LPDDR4 1 GB LPDDR2 N.A
LPDDR4137GB/s 59.7 GB/s 25.6 GB/s (900 MHz)
Display 3 × eDP 1.4, DP 1.2, 2 × DSI, 2 × DP 1.2, HDMI 2.0, eDP 1.4 HDMI, DSI N.A
HDMI 2.0 HDMI2.0, eDP 1.4
Data Storage 32 GB eMMC 5.1 32 GB eMMC, SDIO, microSD microSD N.A
SATA​
USB USB C USB 3, USB 2 USB 3, USB 2 USB 2 N.A
Connectivity 1 Gigabit Ethernet 1 Gigabit Ethernet, Gigabit Ethernet 100 Base Ethernet, USB 3
802.11acWLAN, 2.4GHz802.11n wire-
Bluetooth less
Mechanical 105 mm × 105 mm 50 mm × 87 mm 100 mm × 80 mm 56.5 mm × 85.60 mm 72.5 mm × 27 mm
Power 10 W, 15 W, 30 W 7.5 W 5–10 W 5W 1W
Price 1299 USD 599 USD 99 USD 35 USD 99 USD

Fig. 5  ITS vs DL models—a traffic character, b traffic incident, c vehicle ID, d traffic signal, e public transport, f visual recognition

adversarial attacks is much less than that in the case of high whereas white-box attacks are when the attacker is aware
magnitude noise. Another downside of DL classification of all relevant information such as the training dataset, the
methods is that adversarial attacks can be independent of model, etc. For example, Madry et al. (2017) has used a
the classification model, meaning that one can generate an projected gradient descent (PGD) form of attack, which is
adversarial attack that can fool a machine learning system different from related work that has mostly used a form of
without any access to the model. These are called black-box attack involving the Fast Gradient Sign Method (FGSM).
attacks, a concept first introduced by Papernot et al. (2016), Also, Moosavi-Dezfooli et al. (2017) has come up with a

13
 Journal of Big Data Analytics in Transportation

systematic way to compute universal attacks that are small In summary, though much research is happening in vari-
image-agnostic perturbations that have a high probability ous domains of ITS using a variety of DL models, the focus
of breaking most classifiers. Concurrent to research regard- of future research in DL for ITS should encompass the fol-
ing designing attacks and understanding the vulnerability lowing: how to develop DL models able to efficiently use
of neural networks to them, researchers have studied dif- the heterogeneous ITS data generated, how to build robust
ferent ways to defend against adversarial attacks to make detection models, and how to ensure security and privacy in
DNNs robust to them. One of the most popular approaches the use of these models.
to defense against adversarial attacks is to add the adver-
sarial set generated by any algorithm to the training set and
then training the neural network with the new augmented
dataset (Fawcett 2003). Goodfellow et al. (2014b) has shown References
that although this method works for specific perturbations,
networks being trained by this method are not robust to all Abedin MZ, Nath AC, Dhar P, Deb K, Hossain MS (2017) License
plate recognition system based on contour properties and deep
adversaries. For example, while working to mitigate the learning model. In: 2017 IEEE region 10 humanitarian technol-
effect of adversaries using denoising autoencoders (DAEs), ogy conference (R10-HTC), IEEE, pp 590–593
Gu and Rigazio (2014) discovered that the resulting DNN Adu-Gyamfi YO, Asare SK, Sharma A, Titus T (2017) Automated
became even more 17 sensitive to perturbed input data. vehicle recognition with deep convolutional neural networks.
Transp Res Rec 2645:113–122
Around the same time, Bastani et al. (2016) designed a Ali EM, Ahmed MM, Wulff SS (2019) Detection of critical safety
metric to measure the robustness of networks and approxi- events on freeways in clear and rainy weather using shrp2 natu-
mate this using the encoding of their robustness as a lin- ralistic driving data: parametric and non-parametric techniques.
ear program to improve the robustness of the overall DNN. Saf Sci 119:141–149
Alkheder S, Taamneh M, Taamneh S (2017) Severity prediction of
Defense against adversarial attacks can be looked at as a traffic accident using an artificial neural network. J Forecast
robust optimization problem, as Shaham et al. (2018) has 36:100–108
shown that adversarial training using their proposed algo- Arabi S, Haghighat A, Sharma A (2020) A deep-learning-based com-
rithm results in a more robust network achieved by robust puter vision solution for construction vehicle detection. Comput
Aided Civil Infrastruct Eng 35:753–767
optimization theory which results in increasing the accuracy Arabi S, Shafei B, Phares BM (2018) Fatigue analysis of sign-support
and robustness of the DNN. Also, authors in Esfandiari et al. structures during transportation under road-induced excitations.
(2019) achieved an algorithm which can provide comparable Eng Struct 164:305–315
accuracies with State-Of-the-Art algorithms, and save a lot Arabi S, Shafei B, Phares BM (2019) Investigation of fatigue in steel
sign-support structures under diurnal temperature changes. J
of computational overhead accompanied with computing Constr Steel Res 153:286–297
worst case adversarial attacks. They achieved that by looking Arabi S, Shafei B, Phares BM (2017) Vulnerability assessment of
at the robust learning problem from a robust optimization sign support structures during transportation. Technical Report
lens as well. Another recent method to harden DNNs against Arcos-Garcia A, Alvarez-Garcia JA, Soria-Morillo LM (2018) Deep
neural network for traffic sign recognition systems: an analysis
adversarial attacks is defensive distillation which has shown of spatial transformers and stochastic optimisation methods.
outstanding preliminary results in being able to reduce the Neural Netw 99:158–165
adversarial attack success rate from 95 to 0.5% (Papernot Arif M, Wang G, Chen S (2018) Deep learning with non-parametric
et al. 2016), but Carlini and Wagner (2017) defeated this regression model for traffic flow prediction. In: 2018 IEEE 16th
intl conf on dependable, autonomic and secure computing, 16th
method by designing a powerful attack able to break this intl conf on pervasive intelligence and computing, 4th intl conf
defense mechanism. Thus, defense and design against adver- on big data intelligence and computing and cyber science and
sarial attacks remain an open problem in DL applications. technology congress (DASC/PiCom/DataCom/CyberSciTech),
As mentioned above, most studies regarding the applica- IEEE, pp 681–688
Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017)
tion of DL models in transportation have paid no attention to Deep reinforcement learning: a brief survey. IEEE Signal Pro-
robustness. However, in light of emerging malware attacks, cess Mag 34:26–38
the importance of defending models from such attacks Baldi P (2012) Autoencoders, unsupervised learning, and deep archi-
has become increasingly important. These attacks usually tectures. In: Proceedings of ICML workshop on unsupervised
and transfer learning, pp 37–49.
destroy the input data by adding noise to them. These attacks Bang S, Park S, Kim H, Kim H (2019) Encoder–decoder network for
can thus disturb the control unit by causing it to infer wrong pixel-level road crack detection in black-box images. Comput
information from the data, resulting in serious accidents. Aided Civil Infrastruct Eng 34:713–727
Also, another source of noise can be the weather conditions Bao J, Liu P, Ukkusuri SV (2019) A spatiotemporal deep learning
approach for citywide short-term crash risk prediction with
such as rainy or snowy conditions. Increasing the robustness multisource data. Accid Anal Prev 122:239–254
of detection models will enable ITS models to operate better Bastani O, Ioannou Y, Lampropoulos L, Vytiniotis D, Nori A,
in severe conditions and thus improve their performance. Criminisi A (2016) Measuring neural net robustness with

13
Journal of Big Data Analytics in Transportation

constraints. Advances in neural information processing sys- Collobert R, Weston J (2008) A unified architecture for natural lan-
tems. MIT Press, Cambridge, pp 2613–2621 guage processing: deep neural networks with multitask learning.
Behrendt K, Novak L, Botros R (2017) A deep learning approach to In: Proceedings of the 25th international conference on machine
traffic lights: detection, tracking, and classification. In: 2017 learning, ACM, pp 160–167.
IEEE international conference on robotics and automation Constantinescu G, Bhatti A, Phares B (2018) Effect of wind induced
(ICRA), IEEE, pp 1370–1377. unsteady vortex shedding, diurnal temperature changes, and tran-
Bengio Y, Goodfellow IJ, Courville A (2015) Deep learning. Nature sit conditions on truss structures supporting large highway signs
521:436–444 problem statement. Technical Report
Biggio B, Corona I, Maiorca D, Nelson B, Šrndić N, Laskov P, Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bhar-
Giacinto G, Roli F (2013) Evasion attacks against machine ath AA (2018) Generative adversarial networks: an overview.
learning at test time. In: Joint European conference on machine IEEE Signal Process Mag 35:53–65
learning and knowledge discovery in databases, Springer, pp Cui Y, Meng C, He Q, Gao J (2018) Forecasting current and next trip
387–402. purpose with social media data and google places. Transportation
Bulan O, Kozitsky V, Ramesh P, Shreve M (2017) Segmentation-and Research Part C: Emerging Technologies 97:159–174
annotation-free license plate recognition with deep localiza- Cui Z, Ke R, Wang Y (2018) Deep bidirectional and unidirectional
tion and failure identification. IEEE Trans Intell Transp Syst LSTM recurrent neural network for network-wide traffic speed
18:2351–2363 prediction. arXiv preprint. https​://arxiv​.org/abs/1801.02143​
Cai Y, Li D, Zhou X, Mou X (2018) Robust drivable road region Cui Z, Henrickson K, Ke R, Wang Y (2018) Traffic graph convolutional
detection for fixed-route autonomous vehicles using map-fusion recurrent neural network: a deep learning framework for network-
images. Sensors 18:4158 scale traffic learning and forecasting. arXiv preprint. https:​ //arxiv​
Cai Y, Sun X, Wang H, Chen L, Jiang H (2016) Night-time vehicle .org/abs/1802.07007​
detection algorithm based on visual saliency and deep learning. Dabiri S, Heaslip K (2018) Inferring transportation modes from gps
J Sens 2016:1–7 trajectories using a convolutional neural network. Transp Res
Carlini N, Wagner D (2017) Towards evaluating the robustness of neu- Part C Emerg Technol 86:360–371
ral networks. In: 2017 IEEE symposium on security and privacy Dai X, Fu R, Zhao E, Zhang Z, Lin Y, Wang F-Y, Li L (2019) Deep-
(SP), IEEE, pp 39–57 trend 2.0: a light-weighted multi-scale traffic prediction model
Chakraborty P, Adu-Gyamfi YO, Poddar S, Ahsani V, Sharma A, using detrending. Transp Res Part C Emerg Technol 103:142–157
Sarkar S (2018) Traffic congestion detection from camera Dairi A, Harrou F, Senouci M, Sun Y (2018) Unsupervised obstacle
images using deep convolution neural networks. Transp Res Rec detection in driving environments using deep-learning-based
2672:222–231 stereovision. Robot Autonom Syst 100:287–301
Chang J, Wang L, Meng G, Xiang S, Pan C (2018) Vision-based occlu- Dairi A, Harrou F, Sun Y, Senouci M (2018) Obstacle detection for
sion handling and vehicle classification for traffic surveillance intelligent transportation systems using deep stacked autoencoder
systems. IEEE Intell Transp Syst Mag 10:80–92 and k-nearest neighbor scheme. IEEE Sens J 18:5122–5132
Chen L, Hu X, Xu T, Kuang H, Li Q (2017) Turn signal detection dur- Daneshgaran F, Zacheo L, Stasio FD, Mondin M (2019) Use of deep
ing nighttime by CNN detector and perceptual hashing tracking. learning for automatic detection of cracks in tunnels: proto-
IEEE Trans Intell Transp Syst 18:3303–3314 type-2 developed in the 2017–2018 time period. Transp Res Rec
Chen Q, Song X, Yamada H, Shibasaki R (2016) Learning deep rep- 2673(9):44–50
resentation from big and heterogeneous data for traffic accident Das S, Dutta A, Dixon K, Minjares-Kyle L, Gillette G (2018) Using
inference. In: Thirtieth AAAI conference on artificial intelligence deep learning in severity analysis of at-fault motorcycle rider
Chen Y, Shu L, Wang L (2017) Traffic flow prediction with big data: crashes. Transp Res Rec 2672:122–134
A deep learning based time series model. In: 2017 IEEE con- Deng L, Hinton G, Kingsbury B (2013) New types of deep neural net-
ference on computer communications workshops (INFOCOM work learning for speech recognition and related applications: an
WKSHPS), IEEE, pp 1010–1011. overview. In: 2013 IEEE international conference on acoustics,
Chen C, Xiang H, Qiu T, Wang C, Zhou Y, Chang V (2018a) A rear- speech and signal processing, IEEE, pp 8599–8603.
end collision prediction scheme based on deep learning in the Ding F, Zhang Z, Zhou Y, Chen X, Ran B (2019) Large-scale full-
internet of vehicles. J Parallel Distrib Comput 117:192–204 coverage traffic speed estimation under extreme traffic conditions
Chen M, Yu X, Liu Y (2018b) PCNN: deep convolutional networks for using a big data and deep learning approach: case study in china.
short-term traffic congestion prediction. IEEE Trans Intell Transp J Transp Eng Part A Syst 145:05019001
Syst 19(11):3550–3559 Doersch C (2016) Tutorial on variational autoencoders. arXiv preprint.
Chen B, Gong C, Yang J (2018c) Importance-aware semantic segmen- https​://arxiv​.org/abs/1606.05908​
tation for autonomous vehicles. IEEE Trans Intell Transp Syst Dominguez-Sanchez A, Cazorla M, Orts-Escolano S (2017) Pedestrian
20(1):137–148 movement direction recognition using convolutional neural net-
Cho K, Van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, works. IEEE Trans Intell Transp Syst 18:3540–3548
Schwenk H, Bengio Y (2014) Learning phrase representations Dong C, Shao C, Li J, Xiong Z (2018) An improved deep learning
using RNN encoder-decoder for statistical machine translation. model for traffic crash prediction. J Adv Transp 2018:1–13
arXiv preprint. https​://arxiv​.org/abs/1406.1078 Dong Z, Wu Y, Pei M, Jia Y (2015) Vehicle type classification using a
Choi S, Yeo H, Kim J (2018) Network-wide vehicle trajectory predic- semisupervised convolutional neural network. IEEE Trans Intell
tion in urban traffic networks using deep learning. Transp Res Transp Syst 16:2247–2256
Rec 2672:173–184 Dou Y, Fang Y, Hu C, Zheng R, Yan F (2018) Gated branch neural
Chung J, Sohn K (2017) Image-based learning to measure traffic den- network for mandatory lane changing suggestion at the on-ramps
sity using a deep convolutional neural network. IEEE Trans Intell of highway. IET Intel Transp Syst 13:48–54
Transp Syst 19:1670–1675 Dougherty M (1995) A review of neural networks applied to transport.
Cireşan D, Meier U, Schmidhuber J (2012) Multi-column deep neural Transp Res Part C Emerg Technol 3:247–260
networks for image classification. arXiv preprint. https​://arxiv​ Du S, Li T, Gong X, Yang Y, Horng SJ (2017) Traffic flow forecast-
.org/abs/1202.2745 ing based on hybrid deep learning framework. In: 2017 12th

13
 Journal of Big Data Analytics in Transportation

International conference on intelligent systems and knowledge Gurghian A, Koduri T, Bailur SV, Carey KJ, Murali VN (2016) Dee-
engineering (ISKE), IEEE, pp 1–6. planes: end-to-end lane position estimation using deep neural
Duan Y, Lv Y, Liu Y-L, Wang F-Y (2016) An efficient realization networks. In: Proceedings of the IEEE conference on computer
of deep learning for traffic data imputation. Transp Res Part C vision and pattern recognition workshops, pp 38–45.
Emerg Technol 72:168–181 Hao S, Lee D-H, Zhao D (2019) Sequence to sequence learning with
E. Van der Pol, F. A. Oliehoek, Coordinated deep reinforcement learn- attention mechanism for short-term passenger flow prediction
ers for traffic light control, Proceedings of Learning, Inference in large-scale metro system. Transp Res Part C Emerg Technol
and Control of Multi-Agent Systems (at NIPS 2016) (2016). 107:287–300
Elman JL (1990) Finding structure in time. Cognit Sci 14:179–211 Hardy W, Chen L, Hou S, Ye Y, Li X (2016) Dl4md: a deep learning
Esfandiari Y, Balu A, Ebrahimi K, Vaidya U, Elia N, Sarkar S (2019) framework for intelligent malware detection. In: Proceedings of
A fast saddle-point dynamical system approach to robust deep the international conference on data mining (DMIN), The Steer-
learning. arXiv preprint. https​://arxiv​.org/abs/1910.08623​ ing Committee of The World Congress in Computer Science,
Fang J, Zhou Y, Yu Y, Du S (2016) Fine-grained vehicle model recog- computer engineering and applied computing (WorldComp), p
nition using a coarse-to-fine convolutional neural network archi- 61.
tecture. IEEE Trans Intell Transp Syst 18:1782–1792 Hasselt HV (2010) Double q-learning. Advances in neural informa-
Fawcett T (2003) In vivo spam filtering: a challenge problem for kdd. tion processing systems. MIT Press, Cambridge, pp 2613–2621
ACM SIGKDD Explor Newsl 5:140–148 El Hatri C, Boumhidi J (2018) Fuzzy deep learning based urban traffic
Fouladgar M, Parchami M, Elmasri R, Ghaderi A (2017) Scalable deep incident detection. Cognit Syst Res 50:206–213
traffic flow neural networks for urban traffic congestion predic- He M, Luo H, Chang Z, Hui B (2017) Pedestrian detection with seman-
tion. In: 2017 International joint conference on neural networks tic regions of interest. Sensors 17:2699
(IJCNN), IEEE, pp 2251–2258. Hoang TM, Nguyen PH, Truong NQ, Lee YW, Park KR (2019) Deep
Gang X, Kang W, Wang F, Zhu F, Lv Y, Dong X, Riekki J, Pirtti- retinanet-based detection and classification of road markings by
kangas S (2015) Continuous travel time prediction for transit visible light camera sensors. Sensors 19:281
signal priority based on a deep network. In: 2015 IEEE 18th Hochreiter S, Schmidhuber J (1997) LSTM can solve hard long time
international conference on intelligent transportation systems, lag problems. Advances in neural information processing sys-
IEEE, pp 523–528. tems. MIT Press, Cambridge, pp 473–479
Gao J, Shen Y, Liu J, Ito M, Shiratori N (2017) Adaptive traffic sig- Home—transport research international documentation (2017). https​
nal control: deep reinforcement learning algorithm with experi- ://trid.trb.org/
ence replay and target network. arXiv preprint. https​://arxiv​.org/ Hopfield JJ (1982) Neural networks and physical systems with emer-
abs/1705.02755​ gent collective computational abilities. Proc Natl Acad Sci
Genders W, Razavi S (2018) Evaluating reinforcement learning state 79:2554–2558
representations for adaptive traffic signal control. Proc Comput Hosseini SA (2020) Data-driven framework for modeling deteriora-
Sci 130:26–33 tion of pavements in the state of Iowa. Graduate Theses and
Gers FA, Schmidhuber E (2001) LSTM recurrent networks learn sim- Dissertations
ple context-free and context-sensitive languages. IEEE Trans Hosseini SA, Smadi O (2020) How prediction accuracy can affect
Neural Netw 12:1333–1340 the decision-making process in pavement management system.
Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural net- https​://doi.org/10.31224​/osf.io/t28ue​
works. In: Proceedings of the fourteenth international conference Hosseini SA, Alhasan A, Smadi O (2020) Use of deep learning to
on artificial intelligence and statistics, pp 315–323. study modelling deterioration of pavements a case study in
Gong Y, Abdel-Aty M, Cai Q, Rahman MS (2019) Decentralized net- Iowa. https​://doi.org/10.31224​/osf.io/edhvy​
work level adaptive signal control by multi-agent deep reinforce- Hou Y, Edara P (2018) Network scale travel time prediction using
ment learning. Transp Res Interdiscip Perspect 1:100020 deep learning. Transp Res Rec 2672:115–123
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair Hu C, Bai X, Qi L, Chen P, Xue G, Mei L (2015) Vehicle color
S, Courville A, Bengio Y (2014a) Generative adversarial nets. recognition with spatial pyramid deep learning. IEEE Trans
Advances in neural information processing systems. MIT Press, Intell Transp Syst 16:2925–2934
Cambridge, pp 2672–2680 Hu Q, Wang H, Li T, Shen C (2017) Deep CNNs with spatially
Goodfellow I (2016) Nips 2016 tutorial: generative adversarial net- weighted pooling for fine-grained car recognition. IEEE Trans
works. arXiv preprint. https​://arxiv​.org/abs/1701.00160​ Intell Transp Syst 18:3147–3156
Goodfellow IJ, Shlens J, Szegedy C (2014) Explaining and harness- Hu X, Xu X, Xiao Y, Chen H, He S, Qin J, Heng P-A (2018) Sinet: a
ing adversarial examples. arXiv preprint. https​://arxiv​.org/ scale-insensitive convolutional neural network for fast vehicle
abs/1412.6572 detection. IEEE Trans Intell Transp Syst 20:1010–1019
Goudarzi S, Kama M, Anisi M, Soleymani S, Doctor F (2018) Self- Hu W, Zhuo Q, Zhang C, Li J (2017) Fast branch convolutional
organizing traffic flow prediction with an optimized deep belief neural network for traffic sign recognition. IEEE Intell Transp
network for internet of vehicles. Sensors 18:3459 Syst Mag 9:114–126
Graves A, Schmidhuber J (2005) Framewise phoneme classification Huang W, Song G, Hong H, Xie K (2014) Deep architecture for traf-
with bidirectional LSTM and other neural network architectures. fic flow prediction: deep belief networks with multitask learn-
Neural Netw 18:602–610 ing. IEEE Trans Intell Transp Syst 15:2191–2201
Gu Y, Lu W, Qin L, Li M, Shao Z (2019) Short-term prediction of Huang Y, Wu R, Sun Y, Wang W, Ding X (2015) Vehicle logo
lane-level traffic speeds: a fusion deep learning model. Transp recognition system based on convolutional neural networks
Res Part C Emerg Technol 106:1–16 with a pretraining strategy. IEEE Trans Intell Transp Syst
Gu S, Rigazio L (2014) Towards deep neural network architectures 16:1951–1960
robust to adversarial examples. arXiv preprint. https​://arxiv​.org/ Huang R, Hu J, Huo Y, Pei X (2019) Cooperative multi-intersection
abs/1412.5068 traffic signal control based on deep reinforcement learning. In:
Guo J, Liu Y, Wang Y, Yang K (2019) Deep learning based congestion CICTP 2019, pp 2959–2970.
prediction using probe trajectory data. In: 19th COTA interna-
tional conference of transportation professionals

13
Journal of Big Data Analytics in Transportation

Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction LeCun Y, Bottou L, Bengio Y, Haffner P et al (1998) Gradient-
and functional architecture in the cat’s visual cortex. J Physiol based learning applied to document recognition. Proc IEEE
160:106–154 86:2278–2324
Hussain KF, Afifi M, Moussa G (2018) A comprehensive study of the LeCun Y, Kavukcuoglu K, Farabet C (2010) Convolutional networks
effect of spatial resolution and color of digital images on vehicle and applications in vision. In: Proceedings of 2010 IEEE inter-
classification. IEEE Trans Intell Transp Syst 20(3):1181–1190 national symposium on circuits and systems, IEEE, pp 253–256.
Intel® Neural Compute Stick 2 Product Specifications (2020) Ark. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A,
Intel.Com. https​://ark.intel​.com/conte​nt/www/us/en/ark/produ​ Aitken A, Tejani A, Totz J, Wang Z et al. (2017) Photo-realistic
cts/14010​9/intel​-neura​l-compu​te-stick​-2.html ´ single image super-resolution using a generative adversarial
Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation network. In: Proceedings of the IEEE conference on computer
with conditional adversarial networks. In: Proceedings of the vision and pattern recognition, pp 4681–4690
IEEE conference on computer vision and pattern recognition, Lee HS, Kim K (2018) Simultaneous traffic sign detection and bound-
pp 1125–1134 ary estimation using convolutional neural network. IEEE Trans
Jetson AGX Xavier Developer Kit (2020) NVIDIA Developer. https​:// Intell Transp Syst 19:1652–1663
develo​ per.nvidia​ .com/embedd​ ed/jetson​ -agx-xavier​ -develo​ per-kit Lee S, Xie K, Ngoduy D, Keyvan-Ekbatani M, Yang H (2018) A lane-
Jetson Nano Developer Kit (2020) NVIDIA Developer. https​://devel​ based predictive model of downstream arrival rates in a queue
oper.nvidi​a.com/embed​ded/jetso​n-nano-devel​oper-kit estimation model using a long short-term memory network.
Jetson TX2 - Elinux.Org (2020) Elinux.Org. https​://elinu​x.org/Jetso​ Transp Res Proc 34:163–170
n_TX2 Li J, Wang J (2017) Short term traffic flow prediction based on deep
Jia Y, Wu J, Xu M (2017) Traffic flow prediction with rainfall impact learning. In: CICTP 2019, pp 2457–2469.
using a deep learning method. J Adv Transp 207:1–10 Li C, Yang C (2016) The research on traffic sign recognition based on
Jia Y, Wu J, Du Y (2016) Traffic speed prediction using deep learning deep learning. In: 2016 16th international symposium on com-
method. In: 2016 IEEE 19th international conference on intel- munications and information technologies (ISCIT), Qingdao,
ligent transportation systems (ITSC), IEEE, pp 1217–1222. China, 26–28 September 2016. IEEE, pp 156–161
Jo D, Yu B, Jeon H, Sohn K (2018) Image-to-image learning to predict Li L, Lv Y, Wang F-Y (2016) Traffic signal timing via deep reinforce-
traffic speeds by considering area-wide spatio-temporal depend- ment learning. IEEE/CAA J Automatica Sinica 3:247–254
encies. IEEE Trans Veh Technol 68:1188–1197 Li L, Qian B, Lian J, Zheng W, Zhou Y (2017) Traffic scene segmenta-
Jung J, Sohn K (2017) Deep-learning architecture to forecast destina- tion based on rgb-d image and deep learning. IEEE Trans Intell
tions of bus passengers from entry-only smart-card data. IET Transp Syst 19:1664–1669
Intel Transp Syst 11:334–339 Li X, Liu Y, Zhao Z, Zhang Y, He L (2018a) A deep learning approach
Ke J, Zheng H, Yang H, Chen XM (2017) Short-term forecasting of of vehicle multitarget detection from traffic video. J Adv Transp
passenger demand under on-demand ride services: a spatiotem- 2018:1–11
poral deep learning approach. Transp Res Part C Emerg Technol Li Y, Song B, Kang X, Du X, Guizani M (2018b) Vehicle-type detec-
85:591–608 tion based on compressed sensing and deep learning in vehicular
Ketkar N et al (2017) Deep learning with Python. Springer, Berlin networks. Sensors 18:4500
Khajeh Hosseini M, Talebpour A (2019) Traffic prediction using time- Li C, Chen Z, Wu QJ, Liu C (2018c) Deep saliency with channel-wise
space diagram: a convolutional neural network approach. Transp hierarchical feature responses for traffic sign detection. IEEE
Res Rec 2673(7):425–435 Trans Intell Transp Syst 20(7):2497–2509
Kim T, Ghosh J (2016) Robust detection of non-motorized road users Liang X, Du X, Wang G, Han Z (2019) A deep reinforcement learning
using deep learning on optical and lidar data. In: 2016 IEEE network for traffic light cycle control. IEEE Trans Veh Technol
19th international conference on intelligent transportation sys- 68:1243–1253
tems (ITSC), IEEE, pp 271–276. Liang X, Du X, Wang G, Han Z (2018) Deep reinforcement learning
Kim H-K, Park JH, Jung H-Y (2018a) An efficient color space for for traffic light control in vehicular networks. arXiv preprint.
deep-learning based traffic light recognition. J Adv Transp https​://arxiv​.org/abs/1803.11115​
2018:2365414 Liao S, Zhou L, Di X, Yuan B, Xiong J (2018) Large-scale short-term
Kim I-H, Jeon H, Baek S-C, Hong W-H, Jung H-J (2018b) Application urban taxi demand forecasting using deep learning. In: Proceed-
of crack identification techniques for an aging concrete bridge ings of the 23rd Asia and South Pacific design automation con-
inspection using an unmanned aerial vehicle. Sensors 18:1881 ference, IEEE Press, pp 428–433.
Kim EJ, Park HC, Ham SW, Kho SY, Kim DK (2019a) Extracting Lim K, Hong Y, Choi Y, Byun H (2017) Real-time traffic sign recogni-
vehicle trajectories using unmanned aerial vehicles in congested tion based on a general purpose GPU and deep-learning. PLoS
traffic conditions. J Adv Transp 2019:9060797 ONE 12:e0173317
Kim TS, Lee WK, Sohn SY (2019b) Graph convolutional network Lin L-J (1992) Self-improving reactive agents based on reinforcement
approach applied to predict hourly bike-sharing demands learning, planning and teaching. Mach Learn 8:293–321
considering spatial, temporal, and global effects. PLoS ONE Lin Y, Dai X, Li L, Wang F-Y (2018) Pattern sensitive prediction of
14:e0220782 traffic flow based on generative adversarial framework. IEEE
Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv Trans Intell Transp Syst 20:2395–2400
preprint. https​://arxiv​.org/abs/1312.6114 Lin L, He Z, Peeta S (2018) Predicting station-level hourly demand
Koesdwiady A, Soua R, Karray F (2016) Improving traffic flow predic- in a large-scale bike-sharing network: a graph convolutional
tion with weather information in connected cars: a deep learning neural network approach. Transp Res Part C Emerg Technol
approach. IEEE Trans Veh Technol 65:9508–9517 97:258–276
Le QV et al (2015) A tutorial on deep learning part 2: autoencoders, Lin F, Lai Y, Lin L, Yuan Y (2016) A traffic sign recognition method
convolutional neural networks and recurrent neural networks. based on deep visual feature. In: 2016 Progress in electromag-
Google Brain, California, pp 1–20 netic research symposium (PIERS), IEEE, pp 2247–2250
LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Lin Z, Yih M, Ota JM, Owens JD, Muyan-Özçelik P (2019) Bench-
Jackel LD (1989) Backpropagation applied to handwritten zip marking deep learning frameworks and investigating fpga
code recognition. Neural Comput 1:541–551

13
 Journal of Big Data Analytics in Transportation

deployment for traffic sign classification and detection. IEEE Muresan M, Fu L, Pan G (2019) Adaptive traffic signal control with
Trans Intell Veh 4(3):385–395 deep reinforcement learning an exploratory investigation. arXiv
Lipton ZC, Berkowitz J, Elkan C (2015) A critical review of recurrent preprint. https​://arxiv​.org/abs/1901.00960​
neural networks for sequence learning. arXiv preprint. https​:// Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R,
arxiv​.org/abs/1506.00019​ Muharemagic E (2015) Deep learning applications and chal-
Liu L, Chen R-C (2017) A novel passenger flow prediction model using lenges in big data analytics. J Big Data 2:1
deep learning methods. Transportation Research Part C: Emerg- Nam D, Kim H, Cho J, Jayakrishnan R (2017) A model based on deep
ing Technologies 84:74–91 learning for predicting travel mode choice. In: Proceedings of the
Liu Y, Liu Z, Jia R (2019) Deeppf: a deep learning based architecture transportation research board 96th annual meeting transportation
for metro passenger flow prediction. Transp Res Part C Emerg research board, Washington, DC, USA, pp 8–12.
Technol 101:18–34 Natarajan S, Annamraju AK, Baradkar CS (2018) Traffic sign recog-
Liu X, Liu W, Mei T, Ma H (2016) A deep learning-based approach to nition using weighted multi-convolutional neural network. IET
progressive vehicle re-identification for urban surveillance. In: Intel Transp Syst 12:1396–1405
European conference on computer vision, Springer, pp 869–884. Nezafat RV, Sahin O, Cetin M (2019) Transfer learning using deep
Liu Y, Wang Y, Yang X, Zhang L (2017) Short-term travel time pre- neural networks for classification of truck body types based on
diction by deep learning: a comparison of different LSTM-DNN side-fire lidar data. J Big Data Anal Transp 1:71–82
models. In: 2017 IEEE 20th international conference on intel- Nguyen H, Kieu L-M, Wen T, Cai C (2018) Deep learning meth-
ligent transportation systems (ITSC), IEEE, pp 1–8. ods in transportation domain: a review. IET Intel Transp Syst
Loce RP, Bernal EA, Wu W, Bala R (2013) Computer vision in 12:998–1004
roadway transportation systems: a survey. J Electron Imaging Nguyen VD, Van Nguyen H, Tran DT, Lee SJ, Jeon JW (2016) Learn-
22:041121 ing framework for robust obstacle detection, recognition, and
Luo X, Shen R, Hu J, Deng J, Hu L, Guan Q (2017) A deep convolution tracking. IEEE Trans Intell Transp Syst 18:1633–1646
neural network model for vehicle recognition and face recogni- Nguyen H, Bentley C, Kieu LM, Fu Y, Cai C (2019) Deep learning
tion. Proc Comput Sci 107:715–720 system for travel speed predictions on multiple arterial road seg-
Lv Y, Duan Y, Kang W, Li Z, Wang F-Y (2014) Traffic flow predic- ments. Transp Res Rec 2673(4):145–157
tion with big data: a deep learning approach. IEEE Trans Intell Nhat-Duc H, Nguyen Q-L, Tran V-D (2018) Automatic recognition
Transp Syst 16:865–873 of asphalt pavement cracks using metaheuristic optimized edge
Ma X, Dai Z, He Z, Ma J, Wang Y, Wang Y (2017) Learning traffic detection algorithms and convolution neural network. Autom
as images: a deep convolutional neural network for large-scale Constr 94:203–213
transportation network speed prediction. Sensors 17:818 Ni X, Wang H, Che C, Hong J, Sun Z (2019) Civil aviation safety
Ma X, Yu H, Wang Y, Wang Y (2015) Large-scale transportation net- evaluation based on deep belief network and principal component
work congestion evolution prediction using deep learning theory. analysis. Saf Sci 112:90–95
PLoS ONE 10:e0119044 de Oliveira D, Wehrmeister M (2018) Using deep learning and low-cost
Ma X, Zhang J, Du B, Ding C, Sun L (2018) Parallel architecture of rgb and thermal cameras to detect pedestrians in aerial images
convolutional bi-directional lstm neural networks for network- captured by multirotor uav. Sensors 18:2244
wide metro ridership prediction. IEEE Trans Intell Transp Syst Pamula T (2018) Road traffic conditions classification based on mul-
20:2278–2288 tilevel filtering of image content using convolutional neural
Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2017) Towards networks. IEEE Intell Transp Syst Mag 10:11–21
deep learning models resistant to adversarial attacks. arXiv pre- Pan G, Fu L, Thakali L (2017) Development of a global road safety
print. https​://arxiv​.org/abs/1706.06083​ performance function using deep neural networks. Int J Transp
Maŕın-Reyes PA, Bergamini L, Lorenzo-Navarro J, Palazzi A, Calde- Sci Technol 6:159–173
rara S, Cucchiara R (2018) Unsupervised vehicle re-identification Pan G, Fu L, Thakali L, Muresan M, Yu M (2018) An improved deep
using triplet networks. In: 2018 IEEE/CVF conference on com- belief network model for road safety analyses. arXiv preprint.
puter vision and pattern recognition workshops (CVPRW), IEEE, https​://arxiv​.org/abs/1812.07410​
pp 166–1665. Papernot N, McDaniel P, Goodfellow I, Jha S, Celik ZB, Swami A
Mirza M, Osindero S (2014) Conditional generative adversarial nets. (2016) Practical black-box attacks against deep learning sys-
arXiv preprint. https​://arxiv​.org/abs/1411.1784 tems using adversarial examples. arXiv preprint. https​://arxiv​
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare .org/abs/1602.02697​1
MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al Papernot N, McDaniel P, Wu X, Jha S, Swami A (2016) Distillation
(2015) Human-level control through deep reinforcement learn- as a defense to adversarial perturbations against deep neural
ing. Nature 518:529 networks. In: 2016 IEEE symposium on security and privacy
Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting (SP), IEEE, pp 582–597
items for systematic reviews and meta-analyses: the PRISMA Park J, Min K, Kim H, Lee W, Cho G, Huh K (2018) Road surface
statement. Ann Intern Med 151:264–269 classification using a deep ensemble network with sensor fea-
Moll J, Arnold P, Malzer M, Krozer V, Pozdniakov D, Salman R, ture selection. Sensors 18:4342
Rediske S, Scholz M, Friedmann H, Nuber A (2018) Radar-based Polson NG, Sokolov VO (2017) Deep learning for short-term traffic
structural health monitoring of wind turbine blades: the case of flow prediction. Transp Res Part C Emerg Technol 79:1–17
damage detection. Struct Health Monit 17:815–822 Puarungroj W, Boonsirisumpun N (2018) Thai license plate recogni-
Moosavi-Dezfooli S-M, Fawzi A, Fawzi O, Frossard P (2017) Univer- tion based on deep learning. Proc Comput Sci 135:214–221
sal adversarial perturbations. In: Proceedings of the IEEE confer- Qian R, Zhang B, Yue Y, Wang Z, Coenen F (2015) Robust chinese
ence on computer vision and pattern recognition, pp 1765–1773. traffic sign detection and recognition with deep convolutional
Mousavi SS, Schukat M, Howley E (2017) Traffic light control using neural network. In: 2015 11th international conference on natu-
deep policy-gradient and value-function-based reinforcement ral computation (ICNC), IEEE, pp 791–796.
learning. IET Intel Transp Syst 11:417–423 Radford A, Metz L, Chintala S (2015) Unsupervised representation
learning with deep convolutional generative adversarial net-
works. arXiv preprint. https​://arxiv​.org/abs/1511.06434​

13
Journal of Big Data Analytics in Transportation

Rahman M, Islam M, Calhoun J, Chowdhury M (2019) Real-time Proceedings of the IEEE conference on computer vision and
pedestrian detection approach with an efficient data commu- pattern recognition, pp 3521–3529.
nication bandwidth strategy. Transp Res Rec 2673(6):129–139 Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I,
Raspberry (2020) Raspberrypi.Org. https​://www.raspb​erryp​i.org/ Fergus R (2013) Intriguing properties of neural networks. arXiv
produ​cts/raspb​erry-pi-4-model​-b/ preprint. https​://arxiv​.org/abs/1312.6199
Ren H, Song Y, Liu J, Hu Y, Lei J (2017) A deep learning approach Tan C, Eswaran C (2008) Reconstruction of handwritten digit images
to the prediction of short-term traffic accident risk. arXiv pre- using autoencoder neural networks. In: 2008 Canadian con-
print. https​://arxiv​.org/abs/1710.09543​ ference on electrical and computer engineering, IEEE, pp
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang 000465–000470.
Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L Tan KL, Esfandiari Y, Lee XY, Sarkar AS (2020) Robustifying rein-
(2015) ImageNet large scale visual recognition challenge. Int forcement learning agents via action space adversarial training.
J Comput Vis 115:211–252 arXiv preprint. https​://arxiv​.org/abs/2007.07176​
Saadi I, Wong M, Farooq B, Teller J, Cools M (2017) An investiga- Tang T, Zhou S, Deng Z, Zou H, Lei L (2017) Vehicle detection in
tion into machine learning approaches for forecasting spatio- aerial images based on region convolutional neural networks and
temporal demand in ride-hailing service. arXiv preprint. https​ hard negative example mining. Sensors 17:336
://arxiv​.org/abs/1703.02433​ Tang Z, Wang G, Xiao H, Zheng A, Hwang J-N (2018) Single-camera
Sameen M, Pradhan B (2017) Severity prediction of traffic accidents and inter-camera vehicle tracking and 3d speed estimation based
with recurrent neural networks. Appl Sci 7:476 on fusion of visual and semantic features. In: Proceedings of
Schmidhuber J (2015) Deep learning in neural networks: an over- the IEEE conference on computer vision and pattern recognition
view. Neural Netw 61:85–117 workshops, pp 108–115.
Shaham U, Yamada Y, Negahban S (2018) Understanding adver- Tapu R, Mocanu B, Zaharia T (2017) Deep-see: Joint object detection,
sarial training: increasing local stability of supervised models tracking and recognition with application to visually impaired
through robust optimization. Neurocomputing 307:195–204 navigational assistance. Sensors 17:2473
Shen Y, Xiao T, Li H, Yi S, Wang X (2017) Learning deep neural Tawfeek MH, El-Basyouny K (2019) Estimating traffic volume on
networks for vehicle re-id with visual-spatio-temporal path minor roads at rural stop-controlled intersections using deep
proposals. In: Proceedings of the IEEE international confer- learning. Transp Res Rec 2673(4):108–116
ence on computer vision, pp 1900–1909. Tesauro G (1995) Temporal difference learning and td-gammon. Com-
Shi D, Ding J, Errapotu SM, Yue H, Xu W, Zhou X, Pan M (2018) mun ACM 38:58–68
Deep q-network based route scheduling for transportation net- Theofilatos A, Chen C, Antoniou C (2019) Comparing machine learn-
work company vehicles. In: 2018 IEEE global communications ing and deep learning methods for real-time crash prediction.
conference (GLOBECOM), IEEE, pp 1–7 Transp Res Rec 2673(8):169–178
Shin H-C, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura Tian Y, Pan L (2015) Predicting short-term traffic flow by long short-
D, Summers RM (2016) Deep convolutional neural networks term memory recurrent neural network. In: 2015 IEEE interna-
for computer-aided detection: CNN architectures, dataset char- tional conference on smart city/SocialCom/SustainCom (Smart-
acteristics and transfer learning. IEEE Trans Med Imaging City), IEEE, pp 153–158
35:1285–1298 Vincent P, Larochelle H, Bengio Y, Manzagol P-A (2008) Extract-
Shustanov A, Yakimov P (2017) CNN design for real-time traffic sign ing and composing robust features with denoising autoencod-
recognition. Proc Eng 201:718–725 ers. In: Proceedings of the 25th international conference on
Silver D (2015) UCL course on RL, lecture notes: reinforcement learn- machine learning, ACM, pp 1096–1103.
ing. https​://www0.cs.ucl.ac.uk/staff​/d.silve​r/web/Teach​ing.html. Wan C-H, Hwang M-C (2018) Value-based deep reinforcement
Accessed 9 Sep 2019 learning for adaptive isolated intersection signal control. IET
Simoncini M, Taccari L, Sambo F, Bravi L, Salti S, Lori A (2018) Intel Transp Syst 12:1005–1010
Vehicle classification from low-frequency GPS data with Wan Z, Jiang C, Fahad M, Ni Z, Guo Y, He H (2018) Robot-assisted
recurrent neural networks. Transp Res Part C Emerg Technol pedestrian regulation based on deep reinforcement learning.
91:176–191 IEEE Trans Cybern 50:1669–1682
Singh D, Mohan CK (2018) Deep spatio-temporal representation for Wang Z, Schaul T, Hessel M, Van Hasselt H, Lanctot M, De Freitas
detection of road accidents using stacked autoencoder. IEEE N (2015) Dueling network architectures for deep reinforcement
Trans Intell Transp Syst 20(3):879–887 learning. arXiv preprint. https​://arxiv​.org/abs/1511.06581​
Siripanpornchana C, Panichpapiboon S, Chaovalit P (2016) Travel-time Wang J, Gu Q, Wu J, Liu G, Xiong Z (2016a) Traffic speed prediction
prediction with deep learning. In: 2016 IEEE region 10 confer- and congestion source exploration: a deep learning method.
ence (TENCON), IEEE, p. 1859–1862 In: 2016 IEEE 16th international conference on data mining
Soon FC, Khaw HY, Chuah JH, Kanesan J (2018a) Hyper-parameters (ICDM), IEEE, pp 499–508.
optimisation of deep CNN architecture for vehicle logo recogni- Wang H, Cai Y, Chen X, Chen L (2016b) Night-time vehicle sensing
tion. IET Intel Syst 12:939–946 in far infrared image with deep learning. J Sens 2016:3403451
Soon FC, Khaw HY, Chuah JH, Kanesan J (2018b) Pcanet-based con- Wang JG, Zhou L, Pan Y, Lee S, Song Z, Han BS, Saputra VB (2016c)
volutional neural network architecture for a vehicle model rec- Appearance-based brake-lights recognition using deep learning
ognition system. IEEE Trans Intell Transp Syst 20(2):749–759 and vehicle detection. In: 2016 IEEE intelligent vehicles sympo-
Suhao L, Jinzhao L, Guoquan L, Tong B, Huiqian W, Yu P (2018) sium (IV), IEEE, pp 815–820
Vehicle type detection based on deep learning in traffic scene. Wang J, Zheng H, Huang Y, Ding X (2017a) Vehicle type recognition
Proc Comput Sci 131:564–572 in surveillance images from labeled web-nature data using deep
Sussman JS (2008) Perspectives on intelligent transportation systems transfer learning. IEEE Trans Intell Transp Syst 1–10.
(ITS). Springer, Berlin Wang Q, Gao J, Yuan Y (2017b) A joint convolutional neural net-
Suzuki T, Kataoka H, Aoki Y, Satoh Y (2018) Anticipating traffic works and context transfer for street scenes labeling. IEEE
accidents with adaptive loss and large-scale incident db. In: Trans Intell Transp Syst 19:1457–1470

13
 Journal of Big Data Analytics in Transportation

Wang K, Zhang A, Li JQ, Fei Y, Chen C, Li B (2017c) Deep learning Yao Y, Tian B, Wang F-Y (2016) Coupled multivehicle detection and
for asphalt pavement cracking recognition using convolutional classification with prior objectness measure. IEEE Trans Veh
neural network. In: Proceedings of international conference air- Technol 66:1975–1984
field highway pavements, pp 166–177. Yao H, Tang X, Wei H, Zheng G, Yu Y, Li Z (2018) Modeling spatial-
Wang Y, Zhang D, Liu Y, Dai B, Lee LH (2018a) Enhancing trans- temporal dynamics for traffic prediction. arXiv preprint. https​://
portation systems via deep learning: a survey. Transp Res Part arxiv​.org/abs/1803.01254​
C Emerg Technol 99:144–163 Yao H, Wu F, Ke J, Tang X, Jia Y, Lu S, Gong P, Ye J, Li Z (2018)
Wang Y, Wang C, Zhang H (2018b) Ship classification in high- Deep multi-view spatial-temporal network for taxi demand
resolution sar images using deep learning of small datasets. prediction. In: Thirty-second AAAI conference on artificial
Sensors 18:2929 intelligence
Wang H, Yu Y, Cai Y, Chen L, Chen X (2018c) A vehicle recogni- Ye YY, Hao XL, Chen HJ (2018) Lane detection method based on lane
tion algorithm based on deep transfer learning with a multiple structural analysis and CNNs. IET Intel Transp Syst 12:513–520
feature subspace distribution. Sensors 18:4109 Ye T, Wang B, Song P, Li J (2018) Automatic railway traffic object
Wang J, Chen R, He Z (2019) Traffic speed prediction for urban detection system using feature fusion refine neural network under
transportation network: a path based deep learning approach. shunting mode. Sensors 18:1916
Transp Res Part C Emerg Technol 100:372–385 You C, Wen C, Wang C, Li J, Habib A (2018) Joint 2-D–3-D traffic
Weber M, Wolf P, Zollner JM (2016) Deeptlr: A single deep convo- sign landmark data set for geo-localization using mobile laser
lutional network for detection and classification of traffic lights. scanning data. IEEE Trans Intell Transp Syst 20(7):2550–2565
In: 2016 IEEE intelligent vehicles symposium (IV), IEEE, pp Yu B, Guo Z, Asian S, Wang H, Chen G (2019) Flight delay prediction
342–348 for commercial air transport: a deep learning approach. Transp
Wei H, Zheng G, Yao H, Li Z (2018) Intellilight: a reinforcement learn- Res Part E Logist Transp Rev 125:203–221
ing approach for intelligent traffic light control. In: Proceedings Yu S, Wu Y, Li W, Song Z, Zeng W (2017) A model for fine-grained
of the 24th ACM SIGKDD international conference on knowl- vehicle classification based on deep learning. Neurocomputing
edge discovery and data mining, ACM, pp 2496–2505. 257:97–103
Wen J, Zhao J, Jaillet P (2017) Rebalancing shared mobility-on- Yu R, Li Y, Shahabi C, Demiryurek U, Liu Y (2017) Deep learning:
demand systems: a reinforcement learning approach. In: 2017 a generic approach for extreme condition traffic forecasting. In:
IEEE 20th international conference on intelligent transportation Proceedings of the 2017 SIAM international conference on data
systems (ITSC), IEEE, pp 220–225. mining, SIAM, pp 777–785.
Williams RJ, Zipser D (1989) Experimental analysis of the real-time Yu B, Yin H, Zhu Z (2017) Spatio-temporal graph convolutional net-
recurrent learning algorithm. Connect Sci 1:87–111 works: a deep learning framework for traffic forecasting. arXiv
Wu Y, Tan H, Qin L, Ran B, Jiang Z (2018) A hybrid deep learn- preprint. https​://arxiv​.org/abs/1709.04875​
ing based traffic flow prediction method and its understanding. Yuan Y, Xiong Z, Wang Q (2016) An incremental framework for video-
Transp Res Part C Emerg Technol 90:166–180 based traffic sign detection, tracking, and recognition. IEEE
Wu C-W, Liu C-T, Chiang C-E, Tu W-C, Chien S-Y (2018) Vehicle Trans Intell Transp Syst 18:1918–1929
re-identification with the space-time prior. In: Proceedings of Yuan Z, Zhou X, Yang T, Tamerius J, Mantilla R (2017) Predicting
the IEEE conference on computer vision and pattern recognition traffic accidents through heterogeneous urban data: a case study.
workshops, pp 121–128. In: Proceedings of the 6th international workshop on urban com-
Xiang X, Lv N, Guo X, Wang S, El Saddik A (2018) Engineering puting (UrbComp 2017), Halifax, NS, Canada, volume 14
vehicles detection based on modified faster R-CNN for power Yuan Z, Zhou X, Yang T (2018) Hetero-convLSTM: A deep learning
grid surveillance. Sensors 18:2258 approach to traffic accident prediction on heterogeneous spa-
Xie L, Ahmad T, Jin L, Liu Y, Zhang S (2018) A new CNN-based tiotemporal data. In: Proceedings of the 24th ACM SIGKDD
method for multi-directional car license plate detection. IEEE international conference on knowledge discovery and data min-
Trans Intell Transp Syst 19:507–517 ing, ACM, pp 984–992.
Xu C, Ji J, Liu P (2018) The station-free sharing bike demand forecast- Zang D, Chai Z, Zhang J, Zhang D, Cheng J (2015) Vehicle license
ing with a deep learning approach and large-scale datasets. Trans plate recognition using visual attention model and deep learning.
Res Part C Emerg Technol 95:47–60 J Electron Imaging 24:033001
Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional Zeng Y, Xu X, Shen D, Fang Y, Xiao Z (2016) Traffic sign recognition
neural networks: an overview and application in radiology. using kernel extreme learning machines with deep perceptual
Insights Imaging 9:611–629 features. IEEE Trans Intell Transp Syst 18:1647–1653
Yan Z, Feng Y, Cheng C, Fu J, Zhou X, Yuan J (2017) Extensive Zhang X, Cheng L, Li B, Hu H-M (2018) Too far to see? not really!—
exploration of comprehensive vehicle attributes using D-CNN pedestrian detection with scale-aware localization policy. IEEE
with weighted multi-attribute strategy. IET Intel Transp Syst Trans Image Process 27:3703–3715
12:186–193 Zhang K, Cheng H, Zhang B (2018) Unified approach to pavement
Yang Y, Li D, Duan Z (2017) Chinese vehicle license plate recognition crack and sealed crack detection using preclassification based on
using kernel-based extreme learning machine with deep convolu- transfer learning. J Comput Civil Eng 32:04018001
tional features. IET Intel Transport Syst 12:213–219 Zhang Z, He Q, Gao J, Ni M (2018) A deep learning approach for
Yang Y, Luo H, Xu H, Wu F (2015) Towards real-time traffic sign detecting traffic accidents from social media data. Transp Res
detection and classification. IEEE Trans Intell Transp Syst Part C Emerg Technol 86:580–596
17:2022–2031 Zhang D, Kabuka MR (2018) Combining weather condition data to
Yang G, Wang Y, Yu H, Ren Y, Xie J (2018a) Short-term traffic state predict traffic flow: a gru-based deep learning approach. IET Intel
prediction based on the spatiotemporal features of critical road Transp Syst 12:578–585
sections. Sensors 18:2287 Zhang Z, Li M, Lin X, Wang Y, He F (2019) Multistep speed predic-
Yang H, Xie K, Ozbay K, Ma Y, Wang Z (2018b) Use of deep learning tion on traffic networks: a deep learning approach considering
to predict daily usage of bike sharing systems. Transp Res Rec
2672(36):92–102

13
Journal of Big Data Analytics in Transportation

spatio-temporal dependencies. Transp Res Part C Emerg Technol nonoverlapping camera views. In: 2017 IEEE global conference
105:297–322 on signal and information processing (GlobalSIP), IEEE, pp
Zhang A, Wang KC, Fei Y, Liu Y, Tao S, Chen C, Li JQ, Li B (2018) 166–170.
Deep learning–based fully automated pavement crack detection Zhong J, Lei T, Yao G (2017) Robust vehicle detection in aerial
on 3d asphalt surfaces with an improved cracknet. J Comput Civil images based on cascaded convolutional neural networks. Sen-
Eng 32:04018041 sors 17:2720
Zhang W, Wang Z, Liu X, Sun H, Zhou J, Liu Y, Gong W (2018) Zhou F, Li J, Li X, Li Z, Cao Y (2019) Freight car target detection in
Deep learning-based real-time fine-grained pedestrian recogni- a complex background based on convolutional neural networks.
tion using stream processing. IET Intel Transp Syst 12:602–609 Proc Inst Mech Eng Part F J Rail Rapid Transit 233:298–311
Zhang X, Yang W, Tang X, Wang Y (2018) Lateral distance detection Zhou Y, Liu L, Shao L, Mellor M (2018) Fast automatic vehicle anno-
model based on convolutional neural network. IET Intel Trans- tation for urban traffic surveillance. IEEE Trans Intell Transp
port Syst 13:31–39 Syst 19:1973–1984
Zhang L, Zhang G, Liang Z, Ozioko EF (2018) Multi-features taxi Zhu L, Yu FR, Wang Y, Ning B, Tang T (2018a) Big data analytics in
destination prediction with frequency domain processing. PLoS intelligent transportation systems: a survey. IEEE Trans Intell
ONE 13:e0194629 Transp Syst 20(1):383–398
Zhang J, Zheng Y, Qi D (2017) Deep spatio-temporal residual networks Zhu H, Yang X, Wang Y (2018b) Prediction of daily entrance and exit
for citywide crowd flows prediction. In: Thirty-first AAAI con- passenger flow of rail transit stations by deep learning method.
ference on artificial intelligence, IEEE J Adv Transp 2018:1–11
Zhao Z, Chen W, Wu X, Chen PC, Liu J (2017) LSTM network: a Zhuang Y, Ke R, Wang Y (2018) Innovative method for traffic data
deep learning approach for short-term traffic forecast. IET Intell imputation based on convolutional neural network. IET Intell
Transp Syst 11:68–75 Transp Syst 13:605–613
Zhao X, Gu Y, Chen L, Shao Z (2019) Urban short-term traffic flow
prediction based on stacked autoencoder. In: 19th COTA inter- Publisher’s Note Springer Nature remains neutral with regard to
national conference of transportation professionals jurisdictional claims in published maps and institutional affiliations.
Zheng Y, Ozcan K, Velipasalar S (2017) A codebook of brightness
transfer functions for improved target re-identification across

13