This article has been accepted for inclusion in a future issue of this journal.

Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 1

A Survey of Driving Safety With Sensing,

Vehicular Communications, and Artificial
Intelligence-Based Collision Avoidance
Yuchuan Fu , Member, IEEE, Changle Li , Senior Member, IEEE, Fei Richard Yu, Fellow, IEEE,
Tom H. Luan , Senior Member, IEEE, and Yao Zhang

Abstract— Accurately discovering hazards and issuing appro- pursued in the transportation system, which motivates the
priate warnings to drivers in advance or performing autonomous technologies in Collision Avoidance (CA). By applying sens-
control is the core of the Collision Avoidance (CA) system used ing technology, computing technology, and communication
to solve traffic safety problems. More comprehensive environ-
mental awareness, diversified communication technologies, and technology, the primary goal of the CA is to accurately detect
autonomous control can make the CA system more accurate driving hazards and warn the driver or take automatic braking
and effective, thereby improving driving safety. In addition, the measures at the appropriate time, accordingly to reduce the
assistance of Artificial Intelligence (AI) technology can make probability of accidents.
the CA system adapt to the environment and facilitate fast and Existing CA solutions rely on some traffic facilities
accurate decisions. Considering the current lack of a thorough
survey of driving safety with sensing, vehicular communications, (e.g., roundabout, traffic signs, roadside units (RSUs)) and
and AI-based collision avoidance, in this paper, we survey existing on-board sensors of vehicles. Specifically, traditional methods
researches for state-of-the-art data-driven CA techniques. Firstly, such as traffic rules and traffic facilities rely heavily on the
we discuss the major steps of CA and key research issues. driver’s response, weather, and light intensity [4]. For instance,
For each step, we review the existing enabling techniques and the driver’s distraction, misjudgment, and misoperation while
research methods for CA in detail, including sensing and vehic-
ular communication for safe driving, as well as CA algorithm driving will increase the risk of accidents. In addition, some
design. Particularly, we present a comparison between the most sensor-based methods are also trying to improve driving
common AI algorithms for different functions in the CA system. safety. In this method, the current economical vehicles are
Testbeds and projects for CA are summarized next. Finally, equipped with ultrasonic radar sensors and Global Positioning
several open challenges and future research directions are also System (GPS), however, it is far from satisfying the need
outlined.
for collision avoidance of high-speed vehicles. For some
Index Terms— Collision avoidance, edge computing, artificial luxury vehicles, other sensors, such as camera vision, can
intelligence, vehicle-to-everything (V2X), connected autonomous be installed to form the basis of safety-enhancing systems.
vehicle.
Nevertheless, in addition to increasing vehicle costs, the
I. I NTRODUCTION performance of expensive sensors still suffers from several
drawbacks: encountering bad weather, relatively short mon-
W HILE traffic systems have developed for decades,
traffic injuries still represent the top killer globally.
According to World Health Organization (WHO), on average,
itoring ranges, interference of radar signals, and blind angle
during detection [5], [6]. Meanwhile, the effectiveness of some
about 2.4 people are killed every minute on the roads [1], sensor-based systems still relies on drivers’ subsequent actions.
and the number of traffic accidents is still on the rise The application of vehicular communication technology to
yearly [2], [3]. Preventing collision accidents, as an effective intelligent transportation systems (ITS) is expected to reduce
measure to solve driving safety problems, is the basic goal collisions through effective road traffic management. Vehicular
communication is a low-cost and highly reliable technology
Manuscript received December 18, 2019; revised November 2, 2020, that can be combined with existing sensing technologies to
January 30, 2021, and May 5, 2021; accepted May 18, 2021. This work was extend the sensing range and increase the sensing dimension,
supported in part by the National Key Research and Development Program of
China under Grant 2020YFB1807700, in part by the National Natural Science thus improving the timeliness of CA systems [7]–[9]. This
Foundation of China under Grant U1801266, in part by the Youth Innovation in turn has inspired new use cases, ranging from services
Team of Shaanxi Universities, and in part by the Fundamental Research Funds based on vehicle communication to complex systems (such
for the Central Universities under Grant JB210102. The Associate Editor for
this article was S.-H. Kong. (Corresponding author: Changle Li.) as autonomous vehicles). There is no doubt that connected
Yuchuan Fu, Changle Li, and Yao Zhang are with the State Key Laboratory cars will also be needed. the development of Connected
of Integrated Services Networks, Xidian University, Xi’an 710071, China, and Autonomous Vehicles (CAVs) enables vehicles not only to
also with the Research Institute of Smart Transportation, Xidian University,
Xi’an 710071, China (e-mail: [email protected]). recognize the current potential collision hazard and adopt dif-
Fei Richard Yu is with the Department of Systems and Computer Engineer- ferent levels of autonomous control but also to achieve driving
ing, Carleton University, Ottawa, ON K1S 5B6, Canada. safety in a collaborative manner. As vehicles become increas-
Tom H. Luan is with the School of Cyber Engineering, Xidian University,
Xi’an 710071, China. ingly intelligent, it is challenging to efficiently use vehic-
Digital Object Identifier 10.1109/TITS.2021.3083927 ular sensors, communication technologies, and computing
1558-0016 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

Fig. 1. Collision avoidance methods and their characteristics.

technologies to achieve effective and reliable CA [10], [11]. (Section V) and 3) AI algorithms for CA (Section VI).
In the face of a large volume of data and the increasingly com- Section VII describes testbeds and projects for CA. At last,
plex traffic environment, artificial intelligence (AI) technology Section VIII outlines several open challenges, which need to
can quickly and accurately handle a large volume of data and be addressed to make the solution more adaptable to the actual
adapt to the environment, thus further achieving the accuracy traffic environment as well as satisfy efficient and reliable
and comfort goals of CA [12]. Fig. 1 shows the CA methods requirements, followed by concluding remarks in Section IX.
and their characteristics. Fig. 2 outlines the organization of the survey.
Previously, researchers have studied the work related to
traffic safety from a specific perspective, such as motion II. BACKGROUND K NOWLEDGE
prediction and risk assessment algorithms for CA [13], motion In this section, we briefly introduce the background knowl-
planning approaches [14], as well as vision-based vehicle edge of sensing, communication, and computing for CA
detection and tracking for CA systems [15]. In [16], the systems, as well as autonomous vehicles.
authors focused on different detection techniques for vehicle
safety. However, we found that sensing, vehicular communi- A. Sensing, Communication and Computing for CA
cations, and AI are indispensable for the development of CA
systems. Different from previous surveys, this work focuses Using sensing technology, information about the vehicle
and surrounding environment can be obtained. However, sen-
on how to apply AI to vehicle communication and sensing
technology to further improve collision avoidance systems. sors have limitations in line-of-sight characteristics and are
susceptible to be blocked by other vehicles and buildings.
First, we provide a comprehensive survey on major steps,
characteristics, key research issues, and Key Performance By combining communication, vehicles can exchange infor-
Indicator (KPI) of CA, which can provide readers with a mation and the sensing range can be expanded accordingly.
From the perspective of communication, its function is
systematic understanding and basic concepts of this direction.
According to the major steps of the CA system, we will to establish a vehicular communication network that satisfies
introduce in detail the sensing, vehicular communications, the delay sensitivity and high reliability, thereby expand-
ing the range and dimensions of vehicular sensing. During
and AI-based algorithms related to vehicle safety. Considering
that autonomous vehicle control can reduce the risk of driver driving, sensing data and safety-related data require efficient
and real-time transmission, which greatly depends on the
misoperation, some algorithms for autonomous decision strate-
gies are also involved. We also discuss the opportunities and timeliness and reliability of communication.
challenges of the existing technology and emerging directions From the perspective of computing, it is necessary to
efficiently process the data generated by the vehicle and make
in the CA application scenario from the perspective of commu-
nication. Furthermore, we identify and analyze the importance predictions and decisions accordingly. The integration of com-
of AI technology in CA and conduct an extensive survey on munication and computing can enhance the edge processing
capability of the network, reduce the data processing delay,
the implementation of some popular algorithms in different
functions of CA systems. In order to bridge the gap between and enable the sensing data to communicate among a wider
range of vehicles. In addition, the application of AI tech-
academic research and the actual products of the industry,
we also discuss some recent popular testbeds and projects. nology can form innovative prediction and decision-making
The remainder of this paper is organized as follows. First, algorithms to achieve active safety. Moreover, the self-learning
ability and environmental adaptability of AI algorithms can
background knowledge of CAVs, as well as communica-
tion and computing for CA, are introduced in Section II. make the CA system more flexible and accurate.
In Section III, we focus on major steps of CA in terms
of software architecture, KPI, and key research issues. B. Autonomous Vehicles
Subsequently, a series of enabling techniques and individual An autonomous vehicle is defined as an intelligent vehicle
methods are discussed in Section IV to Section VI, including that senses the road environment through an on-board sensor
1) sensing for CA (Section IV), 2), communication for CA system and automatically plans driving routes as well as

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 3

Fig. 2. Road map of the survey.

making decision [17], [18]. The Society of Automotive Engi-

neers (SAE) divided autonomous driving into 6 levels in the
SAE standard for autonomous vehicles released in 2014 and
revised in 2018 [19]. From level 0 to level 3, the role of
the human driver gradually shifts from driving operation to
peripheral monitoring and support. At level 4 and level 5,
no human operation is required. Currently, autonomous driving
technology integrates a variety of technologies, relying mainly
on the vehicle’s sensing, decision-making, and execution to
replace the driver’s operation [20]. With the help of com-
munication technology, CAVs are autonomous vehicles that
maintain a continuous network connection between each other
and the surrounding environment.

III. M AJOR P HASES OF C OLLISION AVOIDANCE

In this section, we will refine the major steps of CA and
investigate the KPI as well as key research issues.

A. Process Diagram Fig. 3. Major phases for collision avoidance system.

Based on the advanced sensor, communication, and com-
puting technologies, the current CA solution has three major both from the vehicles and from the road-side infrastructure
phases, i.e. sensing, communication, and collision avoidance, to surrounding vehicles, road-side infrastructure, edge server
as shown in Fig. 3 and described as follows: and the remote clouds [18]. Vehicles are allowed to share their
The phase Sensing collects and processes surrounding driving intent (e.g., emergency braking) and exchange sensing
environmental and vehicle information, including traffic sta- data, which can provide higher predictability. We will discuss
tus and vehicle status information, through on-board sensors communication technologies for safe driving will in Section V.
and corresponding sensing technology. We will describe the The phase Collision Avoidance contains three steps, namely
sensing technology from the aspects of sensor classification, prediction, risk assessment, and reaction, as shown in Fig. 3,
existing sensing methods, and AI for multi-sensor data fusion which is used to process the sensing data, detect the potential
in Section IV. danger and make the corresponding reaction in time to avoid
The phase Communication mainly relies on different com- or mitigate it [13]. In general, CA algorithms can be run on a
munication technologies, such as dedicated short-range com- vehicle, on an edge server, or on a central cloud server.
munications (DSRC), cellular network, and visible light to 1) Prediction: The prediction stage mainly uses the motion
ensure the real-time and reliable transmission of sensing data model to infer the future trajectory of the vehicle or the

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

driver’s behavior [14], [21]. According to the object being the future trajectory of the vehicle will cross at the same
considered, there are two categories of motion models, that time [27]. Then the information could be used in a vehicle-
is, physics-based prediction and maneuver-based prediction, to-vehicle/vehicle-to-infrastructure (V2V/V2I) system to warn
as the level of abstraction increases [13], [14], [22]. drivers. Joerer et al. exchange vehicles state data, such as
The physics-based prediction model is the simplest, which speed, acceleration, and distance to the intersection, through
can guarantee the accuracy of the short-term prediction [13], intervehicle communication to calculate the collision probabil-
[14], [22]. Berntorp employs the Rao-Blackwellized particle- ity based on the predicted trajectories [40]. Chen et al. estab-
filtering framework for vehicle-motion estimation by using lish a collision probability model to evaluate driving risks with
a kinematic model [23]. Xiang et al. propose to use the drivers’ intentions and vehicular trajectories considered [26].
vehicle kinematics (VK) model for vehicle trajectory predic- The risk-maneuver identification can be defined as a prob-
tion [5]. Ji et al. develop a 3-D potential field to generate lem where researchers want to distinguish between what is
a collision-free trajectory for the autonomous vehicle based a safe behavior and what is not. Al-Sultan et al. propose a
on the boundary conditions of the road and the VK model probabilistic model for inferring normal and abnormal driving
in [24]. Most of the traditional VK models only consider the behavior by combining contextual information [30]. Four types
vehicle factors (e.g. speed and acceleration) and neglect the of driving behavior, such as normal, drunk, reckless, and
human factors. Because this model consideration is relatively fatigue, are defined in advance. However, this method has
simple, reliability and accuracy are affected in complex traffic limitations that cannot fully take into account all possible
environments and in dealing with uncertainty. There are two situations.
main approaches for predicting this type of model. One is to As a kind of safety indicator and a crucial point,
assume that the vehicle parameters are known with accurate time-to-collision (TTC) indicates the remaining time before
data, and directly use the VK formula to derive the vehicle’s the collision. Due to the simple calculation principle and
subsequent state [5], [24], [25]. The second type of approach process, it is widely used in driving behavior analysis and
takes into account the existence of uncertainty. In order to potential collision assessment [41]. In general, there will be
eliminate the uncertainty of prediction, Kalman filter (KF) no danger if the TTC is much larger than time-to-avoidance
and its extended form, such as unscented Kalman filter (UKF) (TTA) [4]. However, since it only considers the speed and
and extended Kalman filter (EKF) are often used for state distance of the vehicles and ignores important factors such as
estimation [13], [22], [26], [27]. acceleration and human factors, the accuracy will be affected.
The maneuver-based prediction model can be used to obtain Schreier et al. promotes TTC as time-to-critical-collision-
the future state of the vehicle by identifying and predicting probability, making it valid for longer prediction horizons [28].
the driver’s intention as well as combining it with the vehicle Another indicator, time-to-react (TTR), is the remaining time
state parameters and road structure information. Therefore, for a human driver to avoid an imminent collision by emer-
this model is capable of performing longer and higher-level gency braking with full deceleration, steering with maximum
predictions in consideration of driver behavior and even inter- lateral acceleration, or a kickdown maneuver [25]. Therefore,
action between vehicles [13], [14], [22]. However, consid- TTR can be approximately expressed as the maximum of
ering the various uncertainties and the interaction between the time-to-brake (TTB), time-to-steer (TTS), and time-to-
multiple moving entities, the computational complexity will kickdown (TTK) [42], [43].
increase. In general, such methods require a large data- 3) Reaction: There are two main ways to deal with an
base of real driving data for model training. Stochastic emergency. For vehicles without automatic control, a warning
processes and AI techniques have spurred many algorithms message is issued to the driver through the user interface (UI)
for predicting driver’s behavior, including but not limited to: or voice prompts [4], [5], [27]. After receiving the warning
Neural Network (NN) [26], Bayesian Network [28], [29], information, the driver can make corresponding judgments
Dynamic Bayesian Network (DBN) [4], [30]–[32], Hidden and actions according to the actual situation. This method
Markov Model (HMM) [33], [37], Support Vector Machine can be adapted to the driver’s own driving habits, but due to
(SVM) [33], and Recurrent Neural Network (RNN) [34]–[36]. excessive dependence on the driver’s operation, timeliness and
In addition to using the above two types of methods alone, reliability cannot be guaranteed. The other way is to trigger
we can also combine physics-based and maneuver-based pre- the vehicle Automatic Brake System (ABS) or Active Control
diction methods to form a more comprehensive integrated System (ACS) to automatically brake to avoid the danger of
prediction method. collision for critical situations [4], [5], [26]. This method is
2) Risk Assessment: Based on predicted results, how to time-sensitive and can reduce the uncertainty of the driver’s
apply the prediction results to assess whether a dangerous operation, but cannot take into account the habits of different
situation occurs is the work done by the risk assessment. drivers and passengers, thus affecting the driving experience.
For risk assessment, we found that the three most popu- In addition, the automatic control system may also go wrong.
lar methods are collision probability calculation [26], [27], We outline the main steps of CA and the mainstream
[40], risk-maneuver identification [4], [30] and other safety methods used in each step are depicted in Table I.
indicators [28], [41]–[43].
The method of collision probability calculation is gener- B. Characteristics and Key Research Issues of CA
ally based on driving trajectory prediction. Barrios et al. 1) Multi-Source Data: High quality data is the premise and
determine whether there is a collision by judging whether foundation of subsequent prediction, control, and management.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 5

TABLE I
A S UMMARY OF C OLLISION AVOIDANCE S TEPS

TABLE II
C OMMUNICATION AND P OSITIONING R EQUIREMENTS FOR T YPICAL S AFETY-R ELATED A PPLICATIONS

In addition, over-the-horizon information exchange will make ITS, is carried out to obtain more comprehensive informa-
the transportation system safer and more efficient. At the same tion, which makes the vehicle make a timely evaluation and
time, however, the volume and variety of data generated by accurate decision to improve the safety [45]–[47].
multiple sources such as sensors, roadside detectors, remote 2) Ultra-Low Delay Requirement: The Internet of Vehicles
sensing, GPS, cell phone tracking, and Unmanned Aerial (IoV), especially in safety-related application scenarios,
Vehicle (UAV) increase dramatically [27], [44], [45]. requires extremely low latency, which is demanding. On one
Using the communication network as the channel of sensing hand, from a communications perspective, guaranteeing
information gathering, the multi-source sensing data can be ultra-low latency in data transmission is an indispensable
collected and exchanged as data support for follow-up work. requirement. Authors in [48], [49] summarize some system
However, due to the different characteristics of various sensors, performance requirements, such as communication mode,
the sensing data has characteristics of a wide variety, large minimum transmission frequency, and critical latency. If the
volume, and scattered storage, etc. On one hand, the attributes requirement of ultra-low latency is satisfied, the driving
of the sensor determine the attributes of the original data, distance before the vehicle receives the instruction will be
including the type, period, information content, and data greatly reduced. To meet this requirement, one of the current
structure. On the other hand, based on accuracy, reliability, research trends in the design of vehicle-to-everything (V2X)
and other requirements, different types of data are determined is to support the ultra-low latency and ultra-high reliability
in terms of importance, timeliness, and transmission reliability. requirements of driving safety [50]–[52]. In addition to trans-
To overcome the limitations of low reliability and small mission delays, the computation time, which mainly depends
effective detection range of the single sensor, multi-source on the hardware facilities and algorithm complexity, also seri-
data fusion, as the inevitable trend of the development of ously affects the effectiveness of safety-related applications.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE III
KPI S FOR C OLLISION AVOIDANCE S YSTEMS

According to [4], the computation process lasts/(does not to predict what might happen in the future, thus understanding
stop) until the danger is eliminated. It is expected that one the performance changes of the system.
computation process does not exceed 200 ms [53]. Ultimately, the performance of the CA system can be
3) Ultra-High Accuracy Requirement: Traffic accident measured by the accuracy of the warning, the real-time of the
prediction and collision avoidance require accurate data related warning, and the number of collisions that are successfully
to scene measurements. However, different sensors will affect avoided. Depending on the characteristics and needs of the
the accuracy of the sensing data. Due to many uncertainties, CA system, KPIs are generally divided into three categories:
driver behavior and vehicle trajectory are often difficult to effectiveness, reliability, and comfort of the passengers [4],
accurately predict, and the consequences of inaccurate sensor [5], [53], [58], [60]. According to the description of KPIs in
data may be more serious [4], [13], [54]. The accuracy of the literature, we list the typical KPI categories and provides
the algorithm can also influence the results of the prediction descriptions and examples for each category in Table III.
and the decision. Therefore, advanced mathematical methods Effectiveness has always been an important indicator of
such as stochastic, nonlinear, and fuzzy theory should be safety-related applications. Specific indicators include but
considered to develop predictive models more accurately [54]. are not limited to communication time and computing
In addition, AI algorithms are used to better adapt to the time [4], [53], [60]. For example, the authors in [60] study the
driving environment, thereby further improving the accuracy effect of the server location (i.e. on the edge and the cloud)
of prediction and decision making. on the effectiveness of collision avoidance.
4) Passenger Comfort Requirement: A good CA system not Accuracy mainly includes the quality of the alarm and
only ensures high accuracy and reliability but also ensures pas- quality of communications. The indicators used by many
senger comfort during collision avoidance. Passenger comfort studies and tests to evaluate the performance of alarm systems
is affected in many ways during collision avoidance. The brake are false positive, false negative, true positive, and true nega-
deceleration is an important indicator for assessing passenger tive [5], [58]–[60]. Evaluation and verification of CA systems
comfort [55], [56]. The existing works are mainly studied from are full of challenges. These four indicators can roughly reflect
the perspective of vehicle control. According to [49], [57], the accuracy of CA systems. After receiving the warning, the
a summary of requirements for some typical safety-related driver (automatic controller) still has the opportunity to avoid
applications is depicted in Table II. Among these applications, collision by steering or braking. In more recent works, the
the communication type required by collision warning-related authors also consider the effectiveness in the quality of the
applications is event-driven, while the communication type of alarm, such as true and timely positives and true but late pos-
other applications is periodic broadcast. In addition, V2V com- itives [60]. Communication accuracy refers to the accuracy of
munication technology is the most widely used. Intersection the transmission of information, mainly expressed as error rate.
collision warning, post-crash warning, and pedestrian warning The accuracy of the transmitted information will directly affect
will use V2I or V2P communication. Safety-related appli- the prediction and decision-making effects of the CA system.
cations have strict requirements for latency and positioning Comfort of passengers is closely related to the performance
accuracy. of the CA system. A highly effective system will have more
time to deal with dangerous situations, thus eliminating the
need for hard braking. Similarly, a highly accurate system
C. Key Performance Indicator (KPI) will produce fewer false positives, which will have less impact
A KPI is any measurable value that demonstrates the on the passenger experience. The comfort of passengers can
performance of CA systems and helps to improve system be measured by the brake deceleration during emergency
performance. We can analyze current and historical KPI data braking [56], [61].

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 7

IV. S ENSING FOR S AFE D RIVING

Sensing of complex driving environment and collecting data
are the necessary prerequisite for accurate decision-making
and safe driving. Therefore, it is necessary to briefly introduce
the sensor technology currently used in the CA system.

A. Sensors for Safety

Through sensors, the distance from the obstacle, vehicle sta-
tus, and position information can be obtained. Therefore, these
sensors can be classified into the following types according to
their functions.
Distance Sensors: Multiple sensors, such as radar sen-
sors, LiDARs, camera vision, and infrared sensors have been
installed on vehicles. All of them are adopted as distance Fig. 4. Multi-sensor data fusion.
sensors for monitoring the area surrounding a vehicle to detect
the hazard. of the sensed object, thus improving the knowability of infor-
Vehicle State Sensors: Speed sensors and accelerometers mation and the detectability of the target [41], [47], [68]. Fig. 4
are considered here to be the most important source of vehicle depicts the pipeline of multi-sensor data fusion.
status information, and they provide the vehicle speed and Current methods are difficult to achieve accurate and com-
acceleration information needed for early warning algorithms. prehensive sensing of complex conditions, and the sensing data
Position Systems: The positioning based on the inertial cannot be fully mined and utilized. Therefore, we need to carry
navigation system (INS) has the characteristics of fast update out further research on data sensing methods in the complex
frequency and high short-term accuracy, but the accumulated traffic environment, as well as to carry out data fusion and
error over time is relatively large [62]. Global Navigation adaptive processing by leveraging AI technology.
Satellite System (GNSS) is the most mature global positioning
technology applied today, but there are many problems in the C. AI for Multi-Sensor Data Fusion
urban canyon environment. Although differential GNSS and Timely and accurate multi-source fusion data can make up
INS combined methods can achieve centimeter-level position- for the defect of single sensor data that maximizes the safety
ing accuracy, this method is expensive and highly dependent and efficiency of the transportation system. Table IV sum-
on GNSS base stations. Currently, it is mostly applied to marizes the related up-to-date research works on data fusion.
algorithm testing of autonomous vehicles [63]. Multi-system Lahat et al. introduce a comprehensive list of challenges
and multi-frequency fusion is the development trend of GNSS in data fusion and corresponding solutions. Extensive works
positioning, and abundant observation data will bring many utilize intelligent technology to efficiently fuse multi-source
advantages to precision positioning technology. With the data efficiently and accurately [68].
continuous development of vision technology, vision-based The probability-based data fusion method is the most classic
positioning technology has been widely used. The relative and simple and has been widely used [69]. Common methods
positioning technology based on vision mainly includes Visual are mainly based on HMM and Bayes. Utilizing a defined
Odometry (VO), as well as Visual simultaneous localization joint density model over multimodal input space, the missing
and mapping (V-SLAM) [64]. inputs can be filled, thus easily fusing multimodal data with
Since the introduction of sensors is not the focus of this noisy or missing input modalities. Finally, the target data are
survey, the detailed introduction of sensor principles and fused based on Bayesian Estimation [70]. Biresaw et al. [71]
characteristics can be referred to [15], [65]–[67]. propose to use two Bayesian trackers with different formulas
for visual tracking. Then various sequences are used to show
B. Environment Sensing Methods the robustness of the fusion framework with challenges such
Benefiting from these huge numbers of vehicular sensors, as occlusion, illumination changes, and shadows. However,
vehicles can be considered to be an important source of this type of method does not accurately represent complete
sensing data. Collecting, sensing, and computing various traffic information and the accuracy needs to be improved. Therefore,
factors around the vehicle as vehicles move is also one of the more works are currently done using learning-based methods,
most widely used methods of data collection. However, the such as KF, SVM, ANN, and Boltzmann Machine.
high dynamic of vehicles makes the sensing and processing of The Kalman filter and its extended forms are some of
data have some limitations. Vehicular communication allows the earliest research and widely used methods in data fusion
data generated by on-board sensors to be propagated to other problems [72]. Conventional state-vector fusion and measure-
vehicles or infrastructure. Specific communication technolo- ment fusion are two kinds of methods for KF-based data
gies will be discussed in Section V. fusion [72]. For example, in order to obtain complete target
Information fusion technology synthesizes information from information and improve perceptual performance, Yu et al.
different sources, different modes, different media, and differ- propose the parallel KF to track the target of the cam-
ent forms of expression to obtain a more accurate description era and radar respectively [70]. Geng et al. present a data

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE IV
S UMMARY OF THE S ELECTED R ESEARCH L ITERATURE AND C ONTRIBUTIONS TO THE D ATA F USION

fusion scheme based on fault-tolerant Kalman filter (FTKF) According to the type of mathematical method used, the
for integrated navigation systems [73]. The scheme enables data fusion method can be divided into the probabilistic
multi-height sensors data fusion, detecting faulty sensors, and method and the learning method. From the results of the
performing fault tolerance in real-time. literature review of data fusion, we can conclude that most
The Bayesian-based algorithm can make a large part of the of the current works tend to be based on learning methods.
prior knowledge, which has great advantages to process small In particular, SVM-based methods are suitable for processing
amounts of data. However, when the data dimension increases, high-dimensional data. The KF-based methods are relatively
there is a limit to the Bayesian-based algorithm, and SVM can simple, and their extended form can be used to process
overcome this defect and play its role. In [74], an SVM-based linear or nonlinear data. The deep belief network-based and
method for fusion hyperspectral and LiDAR data is proposed. BM-based methods are good ways to handle missing data. The
In recent work, Zebin et al. [75] propose a general-purpose ANN-based methods work well with nonlinear data to explore
framework for facilitating the use of machine learning algo- complex data relationships. Compared with probabilistic meth-
rithms, such as SVM and ANN, for data fusion and pattern ods, the accuracy of learning methods is higher while the
recognition. In addition, the authors explore the amount of data complexity and requirements for computing power are higher.
and data fusion such as multi-sensor locations as well as issues
affecting overall performance in detail. In [76], a multisensor V. C OMMUNICATION FOR S AFE D RIVING
and multimodal vehicle detection system is proposed, which In addition to sensors, real-time communication of the vehi-
fuses the data of color camera and 3D-LiDAR. Particularly, cle while driving are also required. With V2X communication,
an MLP is used to learn the nonlinear relationship between a vehicle can “talk” to other vehicles, non-motor vehicles,
data to perform multimodal detection fusion. transportation infrastructure, and pedestrians, even if they are
Since some missing data modalities can be handled, a Deep not within the vehicle’s direct line-of-sight. In this section,
Belief Networks model is proposed for learning a joint repre- we will describe the communication technologies applied
sentation of multimodal data [77]. Similarly, the well-known in CA.
Boltzmann Machine (BM) and its generalizations are suitable
for data fusion. Srivastava and Salakhutdinov propose a multi-
modal Deep Boltzmann Machine (DBM) for learning a gener- A. Communication Technologies Applied in CA
ative model of multimodal data [78]. A DBM [79] is a network As two or more V2X-enabled entities enter the communica-
of symmetrically coupled stochastic binary units. Because tion range, they form an ad-hoc wireless network that allows
of the features of DBM, such as layer-by-layer pretraining them to automatically send real-time dedicated safety-critical
procedure, incorporating top-down feedback for approximate messages. Some communication technologies such as DSRC,
inference, and optimizing jointly parameters of all layers, the cellular network, ZigBee, and visible light communications
DBM can better incorporate uncertainty about missing or noisy (VLC), have been studied or applied to CA systems. Among
inputs, thus suitable for modeling the multimodal data for a them, DSRC and cellular-based V2X (C-V2X) have always
system with multiple sensors. been the focus of research and are being deployed. VLC is

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 9

a new technology using visible light and is in the research 2) Basestation-enabled Technologies:
stage. ZigBee is a low-power technology that is more used in C-V2X is a cellular-based V2X technology, which is
experiments. a term proposed by 5GAA to distinguish cellular-based
1) IEEE 802.11 Family Technologies: V2X from DSRC. Conceptually, it includes LTE-V2X and
DSRC is an efficient wireless communication technol- NR-V2X. LTE-V2X mainly relies on current LTE technolo-
ogy based on the IEEE 802.11p standard that enables gies, as well as the old standards [80]. The LTE-based tech-
the identification and communication of moving targets in nology called LTE-V2X started in 2010. LTE-V2X technology
high-speed motion in specific small areas. The DSRC stan- includes two working modes: mode 3(LTE-V-Cell) and mode 4
dards are not compatible with each other in the United (LTE-V-Direct). The difference between the two modes lies
States, Europe, and Japan. European ITS-G5 reserves 70 MHz in the wireless resource allocation method. Resources are
(5855 MHz–5925 MHz) for V2X communications, of which allocated by the cellular network in mode 3, and mode 4 uses
30 MHz (5875 MHz-5905 MHz) is reserved for traf- distributed scheduling that does not require cellular cover-
fic safety-related applications. The US Federal Com- age scheme. Since V2V security applications cannot rely
munications Commission (FCC) has allocated 75 MHz on infrastructure-based cellular coverage, mode 4 is consid-
(5850 MHz–5925 MHz) in the 5.9 GHz frequency band for ered the baseline mode and an alternative to DSRC [85].
V2X applications and has reserved 5885 MHz–5895 MHz In China, the 5905-5925 MHz frequency has been officially
“control channel” frequency band for traffic safety. In Japan, allocated for LTE-V2X verification. Li et al. [86] analyze and
ARIB STD-T55 (ETC), ARIB STD-T75 (DSRC) and improve the collision avoidance strategy based on LTE-V2X
ARIB STD-T88 (DSRC application sublayer (ASL)) have for an autonomous vehicle, thereby enhancing the practical-
used 80 MHz (5.770-5.850 GHz) DSRC spectrum. In 2011, ity of autonomous driving on the highway. LTE-eV2X R15
ARIB STD-T109 allocated a new frequency range of introduces additional, and LTE-V2X R14 compatible, key
755.5–764.5 MHz for various types of ITS applications [16], enhancements such as 64-Quadrature Amplitude Modulation
[18], [80], [81]. The IEEE 802.11p / ITS-G5 protocol is mainly (64QAM), small-delay Cyclic Delay Diversity (CDD), and
used in the US and Europe. On-Board Unit (OBU), RSU and carrier aggregation, thereby further improving the performance
related protocols are important parts of DSRC. For example, of delay, speed and reliability [87]. Currently, C-V2X can
V2V and V2I communications enable real-time transmission enhance road safety and traffic efficiency, which is also
of image, voice, and data information to organically connect an important part of achieving autonomous driving. Related
vehicles and roads. DSRC-based communications offer the standardization work is also continuing to be discussed and
benefits of low end-to-end latency, self-organization, and low promoted.
cost for V2X applications [80]. According to statistics, it is C-V2X includes PC5 interface and Uu interface, supporting
possible to reduce traffic accidents by 81% using V2X com- smooth evolution from LTE-V2X to 5G-V2X. 5G does not
munication [81]. involve specific standards, but instead of changing the exist-
DSRC-based technology has been the focus of many ing communication architecture (eg, LTE), the next step of
researches supporting vehicle communication in the previ- demands and new technologies [18], [80]. The Recommenda-
ous 20 years. Xiang et al. propose a DSRC-based model for tion ITU-R proposes to support eight key KPI improvements
rear-end collision avoidance [5]. Zhao et al. propose a rear-end in peak data rate, user experienced data rate, latency, mobility,
collision warning system (ReCWS) and explicitly states the connection density, energy efficiency, spectrum efficiency,
functional specifications of the DSRC technology such as and area traffic capacity [88]. The IoV is a very important
transmission delay and propagation delay [82]. In [4], V2I application scenario for 5G. 5G can use the existing LTE
communication is used to collaborate on intersection collision frequency range (600 MHz to 6 GHz) and the Millimeter-Wave
avoidance, which provides a wider range of communication. band (24-86 GHz). Non-orthogonal multiple access (NOMA)
In addition, DSRC-based V2V communication is also used technology is used to improve spectral efficiency. It can
for vehicle cooperative positioning, improving positioning reach a data rate of 10 Gb/s compared to other vehicular
accuracy and indirectly improving traffic safety [83], [84]. communication technologies [18], [89]. With the development
In the industry, Toyota and General Motors have launched of communication technology, authors of [90] put forward a
DSRC-enabled vehicles. However, the number, distance, and theoretical analysis of C-V2X-based basic safety-related ITS
interference of the RSU layout remain to be resolved. There- spectrum requirements, and the results indicate that at least
fore, DSRC performance evaluation, protocol design is worthy 20MHz frequency should be allocated to support V2V/V2I
of continuous research. In addition, another problem that the basic safety applications. 3GPP announced the freezing of
IEEE 802.11p-based DSRC must solve is to provide secure, the R16 standard (NR-V2X), marking the completion of
verified and authenticated messaging and protect user privacy. the first evolutionary version of 5G. The standard supports
Although DSRC/ITS-G5 had long-term plans in the US and V2V and V2I direct communication. By introducing a
Europe, many researchers believe that C-V2X technology will variety of communication methods and technologies, V2X
develop faster in the future. In particular, in November 2020, supports more abundant application scenarios such as platoon,
FCC officially decided to allocate the 5.9 GHz frequency band semi-autonomous driving, and remote driving. However, from
to Wi-Fi and C-V2X, which marked the elimination of DSRC the introduction of the standard to the actual application,
in the US. a strict performance and safety test process is required.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

Therefore, the current application of the IoV depends more communication range, mobility, data rate, and heterogeneous
on the mature LTE-V2X-based technology. communication networks [94].
5G Automotive Association (5GAA) compared DSRC and 4) Low-power Technology:
C-V2X in the laboratory and in the field. It can be seen from ZigBee is a short-range, low-complexity, low-power,
the test results that C-V2X supports longer communication low-data-rate, low-cost wireless network technology, which
distance, better non-line-of-sight performance, stronger reli- is based on the IEEE 802.15.4 standard. ZigBee relies on
ability, higher capacity, and better congestion control [91]. independent nodes, and each node has different functions.
Performance advantages in these aspects are very important It has a communication range from 10 to 100 m, and a
for improving traffic safety. For example, in non-line-of-sight maximum data rate of 250 Kb/s (2.4 GHz band with Offset
scenarios, the performance of on-board sensors is limited. Quadrature Phase Shift Keying). Priyanka and Kumar [98] use
In addition, cost and standardization also make C-V2X Zigbee to transmit information about their speed and position
develop rapidly. C-V2X is based on a cellular network with between vehicles to avoid collisions. In addition, ZigBee can
wide network coverage, low deployment cost, and clear evolu- also be used to connect electronic components of CAVs for
tion route.C-V2X utilizes and reuses the upper-level protocols vehicle monitoring and status reading [18]. If ZigBee terminal
that the automotive industry has defined, including organiza- equipment is distributed along streets, a GPS-like service
tions such as the Society of Automotive Engineers (SAE) and can be provided with more precise and specific information.
the European Telecommunications Standards Institute (ETSI), Although the performance of ZigBee is relatively low, its
as well as the entire industry’s investment in the ITS-G5/DSRC low complexity and low-cost feature make it suitable for
upper-level protocols over the years. communication simulation.
3) Visible Light Technology: Since the scope of this survey is not vehicular communica-
Due to the limited bandwidth available, current networks tion technology, we do not introduce other technologies such
cannot fully meet the growing traffic demands. Ease of use, as Wi-Fi, WiMAX and Bluetooth. In addition, the detailed
low cost, high data rate, and large geographical distribution introduction of vehicular communications can be referred
of VLC make it a great potential to become a wireless to [18], [48], [80]. Communication technology is mainly used
communication technology that complements or replaces exist- in the following aspects to improve traffic safety.
ing radio frequency (RF)-based communications [92]. Some Early Warning Information Release: This is the most
light sources such as headlights, taillights, traffic lights, and important application of communication technologies in CA.
billboards can be used to transmit vehicle or traffic infor- Safety-related applications are characterized by very stringent
mation. Compared to RF communication, VLC uses visible requirements in terms of communication delay, reliability, and
light (380-780 nm) as a carrier for data to provide greater packet error rate, otherwise, they will cause unimaginable seri-
bandwidth. In addition, the cost of this technology is low due ous consequences. Therefore, DSRC, VLC, and 5G are more
to the unregulated spectrum. In addition, the huge available suitable for conducting early warning information releases.
spectrum enables VLC to achieve very high data rates [93]. It can be applied to broadcast abnormal driving behavior,
Finally, VLC technology is also fully compatible with RF com- vehicle overtaking, and other lane changing behaviors, so as
munications and can be part of the future 5G technology [94]. to facilitate early understanding of driving intentions [4],
This feature can be used to build heterogeneous systems con- [5], [26]. In addition, it can be applied to PCW systems to
sisting of VLC and other RF-based communication schemes to alert pedestrians to potential traffic accidents or minimize the
provide reliable communication systems for vehicles, as these impact of the accident [99], [100]. In general, the warning
technologies can complement each other. information consumes little bandwidth, but the vehicle status
The VLC has been the focus of intensive research in the information used to evaluate the risk of accidents between
past few years. Wang et al. [95] propose to use VLC-based vehicles needs to occupy network resources, depending on
ITS for accident avoidance, especially when the truck fleet the data type and the volume of data. According to [80],
is crossing an intersection. Specifically, signals related to the early warning information application requires a latency
acceleration, deceleration, and braking are sent through the of 20-100ms.
VLC to RSUs that can trigger the appropriate signal. By using Traffic Supervision: This type of application is mainly used
the directionality and impermeability of light, Ucar et al. [96] for real-time monitoring of traffic status by traffic regulatory
propose a hybrid communication hybrid protocol based on authorities. It can make information available for the first time
IEEE 802.11p and VLC, namely SP-VLC, for autonomous after the accident, so as to take effective rescue and traffic
vehicle platoon, which mainly solves the security issues caused evacuation measures to avoid secondary accidents [101]. Both
by RF-based communication technology. Uysal et al. [97] V2V and V2I communication are effective methods. This type
evaluate the performance of VLC-based V2V systems, taking of application does not require extremely high latency for
into account the effects of measured headlamp beam patterns early warning information transmission. In addition, traffic
and road reflected light. The results show that according to regulation may rely on images and video data acquired by
the position of the headlights, a data rate of 50 Mbps can cameras, and take the form of remote processing events, thus
be achieved when the distance between the vehicles is 70m. requiring transmission bandwidth and network throughput.
However, this technology is still in its infancy and needs Cooperative Driving: Some cooperative driving applications
to overcome current challenges, such as noise robustness, use vehicular communications, such as vehicle cooperative

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 11

positioning [83], [84], cooperative platooning [102], [103], learning (DRL)-based routing strategy is proposed for
and cooperative adaptive cruise control (CACC) [104], [105], software-defined vehicular networks. The software-defined
can also reduce traffic accidents. Cruz et al. propose to use networking (SDN) controller acts as an agent to learn the most
vehicular communication (i.e. DSRC) and smartphone sensors trusted routing path through DNN. In another work [115],
to improve overall localization performance [83]. In [84], V2V a clustering-based reliable routing algorithm for vehicular
communication is used to share motion states and physical communication is proposed, in which simulated annealing
measurements between vehicles, and the positioning accuracy is used for proper clustering of nodes, and a radial basis
is improved. And similar to safety-related applications, they function neural network (RBFNN) is used for cluster head
also have strict requirements on communication latency and selection.
frequency. According to [80], approximately 5 Mbps through- For resource allocation, He et al. [116] present an integrated
put and latency as low as 2-10 ms for supporting cooperative framework to dynamically allocate networking, caching, and
driving applications. computing resources and the DRL algorithm is used to solve
Cooperative Collision Avoidance: Adding inter-vehicle highly complex joint resource optimization problems. In the
communication to the CA system can form cooperative col- scenario where the RSU has a limited-lifetime power source,
lision avoidance (CCA), allowing vehicles to obtain more a DRL-based scheme is proposed for optimizing the RSU’s
accurate information and expand the perception range. Hafner downlink traffic scheduling during a discharge period [117].
et al. use V2V communication to determine whether vehicles DRL enables RSU to learn and adapt to optimal decisions,
at intersections need automatic braking for collision avoid- extending battery life while meeting the maximum number
ance [106]. On the basis of considering the information shar- of service requests. In another work [118], authors propose a
ing of neighbors, authors of [107] propose a comprehensive decentralized DRL-based resource allocation mechanism for
method for designing and analyzing the behavior of CAVs, V2V communication. In this work, both unicast and broadcast
which is used to study multi-vehicle CCA problems. In [108], scenarios are considered. The decentralized mechanism can
authors use V2X communication to design a real-time and tightly control the transmission overhead.
implementable collision avoidance algorithm and simulated Since the vehicular network is vulnerable, authors use
different collision risks. For intersections, authors set up a the artificial neural network (ANN) to design the VANET
supervisor, which receives status information from cars, then intrusion detection mechanism for Denial of Service (DoS)
estimates possible collisions and sends control signals when attacks [119]. Alheeti et al. [120] use fuzzy logic to detect
necessary [109]. It can be concluded that vehicle communi- packet loss attacks in the vehicular network. Berlin et al. [121]
cation can effectively overcome the defects of sensors and propose a Security Information and Event Management Sys-
improve driving safety. tem (SIEM), which combines the deep learning algorithm
and Big data algorithm to identify attacks. Authors of [122]
B. AI for Vehicular Communication and [123] propose AI-powered jamming DoS detection to
In recent years, some applications of AI technology in enhance the communication security of highway vehicle
vehicular communication have shown tremendous potential to platoon.
utilize multiple generated data sources and store them in the Table V summarizes the related up-to-date research works
network to learn the dynamics in the environment, and then on applying AI to vehicular communication. AI technol-
extract the appropriate features for use in many tasks for com- ogy plays an important role in channel estimation, routing,
munication purpose [110]. Some learning methods are used resource allocation, and network security. However, the cur-
for routing, resource allocation, channel quality estimation, rent standards for V2X systems are not uniform enough,
etc. Together with AI, communication technologies can obtain and the application of communication technologies still faces
information from different sources, expand the range of vehicle challenges. AI technology provides intelligent methods for
sensing, and predict to avoid potential accidents, thereby codec and data transmission for different communication
improving driving comfort, safety, and effectiveness [111]. systems (e.g., DSRC and cellular).
In addition, AI technology can improve the ability to adapt
to the environment in the case of high dynamic scenarios and C. Challenges and Discussion
uncertainties [110]. High Dynamics: The fast-moving speed of the vehicles
In terms of learning-based channel estimation, makes the network have high dynamic characteristics, which
Wen et al. [112] utilize the propagation characteristics will have many influences on the application of commu-
of massive MIMO systems and then uses the sparse nication technologies. For example, the network topology
Bayesian learning method for accurate channel estimation. changes frequently and the communication link life is rela-
Ye et al. [113] propose to use deep learning to perform tively short, which affects the establishment and selection of
channel estimation and signal detection in OFDM systems. routes. Communication between vehicle nodes in the IoV is
The deep learning-based approach can resolve channel disturbed by weather conditions, surrounding buildings, acci-
distortion and detect transmitted symbols with performance dental obstacles, and road traffic conditions. Some emerging
comparable to the minimum mean square error estimator, technologies such as mmWave communication have strong
making the system more robust. directionality but are not suitable for omnidirectional antenna
Another application of AI in vehicular communication is the communication, so there is still a need to study fast beam-
design of routing protocols. In [114], a deep reinforcement forming in high dynamic scenarios.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE V
S UMMARY OF THE S ELECTED R ESEARCH L ITERATURE AND C ONTRIBUTIONS TO THE V EHICULAR C OMMUNICATION

Infrastructure Deployment Issues: For DSRC communi- VI. AI-BASED A LGORITHMS FOR S AFE D RIVING
cations, the deployment of the RSU remains a challenge. In the collision avoidance algorithms, AI technology has
From an economic perspective, the cost of installation and been applied in multiple aspects, ranging from vehicle
maintenance cannot be ignored. For C-V2X, although there trajectory and driving behavior prediction [4], [31]–[39],
are already many mobile network infrastructures in opera- [125]–[129], to platoon and autonomous breaking [102], [104],
tion, it still does not provide comprehensive cellular network [105], [130]–[132].
coverage. In order to meet the communication requirement,
providing a wide coverage and high-capacity communication
network, infrastructure layout still has problems. A. Vehicle Trajectory and Driving Behavior Prediction
Security and Reliability of Communications: The applica- In a CA system, both driver behavior prediction and vehi-
tion of communication technology enables vehicles to provide cle trajectory prediction are indispensable links. Since the
driving safety data to surrounding vehicles in advance, so that previous works are mostly considered from the state of the
surrounding vehicles can predict the driving behavior of other vehicle, the vehicle trajectory prediction study is earlier than
vehicles and implement active safety strategies. At the same the driving behavior prediction. Recent works tend to be more
time, the safety and reliability of communication are of para- predictive of driving behavior, or combined driving behavior
mount importance due to the connectivity between the vehicle with a driving trajectory to get more accurate predictions.
and the environment. For example, due to network secu- In order to improve the safety of intersections,
rity issues, vehicles may receive wrong data or instructions, Aoude et al. [33] develop an HMM-based algorithm to
which may cause more serious traffic accidents. However, estimate the driver’s behavior at the intersection, that is,
safety-related data usually has a short processing period, whether it is violating. For highway scenarios, Lane Keeping
which makes it a big challenge to ensure the security and Assistance Systems (LKAS) is important for driving safety
reliability of information transmission. In addition, it is very and comfort. In [37], a new LKAS system is proposed,
important to perform security authentication on the identity which mainly designed the driver learning model combined
of the participants in the vehicle communication process. with HMM and Gaussian mixture regression (GMR) to
The security of vehicular communication needs to meet the predict the lane departure. Combine on-board sensor data
following requirements, such as the integrity and privacy of such as machine vision and lidar, Laugier et al. [125] use
information, the ability to resist external security attacks, the traffic scene modeling and collision risk assessment to
authenticity of received data, as well as the integrity of data. ensure driving safety in any traffic scenarios. In particular,
In addition, vehicular communications must provide secure, the layered-HMM and Gaussian process (GP) models are
verified and authenticated messaging and protect user privacy. built to predict behaviors in a short term, which can handle
A detailed assessment of communication security is not within uncertainty during driving. Gadepally et al. [126] propose
the scope of this survey. Authors of [124] have conducted a a framework for estimating driver behavior at intersections
complete overview of the security vulnerabilities and solutions based on modeling driver behavior and vehicle dynamics as
of vehicular networks. a hybrid state system (HSS), which can be flexibly applied

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 13

to other scenarios. Among them, HMM is used to estimate predict the driver’s behavior at intersections (i.e., turning or
driver behavior. going straight), and performing performance tests at intersec-
In view of the limited ability of HMM to express, more tions of different shapes.
works have recently focused on a more complex graph model, In addition to these single prediction models, other
i.e., DBN. Unlike the HMM, which uses a variable to represent works take into account the advantages of different models
the implicit state, the DBN uses a set of variables to represent and methods to design a combined model [36], [129].
the implicit state. Specifically, the graph model can be used Bahram et al. [36] fully consider the advantages of
to express the conditional independent relationship between combinations of model-based and learning-based estimation
variables in each unit of the data sequence and the unit. In [4], methods in terms of scalability of expert knowledge and
considering the influence of driver behavior on vehicle status, difficulty of reducing dimensionality hazards, and propose an
the authors propose a DBN-based vehicle state evolution online-capable driving behavior prediction framework. The
model. Similarly, for the intersection scenario, the authors spatio-temporal cost map is used to determine the vehicle
consider that the interaction between vehicles is governed by interaction-aware behavior, and then the Bayesian filter (BF)
traffic rules. Based on this fact, the DBN is used to estimate is used for maneuver prediction. For the problem that a
the state of the vehicle [31]. Gindele et al. [32] build a specific model cannot balance quality and scope, the authors
hierarchical DBN to simulate the driver’s decision-making in [129] propose a model that combines multiple classifiers,
process. In order to make the prediction more accurate, the each of which is suitable for a specific scenario. In particular,
road topology was taken into account in the modeling process. the author uses the scenario model tree (SMT) to organize
In addition to the generation methods such as HMM, the the scenario and scenario-specific classifiers.
discriminative learning algorithm is also widely used. The Table VI summarizes the related up-to-date research works
driver’s behavior is not only related to the current state but that predict vehicles’ trajectory and driving behaviors. As the
also affected by the state of the previous moment. Therefore, number of data increases, driver behavior prediction can avoid
it is necessary to combine the state of the previous moment complex vehicle trajectory prediction and become the pre-
to obtain a more accurate prediction result. The authors ferred choice for many CA algorithms. The probability-based
in [33] also use a discriminative approach, i.e., a SVM-based methods still occupy very important positions and main-
algorithm, for driver behavior classification at intersections. tain good performance in most scenarios. However, clas-
The performance of this method is better than the traditional sic machine learning methods, such as SVM and MLP,
method. For vehicles approaching the traffic light scenario, are not suitable for increasingly complex traffic conditions.
Ortiz et al. [38] use a simple neural network model, that Algorithms based on RNN and deep-RNN will be promising.
is MLP, to predict driver behavior. Due to the simplicity of
the model, it can be extended to large-scale online learning. B. Platoon and Autonomous Braking
In [39], an RVM-based driver behavior prediction method is Although the collision warning system greatly improves
proposed, which can predict the driver’s intention to change driving safety, it still causes problems such as the driver’s
lanes a few seconds in advance. At the same time, this paper nervousness. Vehicle platoon can actively control vehicles
deeply analyzes the challenges from laboratory simulation to to avoid collisions and improve traffic safety. Enabling the
an actual road test. However, these methods are not sufficient connectivity between vehicles and the cloud, vehicles and
to cope with increasingly complex traffic conditions. vehicles, vehicles and the environment, the CACC application
Among many ANN algorithms, RNN is well suited for will reduce crashes by forming a platoon with small space/time
this type of prediction scenario because it allows neu- headway. Long-term research shows that the AI-based methods
ronal connections from a layer of neurons to previous outperform the traditional methods in vehicle platoon. Dou-
layers of neurons. In the transportation domain, it was glass et al. [130] use LSTM Neural Network to predict the
applied for vehicle trajectory prediction and driving behavior cut-ins in multiple trucks in leader-follower platoons, thereby
prediction. In [128], two types of deep learning models, improving the safety of truck platooning. Guo et al. [102] pro-
i.e., Long Short-Term Memory (LSTM) and the Convolutional pose a Chebyshev neural network-based distributed adaptive
Neural Network (CNN) model, are proposed to predict a approach combined with sliding mode technique for handling
human-driven vehicles’ trajectory. In particular, CNN feeds vehicle-following platoons with input saturation. In an earlier
more information to the LSTM by using images, making work [104], an RL-based method is proposed to obtain an
trajectory prediction more accurate. For the unsignalized inter- autonomous longitudinal vehicle controller. Different from
sections scenario, Zyner et al. [127] propose an RNN-based other RL methods, a policy-gradient algorithm is designed
prediction algorithm for predicting driver intention. In partic- that can result in efficient behavior for CACC. More recently,
ular, the data used for forecasting is the Lidar-based track- a supervised reinforcement learning (SRL)-based framework
ing system, which is in line with the future direction of for the CACC system is proposed in [105]. During the training
autonomous vehicles. In [34], RNN is introduced to develop process, the network is updated under the guidance of the
a car-following model to study its effectiveness in predicting supervisor and the gain dispatcher.
the vehicles’ acceleration distribution of highway. Due to the Autonomous braking plays an important role in the CA
variety of shapes and forms of intersections, it is impossi- system. At the same time, this function is also one of the
ble to find general features and models to generalize them. enabling technologies of CAVs. It can effectively brake when
Phillips et al. [35] use RNN to build a predictive model to the driver reacts too late to the dangerous situation or the

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

14 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE VI
S UMMARY OF THE S ELECTED R ESEARCH L ITERATURE AND C ONTRIBUTIONS TO V EHICLE T RAJECTORY AND D RIVER B EHAVIOR P REDICTION

brake is insufficient, thus reducing the speed of the vehicle while ignoring the driver’s factors, the authors in [132] propose
and reducing the consequences of the accident. The DRL an FCW system based on the RL algorithm to handle the
technology is not only widely used in systems such as ACC, driving preferences of different drivers.
but also has significant research progress on autonomous brake Table VII summarizes the related up-to-date research works
control. More recent works have been done in this area. for platoon and autonomous braking. It can be seen that
In [131], an autonomous braking system based on DRL is RL technology is widely used in vehicle automatic control.
proposed to automatically decide whether to take the brake As data acquisition is more convenient, data such as images
according to the actual driving situation. According to the can be used for more accurate vehicle control. The method of
relative position of the obstacle and the speed of the vehicle, combining deep CNN and RL is worth studying.
the DQN is used to learn the optimal braking control strategy
by setting the reward and punishment function. Since one C. Challenges and Solutions
common feature in current ADAS applications is to focus on AI technology has been researched and applied in many
the environment of the vehicle and its surrounding vehicles, aspects of CA systems, but these systems still cannot achieve

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 15

TABLE VII
S UMMARY OF THE S ELECTED R ESEARCH L ITERATURE AND C ONTRIBUTIONS TO P LATOON AND AUTONOMOUS B RAKING

100% collision avoidance, and the characteristics of AI algo- VII. T ESTBEDS AND P ROJECTS FOR C OLLISION
rithms still affect safety. The CA system has strict require- AVOIDANCE
ments for AI-based algorithms, and current researchers are
Recently, research on CA has gradually moved from theory
working hard to overcome the following challenges. The
to practice. In order to better adapt to actual traffic scenarios
first is the verifiability of neural networks. One challenge
and reduce product risks, many testbeds and projects have been
in applying neural networks to systems that are critical to
made by academic and industrial research communities. In this
safety is that it is difficult to provide formal guarantees
section, we will briefly introduce some recent projects and
for their behavior. Katz et al. [133] propose Reluplex, which
tests dedicated to achieving traffic safety in actual scenarios.
combines linear programming technology with SMT (Satisfia-
bility Module Theory) solving technology for neural network
error detection. The second is poor interpretability and non- A. Academic Testbeds and Projects
transparency. Increasing the expressiveness of ML models In 2005, Caltrans collaborated with the Metropolitan Trans-
usually comes at the expense of transparency. Opacity is an portation Commission (MTC) and California PATH program
obstacle to safety because it is more difficult for researchers to of UC Berkeley established the nation’s first public-Connected
establish confidence that the model will perform as expected. Vehicle Test Bed on El Camino Real (State Route 82). And
In [134], authors use the influence function to determine in 2018, they collaborated to update the equipment in the
the most relevant training node for a given prediction result. testbed to meet the latest connected vehicle standards and
In [135], authors combine neural networks and decision trees implementation architecture. The testbed is currently operating
to maintain high-level interpretability while using neural net- at El Camino Real in Palo Alto, which includes 11 signalized
works to make low-level decisions. The proposed NBDT has intersections. It provides an operating environment where
the same interpretability as decision trees and can output intersections and vehicles can communicate with each other
intermediate decisions that predict results. Gehr et al. [136] using DSRC, so that CACC, cooperative intersection collision
propose a system called AI2 to use the abstract inter- avoidance, advanced traffic signal controls using connected
pretation framework to solve the scalability and precision vehicle data, and truck platooning can be implemented. More
challenges of neural networks. The third is the error rate. detailed information can be referred to [139]. At the same time,
AI models usually do not work perfectly and have errors. this team conducted research on deep automotive perception
The sources of errors may be training data and algorithms. to guide autonomous driving, such as efficient pedestrian
Therefore, when designing an algorithm, on the one hand, detection, pedestrian intent detection, and deep driving control
it is necessary to select appropriate training data, on the policies. In Canada, the University of Alberta (UofA) and
other hand, it is necessary to study the propagation and the University of British Columbia (UBC) jointly set up a
amplification of deviation. The fourth point is the instability connected vehicle testbed network called ACTIVE-AURORA,
of the AI algorithm, for example, it may fall into a local which provides a harsh winter climate to test the application
optimum. As a result, even if the training set remains the of ITS and connected vehicles. The current focus of the
same, the training process may produce different results. project is to improve safety by using connected vehicles
In order to solve the problem, a theoretical framework by and DSRC V2I technology to reduce pedestrian collisions at
Monte Carlo dropout sampling in DNN is proposed for signalized intersections. As a result, a DSRC has been used
uncertainty estimation [137]. In [138], Bayesian Deep Learn- to develop a proof of concept for the upcoming pedestrian
ing (BDL) is proposed to solve this problem and improve detection notice for connected vehicles. Carleton University,
safety. Transport Canada, and BlackBerry QNX have collaborated

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

16 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

on both connected vehicle systems and autonomous vehicles Core algorithms include lane detection, obstacle detection,
to achieve safe, automatic navigation and automatic control target tracking, trajectory management, prediction algorithms,
of the autonomous driving system. The system is embedded and control algorithms.
with the AI algorithm for image recognition, as well as an In addition, the industry has some organizations to evaluate
on-board unit (OBU) and RSU for communication. Driving the active safety of automobiles, such as IIHS&NHTSA in
safety is improved through improved GPS positioning and the United States, Euro-NCAP in Europe, J-NCAP in Japan,
communication security [140]. Using this testbed, DRL-based C-NCAP, and i-VISTA in China. The test content of active
automatic driving control, DNN-based driving behavior pre- safety mainly includes forward collision warning (FCW), lane
diction, CNN-based target detection, and other algorithms departure warning (LDW), ACC, etc.
are proposed and tested. The European Commission proposes In 2017, the US Federal Transportation Administra-
the Coordinated Automated Road Transport (C-ART) project, tion (FTA) provided Pierce Transit with a grant of US
which applies scenarios that coexist with AVs and traditional $1.66 million for research and demonstration projects on
vehicles, and fully affirms the effectiveness of driving safety bus collision avoidance. The scope of the project includes
aids such as FCW with V2I communication [141]. Cooperative the pedestrian avoidance safety system (PASS), which uses
ITS and V2X communication are expected to make road traffic lidar sensors to trigger an automatic deceleration and braking
safer. Nanyang Technological University (NTU Singapore) system. The Alpha test stage realized the collision avoidance
and M1 Limited (M1) are cooperating to build a and automatic braking system based on light detection and
5G C-V2X testbed. C-V2X communication is used to enhance ranging, and completed the verification of the system. The
traffic safety by transmitting real-time traffic navigation and potential benefit of this technology is that the technology cost
hazard information to users [142]. From 2011 to 2015, under is low and old cars can be retrofitted [143].
the funding of the European Union, the University of Twente, In Europe, AUDI AG, Ducati, Ericsson, SWARCO,
SIT, AISEC, and other units launched the PRESERVE project the Technical University of Kaiserslautern, and Qualcomm
to solve key issues such as performance, scalability, and launched Europe’s first demonstration of direct C-V2X
deployability of the V2X security system. The project pro- communication between motorcycles, vehicles, and roadside
posed hardware, software, and security certificate architectures infrastructure. The European C-V2X testbed uses Qualcomm’s
that comply with V2X security standards, such as harmonized V2X chipset solution, SWARCO’s V2V and V2I communica-
V2X security architecture, implementation of V2X security tions, and Ericsson’s 4G/5G cellular test network. The demo
subsystem, as well as cheap and scalable security ASIC demonstrates how the Audi Q7, Audi A4, and motorcycles use
for V2X. C-V2X technology for intersection warnings to improve road
safety [144]. In addition, the H2020 5GCroCo project aims
to provide cross-border cooperative, connected and automated
B. Industrial Testbeds and Projects mobility (CCAM) services. Among them, the Anticipated
Generally speaking, the realization of vehicle CA system in Cooperative Collision Avoidance (ACCA) was deployed on
the industry mainly relies on radar or camera. For example, the Barcelona small-scale trial site, including infrastructure,
Volvo’s City Safety system is a combination of monocular vehicles, and ACCA backends deployed in MEC and public
cameras and millimeter-wave radar sensors, and BMW Driver clouds [145].
Assistance Pro uses ultrasonic radar, millimeter-wave radar, In summary, academic projects and testbeds mainly start
and camera fusion. The Google Driverless Car uses the LiDAR from both communication and autonomous driving, fully
on the roof of the vehicle to emit 64 laser beams. These affirming the role of communication technologies in the CA
vehicles have been tested in California for a long time, but how system. The industry is more concerned with the intelligence
to avoid traffic accidents in the coexistence with traditional of the vehicle itself, using the on-board sensors to sense and
vehicles is still a problem to be solved. Tesla is also equipped make decisions about the surrounding driving environment.
with a lot of equipment, including 8 cameras, 12 ultrasonic However, from the current situation that there are still mis-
sensors, and a radar facing forward. The Autopilot algorithm judgments and accidents in autonomous vehicles. We can infer
is tested on real vehicles, and the total number of vehicles that using communication technologies to connect vehicles
is increasing rapidly, which together form an amazing test infrastructure, pedestrians will be the most promising means
platform. It can be seen that the industry is mainly based on to solve the problem.
the combination of AI and data-driven solutions.
The implementation of the autonomous vehicle in the VIII. C HALLENGES AND R ESEARCH D IRECTIONS
industry is based primarily on AI combined with data-driven Through long-term efforts in various research fields, poten-
solutions. On one hand, a variety of sensors make up the tial traffic accidents have been effectively avoided. However,
automatic driving perception system and large-scale user data the current hybrid traffic scenario, inaccurate sensing data,
(Tesla), which can capture a large amount of real test data communication, and computing resources, as well as the
and driving data. These data provide accurate, stable, and design of CA algorithms pose challenges to the complexity,
reliable input to the autopilot decision control module. On the flexibility, accuracy, and effectiveness of CA systems. In addi-
other hand, these collected data are analyzed using deep tion, the security and privacy issues of vehicles and data will
learning AI techniques, and AI algorithms are trained to also bring traffic safety issues. Therefore, an in-depth research
determine the driving environment and scheme of the vehicle. is needed in these aspects.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 17

A. Hybrid Traffic Scenarios D. CA Algorithm Design

For the foreseeable future, there will be a hybrid traffic On one hand, the complexity of the traffic scenario makes
scenario where CAVs and driver-controlled (or common) vehi- the design of the CA algorithm need to consider more influ-
cles coexist. In CA systems, accurate predictions of driving encing factors. On the other hand, the source, type, and volume
behaviors and driving trajectories are important. Although of sensing data will also increase. These characteristics have
algorithms such as HMM, DBN, and RNN [4], [34], [37], higher requirements for the adaptability, flexibility, scalability,
[127] improve the accuracy and effectiveness of predictions, and complexity of the CA algorithms. With deep learning
there are still challenges in applying this mixed scenario. techniques, data processing and analysis performance can be
Unlike CAVs, the drivers’ behavior is flexible and makes increased, making the CA algorithm more suitable for the
predictions more difficult. For the hybrid traffic scenario, game environment. In addition, the adaptability of a single method in
theory is a viable tool for analyzing the interaction between a variable scenario and the performance of extending to a new
CAVs and driver-controlled vehicles [46]. scenario can be greatly challenged. Therefore, when designing
the CA algorithm, we need to consider the characteristics of
B. Inaccurate Data different AI algorithms, so as to quickly and accurately assess
Inaccurate sensing data will seriously affect the accuracy the driving environment and eliminate potential hazards.
of the CA system and driving safety. On one hand, as time
goes on, the aging of the sensor will cause their performance E. Security and Privacy
degradation. On the other hand, external environmental factors
such as light, rain, and snow can reduce the accuracy of Another major area of concern is security and privacy
sensing. In addition, the quality of the communication link issues. Current vehicle information interaction is frequent and
determines whether the data can be transmitted accurately and complicated and faces threats such as information leakage
timely. Many efforts in this area have made efforts to improve and hacker attacks. As vehicles become more intelligent, the
data accuracy to some extent, such as data fusion, and data attack surface is also increasing. In addition to the sensor and
error correction. For example, authors in [146] propose to vehicular communication security, it also includes operating
fuse GPS and IMU information to improve the robustness system and control system security. How to ensure the security
and accuracy of autonomous vehicle positioning. In [147], of additional components and external information required
authors fuse LiDAR, GPS, and IMU data together, and use for safe driving, as well as the new external interface required
convolutional neural networks to generate safe driving routes for autonomous driving, is full of challenges. Key compo-
for vehicles. Therefore, in addition to the sensor itself, the use nent system reinforcement technology, sensor abnormal data
of AI technology to ensure that the system can make accurate recognition technology, CAN bus authentication encryption
judgments in a noisy data environment is still worthy of further technology, OTA data packet encryption, and digital signature
study. technology, and vehicle IDPS technology can be used to
provide security protection for cars. Communication security
C. Scheduling of Communication and Computing Resources also needs to focus on ensuring the confidentiality, integrity,
In actual scenarios, vehicle users and service types are and availability of data. In addition, it is necessary to take
diversified, and the requirements between each other may advantage of the decentralization of emerging technologies
also cross and conflict. The difference between these services such as blockchain to realize secure data storage and interac-
and users will affect the performance of the network [148]. tion. For example, authors in [149] and [150] use blockchain
Authors in [148] propose a dual-importance evaluation method to safely store and share vehicle data.
to study the relationship between the priority of vehicles and
the priority of contents. Safety-related applications need to IX. C ONCLUSION
respond extremely quickly, or they can have serious con- We surveyed the state-of-the-art researches for state-of-the-
sequences. The increase in vehicles and data poses a huge art data-driven CA techniques. The large volume of sensing
challenge to limited communication and computing resources. data places higher demands on data transmission and process-
The allocation of communication resources in traffic scenarios ing, which makes it challenging to implement effective and
will inevitably affect the allocation of computing resources reliable CA using vehicular sensors, communication, and com-
and vice versa. The current communication architecture and puting technologies. We explored the applications of deployed
computing resource allocation schemes cannot effectively meet or candidate sensing technologies, communication technolo-
the requirements of safe driving, and relying on a single gies, and AI-based collision avoidance algorithms in driving
resource cannot provide reliable, efficient, and low-latency safety. By classifying the reviewed literature, we developed a
information services. The lack of information transfer will more comprehensive foundation for the readers to understand
not complete the calculation and processing work quickly. the three basic phases of CA design and how the current
Conversely, communication resources will not provide safer popular AI algorithms are applied in different aspects of the
and better quality communication services if they do not CA system. Finally, we discussed current challenges and future
coordinate computing resources. Future research can achieve research areas for researchers to refer to and carry out further
an efficient and rational allocation of two types of resources studies.
according to the delay sensitivity of the business by integrating The current challenges in improving traffic safety are multi-
the characteristics of different services and different users. faceted, so there is still much room for further research in this

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

18 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE VIII
L IST OF A CRONYMS

area. The intelligence and networking of vehicles contribute to communication performance improvement, and CA algorithm
the improvement of driving safety from different perspectives. design. After investigating related algorithms and progress,
In the future, on one hand, in the face of increasing sensor we summarized the corresponding challenges and unsolved
data, data fusion technology is worthy of attention. The problems and tried to propose preliminary solutions, which
future increase in vehicle sensing data and user service data are convenient for researchers to refer to and conduct further
requires an AI-based solution to process data and learn user research.
requirements in a relatively short period of time. On the other Academically, although many current algorithms can
hand, the application of communication technology realizes accurately identify dangerous situations and take timely and
data sharing, which improves the efficiency and accuracy accurate warning measures in specific scenarios, it is still
of the CA system. However, although research work and necessary to carefully evaluate the scalability and reliability
standard setting have been carried out, more practical testing of these algorithms. The hybrid traffic scenarios where the
is still needed. In addition, AI-based algorithms are a focus of CAVs and the traditional driver-controlled vehicles coexist
this survey, including applications in perception data fusion, will last for a while, which will have huge impacts on the

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 19

design of CA systems. Driver factors are still not negligible [21] J. Kim and D. Kum, “Collision risk assessment algorithm via
in algorithm design, which brings a lot of uncertainty. The lane-based probabilistic motion prediction of surrounding vehicles,”
IEEE Trans. Intell. Transp. Syst., vol. 19, no. 9, pp. 2965–2976,
AI-based algorithms need to be further explored to adapt to Sep. 2017.
current traffic conditions and enhance the ability to handle [22] G. Xie, H. Gao, L. Qian, B. Huang, K. Li, and J. Wang, “Vehicle trajec-
uncertainty. tory prediction by integrating physics- and maneuver-based approaches
using interactive multiple models,” IEEE Trans. Ind. Electron., vol. 65,
no. 7, pp. 5999–6008, Jul. 2018.
A PPENDIX
[23] K. Berntorp, “Joint wheel-slip and vehicle-motion estimation based on
We summarize the definitions of the acronyms that will inertial, GPS, and wheel-speed sensors,” IEEE Trans. Control Syst.
Technol., vol. 24, no. 3, pp. 1020–1027, May 2016.
be frequently used in this paper in Table VIII for ease of
[24] J. Ji, A. Khajepour, W. W. Melek, and Y. Huang, “Path planning and
reference. tracking for vehicle collision avoidance based on model predictive
control with multiconstraints,” IEEE Trans. Veh. Technol., vol. 66,
R EFERENCES no. 2, pp. 952–964, Feb. 2017.
[25] J. Hillenbrand, A. M. Spieker, and K. Kroschel, “A multilevel collision
[1] International Traffic Safety Data and Analysis Group. (2018). mitigation approach-its situation assessment, decision making, and
IRTAD Road Safety Database. [Online]. Available: https://www.itf- performance tradeoffs,” IEEE Trans. Intell. Transp. Syst., vol. 7, no. 4,
oecd.org/IRTAD pp. 528–540, Dec. 2006.
[2] National Highway Traffic Safety Administration. (Sep. 2018).
[26] C. Chen, L. Liu, T. Qiu, Z. Ren, J. Hu, and F. Ti, “Driver’s intention
2016 Summary of Motor Vehicle Crashes. [Online]. Available:
identification and risk evaluation at intersections in the Internet of
https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812580
vehicles,” IEEE Internet Things J., vol. 5, no. 3, pp. 1575–1587,
[3] Canadian Council of Motor Transport Administrators. (Apr. 2018).
Jun. 2018.
Canadian Motor Vehicle Traffic Collision Statistics: 2016. [Online].
Available: https://www.tc.gc.ca/eng/motorvehiclesafety/canadian- [27] C. Barrios, Y. Motai, and D. Huston, “Trajectory estimations
motor-vehicle-traffic-collision-statistics-2016.html using smartphones,” IEEE Trans. Ind. Electron., vol. 62, no. 12,
[4] Y. Fu, C. Li, T. H. Luan, Y. Zhang, and G. Mao, “Infrastructure- pp. 7901–7910, Dec. 2015.
cooperative algorithm for effective intersection collision avoidance,” [28] M. Schreier, V. Willert, and J. Adamy, “An integrated approach
Transp. Res. C, Emerg. Technol., vol. 89, pp. 188–204, Apr. 2018. to maneuver-based trajectory prediction and criticality assessment in
[5] X. Xiang, W. Qin, and B. Xiang, “Research on a DSRC-based rear- arbitrary road environments,” IEEE Trans. Intell. Transp. Syst., vol. 17,
end collision warning model,” IEEE Trans. Intell. Transp. Syst., vol. 15, no. 10, pp. 2751–2766, Oct. 2016.
no. 3, pp. 1054–1065, Jun. 2014. [29] M. Schreier, “Bayesian environment representation, prediction, and
[6] G. Guo and W. Yue, “Sampled-data cooperative adaptive cruise control criticality assessment for driver assistance systems,” Ph.D. dissertation,
of vehicles with sensor failures,” IEEE Trans. Intell. Transp. Syst., Dept. Elect. Eng. Inf. Technol., Tech. Univ. Darmstadt, Darmstadt,
vol. 15, no. 6, pp. 2404–2418, Dec. 2014. Germany, 2016.
[7] How it Works, Connected Vehicle/Automated Vehicle (CVAV) Program, [30] S. Al-Sultan, A. H. Al-Bayatti, and H. Zedan, “Context-aware driver
Ontario Centres Excellence, Toronto, ON, Canada, 2016. behavior detection system in intelligent transportation systems,” IEEE
[8] What Are Connected Vehicles and Why Do We Need Them? Intell. Trans. Veh. Technol., vol. 62, no. 9, pp. 4264–4275, Nov. 2013.
Transp. Syst. Joint Program Office ITS Joint Program Office, U.S. [31] G. Agamennoni, J. I. Nieto, and E. M. Nebot, “Estimation of multi-
DOT, Washington, DC, USA, 2019. vehicle dynamics by considering contextual information,” IEEE Trans.
[9] H. Peng, L. Liang, X. Shen, and G. Y. Li, “Vehicular communications: Robot., vol. 28, no. 4, pp. 855–870, Aug. 2012.
A network layer perspective,” Jul. 2017, arXiv:1707.09972. [Online]. [32] T. Gindele, S. Brechtel, and R. Dillmann, “Learning driver behavior
Available: http://arxiv.org/abs/1707.09972 models from traffic observations for decision making and planning,”
[10] S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, IEEE Intell. Transp. Syst. Mag., vol. 7, no. 1, pp. 69–79, Jan. 2015.
3rd ed. London, U.K.: Pearson, 2016. [33] G. S. Aoude, V. R. Desaraju, L. H. Stephens, and J. P. How, “Driver
[11] J. Guerrero-Ibáñez, S. Zeadally, and J. Contreras-Castillo, “Sensor behavior classification at intersections and validation on large natu-
technologies for intelligent transportation systems,” Sensors, vol. 18, ralistic data set,” IEEE Trans. Intell. Transp. Syst., vol. 13, no. 2,
no. 4, p. 1212, Apr. 2018. pp. 724–736, Jun. 2012.
[12] I. Anagnostopoulos, S. Zeadally, and E. Exposito, “Handling big data: [34] J. Morton, T. A. Wheeler, and M. J. Kochenderfer, “Analysis of
Research challenges and future directions,” J. Supercomput., vol. 72, recurrent neural networks for probabilistic modeling of driver behav-
no. 4, pp. 1494–1516, Apr. 2016. ior,” IEEE Trans. Intell. Transp. Syst., vol. 18, no. 5, pp. 298–1289,
[13] S. Lefèvre, D. Vasquez, and C. Laugier, “A survey on motion prediction Sep. 2017.
and risk assessment for intelligent vehicles,” ROBOMECH J., vol. 1, [35] D. J. Phillips, T. A. Wheeler, and M. J. Kochenderfer, “Generalizable
no. 1, pp. 1–14, Dec. 2014. intention prediction of human drivers at intersections,” in Proc. IEEE
[14] C. Katrakazas, M. Quddus, W.-H. Chen, and L. Deka, “Real-time Intell. Vehicles Symp. (IV), Jun. 2017, pp. 1665–1670.
motion planning methods for autonomous on-road driving: State-of-
[36] M. Bahram, C. Hubmann, A. Lawitzky, M. Aeberhard, and D.
the-art and future research directions,” Transp. Res. C, Emerg. Technol.,
Wollherr, “A combined model- and learning-based framework for
vol. 60, pp. 416–442, Nov. 2015.
interaction-aware maneuver prediction,” IEEE Trans. Intell. Transp.
[15] A. Mukhtar, L. Xia, and T. B. Tang, “Vehicle detection techniques for
Syst., vol. 17, no. 6, pp. 1538–1550, Jun. 2016.
collision avoidance systems: A review,” IEEE Trans. Intell. Transp.
Syst., vol. 16, no. 5, pp. 2318–2338, Oct. 2015. [37] S. Lefevre, Y. Gao, D. Vasquez, H. E. Tseng, R. Bajcsy, and F. Borrelli,
[16] C. Bila, F. Sivrikaya, M. A. Khan, and S. Albayrak, “Vehi- “Lane keeping assistance with learning-based driver model and model
cles of the future: A survey of research on safety issues,” predictive control,” in Proc. 12th Int. Symp. Adv. Vehicle Control,
IEEE Trans. Intell. Transp. Syst., vol. 18, no. 5, pp. 1046–1065, Tykyo, Japan, Sep. 2014, pp. 22–26.
May 2017. [38] M. G. Ortiz, J. Fritsch, F. Kummert, and A. Gepperth, “Behavior
[17] R. Hussain and S. Zeadally, “Autonomous cars: Research results, prediction at multiple time-scales in inner-city scenarios,” in Proc.
issues, and future challenges,” IEEE Commun. Surveys Tuts., vol. 21, IEEE Intell. Vehicles Symp. (IV), Jun. 2011, pp. 1068–1073.
no. 2, pp. 1275–1313, 2nd Quart., 2019. [39] B. Morris, A. Doshi, and M. Trivedi, “Lane change intent prediction
[18] J. Wang, J. Liu, and N. Kato, “Networking and communications in for driver assistance: On-road design and evaluation,” in Proc. IEEE
autonomous driving: A survey,” IEEE Commun. Surveys Tuts., vol. 21, Intell. Vehicles Symp. (IV), Jun. 2011, pp. 895–901.
no. 2, pp. 1243–1274, 2nd Quart., 2019. [40] S. Joerer, M. Segata, B. Bloessl, R. Lo Cigno, C. Sommer, and
[19] SAE International. (2018). (R) Taxonomy and Definitions for F. Dressler, “A vehicular networking perspective on estimating vehi-
Terms Related to Driving Automation Systems for On-Road cle collision probability at intersections,” IEEE Trans. Veh. Technol.,
Motor Vehicles (SAE J3016–201806). [Online]. Available: vol. 63, no. 4, pp. 1802–1812, May 2014.
https://www.sae.org/standards/content/j3016_201806/ [41] R. Labayrade, C. Royere, and D. Aubert, “A collision mitigation system
[20] S. Thrun, “Toward robotic cars,” Commun. ACM, vol. 53, no. 4, using laser scanner and stereovision fusion and its assessment,” in Proc.
pp. 99–106, Apr. 2010. IEEE Intell. Vehicles Symp., Jun. 2005, pp. 441–446.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

20 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

[42] A. Eskandarian, Ed., Handbook of Intelligent Vehicles. London, [65] Z. Zhao, L. Zhou, Q. Zhu, Y. Luo, and K. Li, “A review of essential
U.K.: Springer-Verlag, 2012. technologies for collision avoidance assistance systems,” Adv. Mech.
[43] A. Tamke, T. Dang, and G. Breuel, “A flexible method for criticality Eng., vol. 9, no. 10, pp. 1–15, Oct. 2017.
assessment in driver assistance systems,” in Proc. IEEE Intell. Vehicles [66] Z. Wang, Y. Wu, and Q. Niu, “Multi-sensor fusion in automated
Symp. (IV), Jun. 2011, pp. 697–702. driving: A survey,” IEEE Access, vol. 8, pp. 2847–2868, 2020.
[44] L. Zhu, F. R. Yu, Y. Wang, B. Ning, and T. Tang, “Big data analytics [67] E. Marti, M. A. de Miguel, F. Garcia, and J. Perez, “A review of sensor
in intelligent transportation systems: A survey,” IEEE Trans. Intell. technologies for perception in automated driving,” IEEE Intell. Transp.
Transp. Syst., vol. 20, no. 1, pp. 383–398, Jan. 2018. Syst. Mag., vol. 11, no. 4, pp. 94–108, Sep. 2019.
[45] J. Zhang, F.-Y. Wang, K. Wang, W.-H. Lin, X. Xu, and C. Chen, “Data- [68] D. Lahat, T. Adali, and C. Jutten, “Multimodal data fusion:
driven intelligent transportation systems: A survey,” IEEE Trans. Intell. An overview of methods, challenges, and prospects,” Proc. IEEE,
Transp. Syst., vol. 12, no. 4, pp. 1624–1639, Dec. 2011. vol. 103, no. 9, pp. 1449–1477, Sep. 2015.
[46] A. Ferdowsi, U. Challita, and W. Saad, “Deep learning for reli- [69] F. Alam, R. Mehmood, I. Katib, N. N. Albogami, and A. Albeshri,
able mobile edge analytics in intelligent transportation systems: An “Data fusion and IoT for smart ubiquitous environments: A survey,”
overview,” IEEE Veh. Technol. Mag., vol. 14, no. 1, pp. 62–70, IEEE Access, vol. 5, pp. 9533–9554, 2017.
Mar. 2019. [70] Z. Yu, J. Bai, S. Chen, L. Huang, and X. Bi, “Camera-radar data fusion
[47] N.-E.-E. Faouzi, H. Leung, and A. Kurian, “Data fusion in intelli- for target detection via Kalman filter and Bayesian estimation,” SAE
gent transportation systems: Progress and challenges—A survey,” Inf. Tech. Paper 2018-01-1608, 2018.
Fusion, vol. 12, no. 1, pp. 4–10, Jan. 2011.
[71] T. A. Biresaw, A. Cavallaro, and C. S. Regazzoni, “Tracker-level fusion
[48] K. Zheng, Q. Zheng, P. Chatzimisios, W. Xiang, and Y. Zhou,
for robust Bayesian visual tracking,” IEEE Trans. Circuits Syst. Video
“Heterogeneous vehicular networking: A survey on architecture, chal-
Technol., vol. 25, no. 5, pp. 776–789, May 2015.
lenges, and solutions,” IEEE Commun. Surveys Tuts., vol. 17, no. 4,
pp. 2377–2396, Jun. 2015. [72] M. Liggins, II, D. Hall, and J. Llinas, Handbook of Multisensor Data
[49] K. Lee, J. Kim, Y. Park, H. Wang, and D. Hong, “Latency of Fusion: Theoryand Practice, 2nd ed. Boca Raton, FL, USA: CRC
cellular-based V2X: Perspectives on TTI-proportional latency and Press, 2017.
TTI-independent latency,” IEEE Access, vol. 5, pp. 15800–15809, [73] K. Geng and N. A. Chulin, “Applications of multi-height sensors data
2017. fusion and fault-tolerant Kalman filter in integrated navigation system
[50] R. Molina-Masegosa and J. Gozalvez, “LTE-V for sidelink 5G V2X of UAV,” Procedia Comput. Sci., vol. 103, pp. 231–238, Jan. 2017.
vehicular communications: A new 5G technology for short-range [74] B. Bigdeli, F. Samadzadegan, and P. Reinartz, “A decision fusion
vehicle-to-everything communications,” IEEE Veh. Technol. Mag., method based on multiple support vector machine system for fusion
vol. 12, no. 4, pp. 30–39, Dec. 2017. of hyperspectral and LIDAR data,” Int. J. Image Data Fusion, vol. 5,
[51] A. Ali, H. Cao, J. Eichinger, S. Gangakhedkar, and M. Gharba, no. 3, pp. 196–209, Jul. 2014.
“A testbed for experimenting 5G-V2X requiring ultra reliability and [75] T. Zebin, P. J. Scully, and K. B. Ozanyan, “Inertial sensor based mod-
low-latency,” in Proc. IEEE Int. ITG Workshop Smart Antennas (WSA), elling of human activity classes: Feature extraction and multi-sensor
Berlin, Germany, Mar. 2017, pp. 1–4. data fusion using machine learning algorithms,” in eHealth 360◦ ,
[52] L. Hobert, A. Festag, I. Llatser, L. Altomare, F. Visintainer, and vol. 181. Cham, Switzerland: Springer, 2017, pp. 306–314.
A. Kovacs, “Enhancements of V2X communication in support of [76] A. Asvadi, L. Garrote, C. Premebida, P. Peixoto, and U. J. Nunes,
cooperative autonomous driving,” IEEE Commun. Mag., vol. 53, no. 12, “Multimodal vehicle detection: Fusing 3D-LIDAR and color camera
pp. 64–70, Dec. 2015. data,” Pattern Recognit. Lett., vol. 115, pp. 20–29, Nov. 2018.
[53] A. Tang and A. Yip, “Collision avoidance timing analysis of [77] N. Srivastava and R. Salakhutdinov, “Learning representations for
DSRC-based vehicles,” Accident Anal. Prevention, vol. 42, no. 1, multimodal data with deep belief nets,” in Proc. Int. Conf. Mach. Learn.
pp. 182–195, Jan. 2010. Workshop, 2012, p. 3.
[54] W. Wang, J. Xi, and H. Chen, “Modeling and recognizing driver [78] N. Srivastava and R. R. Salakhutdinov, “Multimodal learning with deep
behavior based on driving data: A survey,” Math. Problems Eng., Boltzmann machines,” J. Mach. Learn. Res., vol. 15, pp. 2949–2980,
vol. 2014, pp. 1–20, Feb. 2014. Oct. 2014.
[55] Y. Fu, C. Li, F. R. Yu, T. H. Luan, and Y. Zhang, “A decision-making [79] R. Salakhutdinov and G. Hinton, “An efficient learning procedure for
strategy for vehicle autonomous braking in emergency via deep deep Boltzmann machines,” Neural Comput., vol. 24, no. 8, p. 1967,
reinforcement learning,” IEEE Trans. Veh. Technol., vol. 69, no. 6, 2012.
pp. 5876–5888, Jun. 2020. [80] Z. MacHardy, A. Khan, K. Obana, and S. Iwashina, “V2X access tech-
[56] A. Ghasemi, R. Kazemi, and S. Azadi, “Directional control of a platoon nologies: Regulation, research, and remaining challenges,” IEEE Com-
of vehicles for comfort specification by considering parasitic time mun. Surveys Tuts., vol. 20, no. 3, pp. 1858–1877, 3rd Quart., 2018.
delays and lags,” PROMET Traffic Transp., vol. 25, no. 5, pp. 20–412, [81] J. B. Kenney, “Dedicated short-range communications (DSRC) stan-
2013. dards in the United States,” Proc. IEEE, vol. 99, no. 7, pp. 1162–1182,
[57] “Vehicle safety communications project task 3 final report,” CAMP Jul. 2011.
Vehicle Saf. Commun. Consortium Consisting BMW, DaimlerChrysler, [82] X. Zhao, S. Jing, F. Hui, R. Liu, and A. J. Khattak, “DSRC-based
Ford, GM, Nissan, Toyota, and VW, Washington, DC, USA, Tech. Rep. rear-end collision warning system—An error-component safety dis-
DOT HS 809859, Mar. 2005. tance model and field test,” Transp. Res. C, Emerg. Technol., vol. 107,
[58] I. Satu and K. Salla, “Key performance indicators for assessing the
pp. 92–104, Oct. 2019.
impacts of automation in road transportation-results of the trilateral
[83] S. B. Cruz, T. E. Abrudan, Z. Xiao, N. Trigoni, and J. Barros,
key performance indicator survey,” VTT Tech. Res. Centre Finland,
“Neighbor-aided localization in vehicular networks,” IEEE Trans.
Espoo, Finland, Res. Rep. VTT-R-01054-18, 2018, pp. 1–37.
Intell. Transp. Syst., vol. 18, no. 10, pp. 2693–2702, Oct. 2017.
[59] A. Eidehall, J. Pohl, F. Gustafsson, and J. Ekmark, “Toward
autonomous collision avoidance by steering,” IEEE Trans. Intell. [84] Y. Wang, X. Duan, D. Tian, M. Chen, and X. Zhang, “A DSRC-based
Transp. Syst., vol. 8, no. 1, pp. 84–94, Mar. 2007. vehicular positioning enhancement using a distributed multiple-model
[60] M. Malinverno, G. Avino, F. Malandrino, C. Casetti, C.-F. Chiasserini, Kalman filter,” IEEE Access, vol. 4, pp. 8338–8350, Nov. 2016.
and S. Scarpina, “Performance analysis of C-V2I-based automotive [85] R. Molina-Masegosa and J. Gozalvez, “LTE-V for sidelink 5G V2X
collision avoidance,” 2018, arXiv:1803.08798. [Online]. Available: vehicular communications: A new 5G technology for short-range
http://arxiv.org/abs/1803.08798 vehicle-to-everything communications,” IEEE Veh. Technol. Mag.,
[61] Z. Chang-Fu, Y. Sheng-Nan, and Z. Hong-Yu, “A control strategy of vol. 12, no. 4, pp. 30–39, Dec. 2017.
electronic braking system based on brake comfort,” in Proc. Int. Conf. [86] J. Li, Y. Zhang, M. Shi, Q. Liu, and Y. Chen, “Collision avoidance
Transp., Mech., Electr. Eng. (TMEE), Dec. 2011, pp. 1265–1268. strategy supported by LTE-V-based vehicle automation and commu-
[62] Z. Liu, L. Wang, K. Li, and J. Sui, “An improved rotation scheme nication systems for car following,” Tsinghua Sci. Technol., vol. 25,
for dual-axis rotational inertial navigation system,” IEEE Sensors J., no. 1, pp. 127–139, Feb. 2020.
vol. 17, no. 13, pp. 4189–4196, Jul. 2017. [87] 5G; Service Requirements for Enhanced V2X Scenarios, document TS
[63] P. D. Groves, Principles of GNSS, Inertial, and Multi-Sensor Integrated 22.186 V15.4.0, (3GPP TS 22.186 version 15.4.0 Release 15), 3GPP,
Navigation Systems. Fitch-Burg, MA, USA: Artech House, 2013. Oct. 2018.
[64] J. Engel, V. Koltun, and D. Cremers, “Direct sparse odometry,” [88] IMT Vision Framework and Overall Objectives of the Future Develop-
IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 3, pp. 611–625, ment of IMT for 2020 and Beyond, document Rec. ITU-R M.2083-0,
Mar. 2018. ITU-R, 2015.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

FU et al.: SURVEY OF DRIVING SAFETY WITH SENSING, VEHICULAR COMMUNICATIONS, AND AI-BASED CA 21

[89] H. Guo, J. Liu, and J. Zhang, “Computation offloading for multi-access [110] L. Liang, H. Ye, and G. Y. Li, “Toward intelligent vehicular networks:
mobile edge computing in ultra-dense networks,” IEEE Commun. Mag., A machine learning framework,” IEEE Internet Things J., vol. 6, no. 1,
vol. 56, no. 8, pp. 14–19, Aug. 2018. pp. 124–135, Feb. 2019, doi: 10.1109/JIOT.2018.2872122.
[90] L. Gao, Y. Li, J. Misener, and S. Patil, “C-V2X based basic safety [111] W. Tong, A. Hussain, W. X. Bo, and S. Maharjan, “Artificial intel-
related ITS spectrum requirement analysis,” in Proc. IEEE 86th Veh. ligence for vehicle-to-everything: A survey,” IEEE Access, vol. 7,
Technol. Conf. (VTC-Fall), Sep. 2017, pp. 1–5. pp. 10823–10843, 2019.
[91] 5GAA. (2020). V2X Functional and Performance Test Report; Test [112] C.-K. Wen, S. Jin, K.-K. Wong, J.-C. Chen, and P. Ting, “Channel
Procedures and Results. [Online]. Available: https://5gaa.org/wp- estimation for massive MIMO using Gaussian-mixture Bayesian learn-
content/uploads/2018/11/5GAA_P-190033_V2X-Functional-and- ing,” IEEE Trans. Wireless Commun., vol. 14, no. 3, pp. 1356–1368,
Performance-Test–Report_final-1.pdf Mar. 2015.
[92] S. Rehman, S. Ullah, P. Chong, S. Yongchareon, and D. Komosny, [113] H. Ye, G. Y. Li, and B.-H. Juang, “Power of deep learning for channel
“Visible light communication: A system perspective, overview and estimation and signal detection in OFDM systems,” IEEE Wireless
challenges,” Sensors, vol. 19, no. 5, pp. 1–22, Mar. 2019. Commun. Lett., vol. 7, no. 1, pp. 114–117, Feb. 2018.
[93] A. T. Hussein, M. T. Alresheedi, and J. M. H. Elmirghani, “20 Gb/s [114] D. Zhang, F. R. Yu, and R. Yang, “A deep reinforcement learning-based
mobile indoor visible light communication system employing beam trust management scheme for software-defined vehicular networks,” in
steering and computer generated holograms,” J. Lightw. Technol., Proc. ACM DIVANet, Montreal, QC, Canada, Oct. 2018, pp. 1–7.
[115] H. Bagherlou and A. Ghaffari, “A routing protocol for vehicular ad hoc
vol. 33, no. 24, pp. 5242–5260, Dec. 15, 2015.
networks using simulated annealing algorithm and neural networks,”
[94] A.-M. Cailean and M. Dimian, “Current challenges for visible light
J. Supercomput., vol. 74, no. 6, pp. 2528–2552, Jun. 2018.
communications usage in vehicle applications: A survey,” IEEE [116] Y. He, N. Zhao, and H. Yin, “Integrated networking, caching, and
Commun. Surveys Tuts., vol. 19, no. 4, pp. 2681–2703, 4th Quart., computing for connected vehicles: A deep reinforcement learning
2017. approach,” IEEE Trans. Veh. Technol., vol. 67, no. 1, pp. 44–55,
[95] N. Wang, C. Liu, Y. Lu, and J. Shen, “A visible light communi-
Jan. 2018.
cation (VLC) based intelligent transportation system for lorry fleet,” [117] R. F. Atallah, C. M. Assi, and J. Y. Yu, “A reinforcement learning tech-
in Proc. 16th Int. Conf. Opt. Commun. Netw. (ICOCN), Aug. 2017, nique for optimizing downlink scheduling in an energy-limited vehicu-
pp. 1–3. lar network,” IEEE Trans. Veh. Technol., vol. 66, no. 6, pp. 4592–4601,
[96] S. Ucar, S. C. Ergen, and O. Ozkasap, “IEEE 802.11p and visible light Jun. 2017.
hybrid communication based secure autonomous platoon,” IEEE Trans. [118] H. Ye, G. Y. Li, and B.-H.-F. Juang, “Deep reinforcement learning
Veh. Technol., vol. 67, no. 9, pp. 8667–8681, Sep. 2018. based resource allocation for V2V communications,” IEEE Trans.
[97] M. Uysal, Z. Ghassemlooy, A. Bekkali, A. Kadri, and H. Menouar, Veh. Technol., vol. 68, no. 4, pp. 3163–3173, Apr. 2019, doi:
“Visible light communication for vehicular networking: Performance 10.1109/TVT.2019.2897134.
study of a V2V system using a measured headlamp beam pat- [119] K. M. A. Alheeti, A. Gruebler, and K. D. McDonald-Maier, “An intru-
tern model,” IEEE Veh. Technol. Mag., vol. 10, no. 4, pp. 45–53, sion detection system against malicious attacks on the communication
Dec. 2015. network of driverless cars,” in Proc. 12th Annu. IEEE Consum.
[98] D. D. Priyanka and T. S. Kumar, “ARM and zigbee based intelligent Commun. Netw. Conf. (CCNC), Jan. 2015, pp. 916–921.
vehicle communication for collision avoidance,” in Proc. Int. Conf. [120] K. M. A. Alheeti, A. Gruebler, K. D. McDonald-Maier, and
Adv. Commun. Control Comput. Technol. (ICACCCT), May 2016, A. Fernando, “Prediction of DoS attacks in external communication
pp. 735–739. for self-driving vehicles using a fuzzy Petri net model,” in Proc. IEEE
[99] K. Dhondge, S. Song, B.-Y. Choi, and H. Park, “WiFiHonk: Int. Conf. Consum. Electron. (ICCE), Jan. 2016, pp. 502–503.
Smartphone-based beacon stuffed WiFi Car2X-communication system [121] O. Berlin, A. Held, M. Matousek, and F. Kargl, “POSTER:
for vulnerable road user safety,” in Proc. IEEE 79th Veh. Technol. Conf. Anomaly-based misbehaviour detection in connected car backends,”
(VTC Spring), May 2014, pp. 1–5. in Proc. IEEE Veh. Netw. Conf. (VNC), Dec. 2016, pp. 1–2.
[100] P.-F. Ho and J.-C. Chen, “WiSafe: Wi-Fi pedestrian collision avoidance [122] N. Lyamin, D. Kleyko, Q. Delooz, and A. Vinel, “AI-based malicious
system,” IEEE Trans. Veh. Technol., vol. 66, no. 6, pp. 4564–4578, network traffic detection in VANETs,” IEEE Netw., vol. 32, no. 6,
Jun. 2017. pp. 15–21, Nov. 2018.
[101] M. Rath, “Smart traffic management system for traffic control using [123] N. Lyamin, D. Kleyko, Q. Delooz, and A. Vinel, “Real-time jamming
automated mechanical and electronic devices,” in Proc. Int. Conf. DoS detection in safety-critical V2V C-ITS using data mining,” IEEE
Mater. Sci. Eng., 2018, vol. 377, no. 1, Art. no. 012201. Commun. Lett., vol. 23, no. 3, pp. 442–445, Mar. 2019.
[102] X.-G. Guo, J.-L. Wang, F. Liao, and R. S. H. Teo, “CNN-based distrib- [124] E. Hamida, H. Noura, and W. Znaidi, “Security of cooperative intel-
uted adaptive control for vehicle-following platoon with input satura- ligent transport systems: Standards, threats analysis and cryptographic
tion,” IEEE Trans. Intell. Transp. Syst., vol. 19, no. 10, pp. 3121–3132, countermeasures,” Electronics, vol. 4, no. 3, pp. 380–423, Jul. 2015.
Oct. 2018. [125] C. Laugier et al., “Probabilistic analysis of dynamic scenes and
[103] S. Santini, A. Salvi, A. S. Valente, A. Pescape, M. Segata, and collision risks assessment to improve driving safety,” IEEE Intell.
R. L. Cigno, “A consensus-based approach for platooning with inter- Transp. Syst. Mag., vol. 3, no. 4, pp. 4–19, Oct. 2011.
[126] V. Gadepally, A. Krishnamurthy, and U. Ozguner, “A framework for
vehicular communications and its validation in realistic scenar-
estimating driver decisions near intersections,” IEEE Trans. Intell.
ios,” IEEE Trans. Veh. Technol., vol. 66, no. 3, pp. 1985–1999,
Transp. Syst., vol. 15, no. 2, pp. 637–646, Apr. 2014.
Mar. 2017. [127] A. Zyner, S. Worrall, and E. Nebot, “A recurrent neural network
[104] C. Desjardins and B. Chaib-Draa, “Cooperative adaptive cruise control: solution for predicting driver intention at unsignalized intersections,”
A reinforcement learning approach,” IEEE Trans. Intell. Transp. Syst., IEEE Robot. Autom. Lett., vol. 3, no. 3, pp. 1759–1764, Jul. 2018.
vol. 12, no. 4, pp. 1248–1260, Dec. 2011. [128] L. Lin, S. Gong, and T. Li, “Deep learning-based human-driven vehicle
[105] S. Wei, Y. Zou, T. Zhang, X. Zhang, and W. Wang, “Design and trajectory prediction and its application for platoon control of connected
experimental validation of a cooperative adaptive cruise control system and autonomous vehicles,” in Proc. Auton. Vehicles Symp., 2018,
based on supervised reinforcement learning,” Appl. Sci., vol. 1014, pp. 1–30.
no. 8, pp. 1–21, 2018. [129] S. Bonnin, T. H. Weisswange, F. Kummert, and J. Schmuedderich,
[106] M. R. Hafner, D. Cunningham, L. Caminiti, and D. Del Vecchio, “General behavior prediction by a combination of scenario-specific
“Cooperative collision avoidance at intersections: Algorithms and models,” IEEE Trans. Intell. Transp. Syst., vol. 15, no. 4,
experiments,” IEEE Trans. Intell. Transp. Syst., vol. 14, no. 3, pp. 1478–1488, Aug. 2014.
pp. 1162–1175, Sep. 2013. [130] S. P. Douglass, S. Martin, A. Jennings, H. Chen, and D. M. Bevly,
[107] H. Xia, C. Zeng, and X. Meng, “Multi vehicles cooperative collision “Deep learned multi-modal traffic agent predictions for truck platoon-
avoidance model in VANETs,” in Proc. CICTP, Intell., Connectivity, ing cut-ins,” in Proc. IEEE/ION Position, Location Navigat. Symp.
Mobility, 2018, pp. 2788–2795. (PLANS), Portland, OR, USA, Apr. 2020, pp. 688–697.
[108] S. Gelbal, S. Zhu, G. Anantharaman, B. A. Guvenc, and L. Guvenc, [131] H. Chae, C. M. Kang, B. Kim, J. Kim, C. C. Chung, and J. W. Choi,
“Cooperative collision avoidance in a connected vehicle environment,” “Autonomous braking system via deep reinforcement learning,” 2017,
SAE Tech. Paper 2019-01-0488, 2019. arXiv:1702.02302. [Online]. Available: http://arxiv.org/abs/1702.02302
[109] H. Ahn, A. Rizzi, A. Colombo, and D. Del Vecchio, “Experimental [132] S. Elmalaki, H.-R. Tsai, and M. Srivastava, “Sentio: Driver-in-the-Loop
testing of a semi-autonomous multi-vehicle collision avoidance algo- forward collision warning using multisample reinforcement learning,”
rithm at an intersection testbed,” in Proc. IEEE/RSJ Int. Conf. Intell. in Proc. 16th ACM Conf. Embedded Netw. Sensor Syst., Nov. 2018,
Robots Syst. (IROS), Sep. 2015, pp. 4834–4839. pp. 28–40.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

22 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

[133] G. Katz, C. Barrett, D. Dill, K. Julian, and M. Kochenderfer, “Reluplex: Changle Li (Senior Member, IEEE) received the
An efficient SMT solver for verifying deep neural networks,” 2017, Ph.D. degree in communication and information
arXiv:1702.01135. [Online]. Available: http://arxiv.org/abs/1702.01135 system from Xidian University, China, in 2005.
[134] P. Koh and P. Liang, “Understanding black-box predictions via influ- He conducted his post-doctoral research in Canada
ence functions,” in Proc. 34th Int. Conf. Mach. Learn. (ICML), 2017, and the National Institute of information and Com-
pp. 1–10. munications Technology, Japan, respectively. He had
[135] A. Wan et al., “NBDT: Neural-backed decision trees,” 2020, been a Visiting Scholar with the University of Tech-
arXiv:2004.00221. [Online]. Available: http://arxiv.org/abs/2004.00221 nology Sydney. He is currently a Professor with
[136] T. Gehr, M. Mirman, D. Drachsler-Cohen, P. Tsankov, S. Chaudhuri, the State Key Laboratory of Integrated Services
and M. Vechev, “AI2: Safety and robustness certification of neural Networks, Xidian University. His research interests
networks with abstract interpretation,” in Proc. IEEE Symp. Secur. include intelligent transportation systems, vehicular
Privacy (SP), San Francisco, CA, USA, May 2018, pp. 3–18. networks, mobile ad hoc networks, and wireless sensor networks.
[137] Y. Gal and Z. Ghahramani, “Dropout as a Bayesian approximation:
Representing model uncertainty in deep learning,” in Proc. 33rd Int.
Conf. Mach. Learn. (ICML), M. F. Balcan and K. Q. Weinberger, Eds., Fei Richard Yu (Fellow, IEEE) received the Ph.D.
2016, pp. 1050–1059. degree in electrical engineering from The University
[138] R. McAllister et al., “Concrete problems for autonomous vehicle safety: of British Columbia (UBC) in 2003.
Advantages of Bayesian deep learning,” in Proc. 26th Int. Joint Conf. From 2002 to 2006, he was with Ericsson, Lund,
Artif. Intell. (IJCAI), Aug. 2017, pp. 1–9. Sweden, and a start-up in California, USA. He joined
[139] Caltrans and California PATH Program at UC Berkeley. Con- Carleton University in 2007, where he is cur-
nected Vehicle Test Bed. Accessed: May 2021. [Online]. Available: rently a Professor. His research interests include
http://www.caconnectedvehicletestbed.org/index.php/ connected/autonomous vehicles, security, artificial
[140] CBC News. (Feb. 15, 2019). BlackBerry, Federal Government intelligence, blockchain, and wireless cyber-physical
Pour $350M Into Autonomous Cars. [Online]. Available: systems. He is a fellow of IET and the Engineer-
https://www.cbc.ca/news/canada/ottawa/blackberry-funding- ing Institute of Canada (EIC). He is an Elected
government-autonomous-vehicles-1.5020137 Member of the Board of Governors of the IEEE VTS. He received the
[141] M. A. Raposo, B. Ciuffo, M. Makridis, and C. Thiel, “The r-evolutionof IEEE TCGCC Best Journal Paper Award in 2019, the Distinguished Service
driving: From connected vehicles to coordinated automated road trans- Awards in 2019 and 2016, the Outstanding Leadership Award in 2013,
port (C-ART),” Publications Office Eur. Union, Luxembourg City, the Carleton Research Achievement Awards in 2012 and 2021, the Ontario
Luxembourg, EUR-Sci. Tech., Res. Rep. EUR 28575 EN, 2017. Early Researcher Award (formerly Premiers Research Excellence Award)
[142] (Oct. 17, 2019). NTU Singapore and M1 Ink Partnership to in 2011, the Excellent Contribution Award at IEEE/IFIP TrustCom’10, the
Develop Singapore’s First 5G C-V2X Research Testbed and Leadership Opportunity Fund Award from Canada Foundation of Innovation
Trials. [Online]. Available: http://news.ntu.edu.sg/pages/newsdetail. in 2009, and the Best Paper Awards at IEEE ICNC’18, VTC’17 Spring,
aspxURL=http://news.ntu.edu.sg/news/Pages/NR2019_Oct17a.aspx& ICC’14, Globecom’12, IEEE/IFIP TrustCom’09, and Int’l Conference on
Guid=ae3f1a25-2615-48b1-af7a-b4fbb13608d4&Category=NTU Networking 05. He has served as the technical program committee (TPC)
[143] H. H. Soule et al., “Testing an automated collision avoidance and co-chair of numerous conferences. He serves on the editorial boards of several
emergency braking system for buses,” Transp. Res. Rec., J. Transp. journals, including the Co-Editor-in-Chief for Ad Hoc & Sensor Wireless
Res. Board, vol. 2674, no. 4, pp. 66–74, Apr. 2020. Networks, a Lead Series Editor for IEEE T RANSACTIONS ON V EHICULAR
[144] ConVeX Consortium Hosts Europe First Live C-V2X Direct Communi- T ECHNOLOGY, IEEE C OMMUNICATIONS S URVEYS & T UTORIALS , and
cation Interoperability Demonstration Between Motorcycles, Vehicles, IEEE T RANSACTIONS ON G REEN C OMMUNICATIONS AND N ETWORKING.
and Infrastructure, Qualcomm Press Release, Qualcomm Incorporated, He is the Editor-in-Chief for IEEE VTS Mobile World newsletter. He has been
San Diego, CA, USA, Jul. 2018. named in the Clarivate Analytics list of Highly Cited Researchers in 2019 and
[145] R. Munoz et al., “5GCroCo barcelona trial site for cross-border
2020. He is an IEEE Distinguished Lecturer of both Vehicular Technology
anticipated cooperative collision avoidance,” in Proc. Eur. Conf. Netw.
Society (VTS) and Communication Society. He is a registered Professional
Commun. (EuCNC), Dubrovnik, Croatia, Jun. 2020, pp. 34–39.
[146] Z. Zhang, H. Wang, and W. Chen, “A real-time visual-inertial mapping Engineer in the province of Ontario, Canada.
and localization method by fusing unstable GPS,” in Proc. 13th World
Congr. Intell. Control Automat. (WCICA), Jul. 2018, pp. 1397–1402. Tom H. Luan (Senior Member, IEEE) received
[147] L. Caltagirone, M. Bellone, L. Svensson, and M. Wahde, “LIDAR- the B.Eng. degree from Xi’an Jiaotong University,
based driving path generation using fully convolutional neural net- China, in 2004, the M.Phil. degree from the Hong
works,” in Proc. IEEE 20th Int. Conf. Intell. Transp. Syst. (ITSC), Kong University of Science and Technology in 2007,
Oct. 2017, pp. 1–6. and the Ph.D. degree from the University of Water-
[148] Q. Luo, C. Li, T. H. Luan, and W. Shi, “EdgeVCD: Intelligent loo, Waterloo, ON, Canada, in 2012. He is currently
algorithm-inspired content distribution in vehicular edge computing a Professor with the School of Cyber Engineer-
network,” IEEE Internet Things J., vol. 7, no. 6, pp. 5562–5579, ing, Xidian University, Xi’an, China. His research
Jun. 2020. mainly focuses on content distribution and media
[149] J. Kang et al., “Blockchain for secure and efficient data sharing in streaming in vehicular ad hoc networks and peer-
vehicular edge computing and networks,” IEEE Internet Things J., to-peer networking, as well as the protocol design
vol. 6, no. 3, pp. 4660–4670, Jun. 2019. and performance evaluation of wireless cloud computing and edge computing.
[150] Y. Fu, F. R. Yu, C. Li, T. H. Luan, and Y. Zhang, “Vehicular He has authored/coauthored more than 40 journal articles and 30 technical
blockchain-based collective learning for connected and autonomous papers in conference proceedings, and awarded one U.S. patent. He served as a
vehicles,” IEEE Wireless Commun., vol. 27, no. 2, pp. 197–203, TPC member for IEEE Globecom, ICC, and PIMRC. He served as a Technical
Apr. 2020. Reviewer for multiple IEEE Transactions, including IEEE T RANSACTIONS
ON M OBILE C OMPUTING , IEEE T RANSACTIONS ON PARALLEL AND D IS -
TRIBUTED S YSTEMS , IEEE T RANSACTIONS ON V EHICULAR T ECHNOL -
OGY , IEEE T RANSACTIONS ON W IRELESS C OMMUNICATIONS, and IEEE
T RANSACTIONS ON I NTELLIGENT T RANSPORTATION S YSTEMS .

Yuchuan Fu (Member, IEEE) received the Ph.D. Yao Zhang received the B.Eng. degree in telecom-
degree from the School of Telecommunications munication engineering from the Xi’an University
Engineering, Xidian University, Xi’an, China, of Science and Technology, China, in 2015. He is
in 2020. From 2018 to 2019, she was a joint currently pursuing the Ph.D. degree in telecommu-
Ph.D. Student with Carleton University, Ottawa, ON, nication engineering with Xidian University, Xi’an,
Canada. She is currently a Lecturer with the State China. His current research interests include com-
Key Laboratory of ISN, School of Telecommuni- munication protocol and performance evaluation of
cation Engineering, Xidian University. Her current vehicular networks, edge caching, and wireless sen-
research interests include algorithm design in vehic- sor networks.
ular networks and autonomous driving.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:11:08 UTC from IEEE Xplore. Restrictions apply.