IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 15, NO.

4, JUNE 2021 865

Automotive Radar Signal Processing: Research

Directions and Practical Challenges
Florian Engels , Member, IEEE, Philipp Heidenreich , Senior Member, IEEE, Markus Wintermantel,
Lukas Stäcker , Student Member, IEEE, Muhammed Al Kadi , and Abdelhak M. Zoubir , Fellow, IEEE

Abstract—Automotive radar is used in many applications of relative velocity, and affordability. An active field of re-
of advanced driver assistance systems and is considered as search and development is to mitigate current limitations of
one of the key technologies for highly automated driving. An automotive radar, which are mainly angular resolution and object
overview of state-of-the-art signal processing in automotive radar
is presented along with current research directions and practi- classification capabilities. Another practically relevant field is
cal challenges. We provide a comprehensive signal model for the interference mitigation. In the sequel, we will briefly outline
multiple-target case using multiple-input multiple-output schemes, each research field.
and discuss a practical processing chain to calculate the target In the early days of automotive radar, it was sufficient to
list. To demonstrate the capabilities of a modern series production resolve entire objects, e.g. two vehicles in neighboring lanes
high-performance radar sensor, real data examples are given. An
overview of conventional target processing and recent research had to be resolved for an adaptive cruise control application.
activities in machine learning and deep learning approaches is Today, high-performance radar is used for environmental scene
presented. Additionally, recent methods for practically relevant understanding in HAD applications [1], [11], and aims to resolve
radar-camera fusion are discussed. multiple reflection points of extended objects so that the structure
Index Terms—Advanced driver assistance systems, automotive of the resulting point set can be used for classification or size and
radar, autonomous vehicles, machine learning, MIMO radar, orientation estimation. Means to improve this resolution ability
radar signal processing. are an increase of bandwidth, array aperture and the number of
array elements, resulting in a better range and angular resolution.
I. INTRODUCTION Whereas the bandwidth is ultimately limited by the available
E witness today an enormous amount of activities in frequency band of 76 to 81 GHz, the physical array aperture is
W the automotive industry, in particular the development
of advanced driver assistance systems (ADAS), with the goal
generally limited by typical requirements of car manufacturers.
To be hardware- and cost-efficient, state-of-the art automotive
to make driving safer and more comfortable. Moreover, the radar sensors use the multiple-input multiple-output (MIMO)
introduction of highly automated driving (HAD) is considered as principle to increase the effective array aperture and number
a topical technology challenge. The performance and reliability of array elements [10]. Additionally, model based parameter
of these systems strongly depend on the capabilities of the estimation with high-resolution capability can be employed.
environmental sensing, for which radar technology is considered A flexible and computationally efficient framework, in which
as indispensable [1]. Consequently, several recent overview and a high-resolution parameter estimation is applied for either
research articles in the signal processing community, includ- range, relative radial velocity, or angular dimension, has been
ing [2]–[10], have addressed automotive radar. Advantages of presented in [3]. The recent development of system-on-chip
radar technologies when compared with LiDAR and camera technology for embedded platforms [8] has further contributed
technologies clearly are the robust and preferred operation in to the evolution of high-performance radar. Some published
adverse weather conditions, higher range, direct measurement examples of high-performance MIMO radar prototype systems
include [12]–[14]. These systems aim at providing target lists
Manuscript received September 6, 2020; revised January 18, 2021; accepted with several thousands detections, similar to LiDAR point
February 22, 2021. Date of publication March 3, 2021; date of current version clouds. In this paper, we present real data examples of a
June 3, 2021. The guest editor coordinating the review of this manuscript and
approving it for publication was Prof. Maria Sabrina Greco. (Corresponding
series production high-performance radar and demonstrate its
author: Florian Engels.) practical benefits, in particular a detection range of 320 m,
Florian Engels, Markus Wintermantel, and Muhammed Al Kadi are robustness to adverse weather conditions, and a general object
with the Continental, Business Unit Advanced Driver Assistance Systems,
88131 Lindau, Germany (e-mail: florian.engels@continental-corporation.
classification capability.
com; [email protected]; muhammed.al. Recently, automotive radar has been increasingly applied for
[email protected]). classification tasks. In particular, specific data features such as
Philipp Heidenreich and Lukas Stäcker are with the Stellantis, Opel
Automobile GmbH, 65423 Rüsselsheim, Germany (e-mail: philipp.
micro-Doppler signatures can be used to improve the detection
[email protected]; [email protected]). and classification of vulnerable road users [15], [16]. The
Abdelhak M. Zoubir is with the Signal Processing Group, Technische evolution of high-performance radar and the recent development
Universität Darmstadt, 64283 Darmstadt, Germany (e-mail: [email protected]
darmstadt.de).
of machine learning and deep learning has spawned a new field
Digital Object Identifier 10.1109/JSTSP.2021.3063666 of research. Exemplary contributions include a semantic

1932-4553 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 866 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 15, NO. 4, JUNE 2021

segmentation of the radar point cloud [17] or a semantic exponential with a fast-time frequency corresponding to range
radar grid [18]. A summary on the resulting semantic scene and a slow-time frequency corresponding to relative radial ve-
understanding has been presented in [19]. To give consideration locity. Subsequently, this model is extended to the MIMO case
to this development, we include a detailed discussion in this for multiple targets. Here, we describe two slow-time MIMO
paper. schemes, namely time multiplexing and phase coding, which
Many modern cars use radar sensors to realize ADAS and are feasible using current state-of-the-art radar solutions. To
HAD [11], [20], [21] so that the occurrence of interference is be more robust to interference, it is advantageous to include
very likely in present-day traffic scenarios, and its mitigation is pseudo-random variations to the modulation parameters and
a field of practical importance. Note that the penetration of radar the slow-time MIMO codes. Other common techniques for
sensors in small and mid-size cars is further promoted by safety interference mitigation are outlined as well. We continue by
ratings of new car assessment programs. A recent overview of presenting a practical processing chain to transform the base-
interference mitigation has been provided in [6], [9]. In this band samples to the 4D radar cube, with dimensions range,
paper, we summarize popular techniques in the context of a state- relative radial velocity, azimuth angle, and elevation angle. The
of-the-art processing chain, and provide an outlook on future radar cube represents a useful raw data interface. To optimize
development in this field. data rate and storage, it is common to create a more compact
Here, we focus on state-of-the-art radar signal processing, target list by employing adaptive target detection. The target
which is practically relevant in automotive radar, and which list is further supplemented by advanced parameter estimation.
can be realized with currently available system-on-chip tech- Note that the target list is sometimes referred to as the radar
nology. To this end, we provide a multiple-target MIMO sig- point cloud. A typical processing of the target list is outlined,
nal model with a chirp sequence modulation, and describe including occupancy grid mapping for stationary targets and
the pre-processing to transform the baseband samples to the object tracking for moving targets. We finish by highlighting
4D radar cube, with dimensions range, relative radial velocity, the need for continuous performance monitoring to be compliant
azimuth angle, and elevation angle. A detection and parameter with functional safety regulations, and present two examples.
estimation stage allows the creation of the so-called target list.
A typical processing of the target list is outlined, including A. Signal Model
object tracking for moving targets and occupancy grid mapping
We present in this section the receive signal model for the
for stationary targets. We also address the practically relevant
chirp sequence modulation combined with slow-time MIMO
aspects of interference mitigation and performance monitoring.
schemes. To this end, we begin by deriving the SISO model
To demonstrate the capabilities of current high-performance
for a single-target, and extend it to the MIMO case for multiple
radar, we present real data examples. To address the topical
targets.
development of machine learning and deep learning techniques
Let the ideal FMCW transmit signal for a sequence of M
for automotive radar, we include a discussion, along with a recent
chirp pulses be
overview of radar-camera fusion. Note that radar and camera is
M
 −1
today’s most relevant sensor combination for ADAS and HAD  m
applications. We conclude the paper with a discussion on future Π t−t
T cos[ϕLO (t − tm )] (1)
research directions. m=0

where Π(·) is the unit pulse, which is one in the interval [0,1)
II. STATE-OF-THE-ART RADAR SIGNAL PROCESSING and zero otherwise, t is the continuous-time variable, tm are the
pulse repetition times (PRTs), T is the pulse duration,
To be hardware- and cost-efficient, current state-of-the-art
automotive radar sensors use frequency modulated continuous ϕLO (t) = 2πf t + παt2
wave (FMCW) radar technology, in particular the chirp sequence
is the phase of the local-oscillator (LO), f is the carrier fre-
modulation with stretch processing [22]. Moreover, the MIMO
quency, and α is the chirp slope.
principle is used to increase the number of effective receive
The transmitted signal is reflected at a target and received
antennas and to realize larger array apertures. A widely used
with propagation delay τ and a complex-valued response param-
monolithic microwave integrated circuit (MMIC) for this task is
eter s. Stretch processing of the receive signal [22], i.e. direct
the AWR2243 [23] and its predecessors. This single-chip radar
IQ-mixing with the LO reference signal and low-pass filtering,
solution is based on RF-CMOS technology and is capable of
yields the baseband signal
operation in the band of 76 to 81 GHz. Its FMCW transceiver
system provides three transmit and four receive channels. It can M
 −1
   
be cascaded in a master slave configuration to realize e.g. a four x(t) = Π t−tTm −τ s ej ϕLO (t−tm )−ϕ

LO (t−tm −τ )
(2)
chip system with 12 transmit and 16 receive channels. Another m=0  
α 2

recent MMIC with comparable properties is the TEF82xx [24]. ej2π f τ +α(t−tm )τ − 2 τ
In the sequel, we focus on signal processing aspects that can Note that in reality the baseband signal is corrupted by noise.
be realized with current state-of-the-art radar solutions, such Since we focus here on the structure of the signal component,
as those in [23] or [24]. We begin by introducing the chirp we omit the noise component for convenience.
sequence modulation and the single-input single-output (SISO) For a target at range r with constant relative radial velocity v,
model for a single target, which is essentially a 2D complex the two-way propagation delay is τ (t) = 2 r+vt c . Substituting in

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 ENGELS et al.: AUTOMOTIVE RADAR SIGNAL PROCESSING: RESEARCH DIRECTIONS AND PRACTICAL CHALLENGES 867

TABLE I active at the same time, which allows an optimal usage of

PRACTICAL SLOW-TIME MIMO SCHEMES
the measurement time and an optimal SNR gain. When using
linear phase coding, the unambiguous measurement interval is
still reduced by factor Q [26]. It is ideal when using pseudo
random phase coding, but at the cost of an imperfect channel
separation, which effectively results in an increased sidelobe
level [27]. Note that the presented slow-time MIMO schemes
can be realized using current state-of-the-art radar solutions,
such as those presented in [23].

B. Interference Mitigation
(2) and sampling per pulse at times tl yields
f f
 As introduced previously, the occurrence of interference is
α f +αtm
x(t) ≈ sej4π c r+ c rtl + c vtm ej4π c vtl (3) very likely in present-day traffic scenarios with several cars
t=tm +tl
employing automotive radar. For radar sensors using the chirp
Note that we have neglected terms with τ 2 and t2l in (3), which is sequence modulation, interference occurs specifically when the
common practice for typical automotive radar system parame- instantaneous frequency of the interfering signal falls into the
ters. The model in (3) comprises a 2D complex exponential with passband of the victims receive filter. This is usually limited to
a constant phase, a fast-time frequency, corresponding to range, short time slots so that only a small number of baseband samples
and a slow-time frequency that corresponds to the relative radial are affected by interference. Thus, a popular and yet simple
velocity. The remaining term corresponds to range migration and interference mitigation technique is the identification and subse-
Doppler-frequency shift effects, which are typically neglected quent zeroing of samples that are affected by interference [28].
for current automotive radar systems. However, future modula- However, in traffic environments with dense target structure,
tion schemes might combine high bandwidth with long coherent interference identification becomes a challenge when strong
integration times. In that case, it becomes necessary to include target responses mask the interference contribution. Therefore,
range migration in the signal model and compensate for it in the [29] and [30] suggest to use the target-free image band of a
corresponding processing strategies [5], [25]. quadrature receiver to improve interference identification. How-
The single-target SISO model in (3) can now be extended ever, to reduce hardware cost and power consumption, most
to K targets and the MIMO case, with Q transmit antennas at off-the-shelf MMICs use real receivers only, so that alternative
positions pq and N receive antennas at positions pn , where approaches become necessary. In [31], it is suggested to estimate
the narrowband assumption holds for both transmit and receive the parameters of a few dominant targets using standard process-
arrays, as follows ing without interference zeroing. After subtracting the dominant
targets from the baseband samples according to the model in (4),
x(l, m, n)
the samples that are affected by interference can be identified
K
 Q  f f
 f
  more reliably, and a second processing and parameter estimation
α T T
= am,q sk ej4π c rk + c rk tl + c vk tm ej2π c ak pn +dk pq is applied after interference zeroing. In [32], a similar approach
k=1 q=1 is presented, but with a focus on reducing the artifacts resulting
(4) from the zeroing operation.
Herein, l = 0, . . . , L − 1 are the fast-time samples per pulse, Further means to avoid high-power spurious peaks are
m = 0, . . . , M − 1 are the slow-time samples, n = 0, . . . , pseudo-random variations of the modulation parameters to coun-
N − 1 are the receive channel indices, and am,q are the slow- teract a coherent integration of interference energy in fast-time
time MIMO codes per transmit channel. Parameters sk , rk , vk , and slow-time Fourier transforms. In particular, pseudo-random
dk , and ak are respectively the complex response parameter, variations of the PRTs and fast-time sampling instances can be
range, relative radial velocity, direction of departure (DOD), employed [33] by incorporating
and direction of arrival (DOA). Note that pq , pn , dk , and ak , tm = mTP + ΔtP,m , tl,m = lTS + ΔtS,m (5)
are 3D vectors in a sensor-centered Cartesian coordinate system.
We would like to point out that a target can be regarded as a in the model in (4). Herein, TP and TS are the constant PRT
resolvable reflection point, and that a physical object may be and sampling interval, respectively, while ΔtP,m and ΔtS,m are
represented by multiple targets. the corresponding pseudo-random variations, which are drawn
To be more specific about the slow-time MIMO schemes, from a zero-mean uniform distribution with appropriate bounds.
different configurations for am,q in model (4) are gathered in Likewise, when using slow-time MIMO with phase coding, the
Table I. When using time multiplexing, only a single transmit phase shifters can be re-used for interference mitigation [34].
channel is active at a time, which results in an optimal channel To this end, the phase codes ϕm,q from Table I are superposed
separation, but a reduced effective measurement time, leading to with an additional pseudo-random variation, which is equal for
a reduced SNR gain and a reduced unambiguous measurement all transmit channels, as follows
interval for the relative radial velocity, both by factor Q. In
contrast, when using phase coding, all transmit channels are am,q = ejϕm,q +jΔϕm (6)

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 868 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 15, NO. 4, JUNE 2021

Fig. 1. Practical processing chain overview.

Herein, Δϕm is the additional pseudo-random variation,

which is drawn from a uniform distribution over [0, 2π). Com-
pensation with e−jΔϕm prior to the slow-time Fourier transform
leads to a spread of interference energy over all frequencies,
whereas the target energy is concentrated at distinct frequency
regions.
An alternative approach for interference mitigation is to
employ frequency hopping techniques [35], which aim to use
unoccupied frequency bands for radar sensing. A cooperative in-
terference avoidance can be achieved by a coordinated allocation
of frequency bands and transmission time slots, and is enabled by
the use of radar waveforms with communication capability [9].
Note that these approaches generally contradict the current trend
towards high-performance radar, in which already large parts of
the available frequency band and transmission time are used
to realize high range and relative radial velocity resolutions,
respectively.
Fig. 2. MIMO URA example: two different physical transmit and receive
C. Processing Chain Overview antenna configurations on the left result in the same virtual URA on the right.
Note that the bottom configuration can be realized with approximately half of
We present a practical processing chain for state-of-the-art the physical space of the top configuration.
automotive radar signal processing in Fig. 1.
The input of the processing chain are baseband samples fol-
lowing the model in (4). As described in the previous section, in- element spacing dy and dz . Assuming direct signal propagation,
terference mitigation can be applied across the whole processing i.e. with equal DOD and DOA, we have
chain and is inherently present in the pseudo-random variations ⎡ ⎤ ⎡ ⎤ ⎡ ⎤
of the modulation parameters and the slow-time MIMO codes. cos(θk ) cos(φk ) 0 0
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
To stress its importance, we have included the interference ak = dk = ⎣ cos(θk ) sin(φk ) ⎦, pn = ⎣ndy ⎦, pq = ⎣ 0 ⎦
zeroing block at the beginning of the processing chain. The sin(θk ) 0 qdz
pre-processing block consists of the fast-time Fourier trans-
Note that the mentioned MIMO array configuration is shown in
form, MIMO demodulation, the slow-time Fourier transform
the top of Fig. 2 for 16 receive and 12 transmit antennas, and
and beamforming. It is used to transform the baseband samples
results in a virtual uniform rectangular array (URA) with 192
to the 4D radar cube, with dimensions range, relative radial
array elements. Substitution in (4) yields
velocity, azimuth angle, and elevation angle. Target detection,
supplemented by parameter estimation, is used to create the K Q−1
 
target list. Note that in a vehicle setup with multiple sensors, the x(l, m, n) = am,q sk ej(λk l+μk m+νk n+ξk q) (7)
target list is an efficient interface for data fusion. The stationary k=1 q=0
targets are further processed using an occupancy grid mapping where sk now includes the constant phase term,
and an obstacle calculation, whereas the moving targets are fur-
ther processed in an object tracking and road-user classification. λk = 4π αc rk TS , μk = 4π fc vk TP (8)
In what follows, we describe each block in detail. are the frequencies corresponding to range and relative radial
velocity, respectively,
D. Pre-Processing
νk = 2π fc dy cos(θk ) sin(φk ), ξk = 2π fc dz sin(θk ) (9)
To conveniently present the pre-processing, we simplify (4)
by confining ourselves to uniform fast-time sampling tl = lTS , are the horizontal and vertical spatial frequencies, respectively,
uniform PRTs tm = mTP , horizontal and vertical uniform lin- φk is the azimuth angle, and θk is the elevation angle. The
ear arrays of receive and transmit antennas, respectively, with model in (7) is a superposition of 4D complex exponentials with

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 ENGELS et al.: AUTOMOTIVE RADAR SIGNAL PROCESSING: RESEARCH DIRECTIONS AND PRACTICAL CHALLENGES 869

frequencies λk , μk , νk , and ξk . Once the Q transmit components is preferable in automotive radar, since it can be realized with
are disentangled via MIMO demodulation, the target parameters approximately half of the physical space.
are obtained via 4D frequency estimation. If we consider (7)
to be corrupted by additive, white Gaussian noise, the optimal E. Target List Creation
solution for this task is obtained via a nonlinear least squares
As introduced earlier, the radar cube represents a useful raw
optimization. A computationally feasible alternative is the max-
data interface, in which the target energy is concentrated at dis-
imization of the corresponding 4D periodogram, given by the
tinct frequency regions, whereas the noise energy is spread over
squared magnitude of the 4D Fourier transform [36].
all frequencies. To obtain a more compact representation, the
In practice, the Fourier transform is calculated using its dis-
target list is created by means of target detection and parameter
crete variant, which can be efficiently realized using the fast
estimation.
Fourier transform (FFT). As introduced in Fig. 1, an FFT along
To be robust to variations of the noise statistics, constant false
the fast-time dimension of the baseband samples is followed
alarm rate (CFAR) power detection is commonly employed.
by a slow-time FFT, interleaved with the MIMO demodulation.
Here, the power values of the radar cube are compared with
To realize the beamforming, a 2D FFT is calculated along the
a threshold that is based on a locally calculated statistic of the
vertical and horizontal array dimensions. Such a processing
noise power. CFAR power detection is typically complemented
sequence allows an optimal use of the available processing time,
by peak finding and sidelobe thresholding, where the latter is
storage, and FFT processors.
used to exclude windowing artifacts from strong targets. Hav-
To be specific, we summarize the pre-processing without the
ing determined all detected peaks, the next step is to perform
beamforming, as follows
parameter estimation.
Y (λ, μ, n, q) Assuming all targets are resolved, the peak position at fre-
quencies λk , μk , νk , and ξk can be directly used to calculate the
L−1
M −1
 corresponding parameters rk , vk , φk , and θk , using (8) and (9).
= wλ (l) wμ (m) a∗m,q x(l, m, n) e−j(λl+μm) (10)
An interpolation in the local neighborhood is commonly used to
l=0 m=0
obtain more accurate parameter estimates.
where In critical automotive-radar use cases, such as closely spaced
λ = 0, 2π 2π 2π 2π targets or multipath propagation, crucial targets may not be
L , . . . , (L − 1) L , μ = 0, M , . . . , (M − 1) M
resolved, which can result in biased peak positions or in the
are the frequency grids corresponding to the range and rela- occurence of spurious peaks. In such cases, the target parameters
tive radial velocity, respectively, and wλ (l) and wμ (m) are the can be obtained by fitting local signal models with multiple
fast-time and slow-time window functions. Note that window targets to the neighborhood of the detected peaks. To this end,
functions are important to control spectral leakage and prevent we substitute (7) and (10) in (11), which yields
a masking of weak targets by strong targets. We remark that the
described MIMO demodulation holds for the variants in Table I, X(λ, μ, ν, ξ)
except that for time multiplexing, a frequency offset per transmit K

channel needs to be compensated for [37]. = sk Wλ (λ − λk )Wμ (μ − μk )Wν (ν − νk )Wξ (ξ − ξk )
The remaining beamforming is calculated over n and q di- k=1
mensions, which correspond to receive and transmit channels, (12)
respectively, and are equivalent to the vertical and horizontal
where Wλ (λ), Wμ (μ), Wν (ν), Wξ (ξ) are the Fourier transforms
array indices in the URA configuration. The result of the beam-
of the fast-time, slow-time, horizontal, and vertical window
forming is the radar cube, as follows
functions, respectively. Evaluating (12) in the vicinity of de-
X(λ, μ, ν, ξ) tected peaks yields local signal models with few targets, which
enable a computationally efficient parameter estimation via
N
 −1 Q−1
 nonlinear least squares optimization. In a practical framework,
= wν (n) wξ (q) Y (λ, μ, n, q) e−j(νn+ξq) (11)
typically two-target models are compared with single-target
n=0 q=0
models, and the best fitting model is chosen [3]. An extension
where of this framework to the multipath model in the MIMO case has
been presented in [38].
ν = 0, 2π 2π 2π 2π
N , . . . , (N − 1) N , ξ = 0, Q , . . . , (Q − 1) Q We wish to remark that an important challenge in automotive
are the horizontal and vertical spatial frequency grids, respec- radar signal processing is computational complexity. In par-
tively, and wν (n) and wξ (q) are the horizontal and vertical ticular, the pre-processing and target list creation need many
window functions, respectively. Note that the considered virtual iterations over the large radar cube with typically several million
URA can be equivalently obtained with a different MIMO array elements. For instance, a computationally efficient calculation of
configuration, such as shown in the bottom of Fig. 2. In this the Fourier transforms can be achieved by means of hardware-
case, the receive and transmit channel combinations have to accelerated FFTs. Likewise, a computationally efficient peak
be re-ordered according to vertical and horizontal array indices finding and detection can be achieved with hardware-accelerated
prior to beamforming in (11). Note that the latter configuration local maxima search and histogram calculation.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 870 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 15, NO. 4, JUNE 2021

F. Target List Processing relevant for lane-accurate position determination, e.g. to realize
a lateral position accuracy of 3.5 m in 320 m distance, an absolute
Before any further processing, the velocity of the ego vehicle
is used to compensate the relative radial velocity measure- accuracy of the azimuth angle of roughly 0.62◦ is required.
The phase noise is typically measured using a feedback chan-
ments in the target list. It is common practice to distinguish
nel of the LO, and is used to monitor the frequency linearity
stationary and dynamic target processing to better address its
characteristics. of the FMCW transmit signal in (1). By calculating a frequency
error signal and the corresponding frequency error spectrum, one
The stationary targets move in the scene according to the
is able to predict the detection sensitivity in the range spectrum
same motion model of the ego vehicle so that they can be
accumulated in a representation of the static environment in and potential artifacts, which may produce false detections or
mask weak targets. Thus, phase noise monitoring represents a
the form of an occupancy grid map. Due to its static nature,
this grid map can be integrated over several processing cycles. powerful diagnostic measure. A recent and detailed overview is
As motion model of the ego vehicle, often a constant turn rate given in [7].
and acceleration motion model [39] is selected. Its propagation
requires measurements of the yaw rate, velocity, and acceleration III. HIGH-PERFORMANCE RADAR EXAMPLE
of the ego vehicle. Standard occupancy grid map methods have
been derived for the LiDAR sensor model. Adaptions for specific In this section, we provide a snapshot of the ongoing develop-
radar sensor models are described in [40]–[42]. ment of automotive radar towards HAD, and present a current
In contrast, the dynamic targets move according to different series production high-performance radar (HPR). We compare
motion models relative to the ego vehicle. Generally, multiple the HPR with a current series production long-range radar (LRR)
targets correspond to one moving object, for which also constant and a short-range radar (SRR) for ADAS applications. To give
turn rate and acceleration motion models are assumed. The an outlook on the achievable performance, we present real data
tracking of moving objects is typically carried out in a Kalman examples of the target list of the HPR.
filter framework. After transforming the measurements to a The SRR and LRR use a single MMIC with three transmit
Cartesian coordinate system, a data association is carried out, and four receive channels to realize 12 MIMO channels via
such as the one described in [43]. Subsequently, Kalman filtering slow-time phase coding. The HPR cascades four MMICs in
by means of state prediction and state estimation is performed a master-slave configuration to realize a total of 12 transmit
for each track, followed by a track management [44]. and 16 receive channels, resulting in 192 MIMO channels. The
transmit and receive antenna placement for the SRR realizes a
virtual 12-element horizontal uniform linear array (ULA) with
G. Performance Monitoring
half-wavelength spacing. The LRR employs a virtual sparse
Online methods for calibration, performance monitoring and linear array, i.e. a ULA with missing elements, to realize a
fault detection are of significant importance in ADAS and HAD larger array aperture and at the same time a large unambiguous
applications. Full functionality and a guaranteed performance field of view (FOV). The HPR implements a two-way array
has to be provided at all times. In compliance with functional approach: a 128-element URA is used in the signal processing
safety regulations, in particular ISO 26 262:2018 and ISO/PAS chain, as shown in Fig. 1, and a 64-element array is used in a post
21 448, known as SOTIF (safety of the intended functionality), processing. In this way, a large array aperture can be realized
a drop in performance or functional fault has to be detected with URA element spacings larger than half a wavelength, and
and adequate measures have to be taken. To this end, the in- the resulting angular ambiguities can be resolved in a post
tegrity of the measurements has to be monitored continuously, processing using the 64-element array.
and build-in self-tests have to ensure the correct operation Table II gathers performance parameters, namely resolution
of all components. We would like to focus on two exam- limits, unambiguous measurement intervals, and the maximal
ples: boresight angle monitoring and phase noise monitoring. number of detected targets, for the series production HPR, LRR,
These two represent practically relevant diagnostic measures and SRR. The resolution limits specify the minimally required
for determining the accuracy and reliability of the sensor’s target separation for successful parameter estimation. Range and
measurements. velocity resolution limits are inversely proportional to the total
Boresight angle monitoring typically exploits the relation bandwidth and transmission time and the limited unambiguous
between the ego vehicle dynamics and the measured azimuth range and velocity measurement intervals are due to fast-time
angle φk , elevation angle θk , and radial relative velocity vk of and slow-time aliasing, respectively. To mitigate range ambi-
stationary targets, see e.g. [45], [46]. For the simplified case guities, a fast-time anti-aliasing filter is employed and velocity
when the ego vehicle is driving straight, i.e. with zero yaw rate, ambiguities can be resolved over several measurement cycles
we have by target tracking or staggered PRT techniques [47]. Angular
resolution limits are determined by the total array aperture. The
vk = vego cos(θk ) cos(φk ) (13)
minimal antenna spacings and corresponding two-way antenna
where vego is the velocity of the ego vehicle. By monitoring the patterns determine the unambiguous angular measurement in-
corresponding fitting errors, and potentially the phases of the tervals. We highlight the main differences of the HPR compared
antennas, we are able to guarantee a certain calibration goodness to the LRR and SRR, namely the range resolution of 0.2 m,
of the azimuth angle determination. Note that this is practically the maximal range of 320 m, and the azimuth resolution of less

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 ENGELS et al.: AUTOMOTIVE RADAR SIGNAL PROCESSING: RESEARCH DIRECTIONS AND PRACTICAL CHALLENGES 871

TABLE II
TYPICAL SYSTEM PARAMETERS

than 2 ◦ . Note that only the HPR provides a single-cycle elevation availability of large annotated datasets, such as KITTI [48], and
angle measurement capability. the rapid development of machine learning, in particular deep
To demonstrate the maximal range of 320 m, a high target convolutional neural networks (CNNs). Unfortunately, a widely
density, and the direct elevation measurement capability, we used and comparable radar dataset is not publicly available, even
show the target list of the HPR in a bird’s eye view (BEV) repre- though modern high-performance radar can provide dense point
sentation for a typical highway scenario in Fig. 3. To distinguish cloud information for advanced scene understanding.
stationary targets from moving ones, the lower left-hand side To the best of the authors’ knowledge, annotated datasets
BEV uses a color coding according to target velocity. The lower with radar data are limited to the NuScenes dataset [49], the
right-hand side BEV provides a color coding according to target Astyx HiRes 2019 dataset [50], and the Oxford radar RobotCar
height. For reference, the top of Fig. 3 provides a camera image dataset [51]. NuScenes employs undisclosed series production
with an enlarged region to reveal details. Observe a car and a radar sensors, which provide at most only 125 detections per
truck on the opposite highway lane, a car and a truck on the same measurement cycle, as per interface description [49]. For the
lane as the ego vehicle, and a bridge over the highway, which front radar, one observes on average only less than 60 valid
are labeled in the camera image as well as in the radar BEV. detections so that the provided data density is worse than today’s
Observe multiple target detections for each vehicle, the high series production radar sensors, and is deemed not sufficient
detection range of 320 m, the direct elevation measurements for for radar-based object detection or semantic segmentation [52].
the bridge in a distance larger than 200 m, and the high density In the Astyx dataset, a pre-production high-resolution radar
of stationary targets. sensor provides several thousand detections per measurement
Fig. 4 shows the same BEV layout for a highway scene cycle, resulting in a LiDAR-like performance [13]. Although
in adverse weather conditions. Note the poor visibility in the the dataset is very small with only 546 frames, a successful
camera image and the almost unchanged detection range in the proof-of-concept application for 3D object detection has been
radar data. Also, note in particular three vehicles above 100 m, presented in [53]. The Oxford radar RobotCar dataset employs
one on the ego lane and two on the rightmost lane, which are a non-automotive type scanning radar so that it is not considered
clearly visible in the radar BEV and invisible in the camera relevant here.
image. In the sequel, we provide an overview of the recent develop-
In Fig. 5, we show the target list in a BEV representation for ment of radar-based object detection and semantic segmentation
an urban scenario. The ego vehicle makes a right turn, while a that has been conducted mostly with proprietary datasets.
vehicle approaches on the opposite lane, and a cyclist is about
to cross from the right. To cover a larger FOV two reference
cameras are used, where one is centered and the other one is A. Static Environment Classification
oriented to the right-hand side of the ego vehicle. To highlight For classification of the static environment, CNN techniques
details of the approaching car and the cyclist, the BEV is cropped are applied to occupancy grids that are computed from sta-
to 40 m x-distance and ±20 m y-distance. However, note that tionary targets to obtain semantic radar grids. In [18], moving
the maximal range of the radar is still 320 m. Observe the high window sections of the radar-based occupancy grid are fed
target density for the vehicle as well as for the cyclist, which into a CNN classifier, with the following object classes: car,
shows a clear potential for road-user classification. building, curbstone, pole, vegetation, and other. This approach
is enhanced in [19], in which an occupancy grid with radar
cross-section (RCS) histogram information is fed into a semantic
IV. MACHINE LEARNING APPLICATIONS segmentation CNN with an encoder-decoder structure, similar
Automotive scene understanding in terms of camera and to [54].
LiDAR based object detection and classification has advanced The calculated occupancy grid map can also be processed for
significantly in the last few years. This is mainly due to the the purpose of road course estimation or freespace estimation.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 872 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 15, NO. 4, JUNE 2021

Fig. 3. Highway data example: the HPR target list is shown in a BEV (bottom) with color coding according to velocity (left) or height (right). For reference, a
camera image with an enlarged area is shown (top). Vehicles and a bridge are labeled in the camera image as well as in the radar BEV. Note the high detection
range of 320 m, the direct elevation measurement capability shown in the bottom right plot, and the high point density.

A deep learning approach for road course estimation has been and then to classify the resulting clusters using handcrafted
presented in [55]. Freespace estimation and road scene under- features. An initial study of this approach is described in [59].
standing is presented with a semantic segmentation CNN in [56]. It has been investigated specifically for pedestrian detection
Lane marking detection using a radar grid map is presented using several classifiers in [60], and for ghost target detection
in [42]. A visionary approach, which uses dedicated radar targets in [61]. A random forest classifier has been applied in [62],
as lane markers, is suggested in [57]. with the following object classes: car, pedestrian, pedestrian
group, cyclist, truck, and other. The approach has been improved
with an advanced feature selection in [63] and extended with
B. Moving Object Classification a two-stage clustering in [64]. An apparent drawback is that
A popular approach for moving object classification is to the clustering can be error-prone so that multiple close objects
first cluster the target data using the DBSCAN algorithm [58] may be clustered together or large objects may be split into

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 ENGELS et al.: AUTOMOTIVE RADAR SIGNAL PROCESSING: RESEARCH DIRECTIONS AND PRACTICAL CHALLENGES 873

Fig. 4. Data example for adverse weather conditions: the HPR target is list shown in a BEV (bottom) with color coding according to velocity (left) or height
(right). For reference,a camera image with an enlarged area is shown (top). Note the poor visibility in the camera image and an almost unchanged high detection
range in the radar data. Also, note in particular the three cars above 100 m, which are invisible in the camera image.

multiple clusters, which may result in false classification and The detection and classification of vulnerable road users with
false object creation. Moreover, optimal clustering parameters distinct Doppler signatures can be improved by incorporating
cannot be found since they generally depend on the object type low-level data in terms of the radar data cube, in particular the
and size. These problems can be addressed by classifying each range-Doppler plane [15]. Several classifiers are studied in [16].
target individually. To this end, the PointNet++ method [65] is A drawback of using such low-level data is that no advanced
applied to the radar point cloud in [17], [19] and [66]. Note parameter estimation can be applied, such as range interpolation
that PointNet++ is a DNN with a hierarchical structure that or high-resolution angle estimation. To address this issue, a
can be applied directly to point clouds to obtain a semantic hybrid approach for moving object classification that uses both
segmentation. Since this approach has been originally developed the target list and an excerpt of the radar data cube was proposed
for dense LiDAR point clouds, adaptions are necessary. recently [67].

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 874 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 15, NO. 4, JUNE 2021

measurement models. A detailed overview of deep learning

fusion approaches for automotive sensor systems is provided
in [69]. Unfortunately, the main focus of current research and
development is on camera-LiDAR fusion, whereas camera-radar
fusion only plays a secondary role, which is probably due to the
lack of a large public dataset with state-of-the-art radar data.
Nevertheless, we aim to summarize and extend [69] in terms of
camera-radar fusion in the sequel.
Interestingly, most of the current research activity uses the
NuScenes dataset [49] and focuses on the task of 2D object de-
tection, which is commonly applied in image-only frameworks
and yields multiple classified 2D bounding boxes per image.
It is noted that this task does not exploit the full potential of
the radar data in terms of its localization capability. There are
only very few approaches which focus on the tasks of BEV and
3D object detection, which are arguably more relevant in auto-
motive applications, as the obstacles in this representation can
be used directly for subsequent maneuver planning or collision
avoidance functions.

A. 2D Object Detection
Two main groups of image-only algorithms can be identified:
two-stage and single-stage methods. In two-stage methods, such
Fig. 5. Urban data example: the HPR target list shown in a BEV with color as R-CNN [70] and its improvements [71], a region proposal
coding according to velocity (bottom). For reference, two camera images are network is used to identify the regions of interest (ROIs) in an
shown to cover a large FOV. Observe multiple detected targets for the car and for
the cyclist, which match the respective object shape. This gives a clear indication
image, which are then classified in a subsequent classification
of classification potential. network. These networks are usually computationally intensive
and cannot be trained end-to-end, so that single-stage networks
have been developed, e.g. SSD [72] or RetinaNet [73]. These
V. RADAR-CAMERA FUSION
networks use predefined anchor boxes of different size and
While most early self-driving car prototypes heavily depend aspect ratio instead of data-dependent ROIs. For each anchor
on high-end LiDAR technology, the majority of ADAS and HAD box, parallel classification and regressor networks calculate
in today’s series production vehicles relies on a fusion of radar classification scores and bounding box coordinates, respectively.
and camera sensors. Notably, also the Mercedes-Benz S 500 Recently developed fusion approaches re-use the described net-
Intelligent Drive research vehicle employed radar and camera work architectures and employ the radar data to enhance their de-
sensors to autonomously drive the Bertha Benz historic route in tection performance. In a sequential fusion, the radar detections
2013 [68]. In the automotive industry, this combination currently are projected to the image plane to create ROIs in a two-stage
appears to be the most relevant sensor combination, enjoying object detection framework [74]. In [75], this idea is further de-
the benefits of moderate cost, a pleasing sensor integration, and veloped and an improved performance on the NuScenes dataset
the provision of complementary environmental information. In is reported. Alternative deep fusion approaches use the projected
particular, radar sensors provide accurate and direct measure- radar targets to generate additional image channels, which are
ments of the distance and radial relative velocity, while cameras fed to modified single-stage object detection networks. In [76],
provide accurate azimuth and elevation measurements and rich three additional channels are generated from the radar range and
semantic context information. lateral and longitudinal relative velocity. Two output branches
Conventional approaches of data fusion are commonly based are used to separately detect small and large obstacles. The
on a Kalman filter framework. After data alignment, i.e., trans- authors extend their approach by semantic segmentation in [77].
forming the measurements to a common coordinate system, In [78], the authors show that radar can be especially useful for
a data association is carried out in terms of measurement- detecting distant vehicles. They project the radar detections as
to-measurement and measurement-to-track association. Sub- extended ellipses to reflect their uncertainty and use the range
sequently, Kalman filtering by means of state prediction and and the range-rate as channels of the radar image. In contrast,
state estimation is performed for each track, followed by track the authors of [79] argue that the radar detections should rather
management. In this methodology, probabilistic measurement be projected as vertical lines, since the radar is mainly uncertain
models are required and the fusion takes place implicitly during in terms of the object’s elevation. They assume a constant object
the measurement updates from the different sensors [44]. height and use the range and RCS as the channels of the radar
In contrast, recent deep learning approaches focus on single image. They also introduce a new training technique BlackIn,
shot fusion and do usually not incorporate explicit probabilistic which blacks out the camera inputs at a fraction of the training

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 ENGELS et al.: AUTOMOTIVE RADAR SIGNAL PROCESSING: RESEARCH DIRECTIONS AND PRACTICAL CHALLENGES 875

steps, to encourage the network to rely more heavily on radar antenna spacings [84]. Sparse arrays, in particular URAs with
data. Instead of simply concatenating the resulting radar image missing elements, generally offer more control options. A sparse
to the camera image, the authors of [80] propose a so-called array design, which allows a flexible tradeoff between resolution
spatial attention function which effectively provides a relevance and ambiguity, and at the same time considers constraints such as
weighting. robustness to production tolerances, feasible beamforming grid
sizes, and acceptable sidelobe levels, is of high practical rele-
B. BEV and 3D Object Detection vance for automotive radar. A further challenge is to maintain an
accurate array calibration after vehicle integration and over the
In [81], the authors have proposed a fusion network which
sensor lifetime. Research on online array calibration techniques
uses the range-azimuth radar images together with the camera
is therefore also of great importance in the automotive industry.
images, inversely projected onto the radar plane, in separate
feature branches of a CNN. The generated features are then fused
and fed to an SSD detection head. For training and evaluation, C. Parameter Estimation in the Presence of Multipath
an own dataset has been used that includes the required radar
In many urban and highway scenarios, the occurance of
cube data. In [53], the AVOD fusion method [82], originally
multipath propagation is very likely, e.g. horizontal multipath
developed for camera-LiDAR fusion, has been applied with the
via guardrails, parked cars, truck trailers, and walls, or vertical
target list of a high-performance radar sensor. Using the Astyx
multipath via the road surface. Automotive radar sensors also
HiRes 2019 dataset [50], it has been demonstrated that the radar-
have to operate reliably in these cases to avoid erroneous target
camera fusion can outperform the camera-LiDAR fusion.
localization or an incorrect handling of ghost targets. Target
angle estimation in the presence of multipath has been presented
VI. FUTURE RESEARCH DIRECTIONS
in [38]. However, a comprehensive integration in an automotive
We have identified several research direction of practical radar signal processing framework, as presented in Section II,
importance and would like to highlight the potential for improve- is an open topic. This includes considering multipath models in
ment. These include interference mitigation and avoidance, the range and relative radial velocity dimension to complement
high-resolution and large-FOV angle estimation, parameter es- a decoupled parameter estimation, as presented in [3]. Addi-
timation in the presence of multipath, and object classification tionally, pre-processing schemes have to be reconsidered, as
and deep learning. In the sequel, we will address each field in conventional beamforming does not concentrate target energy
more detail. along the angular radar cube dimensions, due to a violated virtual
array assumption in the multipath case. This is in particular
A. Interference Mitigation and Avoidance relevant for high-performance automotive radar with a large
number of receive and transmit channels, which makes model
In a recent overview article on interference mitigation [9], the
fitting prior to beamforming computationally impractical. An
authors ask “is interference really a problem?” We believe that
example of an adapted pre-processing is a 4D beamforming with
a comprehensive answer is crucial for the automotive industry
independent DOD and DOA grids, which concentrates target
in order to trade off costs and benefits of advanced interference
energy also for the multipath case.
mitigation and to guide regulatory initiatives. To this end, large-
scale studies have to be conducted with a fine-grained view on
ADAS and HAD requirements and a careful selection of relevant D. Object Classification and Deep Learning
scenarios. This includes the development of appropriate inter-
As mentioned earlier, today’s research activities in the field
ference simulation models to predict the effects of increasing
of deep learning applications are mainly carried out with pro-
future radar deployment.
prietary datasets. Unfortunately, a widely used and large public
Even if a thorough quantitative answer is not yet available,
dataset with state-of-the-art radar data is missing. We remark that
research on interference mitigation is considered highly relevant
a desirable dataset should be radar-centric, i.e. 3D annotations
in both the automotive industry [83] and the research commu-
should primarily take into account radar detections so that radar
nity [6], [9]. Today, the most common interference scenario
advantages at larger distances or in adverse weather conditions
is that interferer and victim both use FMCW-type modulation
can be exploited and demonstrated. Also, semi-automatic la-
schemes. In this case, we consider a parametric approach to
beling techniques and an adequate reference sensor system,
interference mitigation as particularly relevant. This includes re-
involving e.g. a high-performance LiDAR sensor and several
search on appropriate baseband interference models, respective
cameras with telephoto lenses, should be considered. Based on
model parameter estimation techniques, and optimal methods
an evaluation of the current maturity of radar signal simulation,
for interference removal.
an interesting alternative or supplement could be a dataset with
synthetic radar data. A study to answer the question whether
B. High-Resolution and Large-FOV Angle Estimation
synthetic radar data can be used to train and test object detection
A challenge when designing MIMO arrays for a given number and classification networks is in great demand. In summary, we
of receive and transmit channels is to trade off angular resolu- note that the availability of such a dataset, either real or synthetic,
tion and unambiguous FOV. For URAs, the resolution and the would stimulate the research and development of radar-only or
unambiguous FOV are controlled by the horizontal and vertical radar-camera fusion techniques for BEV or 3D object detection.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 876 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 15, NO. 4, JUNE 2021

REFERENCES [26] V. Mecca, D. Ramakrishnan, F. Robey, and J. Krolik, “Slow-time MIMO

space time adaptive processing,” in MIMO Radar Signal Processing. New
[1] J. Dickmann et al., “Automotive radar the key technology for autonomous York, NY, USA: Wiley, 2009, pp. 283–318.
driving: From detection and ranging to environmental understanding,” in [27] W. van Rossum and L. Anitori, “Doppler ambiguity resolution using
Proc. IEEE Radar Conf., 2016, pp. 1–6. random slow-time code division multiple access MIMO radar with sparse
[2] S. Patole, M. Torlak, D. Wang, and M. Ali, “Automotive radars: A review signal processing,” in Proc. IEEE Radar Conf., 2018, pp. 441–446.
of signal processing techniques,” IEEE Signal Process. Mag., vol. 34, no. 2, [28] M. Toth, P. Meissner, A. Melzer, and K. Witrisal, “Performance compar-
pp. 22–35, Mar. 2017. ison of mutual automotive radar interference mitigation algorithms,” in
[3] F. Engels, P. Heidenreich, A. Zoubir, F. Jondral, and M. Wintermantel, Proc. IEEE Radar Conf., 2019, pp. 1–6.
“Advances in automotive radar: A framework on computationally efficient [29] S. Murali, K. Subburaj, B. Ginsburg, and K. Ramasubramanian, “Interfer-
high-resolution frequency estimation,” IEEE Signal Process. Mag., vol. 34, ence detection in FMCW radar using a complex baseband oversampled
no. 2, pp. 36–46, Mar. 2017. receiver,” in Proc. IEEE Radar Conf., 2018, pp. 1567–1572.
[4] I. Bilik, O. Longman, S. Villeval, and J. Tabrikian, “The rise of radar for [30] F. Jin and S. Cao, “Automotive radar interference mitigation using adaptive
autonomous vehicles: Signal processing solutions and future research di- noise canceller,” IEEE Trans. Veh. Technol., vol. 68, no. 4, pp. 3747–3754,
rections,” IEEE Signal Process. Mag., vol. 36, no. 5, pp. 20–31, Sep. 2019. Apr. 2019.
[5] G. Hakobyan and B. Yang, “High-performance automotive radar: A review [31] M. Alhumaidi and M. Wintermantel, “Interference avoidance and mitiga-
of signal processing algorithms and modulation schemes,” IEEE Signal tion in automotive radar, in Proc. 17th Eur. Radar Conf., 2020, pp. 172–
Process. Mag., vol. 36, no. 5, pp. 32–44, Sep. 2019. 175.
[6] S. Alland, W. Stark, M. Ali, and M. Hedge, “Interference in automotive [32] J. Bechter, F. Roos, M. Rahman, and C. Waldschmidt, “Automotive radar
radar systems: Characteristics, mitigation techniques, and current and interference mitigation using a sparse sampling approach,” in Proc. Euro-
future research,” IEEE Signal Process. Mag., vol. 36, no. 5, pp. 45–59, pean Radar Conference, 2017, pp. 90–93.
Sep. 2019. [33] M. Wintermantel and H. Goelz, “Radar system for detecting the surround-
[7] M. Gerstmair, A. Melzer, A. Onic, and M. Huemer, “On the safe road ings with compensation of interfering signals,” U.S. Patent US8203481B2,
toward autonomous driving: Phase noise monitoring in radar sensors for Jun. 19, 2012.
functional safety compliance,” IEEE Signal Process. Mag., vol. 36, no. 5, [34] M. Wintermantel, “Radar system having arrangements and methods for the
pp. 60–70, Sep. 2019. decoupling of transmitting and receiving signals and for the suppression
[8] S. Saponara, M. Greco, and F. Gini, “Radar-on-chip/in-package in au- of interference radiation,” U.S. Patent US9182476B2, Nov. 10, 2010.
tonomous driving vehicles and intelligent transport systems: Opportunities [35] J. Bechter, C. Sippel and C. Waldschmidt, “Bats-inspired frequency hop-
and challenges,” IEEE Signal Process. Mag., vol. 36, no. 5, pp. 71–84, ping for mitigation of interference between automotive radars,” in Proc.
Sep. 2019. IEEE MTT-S Int. Conf. Microw. Intell. Mobility, 2016, pp. 1–4.
[9] C. Aydogdu et al., “Radar interference mitigation for automated driving: [36] P. Stoica and R. Moses, Spectral Analysis of Signals. Englewood Cliffs,
Exploring proactive strategies,” IEEE Signal Process. Mag., vol. 37, no. 4, NJ, USA:Prentice-Hall, 2005.
pp. 72–84, Jul. 2020. [37] M. Wintermantel, “Radar system with interleaved serial transmission and
[10] S. Sun, A. Petropulu, and H. V. Poor, “MIMO radar for advanced driver- parallel reception,” European Patent 3377915B1, Sep. 2019.
assistance systems and autonomous driving: Advantages and challenges,” [38] F. Engels, M. Wintermantel, and P. Heidenreich, “Automotive MIMO radar
IEEE Signal Process. Mag., vol. 37, no. 4, pp. 98–117, Jul. 2020. angle estimation in the presence of multipath,” in Proc. Eur. Radar Conf.,
[11] J. Dickmann et al., “Radar contribution to highly automated driving,” in 2017, pp. 82–85.
Proc. 11th Eur. Radar Conf., 2014, pp. 412–415. [39] R. Schubert, E. Richter, and G. Wanielik, “Comparison and evaluation of
[12] I. Bilik et al., “Automotive multi-mode cascaded radar data processing advanced motion models for vehicle tracking,” in Proc. 11th Int. Conf. Inf.
embedded system,” in Proc. IEEE Radar Conf., 2018, pp. 0372–0376. Fusion, 2008, pp. 1–6.
[13] S. Brisken, F. Ruf, and F. Höhne, “Recent evolution of automotive imaging [40] K. Werber et al., “Automotive radar gridmap representations,” in Proc.
radar and its information content,” IET Radar, Sonar Navigation, vol. 12, IEEE MTT-S Int. Conf. Microw. Intell. Mobility, 2015, pp. 1–4.
no. 10, pp. 1078–1081, Oct. 2018. [41] R. Prophet, H. Stark, M. Hoffmann, C. Sturm, and M. Vossiek, “Adaptions
[14] M. Stolz, M. Wolf, F. Meinl, M. Kunert, and W. Menzel, “A new antenna for automotive radar based occupancy gridmaps,” in Proc. IEEE MTT-S
array and signal processing concept for an automotive 4D radar,” in Proc. Int. Conf. Microw. Intell. Mobility, 2018, pp. 1–4.
15th Eur. Radar Conf., 2018, pp. 63–66. [42] M. Li, Z. Feng, M. Stolz, M. Kunert, R. Henze, and F. Küçükay, “High
[15] E. Schubert, F. Meinl, M. Kunert, and W. Menzel, “High resolution resolution radar-based occupancy grid mapping and free space detec-
automotive radar measurements of vulnerable road users - pedestrians tion,” in Proc. 4th Int. Conf. Veh. Technol. Intell. Transp. Syst., 2018,
and cyclists,” in Proc. IEEE Int. Conf. Microw. Intell. Mobility, 2015, pp. 70–81.
pp. 1–4. [43] F. Folster and H. Rohling, “Data association and tracking for automo-
[16] R. Prophet, M. Hoffmann, A. Ossowska, W. Malik, C. Sturm, and M. tive radar networks,” IEEE Trans. Intell. Transp. Syst., vol. 6, no. 4
Vossiek, “Image-based pedestrian classification for 79 GHz automotive pp. 370–377, Dec. 2005.
radar,” in Proc. 15th Eur. Radar Conf., 2018, pp. 75–78. [44] Y. Bar-Shalom, X.-R. Li, and T. Kirubarajan, Estimation with Applications
[17] O. Schumann, M. Hahn, J. Dickmann, and C. Wöhler, “Semantic segmen- to Tracking and Navigation: Theory Algorthims and Software. New York,
tation on radar point clouds,” in Proc. 21st Int. Conf. Inf. Fusion, 2018, NY, USA: Wiley-Interscience, 2002.
pp. 2179–2186. [45] D. Kellner, M. Barjenbruch, K. Dietmayer, J. Klappstein, and J. Dickmann,
[18] J. Lombacher, K. Laudt, M. Hahn, J. Dickmann, and C. Wöhler, “Semantic “Joint radar alignment and odometry calibration,” in Proc. Int. Conf. Inf.
radar grids,” in Proc. IEEE Intell. Veh. Symp. (IV), 2017, pp. 1170–1175. Fusion, 2015, pp. 366–374.
[19] O. Schumann, J. Lombacher, M. Hahn, C. Wöhler, and J. Dickmann, [46] R. Prophet, J. Martinez, J.-C. Michel, R. Ebelt, I. Weber, and M. Vossiek,
“Scene understanding with automotive radar,” IEEE Trans. Intell. Veh., “Instantaneous ghost detection identification in automotive scenarios,” in
vol. 5, no. 2, pp. 188–203, Jun. 2020. Proc. IEEE Radar Conf., 2019, pp. 1–6.
[20] A. Eskandarian, Handbook of Intelligent Vehicles. London, U.K.: Springer, [47] N. Levanon and E. Mozeson, Radar Signals. Hoboken, NJ, USA: Wiley,
2012. 2004.
[21] H. Winner, S. Hakuli, F. Lotz, and C. Singer, Handbook of Driver Assis- [48] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving?
tance Systems. New York City, NY, USA: Springer, 2016. The KITTI vision benchmark suite,” in Proc. IEEE Conf. Comput. Vis.
[22] M. Richards, J. Scheer, and W. Holm, Principles of Modern Radar: Basic Pattern Recognit., 2012, pp. 3354–3361.
Principles. Raleigh, NC, USA: SciTech Publishing, 2010. [49] H. Caesar et al., nuScenes: A multimodal dataset for autonomous driving,”
[23] Texas Instruments, “AWR2243 single-chip 76- to 81-GHz FMCW in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2020, pp. 11618–
transceiver,” 2020. [Online]. Available: https://www.ti.com/lit/gpn/ 11628.
AWR2243 [50] M. Meyer and G. Kuschk, “Automotive radar dataset for deep learn-
[24] NXP, “TEF82xx: Fully integrated 77 GHz RFCMOS automotive radar ing based 3D object detection,” in Proc. Eur. Radar Conf., 2019,
transceiver,” 2020. [Online]. Available: http://www.nxp.com/TEF82xx pp. 129–132.
[25] Z. Xu, C. J. Baker and S. Pooni, “Range and Doppler cell migration in [51] D. Barnes, M. Gadd, P. Murcutt, P. Newman, and I. Posner, “The Oxford
wideband automotive radar,” IEEE Trans. Veh. Technol., vol. 68, no. 6, radar RobotCar dataset: A radar extension to the Oxford RobotCar dataset,”
pp. 5527–5536, Jun. 2019. in Proc. Int. Conf. Robot. Automat., 2020, pp. 6433–6438.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 ENGELS et al.: AUTOMOTIVE RADAR SIGNAL PROCESSING: RESEARCH DIRECTIONS AND PRACTICAL CHALLENGES 877

[52] N. Scheiner, O. Schumann, F. Kraus, N. Appenrodt, J. Dickmann, and B. [75] R. Nabati and H. Qi, “RRPN: Radar region proposal network for object de-
Sick, “Off-the-shelf sensor vs. experimental radar - How much resolution tection in autonomous vehicles,” in Proc. IEEE Int. Conf. Image Process.,
is necessary in automotive radar classification?” in Proc. 23rd Int. Conf. 2019, pp. 3093–3097.
Inf. Fusion, 2020, pp. 1–8. [76] V. John and S. Mita, “RVNet: Deep sensor fusion of monocular camera and
[53] M. Meyer and G. Kuschk, “Deep learning based 3D object detection radar for image-based obstacle detection in challenging environments,” in
for automotive radar and camera,” in Proc. 16th Eur. Radar, 2019, Proc. Pacific-Rim Symp. Image Video Technol., 2019, pp. 351–364.
pp. 133–136. [77] V. John, M. Nithilan, S. Mita, H. Tehrani, R. Sudheesh, and P. Lalu, “SO-
[54] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for Net: Joint semantic segmentation and obstacle detection using deep fusion
semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, of monocular camera and radar,” in Proc. Pacific-Rim Symp. Image Video
no. 4, pp. 640–651, Apr. 2017. Technol., 2019, pp. 138–148.
[55] T. Giese, J. Klappstein, J. Dickmann, and C. Wöhler, “Road course [78] S. Chadwick, W. Maddern, and P. Newman, “Distant vehicle detection
estimation using deep learning on radar data,” in Proc. Int. Radar Symp., using radar and vision,” in Proc. Int. Conf. Robot. Automat., 2019,
2017, pp. 1–7. pp. 8311–8317.
[56] L. Sless, G. Cohen, B. Shlomo, and S. Oron, “Road scene understanding [79] F. Nobis, M. Geisslinger, M. Weber, J. Betz, and M. Lienkamp, “A deep
by occupancy grid learning from sparse radar clusters using semantic learning-based radar and camera sensor fusion architecture for object
segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. Workshop, detection,” in Proc. Sensor Data Fusion: Trends, Solutions, Appl., 2019,
2019, pp. 867–875. pp. 1–7.
[57] Z. Feng, M. Li, M. Stolz, M. Kunert, and W. Wiesbeck, “Lane detection [80] S. Chang et al., “Spatial attention fusion for obstacle detection us-
with a high-resolution automotive radar by introducing a new type of road ing mmwave radar and vision sensor,” Sensors, vol. 20, no. 4, 2020,
marking,” IEEE Trans. Intell. Transp. Syst., vol. 20, no. 7, pp. 2430–2447, Art. no. 956.
Jul. 2019. [81] T.-Y. Lim et al., “Radar and camera early fusion for vehicle detection
[58] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density-based algorithm in advanced driver assistance systems,” Workshop Mach. Learn. Auton.
for discovering clusters in large spatial databases with noise,” in Proc. Int. Driving 33rd Conf. Neural Inf. Process. Syst., 2019.
Conf. Knowl. Discov. Data Mining, 1996, pp. 226–231. [82] J. Ku, M. Mozifian, J. Lee, A. Harakeh, and S. Waslander, “Joint 3D
[59] E. Schubert, F. Meinl, M. Kunert, and W. Menzel, “Clustering of high proposal generation and object detection from view aggregation,” in Proc.
resolution automotive radar detections and subsequent feature extraction IEEE/RSJ Int. Conf. Intell. Robots Syst., 2018, pp. 1–8.
for classification of road users,” in Proc. 16th Int. Radar Symp., 2015, [83] M. Kunert, “The EU project MOSARIM: A general overview of project
pp. 174–179. objectives and conducted work,” in Proc. 9th Eur. Radar Conf., 2012,
[60] R. Prophet et al., “Pedestrian classification with a 79GHz automotive radar pp. 1–5.
sensor,” in Proc. 19th Int. Radar Symp., 2018, pp. 1–6. [84] H. van Trees, Detection, Estimation, and Modulation Theory—Part 4
[61] J. Garcia, R. Prophet, J.-C. Michel, R. Ebelt, M. Vossiek, and I. Weber, Optimum Array Processing. Hoboken, NJ, USA: Wiley, 2002.
“Identification of ghost moving detections in automotive scenarios with
deep learning,” in Proc. IEEE MTT-S Int. Conf. Microw. Intell. Mobility,
2019, pp. 1–4.
[62] O. Schumann, C. Wöhler, M. Hahn, and J. Dickmann, “Comparison of
random forest and long short-term memory network performances in
classification tasks using radar,” in Proc. Sensor Data Fusion: Trends,
Solutions, Appl., 2017, pp. 1–6.
[63] N. Scheiner, N. Appenrodt, J. Dickmann, and B. Sick, “Radar-based Florian Engels (Member, IEEE) received the M.Sc.
feature design and multiclass classification for road user recognition,” in and Dr.-Ing. degrees in electrical engineering from
Proc. IEEE Intell. Veh. Symp. (IV), 2018, pp. 779–786. Universität Karlsruhe (currently Karlsruhe Institute
[64] N. Scheiner, N. Appenrodt, J. Dickmann, and B. Sick, “A multi-stage of Technology), Karlsruhe, Germany, in 2008 and
clustering framework for automotive radar data,” in Proc. IEEE Intell. 2016, respectively. From 2008 to 2017, he was with
Transp. Syst. Conf., 2019, pp. 2060–2067. Continental, Automotive Distance Control Systems
[65] C. Qi, L. Yi, H. Su, and L. Guibas, “PointNet++ : Deep hierarchical feature GmbH, Germany, as a Radar Algorithm Develop-
learning on point sets in a metric space,” in Proc. 31st Conf. Neural Inf. ment Engineer and Technical Expert for radar signal
Process. Syst., 2017, pp. 5099–5108. processing. From 2017 to 2019, he was with ZF
[66] A. Danzer, T. Griebel, M. Bach, and K. Dietmayer, “2D car detection in Friedrichshafen AG, Germany, as a Lead Engineer
radar data with PointNets,” in Proc. IEEE Intell. Transp. Syst. Conf., 2019, for automotive imaging radar technology. Since 2019,
pp. 61–66. he has been the Head of Future Radar Algorithm Group, Continental, Business
[67] A. Palffy, J. Dong, J. Kooij, and D. Gavrila, “CNN based road user Unit Advanced Driver Assistance Systems, Germany. His research focuses on
detection using the 3D radar cube,” IEEE Robot. Automat. Lett., vol. 5, statistical signal processing with applications to automotive radar.
no. 2, pp. 1263–1270, Apr. 2020.
[68] J. Ziegler, P. Bender, M. Schreiber, H. Lategahn, T. Strauss, C. Stiller et al.,
“Making bertha drive - an autonomous journey on a historic route,” IEEE
Intell. Transp. Syst. Mag., vol. 6, no. 2, pp. 8–20, Summer 2014.
[69] D. Feng et al., “Deep multi-modal object detection and semantic
segmentation for autonomous driving: Datasets, methods, and chal-
lenges,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 3, pp. 1341–1360, Philipp Heidenreich (Senior Member, IEEE) re-
Mar. 2020. ceived the Dipl.-Ing. and Dr.-Ing. degrees in electrical
[70] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies engineering from Technische Universität Darmstadt,
for accurate object detection and semantic segmentation,” in Proc. IEEE Darmstadt, Germany, in 2006 and 2012, respectively.
Conf. Comput. Vis. Pattern Recognit., 2014, pp. 580–587. From 2007 to 2012, he was with the Signal Process-
[71] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time ing Group, Technische Universität Darmstadt, as a
object detection with region proposal networks,” in Proc. 28th Conf. Neural Research Scientist. From 2012 to 2015, he was with
Inf. Process. Syst., 2015, pp. 91–99. Continental, Automotive Distance Control Systems
[72] W. Liu et al., “SSD: Single shot multibox detector,” in Proc. Eur. Conf. GmbH, Germany, as a Radar Algorithm Development
Comput. Vis., 2016, pp. 21–37. Engineer. Since 2015, he has been with Opel Au-
[73] T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal loss for dense tomobile GmbH (currently subsidiary of Stellantis),
object detection,” in Proc. IEEE Int. Conf. Comput. Vis., 2017, pp. 2999– Germany, as a Systems Engineer of automotive active safety applications and
3007. advanced technology. In 2020, he became a Technical Specialist of advanced
[74] F. Gaisser and P. P. Jonker, “Road user detection with convolutional neural driver assistance systems, data fusion and localization. His research interests
networks: An application to the autonomous shuttle WEPOD,” in Proc. include statistical and array signal processing with applications to automotive
IEEE Int. Conf. Mach. Vis. Appl., 2017, pp. 101–104. radar. From 2020 to 2021, he was the Guest Editor of the IEEE JOURNAL OF
SELECTED TOPICS IN SIGNAL PROCESSING.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.
 878 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 15, NO. 4, JUNE 2021

Markus Wintermantel received the Dipl.-Ing. and Abdelhak M. Zoubir (Fellow, IEEE) has been a
Dr.-Ing. degrees in electrical engineering from Uni- Professor of signal processing and the Head of Signal
versität Stuttgart, Stuttgart, Germany, in 1992 and Processing Group with Technische Universität Darm-
2000, respectively. From 1992 to 2000, he was with stadt, Darmstadt, Germany, since 2003. He authored
the Institute of Network and System Theory, Univer- or coauthored more than 400 journal and conference
sität Stuttgart, as a Research Scientist. Since 2000, he papers on the above areas. His research interest fo-
has been with Continental, Business Unit Advanced cuses on statistical methods for signal processing with
Driver Assistance Systems, Germany. He is currently emphasis on bootstrap techniques, robust detection
the Principal Expert for automotive radar and LiDAR and estimation and array processing applied to radar,
sensors. He holds more than 20 patents in the area of sonar, telecommunications, automotive monitoring
automotive radar applications. and safety, and biomedicine. He was the General
Chair and the Technical Chair of numerous international IEEE conferences and
workshops, most notably as Technical Chair of ICASSP-14. From 2012 to 2014,
he was also on publication boards or as the Guest Editor of various journals,
Lukas Stäcker (Student Member, IEEE) received
notably as the Editor-in-Chief of the IEEE Signal Processing Magazine. From
the M.Sc. degree in industrial engineering and com-
2010 to 2011, he was the Chair of the IEEE Signal Processing Society (SPS)
putational engineering from Technische Universität
Technical Committee Signal Processing Theory and Methods and from 2015 to
Darmstadt, Darmstadt, Germany, in 2019. Since 2017, was on the Board of Governors of the IEEE SPS as a Member-at-Large.
2020, he has been with Opel Automobile GmbH
From 2017 to 2018, he was the President of the European Association of Signal
(currently subsidiary of Stellantis), Germany, where
Processing. From 2010 to 2011, he is a IEEE Distinguished Lecturer. He was
he is currently working towards the Ph.D. degree. He
the recipient of a various best paper awards and the 2018 IEEE Leo L. Beranek
is also jointly affiliated with the German Research
Meritorious Service Award.
Center for Artificial Intelligence, Kaiserslautern, Ger-
many. His research focuses on automotive radar-
camera sensor fusion.

Muhammed Al Kadi received the Diploma degree

from the Higher Institute for Applied Science and
Technology (HIAST), Damascus, Syria, in 2007, the
M.Sc. degree from Technische Universität Darmstadt,
Darmstadt, Germany, in 2012, and the Dr.-Ing. de-
gree in electrical engineering from Ruhr-Universität
Bochum, Bochum, Germany. From 2007 to 2010 and
from 2012 to 2017, he was a Research Scientist with
HIAST and Ruhr-Universität Bochum. From 2017
to 2019, he was with Hella and ZF Friedrichshafen
AG, Friedrichshafen, Germany, developing automo-
tive radar. He is currently with Continental, Business Unit Advanced Driver
Assistance Systems, Germany, as a Signal Processing Engineer for automotive
radar.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on June 12,2023 at 13:18:50 UTC from IEEE Xplore. Restrictions apply.