See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/364332501

Point cloud-based optimization of roadside LiDAR placement at constructed

highways

Article  in  Automation in Construction · October 2022

DOI: 10.1016/j.autcon.2022.104629

CITATIONS READS

0 312

5 authors, including:

Yang Ma Yubing Zheng

Hefei University of Technology Hefei University of Technology
47 PUBLICATIONS   232 CITATIONS    26 PUBLICATIONS   177 CITATIONS   

SEE PROFILE SEE PROFILE

Shuyi Wang Yiik Diew Wong

Fuzhou University Nanyang Technological University
35 PUBLICATIONS   142 CITATIONS    238 PUBLICATIONS   5,537 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Innovations in Highway Geometric Design Research View project

Traffic Operation and Behaviour Research View project

All content following this page was uploaded by Yang Ma on 15 October 2022.

The user has requested enhancement of the downloaded file.

 Point cloud-based optimization of roadside LiDAR placement at constructed
highways

Yang Ma a,b, Yubing Zheng a,*, Shuyi Wang c, Yiik Diew Wong b, Said M. Easa d

*Corresponding Author, Y.B. Zheng: [email protected]

a School of Automotive and Transportation Engineering, Hefei University of Technology, P.R. China,
230009
b School of Civil and Environmental Engineering, Nanyang Technological University, Singapore, 639798
c School of Transportation, Southeast University, P.R. China, 211189
d Department of Civil Engineering, Toronto Metropolitan University, Toronto, ON, Canada M5B 2K3.
Y. Ma: [email protected]; Y. D. Wong: [email protected]; S. M. Easa: [email protected]

This is a manuscript accepted for publication in Automation in Construction

Highlights
 Introduce point cloud data as road background for optimization of sensors’ placement
 Propose physics-based and deep neural network-based virtual sensor models
 Create a detection matrix to simplify the optimization problem
 Test the feasibility of a neural network as a sensor model

Abstract
Current approaches for optimizing the placement of roadside LiDAR (RSL) at constructed highways
work on handcrafted scenes which fail to precisely map real-world situations. This study proposes
a computer-aided framework to address the issue. First, high-accuracy point cloud data are
introduced to model the as-built highway infrastructures, based on which an unsupervised clustering
approach is applied to segment the target monitoring area (TMA). Then, candidate RSL locations
are generated in a semi-automated manner combining manual delineation and spline resampling.
Next, new deterministic and a U-net-based LiDAR models are separately developed to virtually
estimate candidate RSL’s joint coverage. Finally, based on the proposed sensor models, a detection
matrix is created to facilitate the application of binary integer programming that minimizes the
number of RSL while ensuring complete coverage of TMA. The tests on point cloud data of the
three different sites demonstrate the effectiveness of the proposed workflow.

Keywords: Sensor placement; optimization; roadside LiDAR; point cloud data; deep learning

Abbreviations
AV, autonomous vehicles;
BIP, binary integer programming;
FOV, field of view;
FPFH, fast point feature histogram;
KNN, k-nearest neighbors;
LiDAR, light detection and ranging;
NCS, natural cubic splines;
1
RMSE, root-means square error;
RSL, roadside LiDAR;
RSUs, roadside units;
TMA, target monitoring area;
VLM, virtual LiDAR model;
2D, two-dimensional;
3D, three-dimensional

1 Introduction
1.1 Background

Autonomous vehicles (AV) are increasingly being deployed across the world. With various on-board
sensors and smart algorithms, AV are expected to avoid human errors and respond quicker to
incidents and emergencies, thus improving driving safety. However, it has been recognized that the
on-board perception capability of AV is insufficient to ensure safety in some road scenes (e.g.,
curved roads) where severe occlusion issues are present [1]. Deploying roadside sensors to enhance
the sensing capability of the infrastructure is a well-accepted solution. Roadside units (RSUs) can
be incorporated to eliminate blind spots and achieve seamless coverage of the traffic monitoring
area.
Surveillance cameras, millimeter-wave radar, and light detection and ranging (LiDAR) devices
are three primary roadside sensors for monitoring road traffic. Over the past ten years, due to the
strides in the development of computer vision technologies, roadside surveillance cameras have
played an ever-increasingly important role in multiple road user tracking and recognition,
automated diagnosis of traffic violation incidents, etc. However, video data collected by cameras
are two-dimensional (2D) in nature. In this regard, it is difficult to stitch video flows captured by
different cameras to continuously monitor three-dimensional (3D) road space [2]. Besides, video
cameras may not function well in poor lighting conditions.
Millimeter-wave radars can sense objects at a long distance, and are less susceptible to adverse
weather. Multiple radars can be easily integrated to create a full coverage of the monitoring road
area. Nonetheless, the low resolution of millimeter-wave radars adversely affects their performance
in identifying small and static objects. In addition, the millimeter-wave radars are susceptible to
reflection interference in tunnel-like areas.
In comparison, although LiDAR sensors may suffer accuracy reduction in adverse weather,
they are much more accurate than radars. LiDAR sensors have a wider horizontal field of view
(FOV) than either video cameras or millimeter-wave radars. In this regard, a single LiDAR sensor
can have more coverage of the monitoring area than its counterparts. Compared with high-
resolution video cameras, LiDAR sensors do not capture sharp images of drivers, which will cause
fewer privacy concerns. Besides, similar to radar sensors, many LiDAR sensors can cooperate to
monitor road traffic jointly. At some crucial road infrastructures such as curved roads and
intersections, roadside LiDAR (RSL) is the pillar that augments the infrastructure’s sensing
capability [3]. Therefore, the present study gives special attention to the placement of RSL as RSUs.

1.2 Problem Context

Similar to human eyes, RSL cannot see objects in the occluded areas (see Appendix A). The
procurement and installation of RSL may also incur high costs. Considering a limited budget, it is
necessary to place RSL optimally to yield the maximum engineering benefits. Besides, the optimal
2
placement of RSL can better ensure RSL’s monitoring function. Optimizing the placement of RSUs
is an active and ongoing research domain, where existing studies can be generally categorized into
two types: macroscopic and microscopic. The problems regarding macroscopic optimization of RSUs
have been extensively studied. This cluster of studies is mainly conducted at a road network level,
which usually aims to improve the accuracy of collecting or inferring traffic flow information (e.g.,
origin-destination demand estimation) on a large scale through optimizing the deployment of RSUs
[4-7]. The thrust of these studies lies in constructing more accurate mathematical models that better
describe the optimization problem, where sensor models are significantly simplified. The microscopic
studies focus on optimizing the placement of RSUs at small-scale road infrastructures, which aim
to achieve full coverage of the monitoring area by optimizing RSUs’ locations and numbers [8,9].
The relatively high prices of RSL make them unrealistic to be deployed on a large scale. Therefore,
this study falls in the domain of microscopic optimization of RSUs that optimizes RSL’s placement
at highway segments or intersections.
There are two categories of optimization problems regarding sensors’ deployment [10]. The first
category aims to maximize the data quality (e.g., coverage) for a fixed number of sensors (called
FIX), while the second category aims to minimize the number of sensors needed to cover the entire
region of interest (called MIN). Note that this study addresses the MIN problem.

1.3 Related Work

Road environment, targets and sensor model are three essential components of optimizing RSU on
a small scale. Optimizing the placement of RSU is conventionally known as Art Gallery Problem.
This problem aims to minimize the number of security personnel while ensuring that they can
jointly see the Gallery without any blind spots [11]. In the early stage, researchers focused on
placement of roadside surveillance cameras. Due to limited computational power, road environments
were primitively represented by simple geometry on the 2D plane, a number of polygons delineated
the targets, and the camera sensors’ FOV was simplified as fan-shapes or triangles. Then, the main
goal is to optimize cameras’ locations to ensure that the target area is seamlessly covered by the
sensors’ FOV [12-14]. Afterwards, owing to the improvement of computational capacity, researchers
paid increasing attention to optimizing cameras’ placement in 3D space [15]. Altahir et al. [16]
proposed a 2.5D sensor model that projected a sensor’s FOV onto the horizontal plane. The dynamic
programming method was applied to maximize the fixed camera sensors’ joint coverage of a planar
area. Zhang et al. [17] proposed a 3D camera sensor model that considers the occlusion issue
presented by pillars and walls at a construction site. Setting a 100% coverage of the construction
zone as the objective, they optimized the placement of surveillance cameras using genetic algorithms.
Malhotra1 et al. [15] proposed a ray-casting camera model that considers environmental occlusions
and projected cameras’ view on a 3D voxel map to compute the coverage score. The mixed integer
programming method was applied to minimize the difference between the desired and achieved
coverage. Du et al. [18] considered road vehicles as targets and proposed an occlusion degree model
to describe dynamic occlusions caused by vehicle bodies. Using the occlusion degree model, a
multifactor regression model was then developed to explain how different traffic conditions and
placement schemes affect the performance of the RSU. The optimization models were established
on the multifactor regression model.

3
 (a) (b)
Fig. 1 Difference between video cameras and LiDAR sensors (a) video cameras, (b) LiDAR
Although RSL are an important part of the cooperative vehicle-infrastructure system, their
optimal placement has not received adequate attention from researchers. [18] claimed that RSL as
RSUs were also considered in their study. However, they used the same sensor model for either
surveillance cameras or LiDAR, which created a frustum-like FOV to emulate sensors’ detection of
objects. Nonetheless, the data collection mechanism of LiDAR is quite different from that of cameras
(Fig.1). A LiDAR emits laser pulses following a particular pattern and depicts its 3D surroundings
based on returned signals. Data collected by a LiDAR are organized as discrete point clouds [19].
In this case, even when an object falls within a LiDAR’s FOV without any occlusion, it is possible
that the object cannot be detected because of insufficient returned laser points [20].
Very recently, Vijay et al. [3] noticed the difference between LiDAR and camera sensors. They
proposed a Ray casting-based sensor model for RSL, and applied integer programming to optimize
RSL’s placement towards a 100% coverage of the intersection area. The road environments and
sensor model were all 3D in [3]. However, similar to previous studies, the road scenes in their study
were manually created. More specifically, the authors handcrafted virtual road environments using
MATLAB RoadRunner and exported road models to the PreScan software. However, real-world
road environments are very complex. It is very time-consuming to construct road scenarios which
precisely represent real-world situations, and the costs of handcrafting and assembling road objects
in a virtual environment are very high [21]. The substantial difference between real-world and
virtual road environments may bring significant inaccuracies when optimizing RSL’s placement,
which is why the studies in the AV’s domain strive to create a high-fidelity 3D road scene for
virtual testing.
High density point cloud data enable an accurate and precise depiction of constructed road
infrastructures. Compared with handcrafted models, point clouds data are much easier to update,
which are one of the fundamental data sources supporting the digitalization of transportation
infrastructures [22]. Point cloud data can be collected by depth cameras, multi-view oblique
photogrammetry, and LiDAR devices for surveying purposes. Note that the point cloud data in this
study only refer to those collected by high-accuracy LiDAR devices. Considering that the point
cloud data create a digital twin of the real-world scenes, they have been used as background for
optimizing placement of some wireless transmission sensors [23]. Despite the significant potential of
point cloud data in modeling as-built road infrastructures, they have not been successfully
introduced into the area of optimizing RSL’s placement.
Several studies in the AV safety domain have noted the value of point clouds as a digital twin
of real-world scenes. Researchers have created virtual LiDAR models superimposed on AV to

4
generate synthetic LiDAR scans. More specifically, [21] and [24] proposed cube map-based
projection and ray-casting methods, respectively, to compute intersections between virtual laser
beams and point cloud-modelled objects. However, both methods require rendering point cloud data
first, which substantially adds to the computational costs of emulating LiDAR sensors. Besides, the
rendering operation may need graphic engines, which complexify the sensor models’ real-world
applications. A sensor model that directly works in point clouds without rendering is currently
lacking.

1.4 Objectives and Contributions

To fill in the gaps mentioned above, the main objective of this study is to introduce point cloud
data of real-world road infrastructures into the optimization of RSL’s placement on a small scale.
To reduce manual intervention, this study mainly focuses on highway scenarios where road obstacles
are less complicated than urban streets. The detailed objectives are threefold:
 Propose a semi-automated framework for optimizing RSL’ placement at constructed highway
infrastructures that uses point clouds as background,
 Develop a deterministic LiDAR sensor model that directly uses point clouds as input, and
 As deep neural networks have achieved great success in many engineering areas, this study
also introduces a deep neural network as a sensor model, and thereby test its feasibility.

2 Methodology
2.1 Overview

The proposed framework consists of five main steps as shown in Fig. 2. In Step 1, based on raw
point cloud data of as-built highway infrastructures, a clustering-based approach is applied to
segment paved road surface which serves as the target monitoring area (TMA) in this study. In
Step 2, road edges are manually delineated on the segmented road surface, along which candidate
RSL locations are generated automatically. An array of targets is generated within TMA in Step 3.
Then, two virtual sensor models of RSL that work in point clouds are proposed in Step 4, and they
are placed at each candidate RSL location to examine their joint coverage of targets. Finally, in
Step 5, a detection matrix is constructed that helps convert the optimization issue into a linear
programming problem, and the binary integer programming (BIP) method is applied to obtain the
optimal placement of RSL that minimizes the overall number of RSL. Each step is described in
detail next.

Fig. 2 Proposed framework

5
2.2 Segmentation of TMA

This study focuses on highway scenes, as previously mentioned. Compared with urban streets,
traffic flow on highways is less heavy, which means dynamic occlusion issue caused by vehicles is
less severe in highway scenes. For simplicity, this study emphasizes static road objects as sensing
obstacles for RSL. Regarding highway infrastructures, the paved road surface denotes the main
areas where road users may appear. Therefore, the pavement region is viewed as the TMA in this
study. The workflow for segmenting TMA from raw point cloud data is graphically presented in
Fig. 3.

Fig. 3 Workflow of segmenting TMA from point clouds

Let - - be the coordinate system of the geodetic space. On the - plane, point clouds of
highway infrastructures are divided into numerous grid cells. The length and width of each grid cell
are and , respectively. In this way, raw point clouds are partitioned into many pillars as shown
in Fig.4 in the - - space. In each pillar, only the points whose offsets from the lowest point along
the -axis are within ℎ are retained while others are temporarily discarded. During the pillar-wise
process, most off-ground points are filtered. However, at some positions where ground points are
unexpectedly missing, the remaining points are not real ground points (see Fig.4) which requires
further processing.
The ground is continuous in the real-world, whereas those non-ground points are disconnected
from the main part of ground points. Using the method proposed in [25], the remaining points after
pillar-wise filtering are re-divided into many voxel cells. Each voxel is long, wide, and high.
The connected voxels can be efficiently identified using well-established algorithms [26]. Based on
the connectivity among voxels, those points in the isolated voxels can be removed and the refined
ground points are thus obtained.

(a)

(b)
Fig. 4 Segmentation of pavement points (a) segmentation of ground points, (b) extraction of road
surface points
6
 The roles of the identified ground points are twofold: (1) the extracted ground points can be
used to further segment the paved road surface points, (2) a digital elevation model can be
constructed on the extracted ground points, which is a prerequisite for generating candidate RSL
positions in the subsequent steps.
Unlike the rough and uneven ground points on the roadside, the pavement points are commonly
even and smooth. The substantial difference in the point features can help distinguish road surface
points from rough ground points. Specifically, as shown in Fig.4b, an unsupervised clustering
approach is used to segment pavement points automatically. A Kd-tree structure is constructed on
the ground points. K1 neighbors of each point are then obtained using the k-nearest neighbors
(KNN) searching algorithm. In this study, the Fast Point Feature Histogram (FPFH) was selected
as the local point feature descriptor. Specifically, based on a given point’s K1 neighbors and each
neighbor’s K1 neighbors, the FPFH computes 33 different angle features and develops them into a
histogram. Considering its robustness, the FPFH has been widely applied in point cloud registration
and object identification [27].
FPFH is estimated for each ground point. As such, the ground points are converted to an array
of FPFH feature vectors. Using the FPFH feature array as input, the K-Means clustering approach
is applied to classify ground points into two groups. Due to the inherent difference between
pavement points and rough ground points, the K-Means method can automatically identify the
feature vectors corresponding to the even points. As shown in Fig.4b, mapping the clustering result
into the geodetic space, the road surface area is well demarcated. To facilitate the subsequent
generation of candidate RSL positions, a Euclidean-distance based clustering approach is performed
to further group discrete road surface points into a single cluster. Based on the Kd-tree structure,
the Euclidean-distance based clustering approach classifies two points into the same group if their
distance is less than a predefined threshold . During this process, some isolated even points on
roadside can also be eliminated.
In addition to the proposed workflow shown in Fig.4, manual extraction of TMA from point
cloud data at small-scale as-built road infrastructures (e.g., highway intersections) is also acceptable.
Alternatively, users can train deep learning models to segment TMA from point clouds semantically.

2.3 Generation of Candidate RSL Positions

In this study, the candidate RSL are mainly placed alongside road edges. In theory, road edges can
be automatically delineated using algorithms such as Alpha-Shape based on the extracted pavement
points. However, the automatically generated road edges may not be in good accordance with real-
world situations [28]. Besides, some roadside areas are not allowed for sensors’ placement, making
it a more complex problem to delineate road boundaries for generating candidate RSL positions
automatically. Therefore, a semi-automated way of extracting road edges that combines manual
delineation and spline fitting is proposed in this study.
On the extracted road surface points, road edges are manually delineated in a piecewise manner
to reduce possible errors. Specifically, users may select points in order to trace the road edges.
Unlike automated algorithms, which may fail in some complex scenarios, humans can well
distinguish the road boundaries that separate pavement regions and roadside areas. Then, the
natural cubic splines (NCS) are used to fit those points piecewise, and Ma et al.’s approach [29] is
applied to resample denser points on the fitted NCS. The resampled points that are set ∆ apart
serve as the horizontal positions of candidate RSL. Note that a similar workflow can be adopted if
RSL are required to be placed at non-road edge locations.
7
 Fig. 5 Semi-automated generation of candidate RSL locations
For a given horizontal position of candidate RSL, multiple LiDAR sensors can be installed at
different heights when necessary (see Fig. 5). Suppose the upmost and lowermost allowable heights
of candidate RSL from the ground are ℎ and ℎ , respectively. Due to the presence of obstacles or
other roadside facilities, there may be no space for placing RSL at some heights. Therefore, a multi-
step procedure is proposed as follows for generating the heights for candidate RSL:
Step i: at a given horizontal position, search points within a radius of on the - plane.
Step ii: based on the digital elevation model of ground points (described in Section 2.2 above),
compute the height ℎ of each point within the radius of from the ground.
Step iii: sort all points within the range by increasing ℎ , and obtain the maximum height
ℎ and the minimum height ℎ .
Step iv: if there is no point within the radius of , generate a set of candidate RSL heights
from ℎ to ℎ at an interval of ∆ℎ;otherwise, there are four cases regarding the generation of
candidate RSL heights.
Case i: if ℎ ≤ ℎ and ℎ ≥ ℎ , starting from ℎ , compute the height difference between
two consecutive points after sorting. If there is a point pair whose height difference is larger than
∙ ∆ℎ( ∈ + , ≥ 2), the first height equals the lower height plus ∆ℎ/2. Repeat this process
until ℎ is reached.
Case ii: if ℎ ≤ ℎ and ℎ < ℎ , the process is similar to Case i until ℎ is reached. Then,
starting from ℎ + ∆ℎ/2, the candidate heights are generated at an interval of ∆ℎ until ℎ is
reached.
Case iii: if ℎ > ℎ and ℎ < ℎ , compute the height difference between two consecutive
points after sorting. If there is a point pair whose height difference is larger than ∙ ∆ℎ( ∈
+
, ≥ 2), the first height equals ℎ + ∆ℎ/2. Repeat this process until ℎ is reached. The
remaining steps remain unchanged as Case ii.
Case iv: if ℎ > ℎ and ℎ > ℎ , compute the height difference between two consecutive
points after sorting. If there is a point pair whose height difference is larger than ∙ ∆ℎ( ∈
+
, ≥ 2), the first height equals ℎ + ∆ℎ/2. Repeat this process until ℎ is reached. The
remaining steps remain unchanged as Case i.
In this study, the density of candidate RSL is adjustable by changing ∆ and ∆ℎ. Note that
either ∆ or ∆ℎ has a lower bound because LiDAR sensors cannot be placed too close.

8
2.4 Virtual Sensor Model

The virtual LiDAR model (VLM) plays a crucial role in optimizing the placement of RSL. Current
VLMs are mainly developed for simulation platforms (e.g., PreScan) powered by graphics engines.
However, it is very time-consuming and costly to handcraft road scenes that precisely map real-
world situations using simulation software. Therefore, a new VLM that directly works in point
cloud space is proposed in this section for optimizing RSL’ placement.

2.4.1 Deterministic Sensor Model

Let ( , , ) be the center of a RSL, where the virtual laser beams are emitted. The main
technical parameters of a RSL are graphically shown in Fig.6a, including the horizontal FOV ,
horizontal angular resolution , vertical FOV , vertical angular resolution , and detection
range . When a LiDAR sensor is in operation, it emits high-frequency laser pulses within its
FOV. Once a laser photon hits a non-transparent object, it will bounce back to the LiDAR sensor.
According to the time-of-flight information, the spatial distance can be computed between the
LiDAR sensor and the object. As the orientation information is further considered, the 3D
coordinate of the point on that object can be estimated.
Inspired by the working mechanism of LiDAR sensors, where the raw data collected by a
LiDAR sensor are distance and angle, a new VLM is developed as follows. A local coordinate frame
is constructed where the -axis is perpendicular to the road boundary, and the -axis is normal to
the geodetic plane, as shown in Fig.6b. Let be the points within a range of around
( , , ) in the - - space. The coordinates of in the local space originated at ( , , )
are estimated by:
cos sin 0
= −sin cos 0 ∙ − (1)
0 0 1
where = rotation angle around the -axis, it is positive in a counter-clockwise direction, other
notations remain unchanged.
The local Cartesian coordinates of are further transformed into spherical coordinates by
arctan( ∙ − )
⎡
= ⎢arctan( ∙ ( + )− )⎤ (2)
⎥
⎣ ( + + ) ⎦
where ( , , ) = spherical coordinates of

(a) (b)
Fig.6 Sensor model (a) main technical parameters, (b) coordinate transformation
9
 Then, a new Cartesian coordinate frame is constructed where , , and are its three
coordinate components. The new coordinate system makes it easier to emulate the data collection
mechanism of LiDAR sensors. Suppose the points are divided into many grid cells on the - plane.
The width and height of each grid cell are and , respectively. As such, each grid cell corresponds
to a virtual laser beam, where only the point with the smallest can be captured by the LiDAR
sensor. However, points on the - plane are not evenly spaced, as shown in Fig.7. Specifically,
the points closer to the RSL center (i.e., ( , , ) ) are sparser. In this case, the grid cells in the
sparse area may not have any point and return nothing, which may yield inaccurate detections for
virtual laser beams. To address this issue, a hybrid detection model is proposed in Algorithm 1.

Fig. 7 Sparse area on the - plane

Algorithm 1 Virtual detection model

Step 1: Project points onto the - plane
Step 2: Demarcate the FOV on the - plane using and . Partition the points within the
FOV into many grid cells whose width and height are and , respectively a. As such, the grid
center corresponds to the virtual laser beam. Create a cell matrix to store the - coordinates of
grid centers.
Step 3: Construct a null depth matrix. Each matrix element corresponds to a grid cell.
Step 4: If there are more than b points in the grid cell, the corresponding element in the depth

matrix equals the minimum -value in the grid cell; else, go to Step 5.
Step 5: Obtain K2 c neighbors around each grid center on the - plane using the KNN method.
Map the K2 neighbors into the - - space via point indices. Let ⃑ be the unit vector
indicating the direction of a virtual laser beam. Suppose the spherical coordinate of a neighbor
point is ( , , ) , its distance to ⃑ is estimated by
∙ cos ∙ cos
= ∙ cos ∙ sin (3)
∙ sin
= ‖ × ‖⃑ (4)
where = ( , , ) , ‖. ‖ = a function for obtaining the modulus, and × = cross product

Step 6: Remove the points whose -value exceeds d.

If the number of remaining points is less than 3, it is determined that the virtual LiDAR
sensor will not detect any point, and the corresponding element in the depth matrix equals 0;
else, go to Step 7.
Step 7: Sort the remaining points by increasing their -values. Only the closest three points to
the sensor are retained.
If the spatial distance between each pair of three points is less than , and the grid center
is enclosed by the triangle formed by the three points on the - plane, the corresponding depth
value is obtained by averaging three points’ -values; else, the depth value is 0.
Step 8: The algorithm terminates when all grid cells are processed.
Note: a an efficient algorithm proposed in [25] is applied at this step; b a user-defined threshold about
10
the number of points; c a user-defined number of the nearest neighbors on the - plane; d denotes the
average point spacing in the - - frame
Algorithm 1 can be generally divided into two parts. The first part (Steps 1 to 4) down-samples
the points on the - plane using a partitioning-based method [25] and generates a preliminary
depth map. However, point cloud data are discrete in nature and the point density may vary with
positions. Besides, due to the coordinate transformations, points on the - plane are also unevenly
spaced (see Fig.8). In this case, the first part of Algorithm 1 may not well handle objects on which
points are sparse. Therefore, the second part (Steps 5 to 7) of Algorithm 1 is proposed to perform
further processing. Specifically, Steps 5 to 7 identify the grid cells in the sparse area first. Then, for
each identified grid cell, the neighboring points around its corresponding virtual laser beams are
searched and evaluated. Next, if the distances between pairs of the closest three neighboring points
are all within a spacing threshold, it is determined that the virtual beam can capture a point, and
a depth value can thus be computed.
Before the application of Algorithm 1, some deep learning techniques are recommended to
refine point cloud data of road scenes in the future. For instance, some potential vehicle noises in
point clouds can be identified and removed. Also, some data gaps can be filled using deep neural
networks [30]. As such, it would be easier for Algorithm 1 to generate high-quality depth maps.

(a)

(b)
Fig. 8 Detection model (a) mechanism of collecting points, (b) example of application

11
 The outputs of VLM at different steps are displayed in Fig.8b. The proposed VLM can address
the discrete nature of point cloud and generate a continuous and smooth depth map of the road
environment (see rectangles in Fig.8b). Upon the acquisition of the depth map, as shown in Fig. 9,
the sensor output of RSL in the geodetic space can be generated through transformations from
spherical coordinates to Cartesian coordinates, a process inverse to Eqs. (1) and (2). Different
LiDAR products in the market can be simulated by adjusting the technical parameters.

Fig. 9 Example of sensor output

2.4.2 U-net based Sensor Model

Steps 5 to 7 in Algorithm 1 that perform KNN searching and spatial interpolation involve a large
number of computations, which reduces the efficiency of the proposed VLM. Based on the
preliminary depth map generated in Step 4, as shown in Fig.10a, an image-to-image regression
neural network is introduced to output the final depth map directly. The main goal of this step is
to investigate whether a fast and efficient neural network can help improve the efficiency of VLM
without compromising the accuracy of optimization. Note that the neural network part is optional.
The final depth maps can be generated without any neural networks. This study incorporates a 33-
layer U-net into the sensor model. The architecture of the U-net is shown in Fig.10b. The final
layer is a pixel-level 2D regression layer that outputs the refined depth map. This study empirically
determines the hyper-parameters of U-net (Filter size, number of filters, number of layers, etc.). As
illustrated in Fig.10, a lightweight neural network is used in this study to reduce the number of
parameters.

(a)

12
 (b)
Fig. 10 Introduction of U-net neural network: (a) process and (b) architecture

Creation of training datasets

Training datasets are created from 19 different point cloud datasets (see Fig.11a) of intersections,
roundabouts, divided and undivided highways, and urban streets. The point cloud data were all
collected by high-accuracy mobile mapping system or terrestrial laser scanning system in Nanjing,
P.R. China, Hefei, P.R. China and Singapore. In each point cloud dataset, the virtual LiDAR sensor
is placed at the randomly selected positions. The heights of the virtual sensor range from 4 m to 20
m. The VLM is then applied to generate the preliminary and final depth maps in pairs at each
sensor position. Specifically, at a given sensor position, Steps 1 to 4 in Algorithm 1 generate a
preliminary depth map using raw point clouds as input. Then, the remaining steps in Algorithm 1
output the refined depth map (see Fig.10a). The preliminary and final depth maps are the input
and output of the neural network, respectively.
The variable values of the sensor model at this phase are specified in Table 1. LiDAR products
in the market have different technical parameters. It is challenging to incorporate technical
parameters (especially angular resolutions) of RSL when creating the training set. Therefore, as
noted in Table 1, a wide FOV and fine angular resolutions are used to generate high-resolution
depth maps (size: 500-by-3600 pixels) in this study. K2 and are empirically determined as 10
and 5 in this case, respectively. As such, different LiDAR products can be simulated by processing
full-size depth maps. Specifically, the parameter values in Table 1 are adopted to generate the
preliminary depth maps at the deployment stage. Then, the trained neural network is applied to
refine the depth maps. According to the technical information of the LiDAR sensor to be modeled,
different image processing techniques, including cropping, down-sampling and resizing can be
applied to obtain the final depth map based on the refined depth maps.
Finally, a training dataset containing 604,789 training instances is created where each training
instance comprises an input depth map and a same-size response map in pair. A single training
instance is also shown in Fig. 11b.

13
 (a)

(b)
Fig. 11 Datasets: (a) point cloud datasets, (b) a training instance
Table 1 Variable values in the creation of training dataset
Variables /° /° /° /° /m K2
50 to 200 at an
Values [-180,180] [-25,25] 0.1 0.1 10 5
interval of 10

Training of U-net

The training dataset is split into three parts: training data, validation data and test data, whose
sizes are 423,353, 90,718, and 90,718, respectively. Due to the large input depth map size that
increases the difficulty of training, a random patch strategy is adopted to reduce the computational
burden in the training process. Specifically, during training epoch, multiple patches of specific size
are correspondingly extracted from the input and response depth maps. In this way, each epoch
actually uses a slightly different sub-dataset. All depth maps are Min-Max normalized when fed
into the network to avoid bias.
The patch size in this study is 56-by-56 pixels. The training (10 epochs) was conducted on a
computer (RAM of 32 GB, CPU of Intel ® Xeon ® E5-1650 [email protected], GPU of NVIDA Quadro
P5000) and took about 236 hours. After 6 million iterations, the final mean validation accuracy
reached a root-means square error (RMSE) of 2.53 (pixel level), which is acceptable regarding the
depth map prediction.
In this study, the trained U-net is deployed in two ways, as illustrated in Fig. 12. The first
way applies the trained U-net directly that performs an end-to-end regression. Differently, the depth
values outputted by the neural network are not used in the second way. Instead, an intermediate
binary mask is created by binarizing the depth map. Specifically, all non-zero elements in the
predicted depth matrix are converted to 1. Then, linear image interpolation is conducted to obtain
the depth values using the input depth map as the base map.
14
 Fig. 12 Two ways of deploying the trained U-net

2.5 Creation of Detection Matrix

2.5.1 Generation of Target Array

As previously mentioned, this study focuses on the paved road surface as the TMA. As noted in
Fig.9, the points on the pavement collected by the LiDAR sensor are sparse, and the sparsity
increases with the distance from the sensor. However, it does not mean that objects lying between
scanlines on pavement cannot be detected by the LiDAR sensor. As depicted in Fig.13, several laser
beams may impinge on the target and thus capture points on it. In this case, it is very conservative
to directly use the pavement region as TMA, which is a common practice in previous studies [3,12].
In contrast, pole-like objects with a certain height on the pavement are used to represent targets
in this study.

Fig. 13 Illustration of object detection by RSL

The process of generating the target array is illustrated in Fig.14a. First, divide the road surface
points obtained in Section 2.2 into many grid cells on the - plane. Second, if the point density
in a grid cell exceeds a user-defined threshold , a cylinder whose bottom center overlaps with the
grid cell center is constructed. The radius and height of the cylinder are and ℎ , respectively. An
array of targets is generated after all grid cells are processed. The main objective of the optimization
in this study is to minimize the number of required RSL, supposing all RSL are connected, which
ensures all targets within the TMA are covered without any detection gap.

15
 (a)

(b)
Fig. 14 Pre-processing steps (a) generate target array, (b) create detection matrix

2.5.2 Creation of Detection Matrix

Inspired by [3], which leveraged a visibility matrix to simplify the optimization problem, the present
study constructs a detection matrix to help convert the objective function and constraints into
linear forms. More specifically, a null × matrix × is constructed whose row and column
numbers correspond to the indices of candidate RSL and targets, respectively. Then, whether each
target can be detected by each RSL is estimated in sequence. If the -th target is detected by the
-th RSL, × ( , ) = 1; otherwise, × ( , ) remains 0. The procedure for estimating
whether a target is detectable is crucial in this process.
The proposed workflow for constructing the detection matrix is shown in Fig.15 and involves
five steps.
Step i: Select the -th RSL, and the initial value of is 1.
Step ii: Generate the depth map for the -th RSL using Algorithm 1. Note that it is
unnecessary to output results shown in Fig.9 at this step.
Step iii: Select the -th target, and the initial value of is 1. Generate a same-size depth map
of the -th target using the same technical parameters.
Step : Obtain the matrix positions of the target in and use them to retrieve depth values
in . If the depth value in is smaller than that at the same position in , the point
corresponding to the depth value on the target can be detected by the RSL. In this study, it is
assumed that a target can be ‘seen’ by a RSL (i.e. Ψ × ( , ) = 1) if there are more than
points impinging on it.
Step v: = + 1, and go to Step i; else, = + 1, and go to Step iii. The procedure ends
when all RSL and targets are estimated.

16
 Fig. 15 Process of constructing detection matrix
A detection matrix shown in Fig.14b can be obtained after the procedure shown in Fig.15 is
executed. Before the final optimization of RSL’ placement, a preliminary assessment of all RSL’s
coverage is needed. A 1 × vector is obtained by summing each column of × . If there is any
0-elment in the vector, uncovered targets exists even if all candidate RSL are deployed. In that
case, the horizontal and vertical positions of candidate RSL described in Section 2.3 need to be
densified, and the aforementioned steps are repeated until targets are 100% covered by all candidate
RSL.

2.6 Optimization of RSL’ Placement

It is more straightforward to define the objective function and constraints on the basis of the
detection matrix. Suppose there are kinds of RSL available for placement. An array of binary
variables is established for each type of RSL, where denotes the total number of candidate RSL
positions. Therefore, there are ∙ variables in total. Let { , , … − , } be the cell
array that stores variable arrays. Take for an illustration. is essentially a × 1 binary array.
If the -th element of is 0, it means the -th Type-i RSL is not deployed; otherwise, it is deployed.
In this way, the number of 1-element in is the number of placed Type-i RSL. As such, the
objective function is defined as

= arg min ∑ =
=
∑ =
=
∙ (5)

where = the -th element of the variable array of Type- RSL, its value is 0 or 1, = the cost
of placing Type- RSL, and = total cost. Considering the non-transparent prices of many LiDAR
products in the market, is set to 1 in this study for simplicity. In this case, represents the total
number of placed RSL.
The constraints are twofold in the optimization problem. First, different types of RSL cannot
be placed at the same candidate RSL position, which is expressed as:

∑=
=
≤ 1, ∀1 ≤ ≤ (6)

Second, all pole-like targets are 100% covered by deployed RSL. Specifically, for each target,
it should be detected by at least one RSL, which is expressed as:
=
∑= ∑ ⨀ ≥ 1, ∀1 ≤ ≤ (7)

where = the -th column of × , ⨀ = Hadamard product, which performs element-by-

element product of two matrices.
Eqs. (5) to (7) are all in linear forms, and all variables involved are binary. Therefore, the BIP
solver is applied to solve the optimization problem. An integer programming problem is a
mathematical optimization in which the variables are integers. BIP is a special case of integer
programming in which all unknowns are restricted to either 0 or 1. There are mainly two reasons
for using the BIP solver in this study. First, A binary variable can well represent whether a
candidate RSL is deployed or not. On the basis of the detection matrix, it is more straightforward
to formulate the objective functions and constraints using binary variables. Second, each candidate
RSL corresponds to a binary variable. In this study, the optimization problem may involve a lot of
variables (e.g., 1000). The BIP solver can address large-scale 0-1 integer programs [31]. Therefore,
it was applied to solve the optimization problem in this study. Once the solving process is done,
the optimization results can be combined with the candidate RSL positions to visualize the optimal
17
placement of RSL. In addition, after the acquisition of the optimal RSL positions, a procedure (see
Appendix B) for detecting obstacles is also developed to help users understand the occlusion issue.
Although the proposed method mainly focuses on the MIN problem, it can be adapted to
address the FIX problem. In that case, the total number of RSL is limited to a user-defined integer
(positive), and the objective is to maximize the coverage.

3 Application
3.1 Data Descriptions

Point cloud data of three different road segments located in Nanjing, Jiangsu province, P.R. China
are used to test the proposed framework. Site 1 is an undivided two-lane highway segment with a
tunnel. Site 2 is a divided and curved highway. Site 3 is a X-shape highway intersection. The road
environments of three sites are quite different, as noted in Fig.16. All point cloud data were collected
by the Hi-Scan mobile mapping systems, and detailed technical parameters regarding Hi-Scan are
presented in [32]. Differently, at Sites 2 and 3, the forward and opposing lanes are separately
scanned, and point cloud data are then stitched due to the occlusion caused by median barriers.
Detailed information about each test site is summarized in Table 2. Note that when calculating ,
the points whose lateral distances to the road edges exceed 20 m were excluded at Site 1. At Sites
2 and 3, only the roadside points whose lateral distances to the road edges are within 30 m were
retained in addition to the points inside TMA.
Table 2 Basic information about each test site
Parameters Site 1 Site 2 Site 3
Point count 1,405,000 3,724,184 3,361,904
Average point
0.1 0.05 0.05
spacing /m
Major: 269
Length /m 243 252
Minor: 146
Lane width/m 3.5 3.75 3.75
Major: 4
Number of lanes 2 4
Minor: 2
Radius /m 120 1670 Not applicable

Fig. 16 Point clouds of test sites

3.2 Results and Discussions

3.2.1 Pre-processing Steps

At Sites 1 and 2, the variable values for extracting TMA, generating candidate RSL, and creating
a target array are empirically specified in Table 3. Note that the parameter values for the extraction

18
of TMA should not be lower than in Table 2. At Site 3, the TMA is manually extracted while
other values of the variables remain unchanged. Besides, in regard to operations involving KNN
searching, both K1 and K2 are empirically determined as 10. Based on the proposed framework, the
results at different steps are displayed in Fig. 17.
Table 3 Variable values
Step Variables Value Descriptions
Extraction of TMA , /m 0.2 Parameters in the pillar-based filtering.
Smaller , means higher accuracy, but
may also incurs longer processing time
ℎ /m 0.1 A height threshold for filtering off-ground
objects
/m 0.1 , and are parameters in the
generation of voxels.
/m 0.2 A distance threshold in the distance-based
clustering. If the spacing between two
points is within , they belong to the same
cluster
Generation of ∆ /m 4 Horizontal spacing between candidate RSL
candidate RSL and ℎ /m 4 Upper limit on the height of candidate RSL
ℎ /m 10 Lower limit on the height of candidate RSL
∆ℎ/m 0.5 Vertical spacing between candidate RSL.
∆ℎ and ∆ control the density of candidate
RSL vertically and horizontally, respectively
/m 0.5 Radius for searching points around
horizontal positions of candidate RSL
Creation of detection 5 A user-defined threshold for determining
matrix whether grid cells are in the sparse area
during the process of generating depth maps
25 A user-defined threshold for determining
whether a target can be constructed
/m 1 Width of grid cells in the generation of
target arrays
/m 0.25 Radius of the target
ℎ /m 0.6 Height of the target
8 A threshold about the number of pixels for
determining whether a target can be
detected by a candidate RSL

The automatically segmented road surface points were compared with the ground truth data
at Sites 1 and 2. The results are presented in Table 4. Despite the very different road environments
at Sites 1 and 2, the results show that the K-Means clustering method that uses FPFH as input
can effectively segment pavement areas from unorganized point clouds. Then, the distance-based
clustering approach can further convert discrete pavement points into a single cluster (i.e., TMA).
In essence, the proposed procedure segments TMA based on similarity and connectivity among
points, which may fail in some ambiguous areas. For instance, the emergency parking area in the
tunnel was misclassified as road surface (see the ellipse in Fig.17a). In addition, along the boundaries
where paved road surface adjoins roadside ground at Site 2, some ground points were unexpectedly
classified as pavement, leading to unsmooth boundaries. However, because the road boundaries are
manually delineated in this study, these possible false classifications will not substantially affect the
generation of candidate RSL.
19
Table 4 Comparison with ground truth
Sites Accuracy (%) Recall rate (%) F1 score (%)
Site 1 91.84 88.96 90.38
Site 2 94.45 98.18 96.28

Note that the misclassified points in the extracted TMA can be manually removed. Besides,
some specific areas can also be added according to the road administrator’s requirement. After
manually picking up some points along the road boundaries, the candidate RSL positions can be
automatically generated. The upper and lower parts of Fig.17b are the generated targets and
candidate RSL, respectively. At Site 1, the candidate RSL are mainly deployed alongside the road
edges. In addition to road edge positions, candidate RSL are also placed along the median barriers
at Sites 2 and 3. A close-up view of candidate RSL positions at Site 1 is displayed in Fig.17c. As
noted, the presence of tunnel roof at Site 1 may limit the maximum heights of candidate RSL.
Similarly, power lines at Sites 2 and 3 may also affect the generation of candidate RSL positions.
Finally, the numbers of generated RSL positions at Sites 1, 2 and 3 are 956, 1900, and 2019,
respectively. The number of targets at Sites 1, 2 and 3 are 2045, 6239, and 8219, respectively.

3.2.2 Optimization Results

Because the type of RSL may affect the optimization results, five mainstream LiDAR products in
the market are considered in this study: Velodyne VLP-16, Velodyne Puck 32C, Ouster OS-32,
Ouster OS-64, and Ouster OS-128. The technical parameters of each LiDAR are presented in Table
5.
Table 5 Technical parameters of LiDAR sensors
LiDAR /° /° /°a /° /m
VLP-16 30 ([-15,15]) 2 100
Puck 32C 40 ([-25,15]) 1.29 b 200
OS-32 360([-180,180]) 0.1 45 ([-22.5,22.5]) 1.45 120
OS-64 45 ([-22.5,22.5]) 0.72 120
OS-128 45 ([-22.5,22.5]) 0.35 120
Note: a The values in brackets denote the lower and upper angle limits; b average angular resolution
Suppose there are six deployment plans: 1) place VLP-16 only, 2) place Puck 32C only, 3)
place either Puck 32C or OS-32, 4) place OS-32 only, 5) place OS-64 only, and 6) place OS-128
only. Regarding each plan, the VLM is applied at candidate RSL positions to generate the detection
matrix. The hybrid deployment plan is built on the assumption that different RSL can be connected.
Note that the trained U-net model was not applied in this case. Then, the procedure described in
Section 2.6 is used to minimize the number of RSL while ensuring that all targets are covered
without any gap. Finally, the optimal solutions of each deployment plan are obtained using BIP
(integer gap tolerance: 10-5). Fig. 18a shows the minimum number of required RSL in each plan.

20
 (a)

(b)

(c)
Fig. 17 Results of pre-processing steps (a) segmentation of TMA, (b) target array and candidate
RSL locations, (c) close-up view of candidate RSL

21
 The variation in the optimal number of RSL with deployment plans shows a similar pattern,
though road environments at three sites are very different. Generally, the technical parameters of
LiDAR sensors significantly affect the optimal placement of RSL, as expected. The optimal number
of RSL decreases substantially with increases in the channel number. For instance, at least 31 VLP-
16 RSL are needed at Site 1 to seamlessly cover the TMA, whereas 7 OS-128 RSL can achieve the
same purpose. It is true that high-end LiDAR sensors are more expensive than low-end ones, but
fewer RSL also means fewer installation costs and maintenance in the future. In this light, deploying
high-end RSL is a more desirable plan when budget permitting. When both Puck 32C and OS-32
sensors are considered, 10 out of 13, 7 out of 10, and 12 out of 12 RSL are Puck 32C at Sites 1, 2,
and 3, respectively. Compared with OS-32 sensors, the detection range of Puck 32C sensors is much
broader. The results imply that the optimization procedure will automatically determine better
LiDAR as the dominant sensor in a hybrid deployment plan.
It is noted in Fig.18 that the number of sensors required to cover a road segment is large, even
if high-end LiDAR sensors are used. In this regard, the costs of procuring and installing RSL are
high. Therefore, decision-makers need to weigh the benefits of deploying RSL against the costs
incurred. For instance, it is not cost-effective to have many RSL at a low-traffic-volume highway
segment. Different factors, including operational safety, traffic volume, road functions, etc. should
be considered to justify the deployment of RSL.
In regard to RSL’ optimal locations in different plans, there is no visually significant pattern.
This result indicates that optimizing the placement of RSL is a site-specific task. This is also why
an automated or semi-automated optimization procedure is required. The optimal placement of
RSL can be visualized by retrieving the locations of candidate RSL whose corresponding variable
value is 1, as shown in Fig.18b. The layouts of optimally placed RSL at Sites 1 and 3 are displayed
in Appendix C. Besides, the proposed sensor model can help users understand how multiple RSL
jointly cover the TMA (Fig.18c).

(a)

(b)

22
 (c)
Fig. 18 Optimization results of each plan (a) number of RSL, (b) visualization of optimal
placement, (c) sensor outputs

3.3 Test of U-net Model

3.3.1 Time Performance

The U-net model is introduced to examine whether it can improve the efficiency of generating
detection matrices. The deterministic and U-net sensor models were separately applied to generate
detection matrices at three sites considering different technical parameters. Time curves are plotted
in Fig.19. All computations were executed on a computer with RAM of 16 GB, CPU of Intel®
Core™ i7-10700 @ 2.90GHz, GPU of NVIDA GTX3070.

Fig. 19 Time curves at different sites

Time curves at different sites show very similar patterns. Regarding the deterministic sensor
model, the technical parameters in VLM substantially affect its time performance. Modeling a
higher-level LiDAR product means a longer processing time. The number of grid cells on the φ-ω
plane increases significantly with the angular resolutions, which causes more computations and thus
lower the processing efficiency. In fact, the proposed deterministic VLM is efficient enough. For
instance, generating a detection matrix at Site 2 involves 1900 RSL and 6239 targets. It took
14960.59 s in total to create the detection matrix for OS-128. In this case, the mean processing time
of determining an element value in the detection matrix is only 1.26×10-3 s. It is a large number of
computations that leads to longer processing time.
In comparisons, the U-net based sensor model outperforms the deterministic model on efficiency
in all cases. Besides, the processing time of U-net models varies slightly with LiDAR sensors. The
second way of deploying the U-net takes a somewhat longer time than the direct manner because
of the additional image interpolations. Generally, the deep neural network-based sensor model shows
better performance in terms of efficiency.

23
3.3.2 Accuracy

The detection matrices created with the U-net sensor models are also used to optimize the placement
of RSL. Fig. 20a shows an example of the optimal RSL placement at Site 2 using different sensor
models. Substantial differences are observed between the optimization results obtained using the
deterministic and U-net VLM. The parameter is computed for each deployment plan to
better measure the accuracy of U-net sensor models. Specifically, the optimal RSL placement
obtained via the deterministic VLM is used as a baseline. is then obtained by averaging
distances between each RSL location in the u-net based optimization results and its closest
counterpart in the baseline result. Larger value means being less accurate.

(a)

(b)
Fig. 20 Accuracy tests (a) RSL locations, (b) quantitative results
The quantitative accuracy test results are shown in Fig. 20b. The upper and lower parts of Fig.
20b are optimal number of RSL and -curves, respectively. The results indicate that the U-net
VLM are not reliable enough for the optimization of RSL placement. At all sites, especially Sites 2
and 3, the numbers of RSL in the U-net based optimization results are substantially larger than
the baseline. Although the differences in the RSL number decrease when a higher-end LiDAR is
modeled, values in all cases exceed 3. Although the trained U-net reached an acceptable
RMSE, neither ways of deployment achieved a satisfactory accuracy performance on the
optimization of RSL placement.
In this study, the creation of a detection matrix is sensitive to the accuracy of the depth map.
A slight change in the predicted depth map (i.e. ) may cause some element values in the
detection matrix to switch from 0 to 1 or vice versa. The detection matrix plays a crucial role in
the optimization. In this regard, even a small change in the detection matrix may significantly
24
affect the optimization results. This may account for the poor performance of U-net in this study.

4 Concluding Remarks
This paper introduces point cloud data as road background and proposes a semi-automated
framework for optimizing RSL’s placement. The proposed workflow can assist road administrators
in better making decisions on deploying RSL at constructed highway infrastructures. Based on this
study, the following comments are offered:
 A complete computer-aided framework, from the segmentation of TMA to the application of
BIP, is proposed in this study. The effectiveness of the proposed framework is demonstrated
through tests on point cloud data of three very different highway segments.
 A new deterministic LiDAR model is developed that can work in point clouds without rendering.
The developed VLM can simulate different LiDAR products by adjusting the technical
parameters. In this way, road administrators can compare deployment plans which involve
various LiDAR sensors.
 A U-net sensor model is built based on the proposed deterministic VLM and the trained U-net
is deployed in two different ways. Although the U-net models show better time performance in
generating the detection matrix, the accuracy of meaningful optimization results is
unsatisfactory. In this light, the U-net neural network might not be a reliable sensor model for
optimizing RSL’s placement at point cloud-modelled infrastructures.
 Only the U-net model was considered in this study. The unsatisfactory accuracy performance
of the trained U-net does not take away the value of deep neural networks. More deep neural
network models and different ways of introducing neural networks can be investigated in the
future. In addition to neural networks, more physical ways (e.g., parallel computing) of
improving VLM can be explored.
 The proposed framework involves many variables. A comprehensive investigation of the
impacts of different variable values on the optimization results is expected in future research.
 The study mainly focuses on the static road surface as TMA. In the future, dynamic targets
(i.e. road users) on urban streets need to be seriously considered in the optimization.
 In some cases, RSL are not installed parallel to the geodetic plane, which makes the
optimization of RSL placement a more complex problem. It is desirable to incorporate 3D pose
parameters of RSL in future studies to cover more scenarios.
 At some intersections, it is possible that video cameras, radars, and RSL are combined to
monitor road traffic. A joint optimization framework can be explored to optimize the placement
of multimodal sensors. Besides, some solid-state LiDAR (e.g., Livox) do not collect data in an
organized manner. A portfolio of virtual sensor models shall be developed to address the
optimization of various RSL.

Appendix
A. Examples of obstacles causing occlusions
RSL cannot detect objects in the occluded areas, as illustrated in Fig. A.1. In this regard, it is
difficult for a single RSL to monitor road traffic. Therefore, multiple RSLs need to be combined to
achieve full coverage of the monitoring area.

25
 Fig. A1 Occlusions caused by obstacles.
B. Detecting obstacles for RSL
At a given RSL position, the points of 3D road environment are mapped into the - - frame.
Then, similar to Step 2 in Algorithm 1, the points are partitioned into many grid cells on the -
plane, where each grid cell corresponds to a virtual laser beam. In this case, a cell matrix is
created in which each element stores indices of points in each grid cell. The size of the cell matrix
is the same as that of or (see Section 2.5.2).
As illustrated in Fig. B.1a, let be the depth value of a target point in , and its
corresponding depth value in be . The matrix position of the target point is used to retrieve
the corresponding grid cell from . If < , the points whose -values range from to are
determined as obstacles; otherwise, the target point is not occluded. The detection of obstacles is
conducted target-wise. After all targets are estimated, some examples of detected obstacles can be
visualized as shown in Fig. B.1b. Based on results shown in Fig. B.1b, road administrators can
decide whether it is possible to remove some roadside objects. As such, the number of RSLs required
to cover the TMA can reduced.

(a)

(b)
Fig. B.1 Detecting obstacles: (a) approach and (b) examples

C. Optimal placement of RSL at Sites 1 and 3

The optimal placement of RSL at Sites 1 and 3 is displayed in Fig. C.1. It again shows the irregular

26
pattern regarding the variation of RSL’s optimal placement with sensor types.

(a)

(b)
Fig.C.1 Optimal locations of RSL (a) Site 1, and (b) Site 2

Acknowledgements
This work was jointly supported by the Fundamental research funds for the central universities
[grant numbers: JZ2021HGQA0238, and JZ2022HGTA0338]; the Natural Sciences and Engineering
Research Council of Canada [grant number: Ryerson-2020-04667]

References
[1] Gilroy, S., Jones, E., Glavin, M. 2019. Overcoming occlusion in the automotive environment—
A review. IEEE Transactions on Intelligent Transportation Systems, 22(1), pp.23-35.
https://doi.org/10.1109/TITS.2019.2956813
[2] Tian, Y., Xu, H., Guan, F., Toshniwal, S., Tian, Y.X. 2021. Projection and integration of
connected-infrastructure LiDAR sensing data in a global coordinate. Optics & Laser Technology,
144, 107421. https://doi.org/10.1016/j.optlastec.2021.107421
[3] Vijay, R., Cherian, J., Riah, R., De Boer, N., Choudhury, A. 2021. Optimal Placement of
Roadside Infrastructure Sensors towards Safer Autonomous Vehicle Deployments. In 2021 IEEE
International Intelligent Transportation Systems Conference (ITSC), pp. 2589-2595. IEEE.
https://doi.org/10.1109/ITSC48978.2021.9564822
[4] Danczyk, A., Di, X., Liu, H. X. 2016. A probabilistic optimization model for allocating freeway
27
 sensors. Transportation Research Part C: Emerging Technologies, 67, pp.378-398.
https://doi.org/10.1016/j.trc.2016.02.015
[5] Fei, X., Mahmassani, H. S., Murray-Tuite, P. 2013. Vehicular network sensor placement
optimization under uncertainty. Transportation Research Part C: Emerging Technologies, 29,
pp.14-31. https://doi.org/10.1016/j.trc.2013.01.004
[6] Fu, H., Lam, W. H., Shao, H., Xu, X. P., Lo, H. P., Chen, B. Y., Sumalee, A. 2019. Optimization
of traffic count locations for estimation of travel demands with covariance between origin-
destination flows. Transportation Research Part C: Emerging Technologies, 108, pp.49-73.
https://doi.org/10.1016/j.trc.2019.09.004
[7] Shao, M., Xie, C., Sun, L. 2021. Optimization of network sensor location for full link flow
observability considering sensor measurement error. Transportation Research Part C: Emerging
Technologies, 133, 103460. https://doi.org/10.1016/j.trc.2021.103460
[8] Ahn, J. W., Chang, T. W., Lee, S. H., Seo, Y. W. 2016. Two-phase algorithm for optimal camera
placement. Scientific Programming, 2016, 4801784.1. https://doi.org/10.1155/2016/4801784
[9] Nam, Y., Hong, S. Optimal placement of multiple visual sensors considering space coverage and
cost constraints. 2014. Multimedia Tools and Applications volume, 73, pp.129–150.
https://doi.org/10.1007/s11042-012-1266-y
[10] Zhao, J., Yoshida, R., Cheung, S. C. S., Haws, D. 2013. Approximate techniques in solving
optimal camera placement problems. International Journal of Distributed Sensor Networks, 9(11),
241913. https://doi.org/10.1155%2F2013%2F241913
[11] Urrutia, J. 2000. Art gallery and illumination problems. In Handbook of computational
geometry, pp. 973-1027. North-Holland. https://doi.org/10.1016/B978-044482537-7/50023-1
[12] Altahir, A. A., Asirvadam, V. S., Hamid, N. H. B., Sebastian, P., Saad, N. B., Ibrahim, R. B.,
Dass, S. C. 2018. Optimizing visual sensor coverage overlaps for multiview surveillance systems.
IEEE Sensors Journal, 18(11), pp. 4544-4552. https://doi.org/10.1109/JSEN.2018.2825781
[13] Wang, J., Niu, C., Shen, R. 2009. Priority-based target coverage in directional sensor networks
using a genetic algorithm. Computers & Mathematics with Applications, 57(11-12), pp.1915-1922.
https://doi.org/10.1016/j.camwa.2008.10.019
[14] Yang, X., Li, H., Huang, T., Zhai, X., Wang, F., Wang, C. 2018. Computer‐aided optimization
of surveillance cameras placement on construction sites. Computer-Aided Civil and Infrastructure
Engineering, 33(12), pp.1110-1126. https://doi.org/10.1111/mice.12385
[15] Malhotra, A., Singh, D., Dadlani, T., Morales, L.Y. 2022. Optimizing Camera Placements for
Overlapped Coverage with 3D Camera Projections. International Conference on Robotics and
Automation (ICRA), May 23-27, 2022. Philadelphia, PA, USA, pp.5002-5009.
https://doi.org/10.1109/ICRA46639.2022.9812042
[16] Altahir, A. A., Asirvadam, V. S., Hamid, N. H. B., Sebastian, P., Saad, N. B., Ibrahim, R. B.,
Dass, S. C. 2017. Optimizing visual surveillance sensor coverage using dynamic programming.
IEEE Sensors Journal, 17(11), pp.3398-3405. https://doi.org/10.1109/JSEN.2017.2694385
[17] Zhang, Y., Luo, H., Skitmore, M., Li, Q., Zhong, B. 2019. Optimal camera placement for
monitoring safety in metro station construction work. Journal of Construction Engineering and
Management-ASCE, 145(1), 04018118. https://doi.org/10.1061/%28ASCE%29CO.1943-
7862.0001584
[18] Du, Y., Wang, F., Zhao, C., Zhu, Y., Ji, Y. 2022. Quantifying the performance and optimizing
the placement of roadside sensors for cooperative vehicle‐infrastructure systems. IET Intelligent

28
 Transport Systems, 16(7), pp.908-925. https://doi.org/10.1049/itr2.12185
[19] Ma, Y., Zheng, Y., Easa, S., Wong, Y. D., El-Basyouny, K. 2022. Virtual analysis of urban
road visibility using mobile laser scanning data and deep learning. Automation in Construction,
133, 104014. https://doi.org/10.1016/j.autcon.2021.104014
[20] Wu, Y., Wang, Y., Zhang, S., Ogai, H. 2020. Deep 3D object detection networks using LiDAR
data: A review. IEEE Sensors Journal, 21(2), 1152-1171.
https://doi.org/10.1109/JSEN.2020.3020626
[21] Fang, J., Zhou, D., Yan, F., Zhao, T., Zhang, F., Ma, Y., Yang, R. 2020. Augmented lidar
simulator for autonomous driving. IEEE Robotics and Automation Letters, 5(2), pp.1931-1938.
https://doi.org/10.1109/LRA.2020.2969927
[22] Balado, J., Díaz-Vilariño, L., Arias, P., González-Jorge, H. 2018. Automatic classification of
urban ground elements from mobile laser scanning data. Automation in Construction, 86, pp.226-
239. https://doi.org/10.1016/j.autcon.2017.09.004
[23] Demetri, S., Picco, G. P., Bruzzone, L. 2019. LaPS: LiDAR-assisted placement of wireless
sensor networks in forests. ACM Transactions on Sensor Networks (TOSN), 15(2), pp.1-40.
https://doi.org/10.1145/3293500
[24] Manivasagam, S., Wang, S., Wong, K., Zeng, W., Sazanovich, M., Tan, S., Urtasun, R. 2020.
Lidarsim: Realistic lidar simulation by leveraging the real world. In Proceedings of the
IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11167-11176.
https://doi.org/10.1109/CVPR42600.2020.01118
[25] Ma, Y., Easa, S., Cheng, J., Yu, B. 2021. Automatic framework for detecting obstacles
restricting 3D highway sight distance using mobile laser scanning data. Journal of Computing in
Civil Engineering, 35(4), 04021008. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000973
[26] Cheng, C. C., Peng, G. J., Hwang, W. L. 2008. Subband weighting with pixel connectivity for
3-D wavelet coding. IEEE Transactions on Image Processing, 18(1), pp.52-62.
https://doi.org/10.1109/TIP.2008.2007067
[27] Rusu, R. B., Blodow, N., Marton, Z. C., Beetz, M. 2008. Aligning point cloud views using
persistent feature histograms. In 2008 IEEE/RSJ International Conference on Intelligent Robots
and Systems, pp. 3384-3391. IEEE. https://doi.org/10.1109/IROS.2008.4650967
[28] Rodriguez-Cuenca, B., Garcia-Cortes, S., Ordóñez, C., Alonso, M. C. 2015. An approach to
detect and delineate street curbs from MLS 3D point cloud data. Automation in Construction,
51, pp.103-112. https://doi.org/10.1016/j.autcon.2014.12.009
[29] Ma, Y., Zheng, Y., Easa, S., Hou, M., Cheng, J. 2019. Automated method for detection of
missing road point regions in mobile laser scanning data. ISPRS International Journal of Geo-
Information, 8(12), 525. https://doi.org/10.3390/ijgi8120525
[30] Czerniawski, T., Ma, J. W., Leite, F. 2021. Automated building change detection with amodal
completion of point clouds. Automation in Construction, 124, 103568.
https://doi.org/10.1016/j.autcon.2021.103568
[31] Johnson, E. L., Kostreva, M. M., Suhl, U. H. 1985. Solving 0-1 integer programming problems
arising from large scale planning models. Operations Research, 33(4), pp.803-819.
https://doi.org/10.1287/opre.33.4.803
[32] Hi-Target: Hi-ScanC Mobile Mapping System. https://en.hi-target.com.cn/hiscan-c-mobile-
mapping-system, n.d. (accessed on Nov. 24th 2021).

29

View publication stats