Automation in Construction 138 (2022) 104245

Contents lists available at ScienceDirect

Automation in Construction
journal homepage: www.elsevier.com/locate/autcon

Review

Computer vision-based construction progress monitoring

Varun Kumar Reja a, b, *, Koshy Varghese a, Quang Phuc Ha b
a
BTCM Division, Department of Civil Engineering, IIT Madras, India
b
Faculty of Engineering and IT, University of Technology Sydney, NSW 2007, Australia

A R T I C L E I N F O A B S T R A C T

Keywords: Automating the process of construction progress monitoring through computer vision can enable effective
Progress monitoring control of projects. Systematic classification of available methods and technologies is necessary to structure this
Computer vision complex, multi-stage process. Using the PRISMA framework, relevant studies in the area were identified. The
Automated construction
various concepts, tools, technologies, and algorithms reported by these studies were iteratively categorised,
Data acquisition
developing an integrated process framework for Computer-Vision-Based Construction Progress Monitoring (CV-
3D reconstruction
As-built modelling CPM). This framework comprises: data acquisition and 3D-reconstruction, as-built modelling, and progress
Point cloud assessment. Each stage is discussed in detail, positioning key studies, and concurrently comparing the methods
Scan to BIM used therein. The four levels of progress monitoring are defined and found to strongly influence all stages of the
Literature review framework. The need for benchmarking CV-CPM pipelines and components are discussed, and potential research
Digital Twin questions within each stage are identified. The relevance of CV-CPM to support emerging areas such as Digital
Twin is also discussed.

1. Introduction computing power required for processing. However, with the increasing
ubiquity of both devices and high-performance computing, it has
Monitoring the progress of construction projects is crucial, as it become feasible to implement vision-based technologies in applications
provides vital inputs for managers to make timely and informed de­ that automate construction processes. Computer vision enables com­
cisions. Improper progress monitoring leads to a loss of control of the puters to derive numeric information from digital images, videos, depth
project, resulting in time and cost overruns. Traditional progress- images and 3D point clouds, process the information, and take action.
monitoring methods require manual data entry, which proves to be Sub-domains of computer vision relevant to this paper include scene
tedious, time-consuming, and prone to human error [1]. Therefore, its reconstruction, 3D pose estimation, motion estimation, object detection,
effectiveness can be improved through automation. object recognition and labelling, learning, and 3D scene modelling.
Technologies investigated for their applicability in automated Vision-based technologies have been studied on a wide range of
progress monitoring in construction include RFID, barcodes and QR construction applications, such as workforce tracking, resource tracking,
codes, laser scanning, photogrammetry, videogrammetry, range imag­ condition assessment, quality inspections, safety management, and
ing (RGB–D), web-based CCTV, and structural sensing [2]. Among automated layout generation [3]. Several studies on vision-based con­
these, the impact of vision-based technologies was initially limited due struction progress monitoring have also been reported. However, the
to the sophistication of the data-acquisition devices and the high processes for effective progress monitoring are diverse, intricate, and

Abbreviations: AEC, Architecture, Engineering and Construction; AI, Artificial Intelligence; AR, Augmented Reality; BIM, Building Information Modelling; BoQ,
Bill of Quantities; CAD, Computer-Aided Design; CCTV, Closed-Circuit Television; CMVS, Clustering Multi-View Stereo; CNN, Convolutional Neural Network; CV-
CPM, Computer Vision-based Construction Progress Monitoring; DT, Digital Twin; DL, Deep Learning; GPS, Global Positioning System; GPU, Graphical Processing
Unit; GUI, Graphical User Interface; HSI, Hue-Saturation-Intensity; ICP, Iterative Closest Point; IFC, Industry Foundation Class; IMU, Inertial Measurement Unit; IoT,
Internet of Things; LiDAR, Light Detection and Ranging; LoD, Level of Detail; LPM, Level of Progress Monitoring; MEP, Mechanical, Electrical, and Plumbing; ML,
Machine Learning; MR, Mixed Reality; MVS, Multi-View Stereo; PCA, Principal Component Analysis; PMVS, Patch-based Multi-View Stereo; PSR, Poisson’s Surface
Reconstruction; QR, Quick Response; RANSAC, Random Sample Consensus; RFID, Radio Frequency Identification; RGBD, Red Green Blue Depth; SfM, Structure-
from-Motion; SGM, Semi-Global Matching; SLAM, Simultaneous Localisation and Mapping; SMS, Short Message Service; TLS, Terrestrial Laser Scanning; UAV,
Unmanned Aerial Vehicle; UGV, Unmanned Ground Vehicle; VR, Virtual Reality; XR, Extended Reality.
* Corresponding author at: BTCM Division, Department of Civil Engineering, IIT Madras, India.
E-mail addresses: [email protected] (V.K. Reja), [email protected] (K. Varghese), [email protected] (Q.P. Ha).

https://doi.org/10.1016/j.autcon.2022.104245
Received 8 November 2021; Received in revised form 1 March 2022; Accepted 31 March 2022
Available online 8 April 2022
0926-5805/© 2022 Elsevier B.V. All rights reserved.
V.K. Reja et al. Automation in Construction 138 (2022) 104245

complex. In addition, there are many concepts and technologies that can The organisation of this paper is as follows. Section 2 characterises
be applied to automate each stage of the process. An integrated frame­ and classifies the key studies in this area, based on the three macro-
work to characterise and categorise the current and future studies in this stages of the CV-CPM process. Section 3 expands the three macro-
area will enable systematic investigation and documentation in this stages into a detailed process framework, and for each process in the
domain. pipeline, the relevant concepts, technologies, and tools used are
Review papers in this domain have focused on specific aspects of the compared and discussed. Section 4 discusses a strategy for a systematic
progress monitoring process. Omar et al. have presented a review of data approach in this domain and identifies the areas for further work. The
acquisition technologies [2], while Ma et al. [4] have comprehensively conclusions of the study are presented in Section 5.
studied 3D reconstruction techniques. Ekanayake et al. [5] and Patel
et al. [6] have recently presented a bibliometric analysis of the literature 2. Overview of vision-based monitoring of construction progress
and highlighted specific challenges to be addressed. However, there are
several stages in the process, and a broad study of both literature and This section presents the macro-level stages proposed for vision-
practice has resulted in identifying the following key stages: (i) data based progress monitoring in construction and references the key
acquisition and 3D reconstruction, (ii) generation of as-built models, progress monitoring processes within these stages. Each of these pro­
and (iii) monitoring progress. Currently, there is no integrated frame­ cesses is also referred to as a pipeline. Fig. 1 shows the six macro-level
work that addresses the details of the processes of Computer Vision- processes that constitute vision-based progress monitoring.
Based Construction Progress Monitoring (CV-CPM), from data acquisi­ These processes are grouped into three stages, as follows:
tion to progress estimation. Therefore, to address this gap, the following
objectives are identified: i. Acquisition of as-built data and conversion into a point cloud
using 3D reconstruction techniques.
1. Develop an integrated framework that captures the process re­ ii. Generation of the as-built model from the point cloud (with
quirements of vision-based construction progress monitoring and reference to the as-planned model, if available)
enables the characterisation and categorisation of current and future iii. Comparison of the as-built and as-planned models to assess
work in the area. progress by visualisation and/or by quantifying the work done.
2. Utilise the framework to position and compare the various concepts
and tools adopted by published research studies. All studies on vision-based progress monitoring focus on one or more
3. Identify areas and strategies for future work in the area. of these stages. Table 1, a key output of this study, was developed
through a detailed review of the published works on vision-based con­
As a first step towards these objectives, relevant articles were struction progress monitoring. The articles were identified through the
selected through the broader steps defined in the PRISMA framework. systematic process described in the introduction. The categories shown
These include identification, screening, eligibility, and inclusion. An in Table 1 were refined iteratively based on objectives, solutions and
extensive keyword search in the Scopus and Web of Science (WoS) concepts addressed by the reviewed papers.
reputable scientific databases was conducted. The keywords used were The studies referred to in Table 1 are arranged based on the category
“progress monitoring in construction,” “construction progress moni­ of progress monitoring, as indicated in the last column. The first column
toring,” “progress monitoring,” “automated progress monitoring,” contains the reference to the study, and the subsequent columns contain
“computer vision in construction progress monitoring,” and “vision- the categories and sub-categories based on the macro-level model. The
based progress monitoring in construction.” One hundred eighty-two cells in Table 1 identify the technology, concept, and algorithm explored
articles were retrieved from the keyword search. Primary screening of in each category and sub-category. It can be observed in Table 1 that for
the articles was carried out by reading their abstracts. In order to capture each category, there is a wide range of alternatives for all process
relevant developments in this rapidly evolving area, only key articles components, construction elements, and the key foci that the studies
appearing after 2014 were defined as eligible for meta-analysis. Other have explored, as discussed briefly in the following paragraphs.
articles which address specific parts of the framework were also referred Data Acquisition and 3D Reconstruction: This section of the table
to and positioned appropriately in the manuscript to support the documents the data acquisition technologies, type of mounting, and
framework. For an article to qualify as a key article for review, the usage environment (indoor/outdoor) that the studies use. Depending on
following two criteria were set: the input type, the data undergoes 3D reconstruction using various
commercial software, as shown in Table 1. The workflow from data
• “The article should present an applied computer vision-based prog­ acquisition to 3D reconstruction is discussed in detail in Subsection 3.1
ress monitoring pipeline by demonstrating it with a case study or of this paper.
experimental results.” As-built Modelling: As shown in Table 1, there are varied output
• “Pipelines addressing both indoor and outdoor progress monitoring, data types that are generated by the pipelines. The generation of these
as well as pipelines ending at different stages of a type of progress as-built models requires specific as-built modelling techniques, which
estimation, were considered (i.e., only visualisation as well as visu­ are classified and discussed in detail in Subsection 3.2.
alisation with quantification).” Progress Monitoring: Table 1 also indicates the level of progress
monitoring (LPM) carried out, judging by the types of platforms used to
Twenty-four key articles were selected and included for meta-
analysis to formulate the framework. Among other articles referred for
specific tools and techniques, fifty-four articles are from the civil engi­
neering domain, and eight are from the computer science domain.
The key articles were systematically reviewed, analysed and cat­
egorised based on the process they addressed and the concepts they
used, following the methodology defined by [7]. The framework evolved
as more studies were characterised, categorised, and positioned within
it, and the application contexts of concepts and technologies used were
studied and compared. After the reviewed works were positioned in the
framework, the processes, concepts, and technologies that need to be
further developed/explored were identified. Fig. 1. Macro-level conceptualisation of vision-based progress monitoring.

2
 V.K. Reja et al.
Table 1
Review of pipelines for vision-based progress monitoring.
Data Acquisition Progress Monitoring
3D As-Built Identified Operational
Papers Technology Method Indoor/ Visualisation Comparison Quantification Schedule Key Focus LPM
Reconstruction Output Elements State
Outdoor Updating

Immersive VR
Laser Scanning Extended Target (iVR)–based
and Reality XR (VR Cuboidal visualisation of
[45] Photogrammetry Manual Indoor SLAM 3D Mesh + AR) Visualisation N N Objects N progress
Building
CNN-based object
Virtual Segmented Interactive Object Elements
[13] RGBD Camera Manual Both N N Y detection and use of
photogrammetry Images VRE Detection (Walls, floor,
gaming engine
beams etc.,)
Computer vision-
based element
Image and
Segmented detection using
Commercial Point cloud-
[57] Digital Images UAV Outdoor Image with 3D Viewer N N Columns Y images, camera
Software based Object
Point Cloud poses, as-planned-
Detection
BIM, precedence
relationships
Gaming engine
utility for
Laser Scanning or Commercial Commercial
[44] Manual Both Point Cloud VRE N N Not specified N visualisation of as-
Photogrammetry Software Software
built and as-planned
data L-2
Uses Geometry and Visualisation
Photogrammetry/ Construction
[38] UAV Both SfM & MVS Point Cloud 3D Viewer Thresholding N N Y Appearance based and
LS Materials
reasoning comparison
3

Uses Graph Theory

Projection and precedence
[50] Photogrammetry Manual Outdoor VSfM/SGM Point Cloud 3D Viewer N N Columns N
Thresholding relationships for
Progress
Ontology
Photogrammetry/ Uses Construction
[34] Manual Both SfM & MVS BIM 3D Viewer based N N Not specified Y
LS Sequence Ontology
Deviations
AR-based Progress
monitoring using
Interactive Building
[28] Photogrammetry UAV Outdoor SfM 3D Mesh AR viewer N N N mobile aerial 3D
AR Façade
reconstruction and
visualisation
Photogrammetry
Model-assisted ConstructAide Visual Colour Uses user assisted
[58] (Unordered Manual Both Point Cloud N N Not specified N
SfM GUI Coded SfM method
Images)

Automation in Construction 138 (2022) 104245

Appearance-based
Photogrammetry
web-based Material Construction material
[9] (Unordered Manual Both SfM & MVS BIM N N Y
viewer Recognition Materials classification using
Images)
LM & HSI features

MATLAB-based
MATLAB Columns, element cropping
Object in Image beams, block and quantity
[59] Videography Fixed Both NA Image Image viewer Processing Y N masonry N estimation L-3
Hybrid data Quantification
Laser Scanning Mesh collection using
Manual Overlapping Structural
[60] and UAV Both Model/3D 3D Viewer Y N N drone-based
Modelling & BoQ Components
Photogrammetry BIM photogrammetry
and Laser Scanning
 V.K. Reja et al.
Data Acquisition Progress Monitoring
3D As-Built Identified Operational
Papers Technology Method Indoor/ Visualisation Comparison Quantification Schedule Key Focus LPM
Reconstruction Output Elements State
Outdoor Updating

Object detection,
segmentation and
Object multiple objects
Detected & tracking from images
Segmented Object to detect precast
[61] Video Camera Fixed Outdoor NA Images 3D Viewer Detection Y N Precast Walls N walls.
big regular-
Commercial Mixed Rays Mixed Reality–based
[12] RGBD Camera Manual Indoor 3D Mesh Y N shaped N
Software Reality Thresholding progress monitoring
objects
SfM & MVS Enhanced method for
[31] Photogrammetry Manual Both with Coded BIM 3D Viewer BoQ N N Walls N 3D reconstruction
Targets using coded targets
Extraction of columns
Overlapping
Commercial Rectangular using geometric and
[26] Laser Scanning Manual Both Point Cloud 3D Viewer Distance Y N N
Software Columns relationship-based
Threshold
reasoning
Percentage of
[48] NA NA Both NA Point Cloud 3D Viewer Thresholding Y N Walls N completion of in-situ
cast concrete walls L-3
Uses UAS images, Quantification
VSfM/ Building low-cost
[10] Videogrammetry UAS Outdoor 3D BIM 3D Viewer Thresholding Y N N
4

CMPMVS Façade photogrammetry, and

GIS
Use of feature
Commercial Object Masonry engineering for object
[62] Photogrammetry Manual Both Point Cloud 3D Viewer Y N Y
Software Detection Walls detection and data
classification
Use of Hough
Commercial Object MEP Transform to detect
[11] Laser Scanning Manual Both BIM 3D Viewer Y N N
Software Recognition Components parametric features
with Scan Vs BIM
Use of machine-
Photogrammetry Columns,
D4AR Voxel learning scheme built
[8] (Unordered Manual Both SfM & MVS Point Cloud Y N Foundation N
Viewer Occupancy upon a Bayesian
Images) Walls
probabilistic model
Photogrammetry
Web-based Material Construction Appearance-based
[63] (Unordered Manual Both SfM & MVS Point Cloud Y N Y

Automation in Construction 138 (2022) 104245

viewer Recognition Materials material classification
Images)

Photogrammetry-
based monitoring
Enclosed with automated
L-4
Commercial Volume notification system
Schedule
[47] Photogrammetry Manual Outdoor Software Point Cloud NA Calculation Y Y Columns N and schedule update
Update with
Commercial Continuous
notifications
Commercial Project monitoring using
[14] RGBD Camera Manual Both 4D BIM 3D Viewer N Y Not specified N
Software Management helmet-mounted
Software scanners
V.K. Reja et al. Automation in Construction 138 (2022) 104245

Fig. 2. Integrated framework for computer vision-based construction progress monitoring (CV-CPM).

5
V.K. Reja et al. Automation in Construction 138 (2022) 104245

implement progress estimation. This study classifies these into four A systematic review can be found in [2] on technologies used for
levels. These are: (L-1) only visualising the as-built model, (L-2) visu­ progress tracking in construction. The study finds technology selection a
alisation incorporating a comparison with the as-planned model, (L-3) “multi-faceted issue” which depends on “required degree of accuracy,
quantification of progress, and (L-4) quantification with schedule up­ project size, level of automation, and the ultimate purpose of progress
date and warning notifications. tracking”.
For progress visualisation, different immersive and non-immersive Fig. 3 shows a dual matrix illustrating the data acquisition technol­
environments have been used. For quantification, numerous ap­ ogy and method selection criteria. The technologies and methods are
proaches have been used based on the availability of an as-planned listed on the concentric rings in their respective matrix, where colours
model. The details are discussed in Subsection 3.3. and icons are added for easier comprehension. The key factors that
In addition to the three process components—data acquisition and govern their selection have been identified and are positioned along the
3D reconstruction, as-built modelling, and progress monitoring—T­ outer ring. Eight key factors for the sensor mounting method include
able 1 also characterises the pipelines on two additional parameters. statutory clearance, cost of mounting equipment, range of operation,
First, the types of elements being worked on, and second, whether the preferred use case, operator training, navigation, manoeuvring speed,
structure’s current operational state can be recognised (e.g., a wall’s and accessibility. In addition, twelve key factors for data acquisition
stage can be brickwork- completed, plastered, or painted). technology have been identified. They include the level of automation
As inferred from Table 1, numerous combinations of tools, concepts, for data capture, real-time data availability, range of equipment oper­
and algorithms need to be evaluated to arrive at the appropriate ation, spatial resolution, spatial accuracy, adequate lightning require­
approach for a specific progress monitoring need. Though all the com­ ment, user training requirement, time for data capture, preparations for
binations may not be technically viable, a significantly large set of op­ data capture, computation cost for processing, equipment portability,
tions needs to be experimentally explored and documented to ensure and equipment cost.
systematic progress in this domain. Therefore, an integrated framework This matrix is constructed based on the review and classification of
encompassing all the processes will support the systematic study. literature. The relative comparison indicator for technology versus
In the next section, the three macro-level steps are expanded to sensor mounting method is indicated along the radial direction. The low,
formulate the detailed framework wherein the concepts/processes/ medium, and high ratings are shown using green, yellow, and red colour
technologies mentioned in these studies are characterised, positioned, codes, respectively. The proposed rating is a preliminary step towards
and discussed. assisting in the selection of a combination. As the proposed ratings are
based on the authors’ perspective, experimental studies are required to
3. Integrated framework for computer vision-based construction
progress monitoring (CV-CPM) Table 2
Summary of stage-wise specific contributions.
The integrated framework for Computer Vision-based Construction
Stage Published review papers Detailed contribution of
Progress Monitoring is shown in Fig. 2. This framework was derived this paper
after evaluating the existing literature studies in detail using the macro- Reference Their contribution

level framework lens described in the previous section and the meth­ Data [2] Categorised, listed, • Identified and
odology defined in [7]. This section explores the micro-level details of Acquisition and compared various proposed factors and
data acquisition ratings that can be
each of the three main components. In addition to the basic concepts and
technologies and used to guide
references to the work done in this area, Table 2 summarises the key studies in detail. selection of sensors-
contributions of this paper for the micro-level stages of the framework mount combinations
with reference to other review papers. for data acquisition.
The following three sub-sections are on the various processes, algo­ 3D Reconstruction [4]
Presented and reconstruction using • Comparison SLAM
rithms, methods, and technologies that could be applied to the relevant
overall SfM. Categorised, and SfM technologies
subsections of the framework. pipeline for listed, and compared for construction
3D techniques for SfM progress monitoring.
3.1. Data acquisition and 3D reconstruction technologies and
studied in detail.
As-Built [32] Presented an overview • Defined types of as-
This section addresses the data acquisition (Fig. 2 (a)) and 3D Modelling of the heuristic-based built models based on
reconstruction (Fig. 2 (b)) components of the framework. The data modelling process and existing work and
acquisition depends on the acquisition technology as well as the sensor classification of proposed guidelines
mounting method selected. There are various steps involved in 3D categories. for selecting as-built
model type for prog­
reconstruction algorithms. The decision as to which technologies to use ress monitoring based
is a multi-faceted issue. Defining these based on the characteristics of a on project
construction project and other factors is the primary contribution of this requirements.
section. • Detailed review of as-
built modelling ap­
proaches and ratio­
3.1.1. Data acquisition: technologies and methods nale for using a
Data acquisition depends on selecting the technology and the sensor hybrid approach.
mounting method (Fig. 2 (a)). As seen in Table 1, several combinations Progress NA NA • Categorised progress
of sensing technologies and sensor mounting methods have been Visualisation monitoring into four
Progress Quantification levels and associates
explored. Digital cameras [8,9], video cameras [10], laser scanners [11], these levels as the
and range imaging (or RGB-D cameras) [12–14] are the vision-based requirements driver
technologies used for collecting data for progress monitoring. They for deciding on
generate inputs to the framework in the form of image frames or point options in the
upstream stages of
clouds. The sensor mounting method for data acquisition can be in the
process.
form of fixed devices, handheld devices, robotic systems mounted on NA NA
unmanned ground vehicles (UGV) [15], unmanned aerial vehicles
(UAV) [10], or a combination of these systems [16].

6
V.K. Reja et al. Automation in Construction 138 (2022) 104245

Fig. 3. Dual matrix for sensor mounting method and data acquisition technology selection.

make them more objective. and stereo images, and video frames using SfM for photogrammetry and
its corresponding algorithms for 3D reconstruction. In our paper, only
3.1.2. 3D reconstruction the basic concept of SfM is presented, and a detailed comparison with
Following the image-based input from the previous step, the next SLAM is made to facilitate decision making for 3D reconstruction in
step is to generate a point cloud model from a series of algorithms for 3D construction.
reconstruction (Fig. 2 (b)). As observed from Table 1, most SfM (Visual SFM [18] or Open SfM [19]) is a technique consisting of
photogrammetry-based progress monitoring pipelines use commercially a combination of algorithms for photogrammetric 3D reconstruction
available software to generate a 3D point cloud from optical camera from numerous image frames (Fig. 2 (b). It is an offline approach for
images or depth images. estimating the scene’s camera motion information. The process is to
Optical camera images do not contain depth information. SfM match all the images to each other, find the correspondences, and then
(Structure-from-Motion) and SLAM (Simultaneous Localisation and delete mismatched images to obtain relative camera positions and the
Mapping) are the two conceptual approaches to add depth information structure without any prior geometric or semantic information. The
for sparse 3D reconstruction. concept of SfM is based on stereoscopic photogrammetry, as stated in
Depth images are captured using RGB-D cameras, for example, [20]. A detailed review of the 3D reconstruction by SfM and its algo­
Microsoft Kinect. They have depth information (XYZ coordinates) along rithm can be obtained in [4,21].
with colour information (RGB values) on a per-pixel basis. This infor­ SLAM [22] is a more general framework, in real-time, where a
mation is used for mapping and performing dense 3D reconstruction mapping system starts motion from an unknown location in an unknown
using intrinsic and extrinsic camera parameters [17]. The output from a environment, and during movement, simultaneously keeps track of its
laser scanner is a 3D point cloud, and hence does not require 3D position with respect to the surrounding environment, building an in­
reconstruction. cremental map. SLAM may depend on sensors like cameras, LiDAR, GPS,
The work in [4] focuses on generating point clouds from monocular IMU, etc. Modern visual SLAM has gradually developed into a multi-

7
V.K. Reja et al. Automation in Construction 138 (2022) 104245

compared to SfM and has limitations in loop closure, scaling, and

drifting issues. However, SLAM can overcome SfM’s drawback when
dealing with a featureless or repetitive scene with potential false
matches. Also, SfM has been tried on a larger scale, whereas SLAM can
be used for small-scale tasks.
It is evident from the review that both technologies have their ad­
vantages and disadvantages, and selection should be based on project
requirements and compatibility. There are multiple algorithms for SLAM
[24] and SfM ([18,19]), and when various types of embedded sensors
are included, the options for selection increase. For example, the
embedded camera sensor can be monocular, binocular or RGB–D.
Based on the type of sensor, the core of the reconstruction algorithm
changes, as well as the accuracy and the computation time. Therefore,
numerous combinations of these variations increase the complexity of
the framework and the resulting selection process.

3.1.3. Absolute scale recovery and dense 3D reconstruction

The output obtained by the image-based pipeline is primarily sparse
and not to scale. Therefore, as shown in Fig. 2 (b), it requires absolute
scale recovery and dense 3D reconstruction. Laser scans do not require
this. Table 3 shows the techniques and algorithms that can be used in the
associated sub-processes.
The scale recovery sub-process determines the absolute scale of the
generated sparse point cloud by comparing it with the local coordinates
of the point in the sparse point cloud. Scale recovery has been imple­
mented using, manual techniques, pre-measured objects, and geo-
Fig. 4. Comparison between SLAM and SfM for 3D reconstruction in
registration, as shown in Table 3. The first two techniques require
construction.
manual measurements and feeding ground truth, whereas geo-
registration requires automated sensing of camera parameters to apply
feature, multi-sensor, and deep learning-based method that can be used
the transformation.
for dynamic and haphazard construction environments.
The next sub-process, Dense 3D reconstruction, recovers the scene
As shown in Table 1, the use of Visual SLAM in progress estimation
details. The algorithms used are multi-view stereo (MVS), clustering
has been sparsely explored in construction. With the rapid advances in
autonomous technologies for data acquisition (UAV/UGV based), SLAM-
based reconstruction will become more relevant in construction. Fig. 4
Table 3
shows the comparison between SfM and SLAM to guide the selection Sub-processes and associated techniques.
process for 3D reconstruction applications for progress monitoring. The
Sub-Process Techniques/ Existing Use in Progress Other
three key factors are discussed below in detail, namely:
Algorithms Monitoring Pipelines Works

1. Data sequence: SfM is based on feature matching of image pairs is Manual –

Pre-measured
highly dependent on the quality and the sequence of the image Absolute Scale Object [31,63]
frames obtained. It need not necessarily require ordered image Recovery Geo Registration [10,28] [64]
frames. On the other hand, SLAM builds an incremental map in real- Reconstruction Multi-view Stereo MVS [38,63]
time and requires sequential frames to estimate the previous and Clustering Multi-View
Stereo (CMVS) [31]
next poses. Hence, construction sites collecting images from multiple
Patch-based Multi-View
location-aware sensors should be able to work on SfM. SLAM can be Stereo (PMVS) [31]
selected for sites requiring autonomous navigation of robots in vol­ MVS with voxel coloring [8,9]
atile unmanned environments. Dense 3D SGM [50]
2. Computational cost: The process of SfM is computationally costly Meshing using
PSR [18,19,25] [10,28]
because of unordered data compared to SLAM, which works on or­ Coarse
dered data. Bundle adjustment (non-linear optimisation) [23] in Registration
SLAM is applied only on the last “N keyframes”, as opposed to the Manual [8,38,60]
entire graph in SfM, to give a real-time performance in budget. In Marker-Based [26,28]
Sensor-Based [28]
scenarios where the availability of parallel GPUs is not a constraint
Feature-Based [31,67]
for computation, SfM will give better results than SLAM. Fine Registration
3. Path-planning or Autonomy: Pre-planning the robot surveillance ICP [28,45,50,68,14,57,69]
paths for construction sites is tedious and complex. Planned paths are Image
subjected to uncertainty on construction sites due to their dynamic Registration
Registration Geo-referencing [34,58,63,70,21] [65,66,25]
nature. Both SfM and SLAM can be used if the sensor is mounted on a Manual [60]
UAV or a UGV. SfM is predominantly used in known environments RANSAC+PCA-
where surveillance paths can be planned. In contrast, SLAM is ad­ Noise and based [26]
vantageous for autonomous applications in random locations and Outlier Tensor voting
Removal algorithm [68] [71,72]
unknown environments.
Point-space
Down Sampling strategy Algorithm – [73]
Apart from these three key features, other sub-factors which can be
considered are shown in Fig. 4. In general, SLAM generates a sparse map

8
V.K. Reja et al. Automation in Construction 138 (2022) 104245

views for multi-view stereo (CMVS), patch-based multi-view stereo outlier removal, and down-sampling, along with various techniques and
(PMVS), and semi-global matching (SGM). The open-source algorithms algorithms used in the corresponding literature. The table also shows
which, perform dense 3D reconstruction include VisualSfM [18], studies that have addressed fundamental aspects of these sub-processes.
OpenSfM [19], and Bundler SfM [25]. Registration involves merging multiple partial point clouds obtained
A dense 3D reconstruction step is required if the reconstructed point into a single file or registering an as-built point cloud over an as-planned
cloud is sparse, typically in the case of reconstruction from monocular or BIM model for progress monitoring. There are two stages of registration,
stereoscopic images or video frames. The reconstruction output from coarse and fine.
depth images is already dense enough to be used directly and does not Coarse registration aligns a set of point clouds using correspondences
require dense 3D reconstruction. between them. In contrast, the fine registration algorithm further
Only a handful of CV-CPM pipelines have stated the method used for matches multiple point clouds by estimating a rigid transform and
absolute scale recovery and dense 3D reconstruction; others have not minimising the distance between the corresponding matched points. The
explicitly mentioned it in the paper. The choices available for absolute selection of these approaches depends on the type of data acquisition
scale recovery and dense 3D reconstruction add additional variables to technology used and location capture, as registration depends on
our framework. Once a dense point cloud is generated, the next step is to localising the data. While the manual and marker-based methods require
pre-process it through registration, noise filtering, outlier removal and iterative human efforts, the sensor and feature-based approaches are
down sampling. These sub-processes and model generation are dis­ reported to be more automated. The Iterative Closest Point (ICP) algo­
cussed in the next section. rithm is the most used in CV-CPM pipelines for fine registration. It
searches for an optimum solution, and the convergence speed is directly
proportional to the accuracy of coarse registration. For image-based
3.2. Model generation methods, image registration over the BIM model is made using
georeferencing.
In this section, first, the characteristics of the as-planned model are Next, noise and outliers are unwanted points and are removed to
discussed. All the steps in pre-processing and the process of generating improve the point cloud quality prior to further processing. Finally, due
the as-built model are then discussed in detail (Fig. 2 (d)). The existing to the merging of point clouds, the overlapped regions become much
techniques for as-built model generation are categorised into heuristic- denser, affecting processing efficiency. Hence down-sampling is per­
based and learning-based methods. The definition of these categories formed to make the point cloud uniform. Relevant works which include
and the mapping of existing pipelines to these categories/sub-categories these steps are shown in Table 3.
is the primary contribution of this section. This section also presents the Although pre-processing of data has been presented in some progress
trade-off between the levels of information the various as-built outputs monitoring pipelines, there is no detailed exploration of the available
provide for progress estimation and the computational complexity techniques. There is a need for a comprehensive study and further
required to generate them. research to customise pre-processing methods based on the input pipe­
line’s requirement. There is also a need to explore the other techniques
3.2.1. As-planned model mentioned in Table 3 for better results using the CV-CPM framework.
As-planned models may or may not be available for a particular fa­
cility or project. Generally, they are prepared in the design phase of the 3.2.2.2. Types of as-built models. Existing pipelines have used several
project. These can be 2D/3D CAD or 3D/4D BIM models. The approach combinations of as-built and as-planned model comparisons for progress
to progress monitoring depends on the availability and type of the as- estimation (Table 1). Based on the algorithm and method used, the as-
planned model, as it facilitates comparison with the as-built model. built output can be a pre-processed point cloud [26], voxel model
[27], mesh model [12,28], surface model [29,30], or IFC/BIM models
3.2.2. As-built model [11,14,31] (Fig. 2 (d)).
The following sub-sections include the details for the various sub- The level of progress monitoring proposed for a project plays a vital
processes and options for as-built modelling (Fig. 2 (d)). role in deciding the type of as-built model to be generated. Table 4
compares computational requirements and quality of output for the
3.2.2.1. Point cloud pre-processing. Point cloud pre-processing is a different levels of progress monitoring with the corresponding types of
crucial step towards as-built modelling to improve the point cloud as-built models. Here, green represents a performance improvement,
quality. The level of automation in pre-processing has not been and red indicates a decrement in performance.
adequate, and manual intervention is still used to get the required It can be seen from Table 4 that for L-1 or L-2 based monitoring,
quality. Table 3 outlines the critical sub-processes of point cloud pre- voxel-based, mesh-based, or point cloud models are recommended to be
processing, which broadly consist of registration, noise filtering,

Table 4
Relative computational requirements and output accuracy for the level of progress monitoring, based on the type of as-built models.

Level of progress monitoring

L-1 and L-2 L-3 and L-4
Type of as-built
models Progress Visualization Progress Quantification
Computation for model Computation for Accuracy of Qty. take-
Quality of visual output
generation progress quantification offs
IFC/BIM Very High Very High Very Low Very High
Surface Model High High Low High
Mesh Model Low Medium High Medium
Voxel Model Low Medium High Low
Point Cloud Very Low Low Very High Very Low

9
V.K. Reja et al. Automation in Construction 138 (2022) 104245

used because of low computational requirements. Even if the point cloud

[89]
quality is low, it is adequate for visualisation. While visual output

x
quality is better for IFC-based or surface models, it is not recommended

[88]
for progress visualisation due to very high computing requirements.

x
For L-3 or L-4 based monitoring, while it will take higher computa­

[34]
tion to generate an IFC or surface-based model initially, the computation

x
required for progress quantification will be substantially lower with
[87] higher accuracy; hence it is recommended. On the other hand, the mesh
x
x
model, voxel model, and point clouds are easy to generate but will take
[86]

higher computation to perform the comparison and result in low accu­

x
x
racy. Mesh-based and voxel-based modelling are explored domains that
[85]

are easier to develop using established algorithms and software and

x
x

therefore were briefly introduced.

[29]

x
x

Hence, this trade-off of required computation, accuracy, speed, and

[84]

application case should be considered while designing the progress

x
x

monitoring pipeline for a specific usage requirement. The relative

[27]

comparison provided here is a preliminary step based on the authors’

x
x

perspective. To arrive at more objective ratings, the parameters must be

[83]

assessed quantitatively (for computation and accuracy) and qualita­

x
x

tively (for visual output quality). Further experimentation is required to

[82]

make these assessments.

x
x
x

Progress visualisation (L-1 & L-2) is typically being implemented at

[81]

construction sites. However, to take critical decisions, quantifying

x
x
x

progress is essential. IFC-based as-built models can provide direct and

[80]

accurate progress quantification (L-3) and facilitate schedule updating

x
x
x

and notifications generation (L-4). Therefore, existing studies have

[26]

x
x
x

explored various techniques to achieve point-cloud to IFC/BIM model

[79]

conversion. The following sub-section discusses and classifies these ap­

x
x
x

proaches in detail.
[11]

3.2.2.3. Point cloud to BIM. This section focuses on as-built modelling

[78]

methods resulting in IFC classes (Fig. 2 (d)). Only two existing studies
x

have classified point cloud-to-BIM approaches [32,33]. One has classi­

[8]

fied them as local and auxiliary heuristics based on as-planned BIM

[14]

involvement [32], whereas the other extends the classification based on

x

geometry, rule, model, and feature-based techniques [33].

[12]

Table 5 summarises the approaches taken by the referenced studies

x

to explore the point cloud-to-IFC/BIM conversion. This study broadly

[57]

classifies these approaches as heuristics-based and learning-based and

x

into additional subclasses, extending the classification by [32,33].

[77]

The use of heuristics has been based on geometry, rule-based, rela­

x

tionship-based, and model-based constraints. Learning-based techniques

[76]

can be generally classified as geometric and appearance learning. The

x

following section discusses and analyses the two approaches of element

[75]

recognition in detail.
x
[33]

A. Heuristics-based approach
[39]

x
x

Heuristics-based approaches (Fig. 2 (d)) use pre-coded domain

Parametric as-built modelling approaches in the literature.
[30]

knowledge to identify elements from as-built point clouds. Table 6

x

shows the comprehensive sub-classification of identified heuristics with

[13]

their definitions as formulated by the authors and the context of their

x
x
x

applications in the literature. Following are a few insights from Table 6

[74]

for the four types of heuristics:

x
x
x

x
Learning – Appearance x
Learning – Geometry

1. Geometrical constraints have been applied based on shape, di­

mensions, point density, and associative geometry. They require only
basic geometric information as inputs; therefore, they are widely
Relationship
Geometry

utilised. The challenge lies in retrieving and feeding this information

Model

for all the elements, as construction sites may have unconventional

Rule

elements.
Heuristic Based

Learning-Based

2. Rule-based constraints have been applied based on the direction of

Approaches

the surface normal, orientation, principal axis direction and other

Table 5

10
V.K. Reja et al. Automation in Construction 138 (2022) 104245

Table 6
Classification of applied heuristics to obtain specific elements.
Heuristics Classification Elements and Applied Heuristics

Geometry-based Shape and dimension Wall: dimensional constraints [29,82]

Dimensional and associative restrictions that can be applied on point clouds, Wall: floor to roof distance [87]
segmented planes, objects, or volumetric representation. Column: dimensional constraints [89]
Floor: altitude of plane [87,27]
Floor: location of centroid [86]
Roof: altitude of plane [87]
Roof:location of centroid [86]
Pipe: dimensional constraints [11]
Pipe: circular cross-section [11]
Window: rectangular [87,90]
Door: narrow regions along the trajectory of the
device [84]
Door: rectangle shape constraints [87,91,90]
Opening: shape [87,78]
Secondary components: shape [78]
Point density Floor: point density [89]
Roof: point density [89]
Window: low point density [87,85]
Wall: 2D countour generation [84]
Opening: the location on the wall [87]
Associative geometry Beam: plane projection and line fitting [89]
Window & door: distance from the wall, ceiling
and adjacent wall [87,90]
Rule-based Surface normal Floor: surface normal opposite to direction of
These are the general heuristics most of the building elements follow. gravity [87,27,80,81]
Roof: surface normal in direction of gravity
[87,80,81]
Orientation Wall: orthogonal to ground [87,27]
Wall: verticallity constraint [81,82,27,80]
Window & door: alignment [90]
Principal axis Door: axis constraints [84]
Other Window: never stand-alone [82]
Door: never stand-alone [82]
Relationship-based Time-based Sequential relationships [34]
Relationship-based heuristics can be time-based or space-based. (e.g., the sequence of construction
objects)
Space-based (e.g., connections between Wall: intersects with ground [82,34]
objects or spaces). Roof: on top of walls and intersect on top of walls
[82]
Window: positioned in walls [82,34]
Door: positioned in walls [82,34]
Opening: the location on the wall [87]
Model-based Derived from existing model Pipes: alignment and comparision [11]
Uses as-planned BIM model to detect elements using overlap with as-built data. Elements: voxel occupancy [8]
Elements: ray thresholding [12]
Elements: commercial software [14]
Elements: thresholding [57]
Elements: machine learning [13]
Elements: point intensity [77]
Secondary components: RGB comparison [78]
Door: RGB comparison [91]

rules. Apart from a few complex elements, most elements can be computational challenges in automatic registration as the acquired
identified by applying these rules. point cloud data is voluminous.
3. Relationship-based constraints: The construction sequence is of
utmost importance, as it can accurately define the objects’ de­ It can be summarised that heuristics-based techniques usually
pendencies from schedules, enabling better element detection for require a significant amount of prior information. Generation of this
progress monitoring. These time-based relationships can be helpful information requires domain knowledge about construction and the
while there is limited visibility of the scene or occlusions. Space- type of elements being recognised. A recommended solution is to define
based constraints have been used in the literature using ontological a set of metaheuristics that generally applies to any construction dataset
relations [34] and shape grammar [35], but they are challenging to as a subset of all the heuristics; using them will make element identifi­
formulate and apply in practice. cation viable with less detailed information requirements.
4. Model-based constraints: Model-based element retrieval uses Though using some heuristics makes element identification easier, a
reverse engineering to recognise the elements. The point cloud is pure heuristic-based approach for CV-CPM can be unreliable and chal­
registered over the as-planned BIM model, and the elements are lenging to be implemented.
identified based on overlap occupancy. Although the method results
in high accuracy, its use is limited because it requires a detailed as- B. Learning-based approach
planned model. It has specific point cloud quality and reference
BIM model requirements [36]. The approach also faces Learning-based techniques (Fig. 2 (d), Table 5) are used on point

11
V.K. Reja et al. Automation in Construction 138 (2022) 104245

clouds for 3D shape classification, object detection, and segmentation. heuristics-based approaches may work for general geometric compo­
They are based on training features using a neural network and nents, they fail to perform when the construction scene becomes com­
computed weights to predict object or material classes. Two types of plex and do not follow conventional rules, relationships, and geometries.
learning approaches have been used: Learning-based approaches are more adaptive, as they can generalise
on-site data and are not based on structured domain knowledge.
1. Geometric Learning extracts geometrical features using descriptors Therefore, to utilise the advantages of both the approaches and
and feeds neural networks on statistics like distance, area, and an­ overcome their individual shortcomings, there is a need to explore the
gles. The geometric features are used to identify the element’s hybrid approach, i.e., using heuristics in conjunction with learning
physical presence. algorithms.
2. Appearance Learning extracts features like HSI (Hue-Saturation-
Intensity) colour values, material reflectance, and surface roughness 3.3. Progress monitoring
as features. The appearance-based features describe its operational
state. Progress monitoring requires a visual or quantitative comparison
between the as-planned and as-built models. Four progress monitoring
As observed from Table 5, only a few studies have detected elements levels identified as a part of this study are L-1, L-2, L-3, and L-4, as
for as-built modelling using learning-based techniques in the past. Most discussed in Section 2. The following sub-section discusses the key
of these methods detect geometric features and do not recognise the technologies and methods used for implementing these levels.
elements’ operational state. A few studies recognised the operational
state by using image processing techniques [37,38] and learning 3.3.1. Progress visualisation and comparison (L-1 and L-2)
appearance-based features [39]. L-1 based monitoring is currently the predominant method used at
For progress monitoring applications, learning-based approaches construction sites and only requires visualising the 3D as-built model
have been predominantly used on 2D image data, and there are very few built from spatial data obtained from the site. As an extension to L-1, L-2
studies involving 3D point cloud learning. Recent advances in machine based monitoring requires a visual comparison between as-planned and
learning and deep neural networks have motivated researchers to as-built status at a given time. This is also used on several projects and
explore this area. However, their application in point cloud-based explored by numerous research studies.
element recognition for construction has been limited because of chal­ These visualisations or comparisons can be made using selected en­
lenges such as: vironments, as shown in Fig. 2 (e). The literature review in Table 1
shows that most pipelines have used non-immersive environments such
1. 3D point cloud data are unstructured, large, and with varying point as a 3D viewer or web-based viewer. A few pipelines have used
densities, making 3D feature detection computing-intensive and immersive Extended Reality (XR) environments, which include
time-consuming. Augmented Reality (AR) [21,28], Virtual Reality (VR) [13,44], and
2. Scan data are generally noisy and have occluded components and Mixed Reality (MR) [12,45].
cluttered scenes. Also, the data are often susceptible to changes in While a few studies used traffic light-based coding to show progress
environmental factors such as rain, fog, dust, or mist. [21], the use of comparative and interactive UI has also been investi­
3. Learning models do not perform well when there is a significant gated [28]. Interactive UIs are gaining popularity, as they can pass in­
difference between the training and test data. formation among stakeholders and assign tasks or comments for the
4. Supervised learning requires extensive annotated training and workforce by adding features like annotating PDF notes into the envi­
testing datasets for various categories of incomplete building ele­ ronments. Advancements in gaming engines can be used to represent
ments: currently, the data is limited. point clouds and BIM models by importing and overlaying in a single VR
environment utilising Unity3D [13,44].
ScanNet [40], S3DIS [41], and ModelNET-40 [42] are 3D point cloud Based on the review of literature and practice, this study finds that
datasets consisting of different classes of building elements that are the capability of visualisation environments is underutilised for progress
currently used in studies. However, these data sets represent completed monitoring applications. With advancements in computing skills, rapid
buildings; for construction progress monitoring studies, data sets are development in immersive technologies, and required bandwidths
required to be acquired from buildings under construction. For material supported by the 5G standard, these platforms can provide the required
recognition, one such example of an image-based dataset is the Con­ information in real-time [46]. Focused research is required to generate
struction Material Library (CML) created in [9,37], which consist of interactive UI and integrate AR and VR environments to give the user the
more than 3000 images categorised into 20 construction material best of both worlds and utilise gaming engines to take progress visual­
classes. isation to the next level.
There are two approaches to generate the data. The first approach is
by generating realistic, rendered elements-based graphic models to 3.3.2. Progress quantification, schedule updating, and notifications (L-3
create synthetic data [43]. The second approach is to capture and pro­ and L-4)
cess real-world data with the high computing power of state-of-the-art The next level of progress monitoring, L-3, involves quantifying the
data acquisition devices. completed elements. Therefore, in continuation to Table 4, Fig. 5 shows
If the challenges mentioned earlier are solved, the learning-based a trade-off for selecting the quantity estimation method corresponding
techniques open doors to autonomous element detection and identifi­ to the type of model generated. Here, on the left-hand scale, the green-
cation for robust progress monitoring. to-red colour variation indicates the increase in complexity from low to
high. Similarly, the green-to-red colour variation indicates the decrease
Hybrid approaches in the ease of progress quantification from high to low on the right-hand
scale.
As discussed, both the heuristics-based approach and the learning- While the IFC/BIM models can produce direct BoQs from IFC-based
based approach face a set of challenges. While the former requires element geometries [31], other models require comparison with the as-
extensive hard coding of the domain knowledge, which is complex and planned model. For surface models and mesh models, object detection
challenging to define for all situations faced in construction, the latter (based on models overlapping) [13], detection by ray thresholding [12],
approach requires extensive training datasets, higher computational or enclosed volumetric comparison [47] can be made for estimating
resources, and knowledge of advanced computing techniques. Though quantities of completed elements.

12
V.K. Reja et al. Automation in Construction 138 (2022) 104245

Fig. 5. Trade-off for selecting Quantity Estimation method corresponding to the type of as-built model for progress quantification.

For comparison using voxel occupancy, the as-planned model is or discrepancies in the as-built model from the as-planned model is
voxelised and overlapped with as-built point clouds, after which essential for in-time decision making and can improve the project’s
thresholding is done to detect the occupied voxels [8]. Although it works performance.
well for smaller models, it may be computationally costly for larger In addition to providing updated information on progress status,
models; hence, it is not a preferred option. For direct usage of the point features to suggest interventions and control strategies and forecast
cloud, the most straightforward and widely-used method is Scan-vs-BIM, resulting outcomes will automate project controls even further.
which uses thresholding-based element identification. This method re­ Conventional control decisions taken by the project managers are
quires an as-planned 3D BIM and has its shortcomings [48], but the as- based on the know-how of issues specific to the project. Automated
built model generation step can be skipped, saving computation. Com­ control suggestions can be generated using cognitive computing on the
mercial project management software are also used for detecting de­ progress status and the data from sources such as the organisation’s
viations between as-planned and as-built models that can result in the Enterprise Resource Portal (ERP). Such an approach can recognise pat­
quantification of contractual scheduling metrics [14,44]. terns in activities that suffer delays and suggest interventions to bring
The selection of a model for progress monitoring depends on the the project back on track. There have been selected deployments using
project’s characteristics (type, budget, duration), the required level of AI-based simulation engines for addressing complications inherent in
progress monitoring, and other factors. A detailed study of these factors scheduling and project coordination [53]. Such deployments can also
is required to converge onto an appropriate pipeline for progress evaluate alternatives and provide real-time suggestions. This is an
monitoring based on the project-specific use case. emerging area with significant potential, and further exploration is
For L-4 progress monitoring, updating schedules and generating re­ required.
ports/notifications are included with progress quantification. This is This section presented a micro-analysis of the stages of the assembled
challenging; therefore, as shown in Table 1, only a few pipelines have CV-CPM framework. Various gaps were identified while discussing each
explored this area. Though schedule updating has not been fully auto­ section in detail. The following section discusses the application of the
mated yet in practice, conceptual schemes have been proposed [49]. CV-CPM framework to enable broader evaluation of various pipelines/
Some schemes have utilised the concept of precedence relationship components to develop benchmarks. It also summarises specific chal­
graphs and articulation points, where all objects must be finished before lenges identified within each stage.
the element linked to the articulation point can be built [50]. Few
studies have compared as-planned and as-built BIM for schedule 4. Discussion
updating [31,51]. As the as-built IFC/BIM model is updated, parametric
information has also been utilised to generate the updated schedules As presented and discussed in the previous section, numerous com­
automatically [52]. binations of input-process-output options are possible at each stage of
The primary challenge in schedule updating is that schedule LoD is the CV-CPM framework. For an integrated pipeline across all stages, the
different from BIM LoD, resulting in mismatch and incompatibility. combinations of options are immense. Given the pace at which tech­
Secondly, it requires an as-planned 4D model for smooth implementa­ nology is developing, the number of options will continue to increase.
tion, that is not always available for a construction project. Other However, only a limited combination of options will be feasible, and this
challenges include deciding the frequency of updating 4D-BIM. paper has proposed guidelines to structure these combinations.
Although a few research papers on CV-CPM have mentioned the A systematic approach is essential to explore and document the
concept of schedule updating in their pipelines, they have not specified performance of specific approaches/algorithms and integrated CV-CPM
the details. Therefore, there is a necessity to conduct and integrate pipelines. This section discusses requirements for benchmarking to
focused studies for tackling these specific challenges. enable a systematic study. The section also summarises potential
Likewise, only a few studies have explored an automatic notification research questions within each stage of CV-CPM and discusses its rele­
system that can trigger notifications while monitoring and controlling a vance to support the emerging area of Digital Twin.
construction site [47]. These can be obtained via SMS or e-mail, con­
taining warnings, reports, or graphical outputs based on the settings and
LoD required. The frequency and content of notifications should be 4.1. Benchmarking
selected to give construction managers broad insights about areas to
target for meeting schedules. As discussed earlier, providing managers Benchmarking evaluates pipelines, algorithms, devices, and tools
with real-time information/alerts about any deviation in the schedules using standard datasets and test methods. Fig. 6 outlines the broad re­
quirements to benchmark pipelines and components of CV-CPM, and

13
V.K. Reja et al. Automation in Construction 138 (2022) 104245

Fig. 6. Benchmarking of CV-CPM.

these are discussed below: quantitative metrics, qualitative measures will also be required to
evaluate subjective outcomes. For example, for benchmarking visual­
1. Datasets isation environments, qualitative factors like ease of use, skills required
for navigation, immersive features for comparison, and training re­
Though existing datasets contain relevant information for building quirements can be compared from a user feedback-based evaluation.
elements [40,41], these represent completed elements and not the Correspondingly, for benchmarking the algorithms, the evaluation
schedule-based construction progress of the elements as required for metrics can be in the form of accuracy, processing time, computational
progress monitoring. Hence, there is a requirement for creating an open- cost etc., which are quantitative.
source dataset that contains the data on elements as construction pro­ The integrated CV-CPM framework and the guidelines proposed for
gresses. As shown in Fig. 6, these can be synthetically created or ac­ each stage is expected to assist in developing a strategy and a roadmap
quired from the real world. The dataset type will change based on the for benchmarking. As a wide range of studies is required, the technology
particular stage under investigation. For example, for 3D reconstruction, roadmap needs to be developed collaboratively by the research com­
the type of dataset required for benchmarking reconstruction algorithms munity. A starting point for these studies can be to investigate the
will be in the form of raw images, videos, and depth images. subjective ratings proposed in this study and quantify these ratings
through controlled experimental testbeds. The CV-CPM framework can
2. Testbeds help in structuring and prioritising the areas to be explored in devel­
oping a roadmap so that they form a standard reference for bench­
While existing studies in the area have contributed significantly to marking studies.
knowledge about the applicability of specific algorithms and tools used,
the experimental set-ups and procedures used for these investigations 4.2. Future research directions
vary. As a result, repeatability, and comparison of results across research
groups are not feasible. Hence, for each CV-CPM stage, there is a need to This section highlights potential research questions within the three
design standard testbeds that define the experimental setup, procedures, stages of the CV-CPM framework. Several of these questions can be
and measurement parameters. As shown in Fig. 6, the testbed can be addressed through benchmarking studies, while others require explo­
either computational for benchmarking software (algorithms) or phys­ ration of new concepts.
ical for benchmarking the hardware (devices/sensors). For example, a Stage 1: Data acquisition and 3D reconstruction
testbed for data acquisition can have sensors capturing data with various
physical constraints, like the speed of capture, distance from an object, • How can we objectively assess the factors affecting data acquisition
lightning condition, resolution settings etc., that can be varied to technology and sensor mounting method shown in Fig. 3?
perform multiple experiments. Further, the testbeds should be flexible to • What are the appropriate sensing technology and mounting method
accommodate the requirements for collaborative explorations in the for data acquisition based on project types and project characteristics
area. (ex., Linear, underground, elevated etc.)?
• Based on factors identified in Fig. 4, How can we decide on using SfM
3. Component of CV-CPM and SLAM for a specific project?
• Vision-based data is usually enormous, and storage requirements are
As shown in Fig. 6, for CV-CPM, the components can be bench­ high. How sparse can data be without impacting the output?
marked independently or as pipelines integrating multiple components.
Each component can be evaluated for its independent performance. Stage 2: As-built modelling
Additionally, the pipelines can also be benchmarked for their effec­
tiveness for overall progress monitoring by varying specific components • How can the data be effectively pre-processed to meet the point
within the pipeline. This will standardise the approach to sequence and cloud quality requirements for an as-built model generation?
investigate various input-process-output combinations and enable a • How can a hybrid of heuristics and learning-based approaches be
systematic comparison of results with other studies. deployed, such that it exploits the advantages and overcomes the
shortcomings of each?
4. Evaluation Metrics • How can cognitive computing be successfully implemented to
recognise and measure complex geometries at reduced computa­
There are several factors to be considered for evaluation. While most tional cost accurately?
benchmarking studies within this framework require the definition of

14
V.K. Reja et al. Automation in Construction 138 (2022) 104245

Stage 3: Progress Monitoring papers, including recent papers in this area. Hence, it can be inferred
that it is holistic and robust.
• How can we objectively compare the computational requirements It was found that the four levels of progress monitoring identified in
and output accuracy for the level of progress monitoring based on the this study strongly influence the technology selection for the pipeline at
type of as-built models as shown in Table 4? each stage. For implementing progress monitoring on a construction
• How can virtual and physical environments be seamlessly connected project, it is recommended that the pipeline and components required
to improve visualisation? are selected based on the chosen level.
• How can we select an appropriate as-built model and progress Several areas for further research were discussed in the previous
quantification technique based on project characteristics and the section. Among these areas, advancements in the as-built modelling
trade-off mentioned in Fig. 5? stage are required to facilitate the quantification of progress. To enable
• How can we generate automatic schedules update and control sug­ this, exploring a hybrid approach that combines learning with heuristics
gestions by applying cognitive computing techniques to the progress is recommended.
data produced by CV-CPM? The need and requirements for benchmarking and future research
directions derived from the CV-CPM framework have been presented in
In addition to the specific areas identified above, the CV-CPM the Discussion section. Addressing these requirements and following a
framework is also relevant to support emerging areas of research and well-developed roadmap in this area is essential to move CV-CPM
development, such as Digital Twin. research from laboratory studies to field applications.
Digital Twin technologies enable the built facility’s virtual repre­
sentation to mirror and predict the state/behaviour of the physical fa­ Declaration of Competing Interest
cility. Key technologies for Digital Twin implementation include:
The authors confirm that all authors agreed with the current version
1. continuous streaming and processing of sensing data, and there is no conflict of interest among all authors with respect to this
2. threaded models and simulations for current/future state/behaviour paper.
of the facility,
3. formulating/implementing interventions to limit future state/ Acknowledgements
behaviour within specified limits.
The authors would like to thank the journal editor and reviewers for
Studies have envisioned the utility of Digital Twin in all phases of a their detailed constructive comments, which helped improve the paper.
construction project lifecycle. However, details on twinning an under- Additionally, the support, the first author received from the ‘Prime
construction facility are limited [54]. Commercial platforms such as Minister Research Fellowship (PMRF)’ Scheme from the ‘Ministry of
Autodesk Tandem [55] and Bentley iTwin [56] are available for creating Education, Government of India’ is gratefully acknowledged.
Digital Twin solutions. However, technological features available in
these platforms are currently in the early stages and focus on the oper­ References
ations and maintenance phase of the built asset.
A key issue in creating a Digital Twin for construction progress [1] J. Teizer, Status quo and open challenges in vision-based sensing and tracking of
temporary resources on infrastructure construction sites, Adv. Eng. Inform. 29
monitoring is that the geometry of the facility keeps expanding; hence (2015) 225–238, https://doi.org/10.1016/j.aei.2015.03.006.
the corresponding sensor positioning is also dynamic. In this context, [2] T. Omar, M.L. Nehdi, Data acquisition technologies for construction progress
computer vision-based sensing would be the most appropriate input to tracking, Autom. Constr. 70 (2016) 143–155, https://doi.org/10.1016/j.
autcon.2016.06.016.
capture geometrical attributes directly and rapidly. The proposed CV-
[3] S. Paneru, I. Jeelani, Computer vision applications in construction: current state,
CPM framework identifies the essential technologies to embed prog­ opportunities & challenges, Autom. Constr. 132 (2021), 103940, https://doi.org/
ress monitoring capabilities in a Digital Twin. 10.1016/j.autcon.2021.103940.
[4] Z. Ma, S. Liu, A review of 3D reconstruction techniques in civil engineering and
Pipelines that can stream and process the progress data accurately
their applications, Adv. Eng. Inform. 37 (2018) 163–174, https://doi.org/10.1016/
and rapidly to mirror the progress in the virtual model will support j.aei.2018.05.005.
Digital Twinning for progress monitoring. For this, the process within [5] B. Ekanayake, J.K.-W. Wong, A.A.F. Fini, P. Smith, Computer vision-based interior
and across the stages of the CV-CPM framework needs to be optimised construction progress monitoring: a literature review and future research
directions, Autom. Constr. 127 (2021), 103705, https://doi.org/10.1016/j.
for real-time updates. autcon.2021.103705.
In addition to mirroring the progress, a Digital Twin can forecast [6] T. Patel, B.H.W. Guo, Y. Zou, A scientometric review of construction progress
construction progress as well as simulate and evaluate control measures monitoring studies, Eng. Constr. Archit. Manag. (2021), https://doi.org/10.1108/
ECAM-10-2020-0799.
to bring the project back on track. As the usage of autonomous con­ [7] B. Omair, A. Alturki, An improved method for taxonomy development in
struction equipment increases, the control measures can be transmitted information systems, Int. J. Adv. Comput. Sci. Appl. 11 (2020) 535–540, https://
from the Digital to the physical world to be implemented by the doi.org/10.14569/IJACSA.2020.0110470.
[8] M. Golparvar-Fard, F. Peña-Mora, S. Savarese, Automated progress monitoring
equipment. using unordered daily construction photographs and IFC-based building
information models, J. Comput. Civ. Eng. 29 (2015) 04014025, https://doi.org/
5. Conclusion 10.1061/(ASCE)CP.1943-5487.0000205.
[9] A. Dimitrov, M. Golparvar-Fard, Vision-based material recognition for automated
monitoring of construction progress and generating building information modeling
CV-CPM has the potential of creating an immense impact by from unordered site image collections, Adv. Eng. Inform. 28 (2014) 37–49, https://
providing real-time, accurate, reliable information to construction doi.org/10.1016/j.aei.2013.11.002.
[10] J.R. Bognot, C.G. Candido, A.C. Blanco, J.R.Y. Montelibano, Building construction
managers. Though a significant amount of work has been done in the last progress monitoring using unmanned aerial system (UAS), low-cost
decade, specific challenges remain due to the construction industry’s photogrammetry, and geographic information system (GIS), ISPRS Ann.
dynamic nature and complexities at sites. Some of these challenges have Photogramm. Remote Sens. Spat. Inform. Sci. IV–2 (2018) 41–47, https://doi.org/
10.5194/isprs-annals-IV-2-41-2018.
been addressed; however, significant gaps need to be filled to make
[11] F. Bosché, M. Ahmed, Y. Turkan, C.T. Haas, R. Haas, The value of integrating scan-
pipelines accurate and automated to meet rising user expectations of to-BIM and scan-vs-BIM techniques for construction monitoring using laser
real-time feedback. scanning and BIM: the case of cylindrical MEP components, Autom. Constr. 49
This study has comprehensively reviewed individual pipelines and (2015) 201–213, https://doi.org/10.1016/j.autcon.2014.05.014.
[12] M. Kopsida, I. Brilakis, Real-time volume-to-plane comparison for mixed
formulated an integrated CV-CPM framework, as shown in Table 1 and reality–based progress monitoring, J. Comput. Civ. Eng. 34 (2020) 04020016,
Fig. 2, respectively. The proposed framework positions all the reviewed https://doi.org/10.1061/(ASCE)CP.1943-5487.0000896.

15
V.K. Reja et al. Automation in Construction 138 (2022) 104245

[13] F. Pour Rahimian, S. Seyedzadeh, S. Oliver, S. Rodriguez, N. Dawood, On-demand [39] H. Hamledari, B. McCabe, S. Davari, Automated computer vision-based detection
monitoring of construction projects through a game-like hybrid application of BIM of components of under-construction indoor partitions, Autom. Constr. 74 (2017)
and machine learning, Autom. Constr. 110 (2020), 103012, https://doi.org/ 78–94, https://doi.org/10.1016/j.autcon.2016.11.009.
10.1016/j.autcon.2019.103012. [40] A. Dai, A.X. Chang, M. Savva, M. Halber, T. Funkhouser, M. Niessner, ScanNet:
[14] Z. Pučko, N. Šuman, D. Rebolj, Automated continuous construction progress Richly-annotated 3D reconstructions of indoor scenes, in: Conference on Computer
monitoring using multiple workplace real time 3D scans, Adv. Eng. Inform. 38 Vision and Pattern Recognition (CVPR), IEEE, 2017, pp. 2432–2443, https://doi.
(2018) 27–40, https://doi.org/10.1016/j.aei.2018.06.001. org/10.1109/CVPR.2017.261.
[15] A. Adán, B. Quintana, S.A. Prieto, F. Bosché, An autonomous robotic platform for [41] I. Armeni, O. Sener, A.R. Zamir, H. Jiang, I. Brilakis, M. Fischer, S. Savarese, 3D
automatic extraction of detailed semantic models of buildings, Autom. Constr. 109 semantic parsing of large-scale indoor spaces, in: Conference on Computer Vision
(2020), 102963, https://doi.org/10.1016/j.autcon.2019.102963. and Pattern Recognition (CVPR), IEEE, 2016, pp. 1534–1543, https://doi.org/
[16] K. Asadi, A. Kalkunte Suresh, A. Ender, S. Gotad, S. Maniyar, S. Anand, 10.1109/CVPR.2016.170.
M. Noghabaei, K. Han, E. Lobaton, T. Wu, An integrated UGV-UAV system for [42] Zhirong Wu, S. Song, A. Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, J. Xiao,
construction site data collection, Autom. Constr. 112 (2020), 103068, https://doi. 3D ShapeNets: A deep representation for volumetric shapes, in: Conference on
org/10.1016/j.autcon.2019.103068. Computer Vision and Pattern Recognition (CVPR), IEEE, 2015, pp. 1912–1920,
[17] Q. Wang, Y. Tan, Z. Mei, Computational methods of acquisition and processing of https://doi.org/10.1109/CVPR.2015.7298801.
3D point cloud data for construction applications, Arch. Comput. Method Eng. 27 [43] P. Bhadaniya, V.K. Reja, K. Varghese, Mixed reality-based dataset generationfor
(2020) 479–499, https://doi.org/10.1007/s11831-019-09320-4. learning-based scan-to-BIM, in: Pattern Recognition - Lecture Notes in Computer
[18] C. Wu, VisualSFM: A Visual Structure from Motion System. http://ccwu.me/vsfm/, Science (LNCS), Springer International Publishing, Cham, 2021, pp. 389–403,
2011 (accessed July 19, 2020). https://doi.org/10.1007/978-3-030-68787-8.
[19] Mapillary, OpenSfM: Open Source Structure from Motion Pipeline. https://github. [44] S. Vincke, R. de Lima Hernandez, M. Bassier, M. Vergauwen, Immersive
com/mapillary/OpenSfM, 2018 (accessed July 19, 2020). visualisation of construction site point cloud data, meshes and BIM models in a VR
[20] Y. Furukawa, B. Curless, S.M. Seitz, R. Szeliski, Reconstructing building interiors environment using a gaming engine, Int. Arch. Photogramm. Remote. Sens. Spat.
from images, in: 12th International Conference on Computer Vision, IEEE, 2009, Inf. Sci. XLII-5 (W2) (2019) 77–83, https://doi.org/10.5194/isprs-archives-XLII-5-
pp. 80–87, https://doi.org/10.1109/ICCV.2009.5459145. W2-77-2019.
[21] M. Golparvar-Fard, F. Peña-Mora, S. Savarese, Integrated sequential as-built and [45] A. Khairadeen Ali, O.J. Lee, D. Lee, C. Park, Remote indoor construction progress
as-planned representation with D4AR tools in support of decision-making tasks in monitoring using extended reality, Sustainability 13 (2021) 2290, https://doi.org/
the AEC/FM industry, J. Constr. Eng. Manag. 137 (2011) 1099–1116, https://doi. 10.3390/su13042290.
org/10.1061/(ASCE)CO.1943-7862.0000371. [46] V.K. Reja, K. Varghese, Impact of 5G technology on IoT applications in
[22] T. Bailey, H. Durrant-Whyte, Simultaneous localization and mapping (SLAM): part construction project management, in: 36th International Symposium on
II, IEEE Robot. Automat. Mag. 13 (2006) 108–117, https://doi.org/10.1109/ Automation and Robotics in Construction (ISARC 2019), Banff, Canada, 2019,
MRA.2006.1678144. pp. 209–217, https://doi.org/10.22260/ISARC2019/0029.
[23] B. Triggs, P.F. McLauchlan, R.I. Hartley, A.W. Fitzgibbon, Bundle adjustment — a [47] H. Omar, L. Mahdjoubi, G. Kheder, Towards an automated photogrammetry-based
modern synthesis, in: International Workshop on Vision Algorithms, Greece, 2000, approach for monitoring and controlling construction site activities, Comput. Ind.
pp. 298–372, https://doi.org/10.1007/3–540-44480-7_21. 98 (2018) 172–182, https://doi.org/10.1016/j.compind.2018.03.012.
[24] C.-Z. Sun, B. Zhang, J.-K. Wang, C.-S. Zhang, A review of visual SLAM based on [48] M. Bassier, S. Vincke, L. Mattheuwsen, R. de Lima Hernandez, J. Derdaele,
unmanned systems, in: 2nd International Conference on Artificial Intelligence and M. Vergauwen, Percentage of completion of in-situ cast concrete walls using point
Education (ICAIE), IEEE, 2021, pp. 226–234, https://doi.org/10.1109/ cloud data and BIM, Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. XLII-5
ICAIE53562.2021.00055. (W2) (2019) 21–28, https://doi.org/10.5194/isprs-archives-XLII-5-W2-21-2019.
[25] I. Mitsugami, Bundler: structure from motion for unordered image collections, [49] H. Son, C. Kim, Y. Kwon Cho, Automated schedule updates using as-built data and
J. Inst. Image Inform. Telev. Eng. 65 (2011) 479–482, https://doi.org/10.3169/ a 4D building information model, J. Manag. Eng. 33 (2017) 04017012, https://doi.
itej.65.479. org/10.1061/(ASCE)ME.1943-5479.0000528.
[26] R. Maalek, D.D. Lichti, J.Y. Ruwanpura, Automatic recognition of common [50] A. Braun, A. Borrmann, S. Tuttas, U. Stilla, Big data in smart building operation, in:
structural elements from point clouds for automated progress monitoring and eWork and eBusiness in Architecture, Engineering and Construction, CRC Press,
dimensional quality control in reinforced concrete construction, Remote Sens. 11 2014, pp. 765–804, https://doi.org/10.1201/b17396-20.
(2019) 1102, https://doi.org/10.3390/rs11091102. [51] H. Hamledari, B. McCabe, S. Davari, A. Shahi, Automated schedule and progress
[27] P. Hübner, M. Weinmann, S. Wursthorn, Voxel-based indoor reconstruction from updating of IFC-based 4D BIMs, J. Comput. Civ. Eng. 31 (2017) 04017012, https://
HoloLens triangle meshes, ISPRS Ann. Photogramm. Remote Sens. Spat. Inform. doi.org/10.1061/(ASCE)CP.1943-5487.0000660.
Sci. V-4–2020 (2020) 79–86, https://doi.org/10.5194/isprs-annals-V-4-2020-79- [52] H. Kim, K. Anderson, S. Lee, J. Hildreth, Generating construction schedules
2020. through automatic data extraction using open BIM (building information
[28] S. Zollmann, C. Hoppe, S. Kluckner, C. Poglitsch, H. Bischof, G. Reitmayr, modeling) technology, Autom. Constr. 35 (2013) 285–295, https://doi.org/
Augmented reality for construction site monitoring and documentation, Proc. IEEE 10.1016/j.autcon.2013.05.020.
102 (2014) 137–154, https://doi.org/10.1109/JPROC.2013.2294314. [53] ALICE Technologies, AI for Construction Management. https://blog.alicetech
[29] H. Macher, T. Landes, P. Grussenmeyer, From point clouds to building information nologies.com/whitepapers/ai-for-construction-management-whitepaper, 2021
models: 3D semi-automatic reconstruction of indoors of existing buildings, Appl. (accessed January 10, 2022).
Sci. 7 (2017) 1030, https://doi.org/10.3390/app7101030. [54] C. Boje, A. Guerriero, S. Kubicki, Y. Rezgui, Towards a semantic construction
[30] X. Xiong, A. Adan, B. Akinci, D. Huber, Automatic creation of semantically rich 3D digital twin: directions for future research, Autom. Constr. 114 (2020), 103179,
building models from laser scanner data, Autom. Constr. 31 (2013) 325–337, https://doi.org/10.1016/j.autcon.2020.103179.
https://doi.org/10.1016/j.autcon.2012.10.006. [55] Autodesk Tandem, Autodesk. https://intandem.autodesk.com/, 2021 (accessed
[31] H. Mahami, F. Nasirzadeh, A. Hosseininaveh Ahmadabadian, S. Nahavandi, January 10, 2022).
Automated Progress controlling and monitoring using daily site images and [56] Bentley, An Open Platform for Infrastructure Digital Twins: How Bentley Systems’
building information modelling, Buildings 9 (2019) 70, https://doi.org/10.3390/ iTwin Platform is Enabling a Digital Twin Ecosystem for the World’s Infrastructure,
buildings9030070. A Frost Sullivan White Pap. https://prod-bentleycdn.azureedge.net/-/media/files/
[32] V. Pătrăucean, I. Armeni, M. Nahangi, J. Yeung, I. Brilakis, C. Haas, State of documents/white-papers/wp_frost_sullivan_open_platform_for_infrastructure_di
research in automatic as-built modelling, Adv. Eng. Inform. 29 (2015) 162–171, gital_twins.pdf?la=pt-br&modified=00010101000000, 2021.
https://doi.org/10.1016/j.aei.2015.01.001. [57] A. Braun, S. Tuttas, A. Borrmann, U. Stilla, Improving progress monitoring by
[33] S. Zeng, J. Chen, Y.K. Cho, User exemplar-based building element retrieval from fusing point clouds, semantic data and computer vision, Autom. Constr. 116
raw point clouds using deep point-level features, Autom. Constr. 114 (2020), (2020), 103210, https://doi.org/10.1016/j.autcon.2020.103210.
103159, https://doi.org/10.1016/j.autcon.2020.103159. [58] K. Karsch, M. Golparvar-Fard, D. Forsyth, ConstructAide: analyzing and visualizing
[34] K.K. Han, D. Cline, M. Golparvar-Fard, Formalized knowledge of construction construction sites through photographs and building models, ACM Trans. Graph.
sequencing for visual monitoring of work-in-progress via incomplete point clouds 33 (2014) 1–11, https://doi.org/10.1145/2661229.2661256.
and low-LoD 4D BIMs, Adv. Eng. Inform. 29 (2015) 889–901, https://doi.org/ [59] F. Arif, W.A. Khan, Smart progress monitoring framework for building construction
10.1016/j.aei.2015.10.006. elements using videography–MATLAB–BIM integration, international, J. Civ. Eng.
[35] H. Tran, K. Khoshelham, A. Kealy, L. Díaz-Vilariño, Shape grammar approach to 3D 19 (2021) 717–732, https://doi.org/10.1007/s40999-021-00601-3.
modeling of indoor environments using point clouds, J. Comput. Civ. Eng. 33 [60] S. Kim, S. Kim, D.-E. Lee, Sustainable application of hybrid point cloud and BIM
(2019) 04018055, https://doi.org/10.1061/(ASCE)CP.1943-5487.0000800. method for tracking construction progress, Sustainability 12 (2020) 4106, https://
[36] D. Rebolj, Z. Pučko, N.Č. Babič, M. Bizjak, D. Mongus, Point cloud quality doi.org/10.3390/su12104106.
requirements for scan-vs-BIM based automated construction progress monitoring, [61] Z. Wang, Q. Zhang, B. Yang, T. Wu, K. Lei, B. Zhang, T. Fang, Vision-based
Autom. Constr. 84 (2017) 323–334, https://doi.org/10.1016/j. framework for automatic Progress monitoring of precast walls by using
autcon.2017.09.021. surveillance videos during the construction phase, J. Comput. Civ. Eng. 35 (2021)
[37] K.K. Han, M. Golparvar-Fard, Appearance-based material classification for 04020056, https://doi.org/10.1061/(ASCE)CP.1943-5487.0000933.
monitoring of operation-level construction progress using 4D BIM and site [62] A. Pushkar, M. Senthilvel, K. Varghese, Automated progress monitoring of masonry
photologs, Autom. Constr. 53 (2015) 44–57, https://doi.org/10.1016/j. activity using photogrammetric point cloud, in: 35th International Symposium on
autcon.2015.02.007. Automation and Robotics in Construction (ISARC 2018), Berlin, Germany, 2018,
[38] K. Han, J. Degol, M. Golparvar-Fard, Geometry- and appearance-based reasoning https://doi.org/10.22260/ISARC2018/0125.
of construction progress monitoring, J. Constr. Eng. Manag. 144 (2018) 04017110, [63] K.K. Han, M. Golparvar-Fard, Automated monitoring of operation-level
https://doi.org/10.1061/(ASCE)CO.1943-7862.0001428. construction progress using 4D BIM and daily site photologs, in: Construction

16
V.K. Reja et al. Automation in Construction 138 (2022) 104245

Research Congress 2014, American Society of Civil Engineers, Reston, VA, 2014, Symposium on Automation and Robotics in Construction (ISARC 2021), Dubai,
pp. 1033–1042, https://doi.org/10.1061/9780784413517.106. UAE, 2021, pp. 349–356, https://doi.org/10.22260/ISARC2021/0049.
[64] A. Rashidi, I. Brilakis, P. Vela, Generating absolute-scale point cloud data of built [78] A. Adán, B. Quintana, S.A. Prieto, F. Bosché, Scan-to-BIM for ‘secondary’ building
infrastructure scenes using a monocular camera setting, J. Comput. Civ. Eng. 29 components, Adv. Eng. Inform. 37 (2018) 119–138, https://doi.org/10.1016/j.
(2015) 04014089, https://doi.org/10.1061/(ASCE)CP.1943-5487.0000414. aei.2018.05.001.
[65] M. Bueno, H. González-Jorge, J. Martínez-Sánchez, H. Lorenzo, Automatic point [79] S. Nikoohemat, A.A. Diakité, S. Zlatanova, G. Vosselman, Indoor 3D reconstruction
cloud coarse registration using geometric keypoint descriptors for indoor scenes, from point clouds for optimal routing in complex buildings to support disaster
Autom. Constr. 81 (2017) 134–148, https://doi.org/10.1016/j. management, Autom. Constr. 113 (2020), 103109, https://doi.org/10.1016/j.
autcon.2017.06.016. autcon.2020.103109.
[66] M. Bueno, F. Bosché, H. González-Jorge, J. Martínez-Sánchez, P. Arias, 4-plane [80] S. Ochmann, R. Vock, R. Klein, Automatic reconstruction of fully volumetric 3D
congruent sets for automatic registration of as-is 3D point clouds with 3D BIM building models from oriented point clouds, ISPRS J. Photogramm. Remote Sens.
models, Autom. Constr. 89 (2018) 120–134, https://doi.org/10.1016/j. 151 (2019) 251–262, https://doi.org/10.1016/j.isprsjprs.2019.03.017.
autcon.2018.01.014. [81] C. Wang, Y.K. Cho, C. Kim, Automatic BIM component extraction from point clouds
[67] L. Lei, Y. Zhou, H. Luo, P.E.D. Love, A CNN-based 3D patch registration approach of existing buildings for sustainability applications, Autom. Constr. 56 (2015)
for integrating sequential models in support of progress monitoring, Adv. Eng. 1–13, https://doi.org/10.1016/j.autcon.2015.04.001.
Inform. 41 (2019), 100923, https://doi.org/10.1016/j.aei.2019.100923. [82] S. Pu, G. Vosselman, Knowledge based reconstruction of building models from
[68] C. Kim, H. Son, C. Kim, Fully automated registration of 3D data to a 3D CAD model terrestrial laser scanning data, ISPRS J. Photogramm. Remote Sens. 64 (2009)
for project progress monitoring, Autom. Constr. 35 (2013) 587–594, https://doi. 575–584, https://doi.org/10.1016/j.isprsjprs.2009.04.001.
org/10.1016/j.autcon.2013.01.005. [83] H. Tran, K. Khoshelham, Procedural reconstruction of 3D indoor models from Lidar
[69] C.C. Kim, C.C. Kim, H. Son, C.C. Kim, Automated construction progress data using reversible jump Markov chain Monte Carlo, Remote Sens. 12 (2020)
measurement using a 4D building information model and 3D data, Autom. Constr. 838, https://doi.org/10.3390/rs12050838.
31 (2013) 75–82, https://doi.org/10.1016/j.autcon.2012.11.041. [84] S. Franz, R. Irmler, U. Rüppel, Real-time collaborative reconstruction of digital
[70] S. Roh, Z. Aziz, F. Peña-Mora, An object-based 3D walk-through model for interior building models with mobile devices, Adv. Eng. Inform. 38 (2018) 569–580,
construction progress monitoring, Autom. Constr. 20 (2011) 66–75, https://doi. https://doi.org/10.1016/j.aei.2018.08.012.
org/10.1016/j.autcon.2010.07.003. [85] L. Barazzetti, Parametric as-built model generation of complex shapes from point
[71] J. Lee, H. Son, C. Kim, C. Kim, Skeleton-based 3D reconstruction of as-built clouds, Adv. Eng. Inform. 30 (2016) 298–311, https://doi.org/10.1016/j.
pipelines from laser-scan data, Autom. Constr. 35 (2013) 199–207, https://doi. aei.2016.03.005.
org/10.1016/j.autcon.2013.05.009. [86] L. Díaz-Vilariño, K. Khoshelham, J. Martínez-Sánchez, P. Arias, 3D modeling of
[72] Y.-F. Liu, S. Cho, B.F. Spencer, J.-S. Fan, Concrete crack assessment using digital building indoor spaces and closed doors from imagery and point clouds, Sensors 15
image processing and 3D scene reconstruction, J. Comput. Civ. Eng. 30 (2016) (2015) 3491–3512, https://doi.org/10.3390/s150203491.
04014124, https://doi.org/10.1061/(ASCE)CP.1943-5487.0000446. [87] A. Arnaud, J. Christophe, M. Gouiffes, M. Ammi, 3D reconstruction of indoor
[73] H. Son, C. Kim, C. Kim, Fully automated as-built 3D pipeline extraction method building environments with new generation of tablets, in: Proceedings of the 22nd
from laser-scanned data based on curvature computation, J. Comput. Civ. Eng. 29 ACM Conference on Virtual Reality Software and Technology, ACM, New York, NY,
(2015) 1–9, https://doi.org/10.1061/(ASCE)CP.1943-5487.0000401. USA, 2016, pp. 187–190, https://doi.org/10.1145/2993369.2993403.
[74] Y. Li, W. Li, S. Tang, W. Darwish, Y. Hu, W. Chen, Automatic indoor as-built [88] X. Liu, Y. Zhang, X. Ling, Y. Wan, L. Liu, Q. Li, TopoLAP: topology recovery for
building information models generation by using low-cost RGB-D sensors, Sensors building reconstruction by deducing the relationships between linear and planar
20 (2020) 293, https://doi.org/10.3390/s20010293. primitives, Remote Sens. 11 (2019) 1372, https://doi.org/10.3390/rs11111372.
[75] J. Chen, Z. Kira, Y.K. Cho, Deep learning approach to point cloud scene [89] J. Chen, Y. Fang, Y.K. Cho, Unsupervised recognition of volumetric structural
understanding for automated scan to 3D reconstruction, J. Comput. Civ. Eng. 33 components from building point clouds, in: Computing in Civil Engineering 2017,
(2019) 04019027, https://doi.org/10.1061/(ASCE)CP.1943-5487.0000842. American Society of Civil Engineers, Reston, VA, 2017, pp. 34–42, https://doi.org/
[76] D. Iwaszczuk, Z. Koppanyi, N.A. Gard, B. Zha, C. Toth, A. Yilmaz, Semantic 10.1061/9780784480823.005.
labeling of structural elements in buildings by fusing RGB and depth images in an [90] M. Previtali, L. Díaz-Vilariño, M. Scaioni, Towards automatic reconstruction of
encoder-decoder CNN framework, Int. Arch. Photogramm. Remote. Sens. Spat. Inf. indoor scenes from incomplete point clouds: door and window detection and
Sci. XLII–1 (2018) 225–232, https://doi.org/10.5194/isprs-archives-XLII-1-225- regularization, Int. Arch. Photogramm. Remote Sens. Spat. Inform. Sci. 42 (2018)
2018. 573–578, https://doi.org/10.5194/isprs-archives-XLII-4-507-2018.
[77] V.K. Reja, P. Bhadaniya, K. Varghese, Q.P. Ha, Vision-based progress monitoring of [91] B. Quintana, S.A. Prieto, A. Adán, F. Bosché, Door detection in 3D coloured point
building structures using point-intensity approach, in: Proc. 38th International clouds of indoor environments, Autom. Constr. 85 (2018) 146–166, https://doi.
org/10.1016/j.autcon.2017.10.016.

17