Engineering Applications of Artificial Intelligence

Volume 117, Part A, January 2023, 105478

Survey paper

Computer vision framework for crack detection of civil

infrastructure—A review
Dihao Ai a, d , Guiyuan Jiang b , Siew-Kei Lam c , Peilan He c , Chengwu Li d

Show more

Share Cite

https://doi.org/10.1016/j.engappai.2022.105478 Get rights and content

Abstract

Civil infrastructure (e.g., buildings, roads, underground tunnels) could lose its expected
physical and functional conditions after years of operation. Timely and accurate inspection
and assessment of such infrastructures are essential to ensure safety and serviceability, e.g., by
preventing unsafe working conditions and hazards. Cracks, which are one of the most
common distress, can indicate severe structural integrity issues that threaten the safety of the
structure and people in the environment. As such, accurate, fast, and automatic detection of
cracks on structure surfaces is a major issue for a variety of civil engineering applications. Due
to advances in hardware data acquisition systems, significant progress has been made in the
automatic detection and quantification of cracks in recent decades. This paper provides a
comprehensive review of the research progress and prospects in computer vision frameworks
for crack detection of civil infrastructures from multiple materials, including asphalt,
concrete, and metal-like materials. The review encompasses major components of typical
frameworks, i.e., data acquisition techniques, publicly available datasets, detection algorithms,
and evaluation metrics. In particular, we provide a taxonomy of detection algorithms with a
detailed discussion of the advantages, limitations, and application scenarios of the methods in
each category, as well as the relationships between methods of different categories. We also
discuss unsolved issues and key challenges in crack detection that could drive future research
directions.

Introduction

Civil infrastructure such as highways, pavement, bridges, tunnels, etc., may lose their expected
physical and functional conditions due to long-term operation and exposure to severe
scenarios associated with internal defects, aggressive usage, and continuous load changes.
Inspection and condition assessment of civil infrastructures are necessary to ensure safety and
serviceability (Wu and Liu, 2021, Taheri, 2019, Salehi et al., 2019), where a lack thereof could
lead to disastrous incidents. For example, at 18:01 pm on August 1, 2007, the I-35 W
Mississippi River Bridge connecting the east side of downtown Minneapolis and Mars-
Holmes suddenly collapsed (O’Connell et al., 2007). Dozens of cars fell into the river, causing
13 fatalities and 145 people injured. The accident is one of the most serious bridge collapse
incidents in the United States in recent decades caused by non-natural disasters or external
forces. It was reported that 68% of the major roads and highways in the USA are in poor
condition, which increased vehicle operating costs by USD 61 billion annually. It is estimated
that an additional USD 130 billion is required to improve the pavements’
condition (Gopalakrishnan, 2018). To prevent unsafe working conditions and hazards, there is
a need for continuous and periodic inspection of the infrastructures, which calls for the
development of accurate and fast inspection methods.

One of the most common types of defects in civil infrastructure is cracking, which evolves
rapidly under the impact of heavy traffic, aging of materials, and drastic environmental
changes. Cracks often develop and extend in three-dimension on the surface and inside the
structure, which eventually causes local failure or fracture. Since cracking can accelerate the
deterioration process, the occurrence and severity of cracking serve as important indicators of
the need for maintenance. Crack detection is the first step for several diagnosis and inspection
processes. The detection of cracks not only enables the prediction of future conditions and
predictive maintenance, but also supports investment planning through the management of
limited repair resources, thus ensuring the infrastructure still meets its service requirements
with limited maintenance effort.

Conventional methods for inspection of cracks on infrastructure surfaces rely on professional

engineers with rich experience to record the irregular conditions on structures at regular
times or after disasters. They manually assess the physical and functional conditions of
infrastructures to ensure their safety and serviceability still meet the expected requirements.
Although the manual inspection is widely practiced, it is time-consuming, labor-intensive,
and costly to perform manual inspection and evaluation methods at fixed/regular intervals.
Also, manual inspection is prone to subjectivity and can potentially hinder the dependability
of the quantitative analysis as it completely depends on the specialists’ knowledge and
experience. With the progress of visual sensing technologies (e.g., digital cameras), which are
low cost, high resolution, reliable, and easy-to-use, vision-based methods for crack detection
and structural health inspection are gaining popularity. Fig. 1 shows a typical vision-based
inspection pipeline that consists of five processes, i.e., ‘Acquisition–Detection–Measurement–
Assessment–Decision’ (ADMAD). ADMAD helps agencies monitor the civil infrastructure,
optimize limited maintenance resources, and prevent catastrophic accidents. Visual systems
can be permanently installed and are capable of continuously monitoring structural health.
To facilitate efficient infrastructure maintenance, it is critical to achieve accurate, reliable, and
automatic crack detection, segmentation, and classification. As such, a large number of
automatic crack detection algorithms have been developed over the last two decades. In
particular, recent years have witnessed remarkable developments made by deep learning
techniques for crack detection.

A typical automated crack detection framework takes a video or an image as input and returns
one or more regions or pixels as output representing the defect or damage of materials. The
type, number, width, and length of the cracks on the structural surface show the earliest
degradation level of the structures. However, achieving accurate and efficient detection and
quantification of the cracks (e.g., the type, severity, and extent of surface cracking) on civil
infrastructure elements (e.g., buildings, bridges, tunnels, pipes, and roads) is challenging. This
is because crack detection is affected by many factors such as the complex geometries and
typologies of cracks (shape, magnitude), varying degrees of cracking, irregular patterned
background, camera distance, environmental conditions (lighting and shading), and limited
data, etc. Specifically, (1) The geometries and typologies of cracks (shape, magnitude) are
complex and challenging to describe with quantitative features. Various types of cracks show
diverse morphology (e.g., line, graph, hole, and other irregular geometries), magnitudes
(e.g., number, width, length, etc.), shapes (e.g., longitudinal, transverse, alligator), and
backgrounds (e.g., complex scene, similar texture, uneven illumination, etc.). (2) There are
various background noises of different patterns and textures. Taking the asphalt pavement
images as an example, the pothole, ruts, looseness, and subsidence could appear very similar
to real cracks. Also, concrete bridges may contain spalling/joint spall, efflorescence, and holes,
while metal-like beams usually contain joints, seals, bearings, and other connections that
present more complex geometry. The existence of such noises makes it difficult to distinguish
cracks from the backgrounds. (3) The process of capturing crack images also affects the
subsequent detection process. Specifically, the data acquisition system needs to limit and
adjust the internal and external parameters, including distance, viewing angle, focal length,
lighting conditions, etc. These factors will affect the quality of resultant data. (4) The crack
detection problems also suffer from limited public datasets with labels. The datasets from
different application domains (e.g., asphalt pavement and concrete bridges) have significantly
different characteristics. Images of a crack dataset typically have severe label noise, which
could be generated by inter-observer variability or errors due to human annotators or
computer-generated labels. As such, many existing works allow a tolerance margin in
measuring the coincidence between the detected cracks and the ground truth (Shi et al., 2016,
Li et al., 2017a, Ai et al., 2018). Also, due to the characteristics of cracks, label imbalance in
training data is commonly observed, e.g., the crack pixels are much fewer than the background
pixels. Due to label imbalance, the training model converges to major classes (background)
faster than minor classes (cracks), such that most examples in the major class exhibit small
losses. That is, there is a risk of discarding most examples in the minor class. However, high-
loss examples (e.g., cracks) are favored for the crack detection problem. A large body of work in
computer vision-based methods for automated crack detection has been undertaken in recent
years to address these challenges.

To date, various techniques have been proposed and investigated for crack (defect, distress,
damage, anomaly) detection and physical condition assessment of civil infrastructures. Several
surveys have also been conducted to review the existing image-based automatic crack
detection approaches from various perspectives. These surveys typically focus only on
techniques for specific infrastructures or materials.

For example, some surveys focused on reviewing existing methods for specific infrastructures,
e.g., pavements, tunnels, etc. The performance of six pavement distress segmentation methods
was quantitatively evaluated in Tsai et al. (2009), including dynamic optimization, Canny edge,
statistical thresholding, multi-scale wavelet, seed verification, and iterative clipping, by
adopting a scoring measure. The work in Chambon and Moliard (2011) also focused on
pavement assessment and compared the automatic detection methods that are based on
morphological tools. These surveys only considered the traditional image processing
methods. Wang (2011) discussed technological advances in automated detection of pavements
using 3D laser imaging methodology. Instead of reviewing detection methods, they focused
on the technical details and difficulties in capturing 1-millimeter surface images of pavements
and introducing their 3D laser imaging system called PaveVision3D. Recently, the work
in Hsieh and Tsai (2020) reviewed techniques for processing 3D pavement images and
evaluated the performance of shallow machine learning and deep learning algorithms. The
authors also suggested that solving the false-positive problem is an important direction in
further work.

On the other hand, some other surveys focus on methods that detect cracks on surfaces of a
particular material. For example, Koch et al. (2014) reviewed earlier vision-based inspection
methods for large concrete structures using Building Information Modeling (BIM)
techniques. Payab et al. (2019) briefly conducted the development using image-based methods
for detecting cracks in concrete surfaces. However, these works assumed that cracks propagate
inherently with linear relations and only considered morphological-based methods.
Jahanshahi et al. (2009) reviewed image-based approaches for automatic detection of cracks
and corrosion in bridge structures of steel material. Neogi et al. (2014) reviewed defect
detection and classification algorithms for steel surfaces. Zakeri et al. (2017) focused on
asphalt pavement and reviewed image processing approaches for crack detection using
various data acquisition platforms (e.g., Digital Highway Data Vehicle (DHDV), robotic system,
and Quadcopter Unmanned Aerial Vehicle (QUAV/UAV) system). Koch et al. (2015) also
reviewed vision-based defect detection and condition assessment methods for civil
infrastructures of multiple materials. They considered materials of both concrete and asphalt,
and discussed algorithms for several types of distresses. The survey paid less attention to
crack detection and analysis. Recently, Czimmermann et al. (2020) reviewed defect detection
for several materials, including metals, ceramics, and textiles, with 13 kinds of defects such as
scratch, corrosion, galling, color error, etc.

In addition, the work in Mohan and Poobal (2018) reviewed crack detection techniques of 50
articles. The authors discussed various types of images that have been used for crack
detection. They also presented taxonomies of existing methods based on various factors for
analysis, such as detection objective, accuracy/error level, and image processing techniques.
However, the authors only reviewed traditional methods based on low-level features of cracks.
In addition to crack detection algorithms, it reports several state-of-the-art developments in
the Pavement Asset Management System (PAMS) in the work of Peraka Naga Siva Pavani
(2020). They mainly focus on pavement image acquisition and defect assessment techniques
rather than crack detection techniques.

Next, we will describe the scope of this survey and contrast it with the related surveys
discussed in this subsection.

Crack detection has broad applicability in assessing various civil infrastructures, such as
tunnels, walls, buildings, dams, bridges, beams, pavement, etc. In addition, civil
infrastructures are typically constructed using three types of materials, i.e., asphalt, concrete,
and metal-like.

Due to different construction materials and usage environments, the cracks vary significantly
in shapes/width, continuity, backgrounds, noises, lightning conditions, textures, etc. There are
also many crack types: longitudinal, transverse, and complex (alligator, block, mesh, diagonal).
In this survey, we do not limit our scope to any specific infrastructure, and we review crack
detection of infrastructures that are built from the following materials.

Asphalt has been used in many infrastructures like buildings, dams, and especially in the
construction of pavements. Asphalt pavements are often subjected to large crowds, frequently
in poor condition, chronically underfunded, and are becoming increasingly hazardous. It is
estimated that 1/5 of the highway pavements are in poor condition, and there is a significant
and increasing backlog of rehabilitation needs. Cracks are the most common and critical
defects of pavements which are the primary indicators to reflect the status of the deterioration
pattern. Detecting pavement cracks is important to facilitate road maintenance, provide a
better experience in automatic driving, and reduce the risk of accidents (e.g., collisions, falls).

Concrete is not as strong and tough as steel but has many excellent characteristics,
e.g., resistance to fire and water, corrosion, cyclic loading, etc. Thus, concrete material has
been widely used in many infrastructures including bridges, pavement, tunnels, dams, piles,
foundations, footing, floors, beams, columns, roofs, exterior walls, pipes, sewage systems, etc.
The cracking tendency is one of the most serious deterioration compared to other defects,
which is a major cause of concrete failures due to their low tensile strength. Crack is one of
the most important indicators to characterize the safety of concrete structures. For example, it
is used as one of the few quantitative indicators specified in the bridge maintenance
regulations. Monitoring, prevention, and control of cracks on pavement surfaces play a very
important role in ensuring the safety and normal operation of public transportation.

Metal-like material, e.g., steel, is ubiquitous due to its resistance to natural and artificial wear.
They are among the most important building materials for modern city construction. They
have been widely used in constructing infrastructures like wind turbine blades, nuclear power
plants, bridges, slabs, hot/cold strips, aircraft structures, etc. Typically, metal-like material
could yield defects not only during the forming processes but also during the service period.
The changes in the surface characteristics of steel over its service life promote the ingress of
dissolved contaminants. Thus, it is vital to monitor those variables continually. The common
defects of metal-like material are scratch, scarfing, and cracks. Cracks often appear at the
critical structural joints of steel bridges that are vulnerable to fatigue, and grow in depth,
length, as well as quantity. Detection of cracks is essential to (1) improve the yield of metal
products during processing and production (surface-quality control); (2) monitor the safety
status of metal-like material based infrastructures.

Images of infrastructures of different materials vary significantly in illumination conditions

and surface textures (Chambon and Moliard, 2011, Koch et al., 2015). In contrast, concrete
cracks usually have good continuity with relatively regular width, due to the unique
characteristics of external forces (Li et al., 2019b). Also, concrete cracks show obvious texture
differences from the backgrounds even when the background is smeared. On the other hand,
cracks of metal-like material are typically thin and small (with a high length/width ratio). They
usually appear similar to scratches, making them challenging to detect correctly (Jahanshahi
et al., 2009, Tabernik et al., 2019). In many infrastructures, pavement cracks and concrete
cracks could co-exist and affect each other.

This survey will cover computer vision-based crack detection on all infrastructures
constructed using the three materials discussed above. Different from the existing surveys, we
also review techniques that have been used for data acquisition and present publicly available
datasets that are catered to different application scenarios. We will discuss the intuition and
underlying assumptions of the existing methods, their key ideas and theoretical foundation,
and the advantages and disadvantages. We do not compare the absolute performance of the
reviewed methods as they used the different experimental setups (e.g., data sources, evaluation
metrics, model configurations). However, we will identify and discuss the common techniques
and challenges of the studies, and propose potential future directions to further advance the
field.

Compared to existing surveys, we provide a more comprehensive review of vision-based crack

detection, by covering multiple infrastructure materials (asphalt, concrete, metal-like), various
components of a typical computer vision framework (including data acquisition, public
datasets, algorithms, evaluations), and discuss unsolved issues and future directions. The
contributions of this survey can be summarized as follow: (1) We discuss and compare widely-
used data acquisition techniques for crack detection with regard to different application
scenarios. We also summarize up-to-date popular datasets that are publicly available. (2) We
provide a comprehensive overview of the existing detection algorithms for crack image
analysis, from traditional image processing algorithms and classical machine learning-based
methods to state-of-the-art deep-learning methods. We show a Sankey diagram of the existing
works addressing crack detection problems, which provide insights to efforts made regarding
various infrastructure materials and detection methodologies. Then, we introduce a method-
based taxonomy and review representative algorithms of each category in a systematic
manner. (3) We introduce and discuss various standard evaluation metrics employed in the
literature. (4) Finally, we discuss the unsolved issues in crack detection and provide several
promising directions for future works to advance this research field.

The rest of this review is organized as follows: Section 2 describes the widely adopted image
and video acquisition techniques for civil infrastructures and publicly available crack datasets.
In Section 3, the widely used evaluation metrics are discussed. Section 4 first provides a
taxonomy of crack detection methods proposed over the past two decades, including
threshold-based methods, morphology methods, shallow machine learning methods, and
deep learning methods. Then, each category is discussed in the subsequent subsections.
Section 5 discusses the unsolved issue in this field and promising directions for future work.
Section 6 concludes the paper.

Section snippets

Acquisition techniques
The crack detection problem is a challenging class-imbalance problem due to a small
proportion of cracks. Thus, it is critical to capture high-quality images of the target structures,
which helps to produce accurate ground truths to facilitate crack detection model training.
During the past few decades, many non-destructive vision detection techniques have been
proposed and developed.

As a contactless and cost-effective acquisition method, industrial/ commercial cameras are

easy to deploy and…

Evaluation metrics

To assess the performance of crack detection algorithms, many evaluation metrics have been
proposed based on the confusion matrix as shown in Table 2. The matrix contains two rows
and two columns and reports the number of True Positives ( ), True Negatives ( ), False
Positives ( ), and False Negatives ( ), characterizing the results of a crack detection
algorithm.

Correctly labeling the crack boundary pixels is challenging, and the unreliable labels of
boundary pixels have a negative impact…
Algorithms

Many existing surveys have proposed taxonomy methods for crack detection based on ‘pre-
processing segmentation feature extraction detection’ (Zakeri et al., 2017, Koch et al.,
2015). In this survey, we adopted a new taxonomy based on the development of automatic
crack detection algorithms, as shown in Fig. 4. Generally, the threshold-based methods are
widely used in the early stage (i.e., between 2000 and 2005), which classify crack and non-crack
pixels solely based on pixel intensity…

Discussion
Although researchers have proposed many computer vision-based methods to tackle crack
detection tasks over the past 20 years, this domain has not yet been fully explored, and several
challenges remain unresolved. This section discusses some of the main challenges and
provides insights on future prospects.…

Conclusion

Accurate and automatic detection of cracks is critical for monitoring and guaranteeing the
safety conditions of civil infrastructures. As such, it has attracted significant attention in the
research community, and remarkable achievements have been made in crack detection using
computer vision techniques over the past 20 years. In this survey, we aim to help relevant
researchers and institutions summarize the current studies for civil infrastructures and
identify future research directions. This …

CRediT authorship contribution statement

Dihao Ai: Search of related papers, Methodology, Writing – original draft. Guiyuan Jiang:
Methodology, Search of related works, Writing – original draft, Paper revision. Siew-Kei Lam:
Supervision, Writing – review & editing. Peilan He: Conceptualization, Visualization, Writing
– original draft. Chengwu Li: Writing – review & editing.…

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal
relationships that could have appeared to influence the work reported in this paper.…

Acknowledgments
This research is financially supported by the National Natural Science Foundation of China
(52004292,41976185), Shenzhen Polytechnic (6021310002K, 6022312015K), Post-doctoral Later-
stage Foundation Project of Shenzhen Polytechnic (6021271008K), and the Huangpu
International Sci&Tech Cooperation Fundation of Guangzhou, China
(2021GH12).…
References (240)

LiT.Y. et al.
The wave propagation and vibrational energy flow characteristics of a plate with a part-
through surface crack
Internat. J. Engrg. Sci. (2009)

LiG. et al.
Recognition and evaluation of bridge cracks with modified active contour model and
greedy search-based support vector machine
Autom. Constr. (2017)

LiG. et al.
Long-distance precision inspection method for bridge cracks with image processing
Autom. Constr. (2014)

LiS. et al.
Detection of concealed cracks from ground penetrating radar images based on deep
learning algorithm
Constr. Build. Mater. (2021)

KochC. et al.
A review on computer vision based defect detection and condition assessment of
concrete and asphalt civil infrastructure
Adv. Eng. Inf. (2015)

KimH. et al.
Comparative analysis of image binarization methods for crack identification in
concrete structures
Cem. Concr. Res. (2017)

KhasinV.L.
Stochastic model of crack propagation in brittle heterogeneous materials
Internat. J. Engrg. Sci. (2014)

JiA. et al.
An integrated approach to automatic pixel-level crack detection and quantification of
asphalt pavement
Autom. Constr. (2020)

JahanshahiM.R. et al.
Adaptive vision-based crack detection using 3D scene reconstruction for condition
assessment of structures
Autom. Constr. (2012)

IyerS. et al.
A robust approach for automatic detection and segmentation of cracks in underground
pipeline images
 Image Vis. Comput. (2005)

View more references

Cited by (0)

Recommended articles (0)

View full text

© 2022 Elsevier Ltd. All rights reserved.

Copyright © 2022 Elsevier B.V. or its licensors or contributors.

ScienceDirect® is a registered trademark of Elsevier B.V.