Automation in Construction 132 (2021) 103931

Contents lists available at ScienceDirect

Automation in Construction
journal homepage: www.elsevier.com/locate/autcon

Automated structural design of shear wall residential buildings using

generative adversarial networks
Wenjie Liao a, Xinzheng Lu a, *, Yuli Huang a, Zhe Zheng a, Yuanqing Lin a, b
a
Department of civil engineering, Tsinghua University, Beijing 100084, China
b
China Nuclear Power Engineering Co., Ltd., Zhengzhou Branch, Zhengzhou 450052, China

A R T I C L E I N F O A B S T R A C T

Keywords: Artificial intelligence is reshaping building design processes to be smarter and automated. Considering the
Intelligent structural design increasingly wide application of shear wall systems in high-rise buildings and envisioning the massive benefit of
Shear wall system automated structural design, this paper proposes a generative adversarial network (GAN)-based shear wall
Generative adversarial network
design method, which learns from existing shear wall design documents and then performs structural design
Computer vision
Data and hyper-parametric analytics
intelligently and swiftly. To this end, structural design datasets were prepared via abstraction, semanticization,
classification, and parameterization in terms of building height and seismic design category. The GAN model
improved its shear wall design proficiency through adversarial training supported by data and hyper-parametric
analytics. The performance of the trained GAN model was appraised against the metrics based on the confusion
matrix and the intersection-over-union approach. Finally, case studies were conducted to evaluate the applica­
bility, effectiveness, and appropriateness of the innovative GAN-based structural design method, indicating
significant speed-up and comparable quality.

1. Introduction modeling and are less suitable for engineering calculations. Hence, they
are mainly applied to architecture and have found very few applications
Intelligent design offers advantages in its ability to minimize manual in structural design. In addition, the considerable computational
design work, promote diversity in the design space, and ultimately expense of the underlying algorithms forfeits their use in the scheme
provide optimal design performance [1–5]. As a result of rapid global design stage.
urbanization, the demand for high-rise residential buildings is continu­ Deep learning methods offer a new option to overcome these chal­
ously increasing [6,7]. Reinforced concrete shear wall systems have lenges. They have been used efficiently for various purposes via pre-
been widely used in high-rise designs. The transfer of architectural training [9–12]. The generative adversarial network (GAN) is one of
proposals to construction documents involves arranging structural lay­ the most widely used frameworks [13] for automated architectural
outs, defining the position and orientation of structural systems, and design [14–16]. In a GAN framework, the generative network learns to
controlling the dimensions of structural components [8]. These steps are generate candidates of interest, while the discriminative network dis­
fundamental to the design process. However, because of their iterative tinguishes the generated candidates from the ground truth [13]. The
nature, they are also very time consuming, even when conducted by generator synthesizes images using the initial input noises and enhances
competent engineers. An innovative design approach with greater effi­ the generation quality based on the feedback from the discriminator
ciency is needed, and the intelligent structural design is an emerging until the discriminator fails to judge. Simultaneously, the discriminator
approach. consistently elevates the skill at detecting synthetic outputs by the
Existing approaches are primarily based on generative design. The generator. Finally, GAN converges to a Nash equilibrium between the
commonly used options are as follows: (1) design exploration using to­ discriminator and generator. Adversarial training is applied to both
pology optimization, genetic algorithms, and cellular automata; (2) networks so that the generator and discriminator can master the gen­
design synthesis using generative grammars; and (3) design by analogy eration and discrimination, respectively [13,17,18]. Additionally,
[1–5]. These approaches are particularly favorable for geometric compared with the convolutional neural network (CNN)-generated

* Corresponding author.
E-mail address: [email protected] (X. Lu).

https://doi.org/10.1016/j.autcon.2021.103931
Received 6 February 2021; Received in revised form 5 June 2021; Accepted 27 August 2021
Available online 16 September 2021
0926-5805/© 2021 Elsevier B.V. All rights reserved.
W. Liao et al. Automation in Construction 132 (2021) 103931

images, GAN-synthesized images are more refined and precise [17,18]. openings and the connectivity of structural components [8]. It abstracts
GAN has been successfully adopted in innovative architectural home the architectural schemes, extracts and color-codes the layout patterns,
design [14–16], and barely used in structural design. Hence, this study and make them structurally meaningful. The semanticization process
extends the application of GAN to shear wall structural design. The GAN reduces enormously the data dimension while keeping all the core in­
model improves the design proficiency by learning from existing design formation that can adequately inform the structural design, and ulti­
documents, thus offering a significant improvement in design efficiency mately boosts the design performance. To expedite the learning cycles,
and performance. StructGAN not only adopts the traditional pixel-by-pixel evaluation,
This paper presents the development and optimization of a GAN- which fails to comprehend the overall structural layout. It also adopts
based structural design framework, StructGAN, in Section 2. Then Sec­ intersection over union (IoU) [21] as the core metric. The IoU metric can
tion 3 elaborates the implementation of StructGAN, including dataset properly gauge the overall similarity of the design under evaluation
establishment, the associated engineer-perception-based and computer- against the reference design and provide valuable guidance and feed­
vision-based performance metrics, and design process. Next, Section 4 back to the training direction. These innovations provide a solid foun­
introduces detailed discussions on GAN networks and datasets. More­ dation for the design performance that StructGAN delivers consistently
over, in Section 5, by comparing designs by StructGAN and competent to all types of building structures.
engineers, case studies of the GAN-based framework suggest that it is a The components of StructGAN are summarized in Fig. 1: interpreter,
promising and versatile design approach for the future. Finally, Section designer, and modeler. The interpreter digests and semanticizes the
6 presents the conclusions of the study. architectural sketches. Then, the designer analyzes the semantic draw­
ings, performs the inference, and devises the structural design. Finally,
2. StructGAN method the modeler proposes the design and presents the structural model.
Table 1 compares the performances of StructGAN and the conventional
StructGAN was developed to address the growing concerns in the design process. StructGAN offers a promising increase in speed by a
construction industry. Some of the negative impacts experienced include factor of 10, which is equivalent to saving billions of US dollars per
(1) the lack of an efficient communication link between architects and annum in the industry. As it learns and evolves continuously, StructGAN
engineers, (2) the low cost-effectiveness in the design iterations, and (3) will undoubtedly obtain much higher savings.
the quality uncertainties associated with experiential knowledge
[19,20]. Therefore, process optimization and automation are necessary 3. Implementation of StructGAN
to minimize the total project time and cost. As such, StructGAN attempts
to achieve the highest efficiency via artificial intelligence. It resolves the The StructGAN implementation is illustrated in Fig. 2, including
highly involving correlation between the architectural and structural three dominating steps: (1) datasets and StructGAN training, (2)
fields through sophisticated maps established by deep learning, which StructGAN performance evaluation, and (3) StructGAN application.
converts the iterations to one-step solutions, reduces variations in design
quality, and strengthens the control of design processes.
3.1. Datasets and StructGAN training
The strategic innovations adopted by StructGAN involve a semantic
engagement of prior knowledge and high-level performance metrics. By
GAN learns from previous designs based on maximum likelihood
activating experiential knowledge in structural engineering, StructGAN
estimation, making the probability distributions and qualities of the
recognizes the crucial structural concepts, such as the locations of
StructGAN design directly associated with the training datasets. Thus, to

Discriminative network

Conceptual
architectural sketches …
Semantic drawings
Generative network

StructGAN

Structural designs
Structural models

Interpreter Designer Modeler

Fig. 1. StructGAN. The StructGAN automated structural design framework. Interpreter: digesting and semanticizing the architectural sketches. Designer: analyzing
the semantic drawings and devising the structural design. Modeler: proposing the design and the corresponding structural model.

2
W. Liao et al. Automation in Construction 132 (2021) 103931

Table 1 addition, the maximal building height is 141 m. The high seismic design
The excellent performance of StructGAN with high efficiency and superior intensity dominates the demands for structural seismic resistance in the
quality. 8-degree seismic intensity zones with slight influence of structural
StructGAN High efficiency Superior quality heights; thus, the designs in the 8-degree seismic intensity zones were
performance
20 times 10 times Comparable Excellent
not divided by heights. Consequently, the datasets were classified and
faster faster potential stability named Group7-H1, Group7-H2, and Group8, respectively. GroupMix
seismic loss was built by randomly selecting designs from each group and mixed for
(only 6% contrast, including 26 data from each training dataset and 3 data from
larger)
each test dataset. Thus, 63, 80, 81, and 78 training sets and 8, 8, 8, and 9
Competent 3.5 h/ 300 h/ $9,846,000 Economic and testing sets in Group7-H1, Group7-H2, Group8, and GroupMix, respec­
engineers preliminary complete USD mechanical tively. Typical datasets are shown in Fig. 3.
design design performance of
designs by
Based on the pre-processed datasets, the StructGAN could be trained
inexperienced effectively. In addition to the regular training recommended by pix2­
engineers are pixHD [18], in terms of the determined StructGAN system, this study
inferior to those also proposed performance enhancement approaches by (1) data
by senior
augmentation and (2) parametric adjustment for the generative network
engineers
StructGAN 10 min/ 30 h/ $10,454,500 Consistent design architecture of pix2pixHD. The datasets were augmented by flipping the
preliminary complete USD quality for images vertically and horizontally and rotating the images 180◦ . This
design design structural designs was done because the flip and rotation operations do not change the
with various spatial layout of the structural shear walls in the image, and the number
heights and
seismic design
of Group7-H2 training data reasonably increased from 80 to 320.
intensities Furthermore, the complexity of local features is significantly reduced
owing to the semantic architectural–structural designs; hence, the
The estimation of design efficiency, safety, and economic performance are
generator architecture can be simplified to generate more confined and
elaborated in Section 3.3 StructGAN design. The time consumption of competent
engineers is derived from the study of Chakrabarti [22]. The potential seismic
precise image elements. The numbers of global down-sample layers
losses are analyzed by the widely adopted FEMA P58 method [23], and the (n_downsample_global) and residual blocks in the global generator
values are the mean losses of two typical high-rise shear wall residential network (n_blocks_global) were reduced from 4 to 2 (or 1) and from 9 to
buildings. 6, respectively.

ensure the quality of the source design, this study applied approximately 3.2. Evaluation and metrics
250 pairs of architectural–structural designs from more than ten famous
architectural design and research institutes in China. Moreover, these Accurate quality evaluation of GAN-synthetic images is vital and
designs satisfied all relevant design specifications, were optimized and challenging for GAN-related studies [30,37]. The critical content of
evaluated by experienced engineers, and were adopted in real-world evaluation is the difference quantification of generations and targets.
construction applications with excellent quality. Subsequently, based For images with high-dimensional probability distributions, the detailed
on raw design datasets, semanticization and classification were con­ evaluations are listed in Table 2, including (1) Amazon Mechanical Turk
ducted for the designs. (AMT) perceptual studies, and (2) a computer vision-based assessment
This study adopted the semantic process by extracting essential of synthetic images. However, compared with the image assessments,
architectural and structural elements in design images and coding them the evaluations for structural design are more complicated to reasonably
by color patterns, so that critical design elements and the corresponding consider the structural layouts and their correlations. Hence, based on
structural layout information are maintained. Semantic designs can the widely adopted AMT perceptual evaluation and image pixel-based
effectively reduce the dimension of probability distributions and evaluation, this study developed an engineer perception-based and
enhance training performance. In this study, the red (RGB = (255, 0, 0)), computer vision-based evaluation method, assessing the rationality of
gray (RGB = (132, 132, 132)), green (RGB = (0, 255, 0)), and blue (RGB the structural layout.
= (0, 0, 255)) colors denote the structural shear wall, nonstructural infill Engineer perception-based evaluation: the evaluation based on
wall, indoor window, and outdoor gate, respectively. The reader may engineer perception is the most straightforward method to identify the
note that this study adopted the semi-automated interpreter module to acceptance by engineers of the StructGAN design capabilities. It includes
extract the critical elements of images to ensure that the interpreter- (1) “AI” or “Engineer” judgment, which involves inviting engineers to
processed semantic images for GAN training and testing are clear and distinguish designs produced by StructGAN or competent engineers, and
accurate. Furthermore, the StructGAN interpreter module is ready to be (2) rationality score for designs, which comprises asking for scores given
further enhanced by adopting the existing proposals of CAD extraction by engineers based on their experience and perception. Similar to the
[24]. AMT method, the engineer perception-based evaluation was conducted
In addition, the structural design for shear wall residential buildings on the Questionnaire Star (https://www.wjx.cn/) platform for blind
is directly related to the design conditions of structural heights and tests, and typical parts of the questionnaire are illustrated in Table 3.
seismic design intensities, and design conditions-based dataset classifi­ This study invited 11 senior experts (work experience >15 years), 12
cation can efficiently narrow the learning restriction of StructGAN to practicing engineers, and graduate students to participate in the judg­
promote design precision. According to the Chinese Code for Seismic ment and assessment tasks, and the corresponding metrics were pro­
Design of Buildings [25], the datasets were classified into 7-degree and 8- posed based on the evaluation results. SEP-1 is the metric for the “AI” or
degree seismic intensities. Notably, the corresponding peak ground ac­ “Engineer” judgment, expressed in Eq. (1), and which equally counts the
celeration (PGA) values of the design basis earthquake (i.e., 10% prob­ judgment of experts and ordinary engineers. SEP-2 is the metric for ra­
ability of exceedance in 50 years) are 100 cm/s2 and 200 cm/s2 in the 7- tionality evaluation, expressed in Eq. (2), and which adopts the coeffi­
degree and 8-degree seismic intensity zones, respectively. Subsequently, cient of the variation to weight the scores by experts and ordinary
based on the structural height regulations in the Chinese Technical engineers (Eq. (3)).
Specification for Concrete Structures of Tall Buildings [26], the datasets are
classified into H1 (i.e., height ≤ 50 m) and H2 (i.e., height > 50 m). In

3
W. Liao et al. Automation in Construction 132 (2021) 103931

Legend Improve Correct

or not ?
Shear wall

Architectural Infill wall G Generative D

Generator image Discriminator
Window
& door Input image
Outdoor Improve
gates
Structural Real image
Architectural design drawing Semantic image StructGAN model

Pre-processing of design drawings and StructGAN model training

Ground-truth
Confusion matrix
class 1 class i-1 class i
Inter-
Evaluation class 1 Pii Pij Pij IoU= section Union Integrated
based on human Prediction class i-1 Pji Pii Pij
evaluation based
annotators on weighted
class i Pji Pji Pii
judgment (HAJ) multi-metric
Evaluation based on confusion matrix Evaluation based on IoU

Evaluation of trained StructGAN model

Qualified
StructGAN
model

New design drawing Semantic architectural image Semantic structural image

Application of qualified StructGAN model

Fig. 2. The framework of StructGAN, including training, evaluation, and application.

1 Nex∑
+Nnonex
NF adopted a confusion matrix [39] to assess the generation quality of
SEP− 1 = (1) critical elements in images and IoU of structural layouts to assess the
Nex + Nnonex NF + NT
rationality of the structural design. Subsequently, two methods are
i

( Nimg
) ( Nimg
) weighted to evaluate the comprehensive performance of StructGAN.
1 ∑Nex
1 ∑ 1 N∑ nonex
1 ∑
SEP− 2 = ηex Sj + ηnonex Sj (2) In the confusion-matrix-based assessment (Fig. 4), adopting the
Nex i Nimg j Nnonex i Nimg j classification of each pixel and the correctness judgment of the pixel
type to evaluate the generative quality is the core superiority. First,
ηex =
σnonex /μnonex
,η = 1 − ηex (3) structural shear walls, nonstructural infill walls, indoor windows, and
σ ex /μex + σnonex /μnonex nonex outdoor gates are directly distinguished and separated according to the
hue saturation value (HSV) of each pixel, utilizing the Open Source
where Nex and Nnonex denote the number of experts and non-experts, Computer Vision (OpenCV) library for image processing to convert the
respectively, and NF and NT indicate the number of misjudgments and colors into the HSV mode [40] (Fig. 4a). The extracted elements of the
correct judgments of StructGAN designs, respectively. Nimg is the num­ StructGAN design are compared with those of engineers pixel-by-pixel,
ber of assessed images, Sj is the score of image j, and ηex and ηnonex are the and then the comparison results are used to create a confusion matrix
weight coefficients of the scores of experts and non-experts, respec­ (Fig. 4b and c). Subsequently, based on the confusion matrix, Pixel ac­
tively. σ ex and σ nonex are the standard deviations of the scores of experts curacy (PA), weighted IoU (WIoU), and shear wall ratio (SWratio) are
and non-experts, respectively, and μex and μnonex are their mean values, proposed and used, where PA (Eq. (4)) measures the image clarity, WIoU
respectively. The determination of the weight coefficients in Eq. (3) (Eq. (5)) estimates the generative quality of critical elements, and
refers to the coefficient of the variation-based method proposed by SWratio (Eq. (6)) reflects the total amount of structural shear walls.
Diakoulaki et al. [38], where a smaller coefficient of variation corre­ Owing to the above-proposed metrics derived from image pixel classi­
sponds to a higher weight. fication, the use of the confusion-matrix-based evaluation could mea­
Computer vision-based evaluation: the integrated consideration sure the comprehensive quality of the generated image.
of the generated image quality and rationality of the structural design by
StructGAN is a significant advantage of the proposed computer vision-
based evaluation method. In general, the simultaneous quality evalua­
tion of synthetic images and structural designs is challenging. This study

4
W. Liao et al. Automation in Construction 132 (2021) 103931

(a) Training data

Group7-H1 Group7-H2 Group8 Group8

Testing data

Group7-H1 Group7-H2 Group8 Group8

Training Testing
(b) Category
Quantity SWratio Quantity SWratio
Group7-H1 63 0.43 r 0.08 8 0.40 r 0.10
Group7-H2 80 0.57 r 0.08 8 0.57 r 0.08
Group8 81 0.67 r 0.14 8 0.66 r 0.11
GroupMix 78 0.55 r 0.14 9 0.53 r 0.18

Fig. 3. Datasets for GAN training. (a) Typical training-testing sets. (The top row A shows the input architectural design images, and the bottom row B is the cor­
responding target structural design images). (b) Different datasets classified by structural height and seismic design intensity. (SWratio (Eq. (6)) denotes the ratio of
shear walls to total walls, reflecting the area proportion of the shear walls to the total walls).

∑
k the consistency measurement of the structural layouts designed by
pii
StructGAN and experienced engineers. Detailed steps for the structural
PA = i=0
(4) IoU-based evaluation are illustrated in Fig. 5a, and the corresponding
∑
k ∑
k

i=0 j=0
pij metric is named structural intersection over union (SIoU). First, the
images are subdivided into multiple sub-images to reduce the number of
∑
k structural shear walls in each image and elevate the edge capture pre­
ωi pii
WIoU = (5) cision of the contour detection algorithm. Subsequently, the shear wall
∑
k ∑
k
i=0
pij + pji − pii elements of each sub-image are extracted based on the HSV color mode,
and their contour coordinates are identified by the contour detection
j=0 j=0

API “OpenCV.findContours (image).” Then, the total intersection area of

Aswall
SWratio = (6) the shear walls in the StructGAN design and design by engineers are
Aswall + Ainwall
obtained using the Shapely API “shapely.geometry.Polygon (coordinates)”
where (k + 1) is the total class (class 0 is background, class 1 is shear and SIoU is calculated using Eq. (7).
wall, class 2 is infill wall, class 3 is window, class 4 is outdoor gate), pij is Ainter
the number of pixels of class i inferred to belong to class j. In other SIoU = (7)
Aunion
words, pii represents the number of true generated pixels, whereas pij and
pji are usually interpreted as false generated pixels, respectively (Fig. 2). where Ainter is the intersection area of the walls in the GAN-synthetic and
ω0 = 0, ω1 = 0.4, ω2 = 0.4, ω3 = 0.1, and ω4 = 0.1; the synthetic results target designs, Aunion is the union area of the walls in the GAN-synthetic
of the shear wall and infill wall are the most essential, and hence, their and target designs, Aunion = Atarget + AGAN − Ainter, and Atarget, and AGAN
weights are the largest. Aswall and Ainwall are the total areas of the shear denote the shear wall area of the target design and the GAN design,
wall and infill wall, respectively. respectively.
In the structural IoU-based evaluation (Fig. 5), the core superiority is The proposed GAN-based structural design methods are yet to be

5
 W. Liao et al. Automation in Construction 132 (2021) 103931

Table 2 target designs, respectively. ηSIoU and ηWIoU are the weighted coefficients
Review of quality evaluation methods. of SIoU and WIoU, respectively, both equal to 0.5.
References Pascal tests (quantitative Perceptual tests For the evaluation of the StructGAN-designed shear wall layouts,
evaluation) (qualitative WIoU, which reflects the generation quality of the overall walls in im­
Method Metric
evaluation) ages, and SIoU, which reflects the quality of the auto-designed structural
shear wall layouts, are equally important. Thus, the weights ηSIoU and
Springenberg / Log-likelihood /
(2015) [27] estimate
ηWIoU are set to 0.5. Moreover, the ηSWratio denotes the correction coef­
Classification Classification ficient for the overall quantity of shear walls. The values of WIoU and
error SIoU increase with the increment in shear walls, which is unfavorable for
Salimans et al. / exp(ExKL(p(y|x)‖ AMT the evaluation. Hence, the difference in the total shear wall area be­
(2016) [28] p(y)))
tween the synthetic image and target image is adopted as the correction
Wang & Gupta Classification Maximum norm AMT
(2016) [29] ‖∙‖∞ coefficient, and a smaller diversity corresponds to a larger ηSWratio.
Detection Number of
objects 3.3. StructGAN design
Zhang et al. (2016) Classification mAP AMT
[30] Detection
Segmentation IoU
StructGAN is primarily developed for the preliminary schematic
Zhu et al. (2017) Segmentation Per-pixel acc., AMT design, and partly for structural design development, following the
[31] IoU guide in Fig. 2. Before StructGAN-based structural design starts, the
Isola et al. (2017) Segmentation Per-class acc., AMT architectural drawing, seismic design intensity, and structural height
[17] IoU
should be obtained. The structural design result can be obtained by
Wang et al. (2018) Segmentation Pixel acc., IoU AMT
[18] inputting the semantic architectural drawing into StructGAN. Presently,
due to the end-to-end deep learning feature, StructGAN has not yet
Segmentation is derived from the studies of semantic segmentation [32,33];
provided interaction with users, which is different from the conven­
Detection is derived from the studies of object detection [34–36]. “mAP” denotes
tional generative design tools proposed by Shea et al. [41]. Furthermore,
mean average precision; “acc.” denotes accuracy; “IoU” denotes intersection
over union; “AMT” denotes Amazon mechanical turk. the detailed process of converting the structural plan design into an
overall structural analysis model is as follows:

Table 3 • Quantitative evaluation of the design images based on ScoreIoU: The

Evaluation based on engineer perception. structural plan design is first obtained by StructGAN and evaluated
1. Please distinguish the following images: AI-generated or Engineer-designed? (Red - by comparison with designs by engineers.
shear wall; Gray - infill wall; Green - window; Blue - outdoor gate) • Establishment of the corresponding structural analysis models based
on the StructGAN designs: The detailed process of converting the
bitmap of the shear wall design into a structural analysis model is
shown in Appendix A. Based on the structural design model, PKPM
can automatically conduct steel reinforcement design according to
Chinese design codes [42]. Additionally, related studies on optimi­
zation design methods for steel reinforcement [43] can also be in­
tegrated into StructGAN in the future to improve StructGAN design
capability.
• Performance analysis of structures: This study used the PKPM soft­
ware to conduct overall structural design and time-history analyses
[42], and then yielded the direct economic loss under the maximum
considered earthquake using the FEMA P58 method [23].
[Choice] *
○ AI- Engineer-designed
Note that, it is also easy and friendly to add new data for training, by
○

generated
adopting the pre-processing method for semantic architectural and
2. Reasonability score of the shear wall design (1 - Unreasonable, 5 - Reasonable) structural CAD drawings to create new training data (approximately cost
[Choice] * 5–10 min) and then adding them into datasets and training. For further
○ 1 ○ 2 ○ 3 ○ 4 ○ 5 application, corresponding GUI and datasets will be developed.
The StructGAN application results will be elaborated in Section 5
(StructGAN Performance), including 16 structural plan designs and two
mature, and so are the relevant metrics. Therefore, this study attempts to
overall structural designs for shear wall residential buildings. This study
make some progress by proposing holistic metrics to conduct a more
ensured no data crossover between the training sets and 16 testing cases
objective and integrated performance evaluation. Inferring from the
for testing reliability. To further evaluate the generalization perfor­
above studies of single metrics, PA is the traditional pixel-by-pixel
mance of StructGAN, two overall structural design cases supplied by an
evaluation to assess the image quality, which is abandoned because of
architectural design firm were used. These designs were excluded from
the failure to comprehend the overall structural layout. Moreover, SIoU,
the training and test sets. Furthermore, these application results will
WIoU, and SWratio can only evaluate the confined properties of struc­
show the StructGAN with a powerful generalization ability for different
tural layouts of StructGAN. Hence, by combining the qualities of the
design conditions.
generated critical elements in images and structural designs, this study
proposed the weighted multi-metric ScoreIoU (Eq. (8)) to assess the
4. Discussions of networks and datasets
design performance comprehensively.
ScoreIoU = ηSWratio × (ηSIoU × SIoU + ηWIoU × WIoU) (8) Based on the proposed StructGAN and its implementation methods,
detailed discussions and analyses on the StructGAN were conducted to
where ηSWratio = 1 − |SWratioGAN − SWratiotarget|/SWratioGAN. SWratio obtain a high-performance system, whose most critical parts are the
GAN,and SWratiotarget are the shear wall ratios of the GAN designs and GAN algorithm and dataset. Hence, the Discussion section presents the

6
­
W. Liao et al. Automation in Construction 132 (2021) 103931

(a) Background (white) Shear walls (red) Infill walls (gray) Windows (green) Outdoor gates (blue)
Hmin 0 0 0 156 35 100
Hmax 180 180 10 180 77 124
Smin 0 0 43 43 43
Smax 30 43 255 255 255
Vmin 221 46 46 46 46
Vmax 255 220 255 255 255
(b) Target (c) Generated

Target image Synthesized image

Shear walls Shear walls

Infill walls Infill walls

Windows Outdoor gates Windows Outdoor gates

(d) Background Shear walls Infill walls Windows Outdoor gates

(white) (red) (gray) (green) (blue)

Background 489246 1414 42 228 27

Shear walls 824 6706 958 37 0

Infill walls 948 4559 9292 51 3

Windows 386 21 2 8726 0

Outdoor gates 157 15 1 0 645

Fig. 4. Information on confusion-matrix-based assessment. (a) The value range of 5 colors in the HSV color mode. (b) The critical extracted elements of the target
image. (c) The critical extracted elements of the generated image. (d) Confusion matrix used to obtain PA, WIoU, and SWratio.

analysis for the selection of GAN algorithm and the corresponding performance GAN algorithms. Compared with pix2pix, pix2pixHD is
parametric adjustment along with the pre-processing of the datasets, an improved algorithm that can generate high-resolution photo-realistic
based on the developed performance evaluation methods and metrics. images with significantly higher computational demands [18]. In the
above two algorithms, the characteristic “structures loss” can effectively
4.1. GAN algorithm and parametric adjustment reflect the physical position relationship of pixels in an image [17].
Thus, deduced from the “structures loss,” pix2pix and pix2pixHD can
The pix2pix [17] and pix2pixHD [18] algorithms are typical high- capture the potential spatial position distribution of structural layouts.

7
W. Liao et al. Automation in Construction 132 (2021) 103931

(a) (b) Case 1 Case 2 Case 3

Step 1
Target
subdivision
image

Pre-subdivision Post-subdivision

Generated
Step 2 image
contour
detection

The contour detection results

SIoU
Generated
Step 3 Target
SIoU =
Coincidence
calculation SIoU1 = 0.65 SIoU2 = 0.61 SIoU3 = 0.00

Fig. 5. Information on IoU-based evaluation. (a) Detailed steps to get SIoU. (b) Typical cases of SIoU.

The structural distribution correlation can contribute to establishing a structural design quality (Section 3.2). Pixel accuracy (PA) evaluates the
direct map relationship for StructGAN to convert the crucial architec­ overall clarity of the generated image, weighted IoU (WIoU) focuses on
tural elements into the corresponding structural layouts. Furthermore, assessing the generative qualities of critical elements in images, struc­
their significant performance in generative architectural design has tural IoU (SIoU) explicitly measures the structural layout consistency
proved the applicability of pix2pix and pix2pixHD [14,15]. between the designs by StructGAN and engineers, and the difference in
This study compared pix2pix and pix2pixHD with one of the most SWratio estimates the discrepancy of the total structural layout area
famous CNN-based image synthesis methods (i.e., UNet), to validate the between two designs. As these metrics show, a high PA, WIoU, and SIoU
advantage of pix2pix and pix2pixHD in generative structural design. See and low difference in SWratio indicate that the designs by StructGAN
the study of Ronneberger et al. [44] for more details of UNet. UNet, and engineers are highly consistent. Subsequently, this study discussed
pix2pix, and pix2pixHD are all trained to the best state using the same the influence of (γ GAN / γL1) and γ FM based on the proposed evaluations,
training datasets, and the design results are shown in Fig. 6. From the where γGAN was fixed as 1, and the Group7-H2 dataset of shear wall
perspectives of objective scoring and subjective judgment, pix2pix and residential buildings was adopted.
pix2pixHD both perform better than UNet. Moreover, as the UNet design Fig. 7a and b show the typical design performance of pix2pix and
demonstrates, all positions are assigned with shear wall components, pix2pixHD with various hyperparameters. When γ L1 = 0, pix2pix loses
which means UNet has not learned the principle of where to assign shear its control on the global quality of image clarity, causing enormous noise
wall and how many shear walls to set. The primary reason is that UNet in the generated image. As γL1 increases, the automated design quality is
lacks an adversarial training mechanism, inducing the high-dimensional improved until it becomes stable, with the clarity of image and the
probability distribution of UNet-synthesized images different from those structural design rationality improving significantly. In comparison, γFM
of target images, and the corresponding structural components design is only slightly influenced the automated design quality because it mainly
not consistent. Consequently, this work adopted pix2pix and pix2pixHD affects feature mapping and this study involved the input of architec­
as core algorithms for StructGAN. tural semantic images containing few local features. Moreover, quanti­
As recommended by Isola et al. [17] and Wang et al. [18], the per­ tative evaluations were conducted to quantify the design qualities of
formances of pix2pix and pix2pixHD are influenced by critical param­ StructGAN with different hyperparameters. As Fig. 7c–f illustrate, PA,
eters; therefore, parametric studies should be conducted. In the pix2pix WIoU, and SIoU grow, and the difference in SWratio degrades with
algorithm, the local features and global clarity of image qualities are increasing γ L1 and γFM. The evaluation results were consistent with the
primarily determined by the relative values of γGAN and γ L1 (i.e., γ GAN / perceptual results in Fig. 7a and b, revealing that these methods and
γ L1) [17]. In the pix2pixHD algorithm, the vital hyperparameter γFM metrics are reasonable and applicable. Furthermore, both the image
affects the overall quality of the generated image by adjusting the pro­ generation and structural design qualities of pix2pixHD are significantly
portion of the feature matching loss to the total loss [18]. Furthermore, better than those of pix2pix, with small design dispersion and high
this work proposed a computer vision-based evaluation method and stability. The WIoU and SIoU of the pix2pixHD design with the optimal
several metrics such as PA, WIoU, SIoU, and SWratio to pad the blank of parameters both exceed 0.5, indicating that the structural layout is very
the quantitative evaluation method and associated metrics for the reasonable. The total amount of structural shear walls designed by

Target (NO.11) Unet Pix2pix pix2pixHD

ScoreIoU = 0.32 ScoreIoU = 0.43 ScoreIoU = 0.63

Fig. 6. Comparison on structural designs using UNet, pix2pix, and pix2pixHD.

8
W. Liao et al. Automation in Construction 132 (2021) 103931

Output (γL1=0) Output (γL1=10) Output (γL1=30)

Target (NO.11)

(a) Output (γL1=60) Output (γL1=100) Output (γL1=150)

Output (γFM=0) Output (γFM=1) Output (γFM=3)

Target (NO.11)

(b) Output (γFM=6) Output (γFM=10) Output (γFM=15)

pix2pix pix2pixHD pix2pix pix2pixHD pix2pix pix2pixHD pix2pix pix2pixHD

0.8 0.8

0.6 0.6
0.5 0.5
SIoU

WIoU

0.4 0.4

0.2 0.2

0 0
0 (0) 10 (1) 30 (3) 60 (6) 100 (10) 150 (15) 0 (0) 10 (1) 30 (3) 60 (6) 100 (10) 150 (15)
(c) Weight γL1 (γFM) (d) Weight γL1 (γFM)
pix2pix pix2pixHD pix2pix pix2pixHD pix2pix pix2pixHD
1 80%
Difference of SWratio

60%
0.8
40%
PA

0.6 20%

0%
0.4 0 (0) 10 (1) 30 (3) 60 (6) 100 (10) 150 (15)
0 (0) 10 (1) 30 (3) 60 (6) 100 (10) 150 (15) -20%
(e) Weight γL1 (γFM) (f) Weight γL1 (γFM)

Fig. 7. Analyses of the hyperparameters and quantification of testing results for different GAN algorithms with various parameters. (a) Comparison of the pix2pix-
generated images with different parameters. (b) Comparison of the pix2pixHD-generated images with different parameters. (c)–(f), Comparisons of SIoU, WIoU, PA,
and the difference in SWratio between pix2pix and pix2pixHD with various parameters. The evaluation results show that the designs by StructGAN equipped with
pix2pixHD coincide well with those by engineers, with high stability. However, the performance of pix2pix needs further enhancement in future studies.

StructGAN is comparable to those designed by competent engineers, et al. [18], StructGAN adopts pix2pixHD, with γFM = 10.
with a difference in SWratio smaller than 15%. Consequently, the eval­
uation results show that the designs by StructGAN equipped with
pix2pixHD coincide well with designs by engineers, with high stability. 4.2. Datasets analysis
Meanwhile, the performance of pix2pix still needs to be enhanced in
future studies. Based on the discussions and recommendations by Wang The maximum likelihood estimation is the basis of GAN algorithms
[13], making the probability distribution of designs and design quality

9
W. Liao et al. Automation in Construction 132 (2021) 103931

closely related to the datasets; hence, this study discussed the influence were divided into Group7-H1 (seismic design intensity = 7-degree, and
induced by datasets under different design conditions. Building heights height ≤ 50 m), Group7-H2 (seismic design intensity = 7-degree, and
and seismic design intensities were adopted as the classification criteria height > 50 m), and Group8 (seismic design intensity = 8-degree). In
of the datasets because they are the critical factors that determine the addition, for contrast, Group Mix (mixed dataset) was composed of
mechanical performance of building structures. Higher heights and various data.
seismic design intensities correspond to increased requirements for Based on different training sets, this work obtained the GAN models
structural components [8]. Notably, utilizing mixed design datasets with called M-G7H1, M-G7H2, M-G8, and M-Mix. Subsequently, the testing
different heights and seismic design intensities for training, the final sets of Group7-H1, Group7-H2, and Group8 were adopted to evaluate
probability distributions of the automated designs were consistent with the design quality of the trained models. The testing results in Fig. 8
the average probability distribution of the mixed data, which cannot indicate that the best designs for the testing sets of Group7-H1, Group7-
satisfy the demand for different design conditions. Therefore, datasets H2, and Group8 were produced by M-G7H1, M-G7H2, and M-G8,

Fig. 8. Typical automated designs by different trained StructGAN models. (a) Testing groups of Group7-H1 (b) Testing groups of Group7-H2 (c) Testing groups
of Group8.

10
W. Liao et al. Automation in Construction 132 (2021) 103931

respectively, and the corresponding quantitative evaluation results are structural layout design, which takes approximately 30 s. Compared
shown with blue backgrounds. Specifically, in the designs of Fig. 8, the with the design of experienced engineers, adopting StructGAN for pre­
best designs by StructGAN are highly consistent with the designs by liminary structural design can reduce the time consumption from 3.5 h
experienced engineers, with the positions and lengths of structural shear to 10 min. Additionally, StructGAN can also improve the efficiency of
walls highly comparable. SIoU and WIoU are higher than 0.5, and the the overall structural design. As an estimation of this study, time con­
SWratios of StrutGAN designs are close to those of designs by engineers, sumption can be reduced from 300 h to 30 h with ten times efficiency
indicating that the qualities of structural layouts are excellent. Overall, enhancement. Furthermore, standardizing the semanticization process
when the test design conditions match the training structural design of architectural drawings can shorten the time of preliminary structural
conditions, both the layouts and total numbers of the structural shear design to 3 min, with higher overall design efficiency. StructGAN ac­
walls designed by StructGAN are comparable to those in the designs by celerates the entire structural design process by a factor of 10, which is
competent engineers. equivalent to a saving of approximately 100 million USD per annum in
Additionally, the quantitative evaluation results for all structural the industry, according to the statistical estimation of this work. See
designs are shown in Table 4, indicating that the best designs were Appendix B. [23] for additional estimation information.
devised by the StructGAN trained under the consistent design condi­ In addition to design efficiency and economics, design quality is
tions, with the largest corresponding ScoreIoU. In addition, the design equally critical. The comparison is performed on the drawings first,
qualities of the adequately trained StructGAN under different design followed by an evaluation of the structural behavior. Two approaches
conditions are relatively stable, with their ScoreIoU > 0.5. Hence, the based on computer vision and engineer perception were developed to
training datasets for StructGAN should be classified based on design support a comprehensive assessment. Subsequently, two overall struc­
conditions, and the design conditions of the adopted StructGAN should tural designs by StructGAN and engineers are compared to show that the
be matched with those of architectural sketches in a structural design StructGAN designs are not only safe but also economical. The observa­
application. In contrast, the designs by StructGAN with unmatched tions of similar material takeoffs and seismic resilience suggest that the
design conditions are irrational. The layouts and length of the structural structural design by StructGAN is as great as those optimized by
shear walls in the designs by StructGAN are inconsistent with those in competent engineers.
the designs by competent engineers, as shown in Fig. 8. Furthermore, the
designs developed by M-Mix are close to the average of mixed designs,
5.1. Plan drawing design performance
not precisely satisfying the demands of different design conditions.
Consequently, refined design conditions-based data classification nar­
In total, 16 structural design drawings by StructGAN and engineers
rows the restriction of the maximum likelihood estimation and further
were compared. Fig. 9a illustrates four typical structural plan designs for
improves the precision performance of StructGAN. Meanwhile, the
shear wall residential buildings, depicting the high consistency between
design quality evaluation results confirmed the precise and stable design
the designs by StructGAN and experienced engineers.
performance of StructGAN.
The computer-vision-based evaluation was performed first. The
quality evaluation results of structural designs by StructGAN are shown
5. StructGAN performance
in Fig. 9b. The metric ScoreIoU considers the comprehensive performance
of structural design and image generation. The consistency measure­
This study validated the efficiency and accuracy of the proposed
ment basis for ScoreIoU is the IoU, which denotes the ratio of the inter­
StructGAN framework via discussions and analyses. The excellent design
section area to the union area of structural designs by StructGAN and
performance was obtained through the optimal algorithm, the most
engineers. Previous studies indicated that an IoU > 0.5 corresponds to a
applicable hyperparameters, the proposed dataset split method, and
high consistency [45–47]. Moreover, in the comparative study of
rational evaluation methods. This section further demonstrates the su­
structural plan layouts, 16 buildings were adopted as cases and divided
perior performance of StructGAN by comparing it against the structural
into two groups according to their different design conditions. Thus, the
designs of competent engineers.
ScoreIoU in Fig. 9b is the average of each group, and it is higher than 0.5,
Specifically, in the StructGAN application, the semanticization of
indicating that the structural designs by StructGAN are comparable to
architectural drawings requires approximately 9 min, and then the se­
those by experienced engineers and are outstanding.
mantic drawings are input into StructGAN to generate the corresponding
Furthermore, engineers were also invited to judge the “AI” or

Table 4
Quantitative evaluation results for all automated designs by different trained StructGAN models for testing groups of Group7-H1, Group7-H2, and Group8.
Testing sets SIoU WIoU

M-G7H1 M-G7H2 M-G8 M-Mix M-G7H1 M-G7H2 M-G8 M-Mix

T-G7H1 0.41 0.46 0.44 0.52 0.59 0.55 0.47 0.57

T-G7H2 0.38 0.58 0.62 0.57 0.53 0.63 0.61 0.63
T-G8 0.37 0.60 0.74 0.67 0.51 0.62 0.72 0.64

Testing sets SWratiotarget M-G7H1 M-G7H2 M-G8 M-Mix

SWratioGAN 1 − ηSWratio SWratioGAN 1 − ηSWratio SWratioGAN 1 − ηSWratio SWratioGAN 1 − ηSWratio

T-G7H1 0.41 0.50 20% 0.69 41% 0.80 49% 0.65 37%
T-G7H2 0.58 0.44 35% 0.66 14% 0.74 23% 0.64 12%
T-G8 0.66 0.43 59% 0.67 14% 0.76 13% 0.71 9%

Testing sets ScoreIoU

M-G7H1 M-G7H2 M-G8 M-Mix

T-G7H1 0.40 0.31 0.24 0.35

T-G7H2 0.30 0.52 0.48 0.53
T-G8 0.18 0.53 0.65 0.61

The most critical metric is ScoreIoU, with value > 0.5 denoting excellent quality. ScoreIoU = ηSWratio × (ηSIoU × SIoU + ηWIoU × WIoU); ηSWratio = 1 − |SWratioGAN −
SWratiotarget|/SWratioGAN; ηSIoU = ηWIoU = 0.5.

11
W. Liao et al. Automation in Construction 132 (2021) 103931

(a)
Input

StructGAN designs

Optimal engineer designs

(b)
Datasets KSWratio SIoU WIoU ScoreIoU Datasets KSWratio SIoU WIoU ScoreIoU

Group7-H2 0.72 0.68 0.64 0.52 Group8 0.89 0.75 0.70 0.65
(c)
“AI” or “Engineer” judgment Rationality quantification

Engineers % of AI labeled Engineer design StructGAN design SEP-2 of SEP-2 of Difference

SEP-1
as Engineer score (out of 5) score (out of 5) engineer design StructGAN design of SEP-2
Experts 32.6% ± 19.6% 3.21 ± 0.46 2.84 ± 0.35
Non- 30.1% 3.51 3.05 −12.96%
27.8% ± 15.3% 3.84 ± 0.61 3.24 ± 0.46
Experts

Fig. 9. Evaluation results of the optimal StructGAN. (a) Typical structural designs by StructGAN (red, gray, blue, and green denote structural shear wall,
nonstructural infill wall, indoor windows, and outdoor gates, respectively). (b) Computer vision-based evaluation results (ScoreIoU = ηSWratio × (ηSIoU × SIoU + ηWIoU
× WIoU); ηSWratio = 1 − |SWratioGAN − SWratiotarget|/SWratioGAN; ηSIoU = ηWIoU = 0.5). (c) Engineer perception-based evaluation results, including “AI” or “Engineer”
judgment and rationality quantification. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

“Engineer” of designs and assess the rationality of StructGAN designs Notably, the corresponding peak ground acceleration (PGA) values of
based on their straightforward perception. In the “AI” or “Engineer” the design basis earthquake (i.e., 10% probability of exceedance in 50
judgment, engineers need to figure out the ones designed by StructGAN, years) are 100 cm/s2 and 200 cm/s2 in the 7-degree and 8-degree
and the judgment score is SEP-1. Simultaneously, in the design rationality seismic intensity zones, respectively.
assessment, the rationality score is SEP-2. This study invited 11 senior
experts (work experience >15 years), 12 practicing engineers, and
graduate students to participate in the judgment and evaluation work. 5.2. Overall structural design performance
Two important conclusions can be drawn from the results shown in
Fig. 9c. (1) Approximately 30% of the StructGAN designs were appraised Subsequently, two overall structural designs by StructGAN and en­
as designs by engineers, with the corresponding SEP-1 equal to 30%, gineers were compared. The safety and economic properties are directly
which is notably better than the highest Amazon Mechanical Turk associated with seismic deformation and material consumption,
(AMT) test result of 22.5% in the pix2pix study [17]. The above result respectively. Adopting StructGAN, this study developed two structural
indicates that it was challenging for humans to distinguish the designs designs following the guide in Fig. 1, and then compared their safety and
by engineers and StructGAN accurately. (2) The difference in rationality economic properties with those of the designs by competent engineers.
quantification for structural shear wall design between the StructGAN The two buildings are shown in Fig. 10a and b.
and engineers was approximately 12%, confirming that the StructGAN First, structural safety is primarily evaluated by the seismic story
structural designs were excellent and highly accepted by engineers. drift ratio because excessive story deformation under an earthquake can
Additionally, the architectural layouts of the 16 buildings are induce damage to structural components and facilities and even cause a
completely different, with seismic design intensities of 7-degree and 8- large number of casualties. Hence, ensuring that the seismic deforma­
degree, and heights of 28–140 m. The 16 cases were subdivided into tion of buildings satisfies the specifications is essential in the overall
groups of Group7-H2 and Group8, where Group7-H2 denotes buildings structural design. The comparisons of seismic story drift ratios for two
designed under 7-degree seismic intensity and heights over 50 m, and buildings are shown in Fig. 10d and e. As depicted in the figures, the
Group8 denotes buildings designed under 8-degree seismic intensity. maximum seismic deformation of the StructGAN design is only 11%
larger than that of the design by engineers, which is perfectly acceptable

12
W. Liao et al. Automation in Construction 132 (2021) 103931

(a) (b) (c) Case-7degree Case-8degree

Engineer StructGAN Percentage Engineer StructGAN Percentage

design design difference design design difference

Material consumptions (ton)

14099 14875 5% 13205 12784 −3%

Max story drift ratio (rad). Story drift ratios are shown in sub-figures (d) and (e)

0.00099 0.00109 0.00100 0.00112

10% 11%
y-direction y-direction y-direction y-direction

z Seismic repair costs (US dollars)

x y 10,627,106 11,829,493 11% 9,065,052 9,079,461 0%

(d) (e)
Engineer design StructGAN design Engineer design StructGAN design

40 40 35 35
35 35 30 30
30 30
25 25
25 25
Story

20 20
Story

Story

Story
20 20
15 15
15 15
10 10
10 10
5 5 5 5

0 0 0 0
0.00%0.03%0.05%0.08%0.10% 0.00% 0.05% 0.10% 0.15% 0.00%0.03%0.05%0.08%0.10% 0.00% 0.05% 0.10% 0.15%
x-direction story drift ratio y-direction story drift ratio x-direction story drift ratio y-direction story drift ratio

Fig. 10. Comparisons between the StructGAN designs and designs by engineers. The two cases are named Case-7degree and Case-8degree, with heights of
approximately 100 m and seismic design intensities of 7-degree and 8-degree, respectively. (a) 3D view of Case-7degree. (b) 3D view of Case-8degree. (c) Com­
parisons of overall performance in Case-7degree and Case-8degree. (d) Comparisons of story drift ratios in Case-7degree. (e) Comparisons of story drift ratios in
Case-8degree.

in the preliminary design and meets the safety requirements. Subse­ of qualified engineers. Hence, it is promising that the combination of
quently, this work compared the StructGAN designs and designs by StructGAN and automated optimization methods [49–51] can both
engineers with respect to material consumption, and the results are improve the design efficiency and quality in the future.
shown in Fig. 10c. The maximum percentage difference is within 5% in
the two cases, indicating that the automated designs consume almost the 6. Conclusions
same amount of materials as manual designs and fully meet the eco­
nomic requirements. Additionally, the seismic repair costs of the A GAN-based method for the structural design of high-rise shear wall
StructGAN designs and designs by engineers are comparable, as illus­ residential buildings (i.e., StructGAN) was proposed in this study,
trated in Fig. 10c. It is worth noting that, the seismic repair cost is mastering the direct map relationship for converting critical architec­
calculated based on the FEMA P-58 method [23], where it is determined tural elements into the corresponding structural designs. Moreover, a
by the story drift ratio, the story floor acceleration response, and the reasonable evaluation system and the corresponding metrics were
fragility of the structural components. Hence, in Case-8degree, the developed and adopted in the discussions and analyses of GAN algo­
phenomenon that the percentage difference of story drift ratio is 10% rithms and datasets, enhancing the learning and design performance of
while that of seismic repair cost is 0% is rational [48]. StructGAN. The outstanding StructGAN provides preliminary structural
Consequently, the economic and safety differences in the structural design schemes for architects and structural engineers, improving the
designs by StructGAN and experienced engineers are relatively slight, design efficiency and quality of building structures. The conclusions
and the StructGAN design meets the requirements of high efficiency and drawn are as follows:
high quality in the preliminary structural design. Presently, the struc­
tural mechanics performance of StructGAN-generated preliminary 1. Semantic designs can reduce the probability distribution dimension
design is not as excellent as that of engineers-optimized design. How­ of the StructGAN training dataset and enhance its study perfor­
ever, it is believed that the preliminary design by StructGAN would be a mance. Dataset classification by building heights and seismic design
great starting point for the follow-up optimization. Moreover, the effi­ intensities can narrow the restriction of the maximum likelihood
ciency of the StructGAN design is nearly 10 to 20 times higher than that

13
W. Liao et al. Automation in Construction 132 (2021) 103931

estimation, improving the precision of generative designs by (or extension) of the shear wall length, the connected beams are
StructGAN. extended (or shortened) to maintain the completeness of the structural
2. pix2pixHD (γ FM = 10) is recommended for StructGAN owing to its analysis model.
high-quality design capability with high efficiency and stability. An Note that, this study adopted the semi-automated modeler module to
appropriate simplification of the generative network architecture of guarantee the generated structural analysis model reliable. Moreover,
pix2pixHD is beneficial for further enhancing the design precision of the StructGAN modeler module is ready to be improved by adopting the
StructGAN. existing method of 3D model generation from 2D drawings [52].
3. Computer vision-based and engineering-perception-based evalua­
tion methods were developed and adopted in this study. The com­ Appendix B. Potential profit from StructGAN
puter vision-based evaluation quantifies and confirms the design
quality of StructGAN, and the engineer perception-based evaluation Approximately $265 billion annual profit pool awaits disrupters in
indicates that engineers highly accept the StructGAN designs. the construction industry, according to McKinsey [20]. The total cost of
architectural design accounts for approximately 2.5% of the entire
Notably, this study is the first to propose a GAN-aided structural construction cost, and the preliminary design accounts for approxi­
design method by establishing the complicated fuzzy map relationship mately 5% of the architectural design cost. StructGAN accelerates the
for converting semantic architectural sketches into structural layouts, design process and reduces the time consumption by approximately 1/3.
which can also be an automated design basis for other structural sys­ Hence, the potential profit created by StructGAN equals 110 million USD
tems, bridges, and tunnels. Moreover, as the design data and GAN (= 265 billion × 2.5% × 5% × 33.3%).
capability increase, the design performance of StructGAN is continu­
ously enhanced. Appendix C. Data availability statement

Declaration of Competing Interest The structural design datasets are collected from architectural design
and research institutes in China. The dataset and all codes for this study
The authors declare that they have no known competing financial are available on the authors’ GitHub page (https://github.com/wenjie-li
interests or personal relationships that could have appeared to influence ao/StructGAN_v1). The adopted computer hardware information: 6-core
the work reported in this paper. Intel Xeon CPU E5–2620, 32 GB memory, NVIDIA GeForce GTX TITAN X
(12 GB memory). The main software information: Windows 10, Python
Acknowledgments 3.6 with Pytorch 1.5, pix2pixHD algorithm [18]. All inquiries regarding
this content should be directed to the corresponding author.
This work was supported by the National Natural Science Foundation
of China (No. 51778341) and the Tencent Foundation through the References
XPLORER PRIZE. The authors would like to acknowledge Prof. Kincho H
Law (Stanford University) for his constructive suggestions for this work. [1] A. Chakrabarti, K. Shea, R.B. Stone, J. Cagan, M.I. Campbell, N.V. Hernandez, et
al., Computer-based design synthesis research: an overview, J. Comput. Inf. Sci.
Furthermore, the authors also appreciate Mr. Jie Jing from Architectural Eng. 11 (2011), 021003, https://doi.org/10.1115/1.3593409.
Design and Research Institute of Tsinghua University Co., Ltd., Mr. [2] S. Ahmed, M. Weber, M. Liwicki, C. Langenhan, A. Dengel, F. Petzold, Automatic
Zhaoqing Zhu from Anhui Provincial Architectural Design and Research analysis and sketch-based retrieval of architectural floor plans, Pattern Recogn.
Lett. 35 (2014) 91–100, https://doi.org/10.1016/j.patrec.2013.04.005.
Institute Co., Ltd., Mr. Hongjing Xue from Beijing Institute of Architec­ [3] N.C. Brown, C. Mueller, Design for structural and energy performance of long span
tural Design, Mr. Dongya An from East China Architectural Design & buildings using geometric multi-objective optimization, Energy Build. 127 (2016)
Research Institute Co., Ltd., Mr. Shaohua Pang from Guangxi Hualan 748–761, https://doi.org/10.1016/j.enbuild.2016.05.090.
[4] C.M. Herr, R.C. Ford, Cellular automata in architectural design: from generic
Design & Consulting Group, Mr. Zhenlin Han from Shandong architec­ systems to specific design tools, Autom. Constr. 72 (2016) 39–45, https://doi.org/
tural design and research institute Co., Ltd., Mr. Nan Zhao from Walton 10.1016/j.autcon.2016.07.005.
Design & Consulting Engineering Co., Ltd., Mr. Qi Wang from China [5] S. Tafraout, N. Bourahla, Y. Bourahla, A. Mebarki, Automatic structural design of
RC wall-slab buildings using a genetic algorithm with application in BIM
Architecture Design & Research Group, and Mr. Yifeng Liu China environment, Autom. Constr. 106 (2019) 102901, https://doi.org/10.1016/j.
Southwest Architectural Design and Research Institute Co., Ltd. for autcon.2019.102901.
providing building design blueprints that were used in this work. [6] CTBUH, Tall Buildings in 2019: Another Record Year for Supertall Completions,
CTBUH Research, 2019. https://www.skyscrapercenter.com/research/CTBUH_Re
searchReport_2019YearInReview.pdf.
Appendix A. Extension of the structural plane design to the [7] R. Perez, A. Carballal, J.R. Rabuñal, M.D. García-Vidaurrázaga, O.A. Mures, Using
overall structural design AI to Simulate Urban Vertical Growth, CTBUH Journal, Issue III, 2019. https://gl
obal.ctbuh.org/resources/papers/download/4212-using-ai-to-simulate-urban-vert
ical-growth.pdf.
Step1: The bitmap of the shear wall design was attached to the [8] Q.X. Shi, X.W. Liang, Design of Tall Building Structures, Ssecond edition, Science
AutoCAD drawing of the original architectural design using the attach Press, 2012 (in Chinese).
function in AutoCAD. [9] H. Zhang, T. Xu, H. Li, S. Zhang, X. Wang, X. Huang, et al., Stackgan++: realistic
image synthesis with stacked generative adversarial networks, IEEE Trans. Pattern
Step2: The coordinates and length of the StructGAN-designed shear Anal. Mach. Intell. 41 (2018) 1947–1962. https://ieeexplore.ieee.org/abstra
walls are obtained using the dimension function in AutoCAD. ct/document/8411144.
Step3: The structural analysis model is established based on the [10] X.Z. Lu, W.J. Liao, W. Huang, Y. Xu, X.Y. Chen, An improved linear quadratic
regulator control method through convolutional neural network-based vibration
original structural model and the shear wall coordinates obtained in identification, J. Vib. Control. 1077546320933756 (2020), https://doi.org/
Step 2 using the PKPM software. 10.1177/1077546320933756.
Step4: The following principles are adopted for establishing the [11] C. Xiong, Q.S. Li, X.Z. Lu, Automated regional seismic damage assessment of
buildings using an unmanned aerial vehicle and a convolutional neural network,
structural analysis model. In the event that certain pixels of the auto- Autom. Constr. 109 (2020) 102994, https://doi.org/10.1016/j.
designed shear wall are missing, the shear wall is considered contin­ autcon.2019.102994.
uous if pixels of the shear wall exist within that length. Shear walls with [12] Y.J. Xu, X.Z. Lu, B. Cetiner, E. Taciroglu, Real-time regional seismic damage
assessment framework based on long short-term memory neural network, Comput.-
a length shorter than the wall thickness (i.e., 200 mm) are excluded.
Aided Civil Infrastruct. Eng (2020), https://doi.org/10.1111/mice.12628.
Only the shear wall length of the structural design of the StructGAN is [13] I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et
adjusted without altering any other properties, such as the section al., Generative Adversarial Networks, arXiv preprint, arXiv:1406.2661, 2014, htt
thickness and material properties, to maintain a better comparison of the ps://arxiv.org/abs/1406.2661.

designs of the StructGAN and engineered design. After the shortening

14
W. Liao et al. Automation in Construction 132 (2021) 103931

[14] H. Zheng, W.X. Huang, Architectural drawings recognition and generation through [34] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Y. Fu, A.C. Berg, SSD: single
machine learning, in: Association for Computer Aided Design in Architecture 2018, shot multibox detector, Eur. Conf. Comp. Vis. (2016) 21–37. https://link.springer.
2018. http://papers.cumincad.org/data/works/att/acadia18_156.pdf. com/chapter/10.1007/978-3-319-46448-0_2.
[15] S. Chaillou, ArchiGAN: A Generative Stack for Apartment Building Design. https [35] R. Girshick, Fast R-CNN, IEEE Int. Conf. Comp. Vis. (2015) 1440–1448. https://ope
://devblogs.nvidia.com/archigan-generative-stack-apartment-building-design/?lin naccess.thecvf.com/content_iccv_2015/papers/Girshick_Fast_R-CNN_ICCV_2015_
kId=70968833, 2019. paper.pdf.
[16] N. Nauata, K. Chang, C. Cheng, G. Mori, Y. Furukawa, House-GAN: relational [36] J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: unified, real-
generative adversarial networks for graph-constrained house layout generation, time object detection, IEEE Conf. Comp. Vis. Pattern Recognit. (2016) 779–788.
Eur. Conf. Comp. Vis. (2020) 162–177, https://doi.org/10.1007/978-3-030- https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Redmon_
58452-8_10. You_Only_Look_CVPR_2016_paper.pdf.
[17] P. Isola, J.Y. Zhu, T. Zhou, A.A. Efros, Image-to-image translation with conditional [37] J. Gui, Z. Sun, Y. Wen, D. Tao, J. Ye, A Review on Generative Adversarial
adversarial networks, IEEE Conf. Comp. Vis. Pattern Recognit. (2017) 1125–1134. Networks: Algorithms, Theory, and Applications. https://arxiv.org/abs/2
https://openaccess.thecvf.com/content_cvpr_2017/papers/Isola_Image-To-Image_ 001.06937, 2020.
Translation_With_CVPR_2017_paper.pdf. [38] D. Diakoulaki, G. Mavrotas, L. Papayannakis, Determining objective weights in
[18] T.C. Wang, M.Y. Liu, J.Y. Zhu, A. Tao, J. Kautz, B. Catanzaro, High-resolution multiple criteria problems: the critic method, Comput. Oper. Res. 22 (1995)
image synthesis and semantic manipulation with conditional GANs, IEEE Conf. 763–770, https://doi.org/10.1016/0305-0548(94)00059-H.
Comp. Vis. Pattern Recognit. (2018) 8798–8807. https://openaccess.thecvf.co [39] G.M. Foody, Status of land cover classification accuracy assessment, Remote Sens.
m/content_cvpr_2018/papers/Wang_High-Resolution_Image_Synthesis_CVPR_ Environ. 80 (2002) 185–201, https://doi.org/10.1016/S0034-4257(01)00295-4.
2018_paper.pdf. [40] A. Rosebrock, Practical python and openCV: An Introductory, Example Driven
[19] M.M. Ivashkov, ACCEL: A Tool for Supporting Concept Generation in the Early Guide to Image Processing and Computer Vision (3rd Edition), Pyimagesearch,
Design Phase, [Doctoral Dissertation, Technische Universiteit Eindhoven]. htt 2016. https://www.pyimagesearch.com/practicalpython-opencv/.
p://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.8104&rep=rep1 [41] K. Shea, R. Aish, M. Gourtovaia, Towards integrated performance-driven
&type=pdf, 2004. generative design tools, Autom. Constr. 14 (2005) 253–264, https://doi.org/
[20] McKinsey & Company, The Next Normal in Construction. https://www.mckinsey. 10.1016/j.autcon.2004.07.002.
com/business-functions/operations/our-insights/the-next-normal-in-constructi [42] PKPM, Software Manual-Structural Analysis and Design Software for Multistory
on-how-disruption-is-reshaping-the-worlds-largest-ecosystem, 2020. and High-Rise Buildings SATWE, Beijing Glory PKPM Technology Co., Ltd, 2020
[21] H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, S. Savarese, Generalized (in Chinese).
intersection over union: a metric and a loss for bounding box regression, Comp. [43] M. Mangal, J.C. Cheng, Automated optimization of steel reinforcement in RC
Vis. Pattern Recognit. (2019) 658–666. https://openaccess.thecvf.com/content_ building frames using building information modeling and hybrid genetic
CVPR_2019/papers/Rezatofighi_Generalized_Intersection_Over_Union_A_Metric_a algorithm, Autom. Constr. 90 (2018) 39–57, https://doi.org/10.1016/j.
nd_a_Loss_for_CVPR_2019_paper.pdf. autcon.2018.01.013.
[22] A. Chakrabarti, S. Morgenstern, H. Knaab, Identification and application of [44] O. Ronneberger, P. Fischer, T. Brox, U-net: convolutional networks for biomedical
requirements and their impact on the design process: a protocol study, Res. Eng. image segmentation, in: International Conference on Medical Image Computing
Des. 15 (2004) 22–39. https://link.springer.com/article/10.1007%2Fs00163-00 and Computer-Assisted Intervention, Springer, Cham, 2015, pp. 234–241. htt
3-0033-5. ps://link.springer.com/chapter/10.1007/978-3-319-24574-4_28.
[23] FEMA, FEMA P-58-1, Seismic Performance Assessment of Buildings, Volume 1 – [45] M. Everingham, L. Van Gool, C.K. Williams, J. Winn, A. Zisserman, The pascal
Methodology, Second Edition, California, 2018. visual object classes (VOC) challenge, Int. J. Comput. Vis. 88 (2010) 303–338.
[24] M. Oh, J. Lee, S.W. Hong, Y. Jeong, Integrated system for BIM-based collaborative https://link.springer.com/article/10.1007%252Fs11263-009-0275-4.
design, Autom. Constr. 58 (2015) 196–206, https://doi.org/10.1016/j. [46] M. Everingham, S.A. Eslami, L. Van Gool, C.K. Williams, J. Winn, A. Zisserman, The
autcon.2015.07.015. pascal visual object classes challenge: a retrospective, Int. J. Comput. Vis. 111
[25] GB50011–-2010, Code for Seismic Design of Buildings, China Architecture & (2015) 98–136. https://link.springer.com/article/10.1007%252Fs11263-014-07
Building Press, Beijing, 2010 in Chinese. 33-5.
[26] JGJ3–-2010, Technical Specification for Concrete Structures of Tall Building, China [47] H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, S. Savarese, Generalized
Architecture & Building Press, Beijing, 2010 in Chinese. intersection over union: a metric and a loss for bounding box regression, in:
[27] J.T. Springenberg, Unsupervised and Semi-Supervised Learning with Categorical Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Generative Adversarial Networks. https://arxiv.org/abs/1511.06390, 2015. Recognition, 2019, pp. 658–666. https://openaccess.thecvf.com/content_CVPR_
[28] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, X. Chen, Improved 2019/papers/Rezatofighi_Generalized_Intersection_Over_Union_A_Metric_and_a_
techniques for training GANs, Adv. Neural Inf. Proces. Syst. (2016) 2234–2242. htt Loss_for_CVPR_2019_paper.pdf.
ps://arxiv.org/abs/1606.03498. [48] X. Lu, H. Guan, Comparison of seismic design and resilience of tall buildings based
[29] X. Wang, A. Gupta, Generative image modeling using style and structure on Chinese and US design codes, in: Earthquake Disaster Simulation of Civil
adversarial networks, Eur. Conf. Comp. Vis. (2016) 318–335. https://link.springer. Infrastructures, Springer, Singapore, 2021, pp. 171–222. https://link.springer.
com/chapter/10.1007/978-3-319-46493-0_20. com/chapter/10.1007/978-981-15-9532-5_4.
[30] R. Zhang, P. Isola, A.A. Efros, Colorful image colorization, Eur. Conf. Comp. Vis. [49] M. Fragiadakis, N.D. Lagaros, An overview to structural seismic design
(2016) 649–666. https://link.springer.com/chapter/10.1007/978-3-319-46487-9 optimisation frameworks, Comput. Struct. 89 (2011) 1155–1165, https://doi.org/
_40. 10.1016/j.compstruc.2010.10.021.
[31] J.Y. Zhu, T. Park, P. Isola, A.A. Efros, Unpaired image-to-image translation using [50] G.S. Papavasileiou, D.C. Charmpis, Seismic design optimization of multi–storey
cycle-consistent adversarial networks, IEEE Int. Conf. Comp. Vis. (2017) steel–concrete composite buildings, Comput. Struct. 170 (2016) 49–61, https://
2223–2232. https://openaccess.thecvf.com/content_ICCV_2017/papers/Zhu_ doi.org/10.1016/j.compstruc.2016.03.010.
Unpaired_Image-To-Image_Translation_ICCV_2017_paper.pdf. [51] S.A. Mirfarhadi, H.E. Estekanchi, M. Sarcheshmehpour, On optimal proportions of
[32] J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic structural member cross-sections to achieve best seismic performance using value
segmentation, IEEE Conf. Comp. Vis. Pattern Recognit. (2015) 3431–3440. http based seismic design approach, Eng. Struct. 231 (2021) 111751, https://doi.org/
s://openaccess.thecvf.com/content_cvpr_2015/papers/Long_Fully_Convolutional_ 10.1016/j.engstruct.2020.111751.
Networks_2015_CVPR_paper.pdf. [52] J. Lim, P. Janssen, R. Stouffs, Automated Generation of BIM Models from 2D CAD
[33] A. Garcia-Garcia, S. Orts-Escolano, S. Oprea, V. Villena-Martinez, J. Garcia- Drawings. Proceedings of the 23rd CAADRIA Conference, Beijing, China, 2018,
Rodriguez, A Review on Deep Learning Techniques Applied to Semantic pp. 61–70.
Segmentation. https://arxiv.org/abs/1704.06857, 2017.

15