Research Article

www.advmat.de

Studying Complex Evolution of Hyperelastic Materials under

External Field Stimuli using Artificial Neural Networks with
Spatiotemporal Features in a Small-Scale Dataset
Songlin Yu, Haiyang Chai, Yuqi Xiong, Ming Kang, Chengzhen Geng, Yu Liu,*
Yanqiu Chen, Yaling Zhang, Qian Zhang, Changlin Li, Hao Wei, Yuhang Zhao,
Fengmei Yu,* and Ai Lu*

1. Introduction
Deep-learning (DL) methods, in consideration of their excellence in dealing
with highly complex structure–performance relationships for materials, are Deep-learning (DL) based on statistical
expected to become a new design paradigm for breakthroughs in material algorithms is an effective means of mod-
performance. However, in most cases, it is impractical to collect massive- eling complex relationships among data
without any prior physical knowledge.[1–3]
scale experimental data or open-source theoretical databases to support Upon the considerable expansion of the
training DL models with sufficient prediction accuracy. In a dataset consisting material structure parameter space as
of 483 porous silicone rubber observations generated via ink-writing additive realized by additive manufacturing,[4–6]
manufacturing, this work demonstrates that constructing low-dimensional, DL shows great promise for overcoming
accurate descriptors is the prerequisite for obtaining high-precision DL the limitations of traditional phenomeno-
logical constitutive relationships or ideal-
models based on small experimental datasets. On this basis, a unique con-
ized physical models, and establish a new
volutional bidirectional long short-term memory model with spatiotemporal design paradigm for the development of
features extraction capability is designed, whose hierarchical learning mecha- materials with excellent performance.[7,8]
nism further reduces the requirement for the amount of data by taking full However, DL’s current applications
advantage of data information. The proposed approach can be expected as a in material design face the following
powerful tool for innovative material design on small experimental datasets, dilemma. On the one hand, high-precision
DL models often require datasets with
which can also be used to explore the evolutionary mechanisms of the struc- large sample sizes to supply sufficient
tures and properties of materials under complex working conditions. information for their training.[9,10] Cur-
rent material design methods based on
DL mainly rely on large-scale open-source
S. Yu, Y. Xiong, C. Geng, Y. Zhang, Q. Zhang, C. Li, H. Wei, Y. Zhao, theoretical calculation databases[11,12] or advanced simulation
F. Yu, A. Lu tools[1,13–16] to obtain sufficiently large datasets, which originate
Institute of Chemical Materials from highly idealized molecular or structural models and differ
China Academy of Engineering Physics from datasets based on actual experimental measurements.
Mianyang, Sichuan 621900, P. R. China
E-mail: [email protected]; [email protected]
The distortion of these datasets is then also naturally inher-
S. Yu, M. Kang, C. Li, H. Wei, Y. Zhao, A. Lu
ited by the DL models that are trained on them.[12,17] To make
State Key Laboratory of Environment-Friendly Energy Materials matters worse, there will be a certain shielding effect on the
School of Materials Science and Engineering structure–performance response in the actual sample space,
Southwest University of Science and Technology and thus, new discoveries may be lost due to the simplifica-
Mianyang, Sichuan 621010, P. R. China tion and abstraction of the applied models. On the other hand,
H. Chai additive manufacturing is being widely considered to provide
School of Big Data and Software Engineering
Chongqing University high-throughput experimental results with accurate control-
Chongqing 401331, P. R. China ling of the structural parameters and extensive expansion of the
Y. Liu, Y. Chen parameter space. However, it is still impractical to collect mas-
School of Mechanical Engineering sive-scale experimental data to train DL models. Although deep
Jiangnan University transfer learning reduces the scale of the experimental data
Wuxi, Jiangsu 214000, P. R. China
E-mail: [email protected]
as required in material design, this approach still needs con-
siderable theoretical calculation data or open-source database
The ORCID identification number(s) for the author(s) of this article
can be found under https://doi.org/10.1002/adma.202200908. support.[12] However, in most material-related research work,
expensive computing costs and a lack of open-source theoretical
DOI: 10.1002/adma.202200908 databases restrict the applications of DL for material design.

Adv. Mater. 2022, 34, 2200908 2200908 (1 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Therefore, building a high-quality DL prediction model from (BLSTM) algorithm, is designed to capture the structure evolu-
small-scale experimental data is a universally beneficial strategy tion of porous silicone rubber under external force. iii) Hierar-
to promote the development of DL in materials science.[18,19] chical learning mechanism—static spatial structure features and
In this work, we take the nonlinear stress and strain behavior serialized stimulus signals are input separately, further reducing
of hyperelastic materials as the focus of our research to achieve the difficulty of the model’s understanding of the task. These
high-precision DL prediction models based on a small sample three strategies synergistically improve the accuracy of DL pre-
space. This example is of general interest in material mechan- diction models and reduce data volume requirements.
ical experiments because it can be regarded as a universal case The specific workflow schematic of DL modeling in this work
in the mechanical response process of materials whose spatial is shown in Figure 1. First, experimental datasets were collected,
structure features continuously evolve under sustained external which included sample static spatial structures, serial external
stimuli. More importantly, the full response process can obtain stimulus signals (serial strains), and the corresponding stresses
more key characteristics of materials compared with extreme with context relations. Subsequently, precise, low-dimensional
point researches.[20] Hyperelastic materials (a typical example as descriptors were used to capture the static spatial structure
porous elastomer), which undergo large deformation under the features of the samples, and a CBLSTM model consisting of
action of external force and fully recover to the beginning after both CNN and BLSTM units was employed to abstract samples’
the withdrawal of external force, are a class of materials with temporal (BLSTM), and spatial (CNN) changes under uniaxial
stress–strain relations that exhibit nonlinear characteristics. In compression. Finally, in accordance with the obtained CBLSTM
recent decades, important mathematical and physical models prediction model, a hyperelastic porous silicone sample with a
for studying hyperelastic materials have been proposed.[21–24] prominent hyperelastic stress platform based on a straightfor-
However, complete constitutive relations suitable for multiple ward stacking structure is designed and fabricated by additive
deformation modes and full deformation range are still a cru- manufacturing. This goal is generally considered to require
cial problem to be solved in this field. Although considerable extremely complex spatial structure design[29,30] and cannot be
progress has been achieved in hyperelastic finite deformation achieved by traditional porous materials. Thus, the CBLSTM
theory, the equation given by finite elasticity theory is complex model in our work is expected to alleviate the current issues
and challenging to solve.[25] The abstraction and simplification with high experimental and computational costs when facing
of the related structural parameters inevitably cause a reduc- the implementation of high-precision DL in material design
tion in the actual performance space, which offsets the range and to speed up the discovery of innovative materials by virtue
of possible performance that can be achieved by realizing a of its excellent performance.
large number of structures utilizing digital preparation tech-
nology.[26] Expensive data collection costs still impede DL as a
powerful tool for constructing constitutive relation for additive 2. Results and Discussion
manufacturing.[7] Previous studies have sought to augment data
to improve DL model accuracy by data augmentation strate- 2.1. Dataset
gies, including generative adversarial networks[17,27] (GAN) and
decoder.[7,28] In light of these effective methods being subject The dataset used in this paper was constructed based on the
to the validity of the data generator, we propose a unique per- observed compressive stress–strain curves of a typical hyper-
spective that hyperelastic materials’ stress and strain behavior elastic material, that is, porous polysiloxane elastomer, with a
can be regarded as a change in time and space, and we reduce series of simple stacked structures that were prepared by means
the data requirements for DL by manipulating neural networks of direct ink writing (DIW) additive manufacturing (Figure S1,
with spatiotemporal features and hierarchical learning mecha- Supporting Information). Optical microscopy images of porous
nisms to maximize data information. silicone rubber samples obtained via 3D printing are shown
This paper is organized to present a high-precision DL model in Figure S2, Supporting Information. For more 3D structural
for the constitutive relations of hyperelastic materials based on details, see Movie S1, Supporting Information (porous silicone
a small sample space, starting from the following three points: rubber with a simple cubic structure) and Movie S2, Supporting
i) Descriptors of moderately coarse sample structural features Information (porous silicone rubber with a simple dislocation
are applied. The main type of investigated structure is the porous structure). The experimental dataset consists of 483 samples
structure of hyperelastic materials. Although porous structures whose spatial structure features and corresponding stress–
constructed based on additive manufacturing can be accurately strain columns were recorded. To account for the Mullins effect
described using geometric coordinate systems, it is expected in silicone rubber, each sample was tested three times under
that a large sample space should be needed to obtain an accu- the same conditions, and the third results were written into the
rate DL model. Using a simple descriptor such as density, a clas- dataset. Finally, all stress–strain curves are represented as arrays
sical spatial structure parameter which appears in the traditional of stress values evaluated at 50 strain points. The main reason is
constitutive relation of stochastic porous materials, will lose a that although the stress–strain curves of hyperelastic materials
large amount of structural information and lead to unacceptable during the compression process can be roughly divided into
errors. ii) The sample structure and its stress–strain response can an elastic deformation zone, a buckling platform zone, and a
be regarded as processes of spatial and temporal changes, respec- densifying zone, there are often no obvious boundaries between
tively. A convolutional bidirectional long short-term memory these zones in actual materials, so it is difficult to use corre-
(CBLSTM) model, which integrates the benefits of convolutional sponding characteristic points to describe the curves. Given
neural network (CNN) and bidirectional long short-term memory the limitations imposed by the frame rigidity of the instrument

Adv. Mater. 2022, 34, 2200908 2200908 (2 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Figure 1. Workflow schematic of high-quality DL modeling based on a small-scale experiment dataset for mechanical property design of porous silicone
rubber.

used for mechanical testing, we consider only stress data points of structural information for the sample. ii) Spatial structure
below a stress level of 0.8 MPa to avoid measurement error due parameters. Considering the more precise and controllable
to instrument creep, and each stress–strain curve is divided spatial structures of samples that are printed layer by layer by
into 50 columns based on an equal-difference strain sequence means of additive manufacturing, we choose a group of spa-
to ensure that the stress–strain columns in the dataset are of tial structure parameters that can be intuitively described in
equal length. Because the stress values corresponding to high- accordance with natural human understanding as more elabo-
and low-strain areas span several orders of magnitude, we nor- rate descriptors, consisting of the strand diameter (the diam-
malize all inputs to reduce the computational burden of the eter of a single wire for DIW), the line spacing (the distance
model. The standardization formula is as follows the equation between two adjacent strands), the layer height (the distance
between neighboring layers), the dislocation (the lateral shift
x−µ between one layer and the next) and the number of layers (the
z=  (1)
δ number of layers constructed via additive manufacturing), as
shown in Figure S3, Supporting Information. iii) Spatial coor-
x represents the original data, z represents the data after stand- dinates combined with strand diameter. The most accurate
ardization processing, and μ and δ are the mean and standard way to describe the spatial structure of a sample is to trans-
deviation, respectively, of all data in the same category. form its complex geometric structure features into 3D spatial
coordinates, which computers can easily process compared
with complex geometric structure input as an image. Thus, we
2.2. Descriptors introduce the coordinates of the spatial structure of a sample
as the most accurate descriptors, which also include the strand
Although the research on descriptors is fruitful,[10,31] the fields diameter because the strands constituting the spatial structure
of study in material science are rich with great differences and are of finite thickness. Specifically, as shown in Figure S4, Sup-
small correlations.[19] Hence, the primary task is to choose a porting Information, we construct a spatial coordinate system
suitable descriptor for our objective. We use three groups of with its origin at the far left of the bottom of the sample to
descriptors with gradually increasing degrees of refinement to describe the spatial structural properties of the sample, and the
extract the spatial geometric features of the samples. i) Density. structure of each layer was depicted using three spatial coor-
For systems with conventionally irregular porous structures, the dinates and strand diameter. Considering that the output is a
density, is easy to handle in analytically theoretical equations serialized process prediction of 50 time-steps and the input is a
and measure and control in practical applications. Also, it is serialized stimulus signal of 50-steps, the descriptors are repli-
the key parameter in the classical constitutive equations used to cated to match the dimensions of the stimulus signal and form
characterize the geometric features of cellular solids. Therefore, the input matrix. Hence, the input matrix of density, spatial
for ease of comparison, we choose the density of a sample as structure parameters, and spatial coordinates combined with
the coarsest descriptor, which actually discards a large amount strand diameter are 50 × 2, 50 × 6, and 50 × 13, respectively.

Adv. Mater. 2022, 34, 2200908 2200908 (3 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Table 1. Evaluation of the impact of different descriptors.

Tenfold cross-validation results Test set results

Descriptor(s) MSE MAE MSE MAE Dimensions
Density 0.1069 ± 0.0155 0.0223 ± 0.0149 0.1050 ± 0.0139 0.2106 ± 0.0135 50 × 2
Spatial coordinates+ diameter 0.0823 ± 0.0125 0.0172 ± 0.0152 0.0806 ± 0.0147 0.1692 ± 0.0164 50 × 13
Structural parameters 0.0582 ± 0.093 0.1522 ± 0.0102 0.0687 ± 0.0127 0.1572 ± 0.0143 50 × 6

2.3. Effect of Descriptors on Model Accuracy lowest for any training set size, and the model using the spatial
coordinates combined with the strand diameter lies in accuracy
A BLSTM model (for details, see Experimental Section and between the other two models. Notably, the MAE curve of the
Figure S5, Supporting Information), whose accuracy is evalu- model using the structural parameters crosses the horizontal
ated with the mean squared error (MSE) and the mean absolute dashed line corresponding to the performance of the density-
error (MAE) as the loss function, is employed to compare the based model trained on 432 data samples at a training set size
accuracy of models based on descriptors of different precisions, of ≈85 (Figure ), implying that the model based on the single
where lower MAE and MSE values imply that the predictions density descriptor needs approximately five times as many data
are closer to the ground truth. The results obtained through for training to obtain the same accuracy as the model based on
tenfold cross-verification are recorded in Table 1, and this table the second group of descriptors. Similarly, the structural para-
shows that the prediction model built using only the density meter descriptors model requires fewer data (approximately
as the sample structure descriptor exhibits the worst accuracy half) to achieve the same MSE value as the model using the spa-
among the models based on the different groups of descrip- tial coordinates combined with the strand diameter (Figure 2B).
tors. This finding indicates that using more precise descriptors It can be attributed that the accuracy of the models generated
can improve the model’s generalization ability. However, the based on the different descriptors, varies in sensitivity to the
model accuracy based on descriptors consisting of the spatial dataset size. If the training set contains sufficient data to train
coordinates and the strand diameter is lower than that of the models that can map each characteristic structure to its final
model based on the intuitive spatial structural parameters. This properties, the most accurate descriptors are undoubtedly the
fact suggests that increasing the dimensionality to improve the best choice. However, collecting such large-scale data is an
description accuracy of the sample features does not always arduous task for most material-related experiments; therefore,
yield positive returns on limited experimental datasets. the dimensionality and accuracy of the descriptors should
To further investigate the effect of the descriptor dimen- instead be suitably balanced. Therefore, appropriate low-dimen-
sions on the model accuracy for small experimental datasets, sional, accurate descriptors are a key requirement for applying
the Monte-Carlo sampling was used to divide the training DL based on small-scale experimental datasets.
and test set. On which the test set has 48 data, the number of
training data ranges from 48 to 432. The results for each scale
are the average of ten Monte-Carlo samplings and are depicted 2.4. Model Building and Evaluation
in Figure 2. For each training set size, the model based on the
spatial structure parameters offers significantly higher predic- To enhance the prediction accuracy for the complex nonlinear
tion accuracy than the other two approaches. Meanwhile, the features of porous silicone rubber, we have designed a CBLSTM
accuracy of the model using the density descriptor solely is the model to extract the temporal and spatial characteristics of the

Figure 2. Impact of the descriptors on the prediction results. A,B) Performance of models based on the three types of descriptors with different training
set sizes in terms of the MAE (A) and the MSE (B) as indicators. Note that the intersection of the orange curve with the horizontal green (blue) line
represents the amount of data required for the model based on spatial structure parameters to achieve the best accuracy achieved by the model based
on the density descriptor (spatial coordinates combined with the strand diameter).

Adv. Mater. 2022, 34, 2200908 2200908 (4 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Figure 3. Schematic illustrating the CBLSTM model. The model consists of CNN and BLSTM units, where the CNN units are used to extract the static
spatial features of 3D-printed porous silicone rubber samples and the BLSTM units are used to process the serial stimulus process data.

samples. Figure 3 plots a schematic of the CBLSTM model, and into 10 equal parts, 9 of which are used to train the model, and
the details of its construction are described in Experimental the last part is employed to validate the trained model, the pro-
Section. Meanwhile, classic BLSTM (Figure S5, Supporting cedure is repeated ten times by using different data combina-
Information, construction details provided in Experimental tions in one cross-validation. The MSE, MAE, and Huber loss
Section) and CNN (Figure S6, Supporting Information, con- functions (for details, please see Experimental Section), which
struction details provided in Experimental Section) models, are widely used in regression task, were used for model evalu-
which are widely used in materials science, are considered ation.[34] All models were trained from scratch, and the model
for comparison. Considering our small-scale dataset, the with the lowest values of the evaluation metrics on the test
rational network architecture design of the model is critical for set during 300 repeated epochs of training was saved. Density
improving model accuracy and avoiding the risk of overfitting. scatter plots of the prediction error and line plots of the cumu-
Here, we performed exhaustive ablation experiments to deter- lative distribution functions (CDFs) of the different models
mine the optimal combination of models hyperparameters (see on each test set in tenfold cross-validation are presented in
Experimental Section), and recorded the results in Table S1, Figure 4. A CBLSTM network with four convolutional layers
Supporting Information. The optimal combination of hyper- and two BLSTM units, in which the spatial structure param-
parameters for CNN is 4 convolutional layers with a learning eters and strain were used as the inputs to the convolutional
rate of 0.0001 and a batch size of 128; the optimal combina- layers and BLSTM units, respectively, achieved MAE = 0.1457
tion of hyperparameters for BLSTM is 3 BLSTM units with a (Figure 4A), MSE = 0.0611 (Figure 4B), and Huber = 0.0267
learning rate of 0.0005 and a batch size of 32; as for CBLSTM, (Figure 4C). For comparison, the spatial structure parame-
the optimal combination of hyperparameters is 4 convolutional ters and strain together were also used as the inputs to CNN,
layers and 2 BLSTM units, a learning rate of 0.0005 and a batch BLSTM, and LSTM models, and the results show that the loss
size of 32. Furthermore, the accuracy of the three models did values of the CNN (Figure 4E–G), BLSTM (Figure 4I–K) models
not further improve with the increase of trainable parameters. are all higher than those of the CBLSTM model.
More notably, compared with CNN and BLSTM alone, the inte- Moreover, we can observe the same tendency from the den-
gration of CNN and BLSTM has higher accuracy in most cases, sity scatter plots; although most of the scatter is concentrated
which owing to its full utilization of data information. along the diagonal for all models, the points predicted by the
Since outliers in small datasets with fixed splits can seriously CBLSTM model are the closest to the actual experimental values
affect the evaluation of the model,[32] the ten times k-fold cross- when MAE, MSE, and Huber as loss functions. In addition, we
validation was performed to exhaustively evaluate the accuracy further compared the BLSTM and LSTM models to evaluate
of the various DL models to predict the stress–strain curve. the benefit of a bidirectional network. These two models have
Before each k-fold cross-validation, a new test set and valida- almost identical frameworks; the only difference is that BLSTM
tion set are obtained through Monte Carlo sampling.[33] The k units are substituted for LSTM units. As shown in Figure 4I–K
value is highly related to the accuracy and robustness of the and Figure S8A–C, Supporting Information, the BLSTM model
model. Thus, we compared the effect of different k values on has lower MAE and MSE loss values, indicating that the bidi-
the model (Figure S7, Supporting Information). When k = 10, rectional network has a better ability to capture features than
the model has the best accuracy and modest robustness. There- the network with only unidirectional LSTM units. Meanwhile,
fore, we set k as 10, namely, the dataset was randomly divided to verify the merit of the hierarchical learning mechanism,

Adv. Mater. 2022, 34, 2200908 2200908 (5 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Figure 4. Prediction error analysis of different models. The density scatter plots contain the prediction results of each test set in a tenfold cross-
validation of closest the mean, and the color gradually deepens in accordance with the color scale presented on the right side of the scatter chart,
indicating increasing density. The three rows represent the three models: A–D) a CBLSTM model, E–H) a CNN model, and I–L) a BLSTM model. In
each row, the first (A, E, and I), second (B, F, and J) columns, and third (C, G, and K) columns represent the prediction results with the MAE, MSE,
and Huber, respectively, as the loss function, and the fourth columns (D, H, and L) depicts the corresponding CDFs of the model prediction errors,
and insert is the partial enlargement of CDF.

we attempted to use the sequential characteristics and spatial 2.5. Prediction of Nonlinear Characteristics
structure parameters together as the input to a CBLSTM model
(denoted by CBLSTM-1). Although the MAE, MSE, and Huber Prediction of an entire response process is beneficial for a com-
of CBLSTM-1 are lower than using CNN or BLSTM alone, they prehensive understanding of material properties.[20] Since intri-
are higher than those of a CBLSTM model that extracts features cate constitutive relation between structure and performance,
separately (Figure S9, Supporting Information), which is attrib- the effective analysis of nonlinear characteristics still is the
uted to the hierarchical learning mechanism helps to reduce main challenge.[24] In pioneer works, researchers focused on
the difficulty of the model’s understanding of the task.[35] insights into the relationship between spatial structure and cor-
To select the most suitable loss function for different models, respondence performance, ignoring the underlying logical rela-
we plotted the prediction error CDFs of models with the MAE, tionships between performances under an external field.[20,36–38]
MSE, and Huber as the loss functions (Figure 4D,H,L). The Here, we employ the CBLSTM model to predict the nonlinear
curve of MAE is closer to the y-axis, meaning that the MAE behavior of foam rubber silicone, the CNN and BLSTM model
predicts the stress–strain relation with higher precision than as a comparison. We choose the MAE as the loss function of
the MSE and the Huber in three models. Therefore, the our model because of its excellent performance in different
results show that the MAE is a more suitable loss function for models (Figure 4D,H,L). We further evaluate the predictive
CBLSTM, CNN, and BLSTM. Additionally, we compare the accuracy of various models for stress–strain profiles with other
prediction error CDFs of the different models with the MAE nonlinear characteristics, including three representative stress–
(Figure S10, Supporting Information). The CBLSTM model strain profiles with different mechanical strengths (the first and
with MAE as loss function clearly has significantly lower pre- third columns of Figure 5) and a hyperelastic stress plateau
diction errors than the other models, illustrating that the (the second column of Figure 5). It should be noted that these
CBLSTM model is the best prediction model for our task of three stresses–strain curves are independent of both the test
stress–strain. set and the training set. We choose the model corresponding to

Adv. Mater. 2022, 34, 2200908 2200908 (6 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Figure 5. Evaluation of the prediction of the entire stimulus-response process using three models. Here, three representative response curves are
selected as the objects of prediction. The three rows present the prediction results of three models: A–C) a CNN model, D–F) a BLSTM model, and
G–I) a CBLSTM model.

the lowest MAE value on the test set in 10-fold cross-validation hyperelastic stress plateau height are accurately predicted. Now
as the predictive model. The different rows of Figure 5 show let us consider the BLSTM model, which adopts the same input
comparisons of the curves predicted by the CNN, BLSTM, and method as the CNN model except that during training, the input
CBLSTM models with the experimental test curves. As shown is further divided into single time steps in accordance with the
in Figure 5A–C, although the trend of the entire forecast curve sequential characteristics and the output from the previous
roughly follows the test curve in each case, the CNN-predicted time step is added to the training data for the current time step;
curves are tortuous zigzag lines. This result is consistent with the results of using this model to predict the same stress–strain
previous work that did not use temporal features,[20,38] which curves are shown in Figure 5E–G. The BLSTM model, which
can be attributed to the working mechanism of the CNN. When takes both forward and reverse sample inputs, can simultane-
the sequential features and spatial structure features are used ously capture contextual information; as a result, the prediction
together as the input to the CNN model, the first-layer convo- curves are smoother than those of the CNN model. However,
lution kernel extracts all features simultaneously, and each fol- the ability of a BLSTM network to extract spatial features is
lowing convolution kernel further extracts features from the insufficient compared with that of a CNN, which has been
results of the previous layer, thereby ignoring the serial fea- proven to be effective at extracting multidimensional structural
tures. Nevertheless, due to the powerful feature extraction capa- features. Finally, our CBLSTM network, combining the capabil-
bilities of the CNN model, the sample mechanical strength and ities of CNN and BLSTM units, exhibits better performance in

Adv. Mater. 2022, 34, 2200908 2200908 (7 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

continuous process prediction (Figure 5G–I), as it inherits the validation loss during part of the training process were shown
outstanding multidimensional structure extraction ability of a in Figure S13, Supporting Information to evaluate different
CNN and the exceptional sequential feature capturing ability of models’ risk of overfitting in a small dataset. Even if 48 data
a BLSTM network. It should be that the input structure of the were used as a training set, the validation loss still keeps stable
CBLSTM network is highly correlated with the final properties with the increase of training epochs. These models have no
of the model. In the CBLSTM model, CNN units first extract significant risk of overfitting, which benefit from: 1) accurate
structural features; subsequently, the sequential characteristics and low-dimensional descriptors; and 2) few-layer neural net-
and the CNN output are together sent to BLSTM units for fur- work architecture design. These findings demonstrate that
ther feature extraction. This unique input approach maximizes the CBLSTM model can further reduce the demand for data
the use of both the CNN and BLSTM network characteristics, and its reliability in small-scale datasets, paramount in mate-
ensuring the capture of spatial structure features and avoiding rials-related experiments. To further analyze the impact of the
the problem of the distortion of serial features caused by con- dataset size, we draw the predicted stress–strain curves for
volution kernel processing. Moreover, the CDFs of the various nonlinear behavior corresponding to different training set sizes
models are shown in Figure S11, Supporting Information, from (Figure 6B and Figure S14A,B, Supporting Information). We
which it can be seen that the prediction error of the CBLSTM find that the prediction for a curve without a hyperelastic stress
model is lowest compared with the other models. Meanwhile, plateau is highly consistent with the test curve (Figure S14A,B,
we tried more combinations of model hyperparameters to pre- Supporting Information) when only 48 training data samples
dict the stress–strain process (Figure S12, Supporting Informa- are used, whereas the predicted and test curves for a sample
tion), can reach the same conclusion as above. with a hyperelastic plateau feature almost overlap (Figure 6B)
when the training set size reaches 240. These results illustrate
that complex nonlinear mechanical behavior can also be fore-
2.6. Impact of the Training Data Size on the CBLSTM Model casted even in the case of an extremely small experimental
dataset. Although we have 483 data samples, we can actually
The training set size is a crucial indicator for the accuracy that implement a high-precision DL prediction model based on an
can be achieved by a DL model; however, the collection of large- even smaller dataset.
scale experimental data is a daunting challenge that hinders
the application of DL in materials science.[19,26] To demonstrate
that our method can deliver high-accuracy predictive models 2.7. Material Design to Find Structures for Samples
based on limited datasets, we use Monte-Carlo sampling to with Ultralong Stress Plateaus
obtain new pairs of training and test sets.[33] These test sets
cover 48 data, and the size of these training sets varies from In comparison to traditional material design methods, the
48 to 432. We choose MAE as the evaluation indicator since it revolution in material design enabled by DL offers far greater
has the best performance in all models (Figure 4D,H,L). The efficiency in exploring material properties of interest in a large
variation in the MAE on the test set with the training set size potential design space. Here, we predicted the stress–strain
is shown in Figure 6. Noting, each result of different scales was curves of 21 500 groups of random structural parameters using
the mean of ten Monte-Carlo samplings. We observe a signifi- the trained CBLSTM model, and the time spent on the whole
cant impact of the size of the training dataset; as the training forecasting process was only 3.55 s (graphics card configura-
set size increases, the overall loss value shows a downward tion: MSI GeForce RTX 3060Ti). Moreover, the structure of a
trend. Of particular note, each loss value of the CBLSTM model porous elastomer sample with an ultralong stress plateau in the
is lower than those of the other models for the same scale of compressive stress–strain curve, which has been suggested by
the training dataset. Moreover, changes in the training loss and Gibson and Ashby[39] but hardly appears in stochastic porous

Figure 6. Performance of the CBLSTM model with different training set sizes. A) Comparison plot of the prediction accuracy of three models with dif-
ferent training set sizes. B) Comparison chart of the results of the CBLSTM model trained on training sets of different sizes for predicting a response
curve with a hyperelastic stress plateau. The inset shows the distributions of the errors between the predicted and actual values.

Adv. Mater. 2022, 34, 2200908 2200908 (8 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Information. The CBLSTM has the best performance in whole

stimulus response process prediction, which agrees with our
conclusion in the prediction of stress–strain behavior. Hence,
the results indicate that the CBLSTM model can also be applied
to other stimulus-response tasks of material science and can
get a higher predictive accuracy compared to CNN or BLSTM
model.

3. Discussion
Obtaining DL models with satisfactory prediction accuracy and
generalization ability based on small experimental datasets is
Figure 7. Material design to find a structure of interest. Design and one of the most critical issues to be solved for a material design
testing of a Gibson–Ashby ideal stress–strain curve for a porous elas- using DL models. More specifically, three points should be
tomer with an ultralong stress plateau. taken into account: i) the development of a group of descriptors
that can extract sufficiently accurate features of a sample, ii) the
silicone rubber, was successfully predicted, printed and tested; reduction of the data dimensionality as much as possible while
see Figure 7. This material’s stress remains stable for an exten- ensuring the accuracy of the descriptors, and iii) the establish-
sive range of strains after the initial short-lived linear region, ment of an optimal algorithm to maximize data information.
even at close to 40 percent strain. More importantly, the pre- This work presents a universal CBLSTM model for using small
dicted stress–strain curve of the porous silicone rubber with the experimental datasets to build high-precision DL models to
designed structure almost coincides with the test curve, sug- predict the nonlinear mechanical property of hyperelastic mate-
gesting that our DL model can be used as an efficient predic- rials based on an appropriate descriptor.
tive tool to design materials with certain properties of interest As the most straightforward 1D descriptor, the density
without extensive trial-and-error experiments. cannot accurately describe the spatial structural characteris-
tics of porous materials prepared through additive manufac-
turing. Therefore, it is challenging to obtain high-precision DL
2.8. Method Transferability models based solely on this parameter. However, increasing the
descriptor dimensions to improve the accuracy of the descrip-
The previous results clearly show that the CBLSTM model is tors for spatial structure extraction does not always yield a posi-
suitable for predicting constitutive (stress–strain) relations tive return on small datasets. In this work, lower-dimensional
for multiple deformation modes and large deformation range structural parameters lead to lower MAE, MSE, and Huber
of porous silicone rubber by small-scale experimental data. values than high-dimensional spatial coordinates combined
In order to validate the universality of CBLSTM model for with the strand diameter, although the latter set of parameters
common stimulus-response tasks, we employ three models to describes the spatial structure more precisely. Thus, the prereq-
predict the broad frequency damping characteristics of porous uisite for obtaining high-precision DL models based on small
silicone rubber (more detail on data collecting, preprocessing, experimental datasets is to construct low-dimensional, accurate
please see Experimental Section). The damping characteristic descriptors.
is a typical stimulus-response process, which can be viewed The predictive models studied here are related to the non-
as the energy dissipation response of materials under external linear stress response of ordered porous structured mate-
dynamic stress in different frequencies. Similar to predicting rials under simulated strain. We demonstrate the benefit of
stress–strain curves, we also select structural parameters as exploiting both the CNN and BLSTM algorithms to build pre-
descriptors. After ten times tenfold cross-validation, the detailed dictive models with better generalization ability than models
results were tabulated in Table 2. Benefiting from the CBLSTM built using either of these algorithms alone. When only a CNN
model integrating CNN powerful spatial structure informa- is used, the strain is also input as part of the material’s struc-
tion extraction capabilities and the excellent sequential feature tural characteristics. Because a CNN has a strong ability to
extraction capability of BLSTM, the CBLSTM model has the extract material structural characteristics, models established in
lowest error of MAE and MSE. Meanwhile, the prediction of the this way can predict the variation trend of the stresses but fail to
entire response process was plotted in Figure S15, Supporting capture the inheritance relationship among the output stresses

Table 2. Evaluation of the prediction accuracy of different models for damping characteristics of foam silicone rubber.

Tenfold cross-validation results Test set results

Model MSE MAE MSE MAE
CNN 0.0046 ± 0.0013 0.0452 ± 0.0055 0.0049 ± 0.0017 0.0453 ± 0.0065
BLSTM 0.0034 ± 0.0006 0.0454 ± 0.0035 0.0033 ± 0.0008 0.0458 ± 0.0046
CBLSTM 0.0026 ± 0.0005 0.0397 ± 0.0038 0.0026 ± 0.0005 0.0393 ± 0.0044

Adv. Mater. 2022, 34, 2200908 2200908 (9 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

due to the loss of the time series characteristics of the input curves as well as wide frequency damping characteristics. Of which,
strains, which causes the predicted curve to be insufficiently the curve of stress–strain was used to verify the validity of the CBLSTM
model, and the wide frequency damping characteristics was used
smooth. The results predicted using DL models without time
to demonstrate the transferability of the CBLSTM model. Data
steps correspond to pioneering work.[20,38] In contrast, although preprocessing for stress–strain is described in the Section 2.1. As for
a BLSTM network can make full use of the time series charac- wide frequency damping characteristics, the test frequency ranges from
teristics of the input strains, it only weakly extracts the material 100 to 0.05 Hz; Considering the instrument test accuracy, data points
structural features. Consequently, prediction models established with frequencies greater than 20 were deleted. Since the value of loss
based on either algorithm alone do not take full advantage of tangent is concentrated in the range of 0 to 0.3, normalization was used
the information contained in the input data, so their predic- to amplify the data values for better model training. Moreover, to ensure
the reliability of the results, all evaluations used multiple sampling (for
tion accuracy based on small experimental datasets is limited. detail please see related content in results and discussion), and the
In contrast, the proposed CBLSTM model with a hierarchical result presentation is mean ± standard deviation (SD).
learning mechanism consisting of both CNN and BLSTM units BLSTM Model: A BLSTM model is a DL tool with context-aware
can simultaneously extract the spatial structure characteristics functionality, which is particularly good at handling tasks related to
of a material sample and the sequential characteristics of the sequences. As a result, BLSTM and LSTM models have achieved
corresponding external stimulus. As a result, the CBLSTM dis- excellent performance in DNA modification detection,[40,41] additive
manufacturing process quality control,[42] brain–computer interface
plays an outstanding ability to generalize in the test set in terms
applications,[43,44] etc.
of both the MAE, MSE, and Huber evaluation indices compared In light of the complex evolution of hyperelastic materials under
to a CNN or BLSTM model alone. More importantly, the hierar- external field stimuli had an inherent logical relationship. In this work,
chical learning mechanism helps CBLSTM to understand the a BLSTM model was built to extract the serial features of data collected
task, resulting in higher accuracy. Even for a small dataset of in experiments, since its excellent temporal feature extraction capability
fewer than 100 data samples, a CBLSTM prediction model with compared with other supervised learning algorithms, such as random
forest, extreme gradient boosting, etc.
an acceptable degree of fidelity can be obtained. Thus, we can
In the BLSTM model, the input structure and the data processing
establish a high-precision model based on a small dataset for were the same as in the CNN input layer, and the output included the
the constitutive relationships of hyperelastic materials using a entire prediction of the stress process corresponding to the input strain.
dual strategy, namely, constructing low-dimensional descrip- The working mechanism of the four BLSTM units between the input and
tors of moderate accuracy and making use of the enhancement output layers is as follows:
effect of the proposed CBLSTM algorithm. Based on our trained
CBLSTM model, we can successfully predict the structure of a i. First, each BLSTM unit consists of two LSTM units. One received the
forward data flow from the input layer, and the output for each time
porous elastomer material with an ideal Gibson–Ashby com- step was co-controlled by the forget gate, input gate, and output gate:
pressive stress–strain curve that exhibits a hyperelastic stress
plateau region with a strain range of nearly 40%, for which the ( (
ht = ot × tanh ( f t × C t−1) + i t × C t ))  (2)
experimental results are in line with expectations.
The nature of complex multiscale structure-mechanical where ft = sigmoid(Wf × [ht−1,xt] + bf) is the output of the forget
response of materials is the evolution of structure and the gate, which determined the part of the cell state from the
corresponding change of performance of materials under previous time step that is discarded; it = sigmoid(Wi × [ht−1,xt] + bi)
and C t = tanh(WC × [ht−1, x t ] + bC ) jointly determined the part of
the stimulation of the external field. Due to the integration
the current input that has an impact on the long-term memory;
of the spatial structural characteristics of materials and the
sequential characteristics of external stimuli, the presented
( )
C t = ( f t × C t−1) + i t × C t is the update of the current cell state; and
ht is the output of each forward time step, which was influenced by
CBLSTM model has the ability to explore the kinetic processes the current input ot = sigmoid(Wo × [ht −1, xt] + bo) and the long-term
of materials and is expected to become a powerful tool for memory Ct.
advanced material design, which can also be used to explore the The other LSTM unit received the reverse data flow from the input
evolutionary mechanisms of structure- properties of materials layer, and its output is ht′, following the same process as above.
under complex working conditions for materials. ii. The forward ht was added to the reverse ht′ to obtain the integrated
result of the BLSTM unit, endowing the BLSTM unit with the ability to
perceive context. After the feature extraction of four BLSTM units, the
final prediction result was output by the terminal BLSTM unit.
4. Experimental Section
Ink Preparation and 3D Printing: The ink used for 3D printing was CNN Model: The CNN algorithm is an artificial neural network
made in the laboratory; the details are as follows. Silicone oil and silica algorithm that is widely used in materials science, and it has also
were fully mixed using a planetary mixer at a mass ratio of 100:35 to achieved outstanding performance in computer vision[45,46] and computer
obtain component A, and hydrogen-containing silicone oil, polysilicon, touch[47] as well as in the extraction of multiscale structure information
thixotropic agent, and Pt catalyst were fully mixed at a mass ratio in materials science.[48–50] In this article, the CNN model consisted of an
of 20:10:2:1 to prepare component B. Then, the ink was obtained by input layer, four convolutional layers, and a fully connected layer. Each of
mixing component A and component B at a mass ratio of 10:1. Porous these three components is described below.
silicone rubber samples were printed using a Musashi ImageMaster
350PC Smart robot; the number of layers, dislocation, line spacing, i. The input to the CNN model was a 50 × 6 matrix composed of
and layer height were controlled by adjusting the system parameters, the structural parameters of a porous silicone rubber sample and
and the line width was controlled by varying the needle’s speed and the a corresponding serial stimulus signal, in which the structural
atmospheric pressure. parameters were copied 50 times to match the 50-dimensional
Data Preparation: First, 483 foam silicone rubbers with different sequential stimulus signal. The input was also normalized to avoid
structures were obtained using 3D printing and test its stress–strain the exploding gradient problem.

Adv. Mater. 2022, 34, 2200908 2200908 (10 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

ii. The convolutional layers gradually extracted features from the input, Moreover, in Section 2.8, ablation were performed experiments using
from simple to complex, by means of convolution kernels: damping data of foam silicone rubber, and all comparisons were based
on the optimal hyperparameter combination of the three models.
2 2
ai , j = ω b + ∑ ∑ω m,n x i +m, j +n (3) Loss function:
n=1m=1
m
1
m∑ i
where ωm,n denotes the value in the mth row and nth column of MAE: S = y − h( x i )  (5)
the filter, xi,j represents the element in the ith row and jth column of i =1

the input matrix, ai,j is the element in the ith row and jth column of m
1
m∑ i
the feature map after convolution, and ωb is the bias parameter of MSE: S = ( y − h ( x i ))2 (6)
the filter. Considering that the convolution operation is only linear i =1

in nature, it has difficulty capturing complex nonlinear relations  1

between structure and performance; hence, the rectified linear unit m 
1  2 i
( y − h ( x i ))2 for y i − h ( x i ) ≤ δ
(ReLU) activation function was used to solve this problem: Huber: S = ∑  (7)
m i =1
 δ y i − h ( x i ) − 1 δ 2 otherwise
f ( x ) = max ( x ,0 ) (4)  2

 fter repeated iterations of alternating convolution and activation

A In these equations, S is the loss value, m is the number of data points
layers, the feature map became an abstraction that is easily in an epoch, yi is the label value, and h(xi) is the value predicted by the
understandable by a computer. model, δ is the hyperparameter of the Huber loss function. Of which,
the MSE loss function had faster convergence, but it endowed more
iii. The 50-dimensional fully connected layer that followed the last weight to outliers; the MAE loss function was more robust and generally
convolutional layer maps the distributed feature representation, after less affected by outliers; the Huber loss function was a modest strategy
flattening, to the sample label space. between the MAE and MSE, which depended on the value of δ, when
prediction error was greater than δ, it was closer to MAE, otherwise was
CBLSTM Model: The integration of CNN and LSTM was widely closer to MSE. Owing to the prediction error of the task mostly lies in
applied for visual recognition and description,[51] inferring missing the range of 0 to 1.0, δ is set to 0.5 in this comparison.
structure scores of RNA,[52] predicting the remaining useful life of
components,[53] sentiment analysis,[54] and monitoring of air quality.[55] In
this work, facing the challenge of data scarcity and complex constitutive
relations between structure and performance in material science, a Supporting Information
CBLSTM model consisting of CNN and BLSTM was manipulated to Supporting Information is available from the Wiley Online Library or
study the complex evolution of hyperelastic materials under external field from the author.
stimuli in a small-scale dataset. Compared with the abovementioned
combination of CNN and LSTM in other fields, the hierarchical learning
mechanism was tactfully used to maximize the data information and
reduce the requirement of data. This is because the hierarchical learning Acknowledgements
mechanism was beneficial to solving the problem by reducing the
difficult of the tasks.[35] The CBLSTM model was designed for common The authors acknowledge financial support from the National Natural
stimulus–response tasks in materials science experiments. Its basic Science Foundation of China (Grant No. U2030203,22075258).
structure included four convolutional layers and two BLSTM units. For
all stimulus–response tasks, the relevant data can be divided into static
sample features and serial external stimulus signals. In the CBLSTM
model, the convolutional layers were used for feature engineering to Conflict of Interest
abstract the static sample features, and the BLSTM units capture the
The authors declare no conflict of interest.
contextual relationships between prediction points. The details are as
follows:

i. The digitized sample feature matrix was first passed through the Data Availability Statement
sequence of convolutional layers and then was flattened and passed
through a fully connected layer to reduce its dimensionality to obtain The data that support the findings of this study are available from the
a 50-dimensional feature vector (for details, refer to the above corresponding author upon reasonable request.
description of the CNN algorithm).
ii. The serial stimulus signal was combined with the abovementioned
feature matrix to serve as the input to the BLSTM units. After
feature extraction by the two BLSTM units, the final prediction of Keywords
the entire response process was obtained, where each predicted
additive manufacturing, deep learning, machine learning, material
response value corresponded to a corresponding stimulus signal (for
design, porous silicone rubber
details, refer to the above description of the BLSTM algorithm).
Received: January 27, 2022
Hyperparameter Settings: Hyperparameters for each model were tuned
Revised: April 17, 2022
using ablation experiments, and hyperparameter combinations were
exhaustively evaluated using tenfold cross-validation. The searching space Published online: May 24, 2022
of CNN and BLSTM included the number of layers in range of [2, 3, 4, 5],
the learning rate in the range of [0.0001, 0.0005, 0.001], the batch size in
range of [32, 64 128]; the searching space of CBLSTM included the number
of combinations of CNN layers and BLSTM units in the range of [3+3, 3+4, [1] P. Z. Hanakata, E. D. Cubuk, D. K. Campbell, H. S. Park, Phys. Rev.
3+5, 4+2, 4+3, 4+4, 4+5, 5+3, 5+4, 5+5], the learning rate in the range of Lett. 2018, 121, 255304.
[0.0001, 0.0005, 0.001], and the batch size in the range of [32, 64 ,128]. [2] T. L. Burnett, P. J. Withers, Nat. Mater. 2019, 18, 1041.

Adv. Mater. 2022, 34, 2200908 2200908 (11 of 12) © 2022 Wiley-VCH GmbH
 15214095, 2022, 26, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202200908 by The Chinese University Of Hong, Wiley Online Library on [22/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

[3] K. H. Tu, H. J. Huang, S. Lee, W. Lee, Z. H. Sun, A. A. Katz, [31] B. Weng, Z. L. Song, R. L. Zhu, Q. Y. Yan, Q. D. Sun, C. G. Grice, Y.
C. A. Ross, Adv. Mater. 2020, 32, 2005713. F. Yan, W. J. Yin, Nat. Commun. 2020,11, 3513.
[4] S. Bonfanti, R. Guerra, F. Font-Clos, D. Rayneau-Kirkhope, [32] P. Ambure, A. Gajewicz-Skretna, M. Cordeiro, K. Roy, J. Chem. Inf.
S. Zapperi, Nat. Commun. 2020, 11, 4162. Model. 2019, 59, 4070.
[5] I. F. Ituarte, N. Boddeti, V. Hassani, M. L. Dunn, D. W. Rosen, Addit. [33] X. L. Zhang, J. F. Xu, J. Yang, L. Chen, H. B. Zhou, X. J. Liu, H. F. Li,
Manuf. 2019, 30, 100839. T. Lin, Y. B. Ying, Anal. Chim. Acta 2020, 1119, 41.
[6] M. A. Bessa, P. Glowacki, M. Houlder, Adv. Mater. 2019, 31, 1904845. [34] A. Botchkarev, Interdiscip. J. Inf., Knowl., Manage. 2019, 14, 045.
[7] X. B. Qi, G. F. Chen, Y. Li, X. Cheng, C. P. Li, Engineering 2019, 5, 721. [35] M. Eppe, C. Gumbsch, M. Kerzel, P. D. H. Nguyen, M. V. Butz,
[8] L. Velasco, J. S. Castillo, M. K. Kante, J. J. Olaya, P. Friedrich, S. Wermter, Nat. Mach. Intell. 2022, 4, 11.
H. Hahn, Adv. Mater. 2021, 33, 2102301. [36] H. Yang, J. L. Li, K. Z. Lim, C. J. Pan, T. Van Truong, Q. Wang, K. Li,
[9] T. Zhou, Z. Song, K. Sundmacher, Engineering 2019, 5, 1017. S. Li, X. Xiao, M. Ding, T. L. Chen, X. L. Liu, Q. Xie, P. V. Alvarado,
[10] G. S. Fanourgakis, K. Gkagkas, E. Tylianakis, G. E. Froudakis, J. Am. X. N. Wang, P. Y. Chen, Nat. Mach. Intell. 2022, 4, 84.
Chem. Soc. 2020, 142, 3814. [37] G. X. Gu, C. T. Chen, D. J. Richmond, M. J. Buehler, Mater. Horiz.
[11] C. Liu, E. Fujita, Y. Katsura, Y. Inada, A. Ishikawa, R. Tanura, 2018, 5, 939.
K. Kinura, R. Yoshida, Adv. Mater. 2021, 33, 2102507. [38] D. J. Roach, A. Rohskopf, C. M. Hamel, W. D. Reinholtz,
[12] D. Jha, K. Choudhary, F. Tavazza, W. K. Liao, A. Chiudhary, R. Bernstein, H. J. Qi, A. W. Cook, Addit. Manuf. 2021, 41, 101950.
C. Campbell, A. Agrawal, Nat. Commun. 2019, 10, 5316. [39] L. J. Gibson, M. F. Ashby, Cellular Solids: Structure and Properties,
[13] S. Ye, B. Li, Q. Y. Li, H. P. Zhao, X. Q. Feng, Appl. Phys. Lett. 2019, Press Syndicate of the University of Cambridge, Cambridge, UK
115, 161901. 1997.
[14] M. Bogojeski, L. Vogt-Maranto, M. E. Tuckerman, K. R. Muller, [40] S. Yadav, A. Ekbal, S. Saha, A. Kumar, P. Bhattacharyya, Knowl.-
K. Burke, Nat. Commun. 2020, 11, 5223. Based Syst. 2019, 166, 18.
[15] J. A. Ellis, L. Fiedler, G. A. Popoola, N. A. Modine, J. A. Stephens, [41] Q. Liu, L. Fang, G. L. Yu, D. P. Wang, C. L. Xiao, K. Wang, Nat.
A. P. Thompson, A. Cangi, S. Rajamanickam, Phys. Rev. B 2021, 104, Commun. 2019, 10, 2449.
035120. [42] J. Zhang, P. Wang, R. X. Gao, Comput. Ind. 2019, 107, 11.
[16] V. L. Deringer, M. A. Caro, G. Csanyi, Adv. Mater. 2019, 31, 1902765. [43] L. P. B, J. S, J. K. Pragatheeswaran, S. D, P. N, Biomed. Signal Pro-
[17] Z. Yang, C. H. Yu, M. J. Buehler, Sci. Adv. 2021,7, 7416. cess. Control 2021, 66, 102501.
[18] Y. Zhang, C. Ling, npj Comput. Mater. 2018, 4, 71. [44] G. K. Anumanchipalli, J. Chartier, E. F. Chang, Nature 2019, 568,
[19] C. C. Gao, X. Min, M. H. Fang, T. Y. Tao, X. H. Zheng, Y. A. Liu, 493.
X. W. Wu, Z. H. Huang, Adv. Funct. Mater. 2021, 32, 2108044. [45] X. Wang, S. F. Zheng, R. Yang, A. H. Zheng, Z. Chen, J. Tang,
[20] C. Yang, Y. Kim, S. Ryu, G. X. Gu, Mater. Des. 2020, 189, 108509. B. Luo, Pattern Recognit. 2022, 121, 108220.
[21] M. Mooney, J. Appl. Phys. 1940, 11, 582. [46] J. Huertas-Tato, A. Martín, J. Fierrez, D. Camacho, Inf. Fusion 2022,
[22] R. W. Ogden, J. Mech. Phys. Solids 1976, 24, 323. 79, 174.
[23] J. P. Wilber, J. C. Criscione, Int. J. Solids Struct. 2005, 42, 1547. [47] S. Sundaram, P. Kellnhofer, Y. Z. Li, J. Y. Zhu, A. Torralba,
[24] M. Destrade, G. Saccomandi, I. Sgura, Proc. R. Soc., Sect. A 2017, W. Matusik, Nature 2019, 569, 698.
473, 20160811. [48] E. H. Cho, L. C. Lin, J. Phys. Chem. Lett. 2021, 12, 2279.
[25] T. H. Weisgraber, T. Metz, C. M. Spadaccini, E. B. Duoss, W. Small, [49] S. Xu, A. S. Mcleod, X. Z. Chen, D. J. Rizzo, B. S. Jessen,
J. M. Lenhardt, R. S. Maxwell, T. S. Wilson, J. Mater. Res. 2018, 33, Z. H. Yao, Z. C. Wang, Z. Y. Sun, S. Shabani, ACS Nano 2021, 15,
309. 18182.
[26] C. Wang, X. P. Tan, S. B. Tor, C. S. Lim, Addit. Manuf. 2020, 36, [50] J. Na, G. Kim, S.-H. Kang, S.-J. Kim, S. Lee, Acta Mater. 2021, 214,
101538. 116987.
[27] S. Muller, C. Sauter, R. Shunmugasundaram, N. Wenzlez, [51] J. Donahue, A. H. Hendricks, M. Rohrbach, S. Venugopalan,
V. De Andrade, F. De Carlo, E. Konukoglu, V. Wood, Nat. Commun. S. Guadarrama, K. Saenko, T. Darrell, IEEE Trans. Pattern. Anal.
2021, 12, 6205. Mach. Intell. 2017, 39, 677.
[28] A. Géron, Hands-on Machine Learning with Scikit-Learn and Tensor- [52] J. Gong, K. Xu, Z. Y. Ma, Z. J. Lu, Q. F. C. Zhang, Nat. Mach. Intell.
Flow: Concepts, Tools, and Techniques to Build Intelligent Systems, 2021, 3, 995.
O’Reilly Media, Boston, MA, USA 2019. [53] M. Ma, Z. Mao, IEEE Trans. Ind. Inf. 2021, 17, 1658.
[29] A. D. Auguste, J. W. Ward, J. O. Hardin, B. A. Kowalski, T. C. Guin, [54] M. E. Basiri, S. Nemati, M. Abdar, E. Cambria, U. R. Acharya, Future
D. Berrigan, T. J. White, Adv. Mater. 2018, 30, 1802438. Gener. Comput. Syst. 2021, 115, 279.
[30] M. L. Ye, L. Gao, H. Li, Mater. Des. 2020, 192, 108751. [55] T. Y. Li, M. Hua, X. Wu, IEEE Access 2020, 8, 26933.

Adv. Mater. 2022, 34, 2200908 2200908 (12 of 12) © 2022 Wiley-VCH GmbH