Research Article

 www.advmat.de

Machine-Learning Assisted Electronic Skins Capable of

Proprioception and Exteroception in Soft Robotics
Sheng Shu, Ziming Wang, Pengfei Chen, Junwen Zhong, Wei Tang,*
and Zhong Lin Wang*

in e-skin is mainly concerned with

Inspired by natural biological systems, soft robots have recently been devel- perceiving external stimuli (exteroception),
oped, showing tremendous potential in real-world applications because of and only a few studies have focused on
their intrinsic softness and deformability. The confluence of electronic skins real-time monitoring of the shape of soft
and machine learning is extensively studied to create effective biomimetic robots (proprioception).[2] Body compliance
and shape adaption enable soft robots to
robotic systems. Based on a differential piezoelectric matrix, this study
perform unique functions in confined
presents a shape-sensing electronic skin (SSES) that can recognize surface spaces,[3] conformable interactions,[4] and
conformations with minimal interference from pressing, stretching, or other walking over rough terrains for delivery.[5]
surrounding stimuli. It is then integrated with soft robots to reconstruct their Koivikko et al. reported a curvature sensor
shape during movement, serving as a proprioceptive sense. Additionally, the based on screen-printed silver conductors
robot can utilize machine learning to identify various terrains, demonstrating to monitor the operating status of soft
grippers.[6] Li et al. integrated a distributed
exteroception and pointing toward more advanced autonomous robots
curvature sensor array with a soft manipu-
capable of performing real-world tasks in challenging environments. lator to reconstruct its shape.[7] Despite
the accuracy of shape sensing in these
studies, additional sensors that can per-
1. Introduction ceive external stimuli are still necessary for these soft robots in
practical applications. Therefore, an e-skin that can reconstruct
Soft robots are extensively studied as promising candidates the shape and simultaneously react to the surrounding environ-
to bridge the gap between machines and biological organ- ment is highly desired for diversifying the capabilities of soft
isms owing to their intrinsic softness.[1] Soft robots are usually robots with a simplified structure.[8]
equipped with various sensors that can provide target infor- In addition to e-skins, progress in machine learning (ML)
mation from the surrounding environment to enable their techniques has provided abundant opportunities for developing
effective performance. As a comprehensive platform that can more advanced soft robots. The combination of e-skin and
accommodate multimodal sensors or even sensor arrays, e-skin ML techniques allows soft robots to perform a given task in a
technology presents a promising strategy for further aug- more intelligent manner, and the results are often beyond our
menting the ability of soft robots. However, current progress expectations. For instance, Tao et al. employed convolutional
neural networks to learn the deformations of a soft gripper
for real-time object classification.[9] Meerbeek et al. imple-
S. Shu, Z. Wang, P. Chen, W. Tang, Z. L. Wang mented a neuro-inspired hyperdimensional computing algo-
CAS Center for Excellence in Nanoscience rithm for the real-time gesture estimation of soft robots.[10]
Beijing Key Laboratory of Micro-nano Energy and Sensor Schlagenhauf et al. exploited reinforcement learning to achieve
Beijing Institute of Nanoenergy and Nanosystems
Chinese Academy of Sciences dynamic manipulation of soft robot hands.[11] The outstanding
Beijing 100083, P. R. China performance of ML in exploiting datasets is suitable for
E-mail: [email protected]; [email protected] analyzing sensing signals, especially in sensor arrays, which are
S. Shu, Z. Wang, P. Chen, W. Tang usually characterized by the generation of numerous data in a
School of Nanoscience and Technology small period. Even small differences in signals that are often
University of Chinese Academy of Sciences
neglected by humans can be precisely discerned by ML. There-
Beijing 100049, P. R. China
fore, the combination of machine learning and shape-sensing
J. Zhong
Department of Electromechanical Engineering e-skins is expected to enable the development of soft robots
Centre for Artificial Intelligence and Robotics University of Macau with advanced artificial intelligence, eventually resulting in arti-
Macao 999078, China ficial systems that can rival the abilities of biological organisms.
Z. L. Wang This study proposes a shape-sensing electronic skin (SSES)
Georgia Institute of Technology based on a differential piezoelectric matrix that can arm soft
Atlanta, GA 30332–0245, USA
robots with both proprioception and exteroception abilities by
The ORCID identification number(s) for the author(s) of this article
integrating machine learning techniques. Each building block
can be found under https://doi.org/10.1002/adma.202211385.
of the SSES contains a conductive fabric sandwiched between
DOI: 10.1002/adma.202211385 two polyvinylidene fluoride (PVDF) membranes. The outputs

Adv. Mater. 2023, 35, 2211385 2211385 (1 of 9) © 2023 Wiley-VCH GmbH

 15214095, 2023, 18, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202211385 by Northeastern University, Wiley Online Library on [15/11/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

of these two PVDF membranes are identical when pressing This approach is effective even in more complex situations,
or stretching is inflicted. Therefore, these interferences can be where bending is inflicted during stretching or pressing. As
easily discerned and minimized using a differential strategy, shown in Figure 1f, the outputs of PVDF A and B exhibit the
resulting in satisfactory robustness for sensing bending. same tendency when a stretching force is applied, and a pair of
Proprioception is enabled for soft robots, as demonstrated by reversed peaks is generated when bending is inflicted. Similar
the reconstruction of surface conformations during crawling profiles are observed in the case of pre-pressing, as shown in
using data collected by the SSES. The precision reaches Figure S3 (Supporting Information). Although the stretching/
0.0025°, and the response time is 36 ms, which is compatible pressing force causes a certain shift in the neutral layer, the
with the majority of the application scenarios of soft robots.[12] differential values remain nearly the same, as illustrated in
Furthermore, based on the acquired variations in shape, this Figure S4 (Supporting Information). A theoretical analysis of
study establishes a dynamic model, which can further deduce the output performance is presented in Note S2 (Supporting
the crawling distance, gesture, and velocity for modulation and Information). Another salient merit of the sandwiched struc-
calibration. Exteroceptive ability is realized when a machine- ture is the apparent improvement in the magnitude of its sig-
learning strategy is introduced to analyze the SSES-acquired nals. As demonstrated in Figure S5 (Supporting Information), a
data. For instance, soft robots can identify various terrains 14-time higher peak-to-peak value is delivered when compared
or barriers, thus exhibiting advanced artificial intelligence. to that of a single-layer structure (7 V vs 0.5 V). This enhance-
We expect that the SSES and its combination with machine ment may be because the conductive fabric serves as an approx-
learning techniques can provide a general approach for imate translational layer (see Note S4, Supporting Information).
augmenting and diversifying the ability of soft robots to thrive The linearity of the output signals is examined in Figure 1g,
in real-world applications. indicating a linear relationship with R2 = 0.9953 for angles
ranging from 24.6° to 172.8°. Meanwhile, the sensitivity is
calculated to be 0.146 V deg−1,[14] laying the foundation for high-
2. Results and Discussion resolution applications of SSES in robotics and healthcare
fields.[15] Figure S6 (Supporting Information) shows that the
2.1. Configuration and Characterization of SSES limiting resolution of the SSES can reach 0.0025°, and it can be
further enhanced by employing a more sensitive signal acquisi-
Figure 1a presents an explosive illustration of the sand- tion system or developing materials with higher piezoelectric
wiched structure of the SSES and a 3D schematic of its properties. The SSES also demonstrates a fast response feature
integration with a typical soft-crawl robot. The enlarged (36 ms), long cycle stability, and satisfactory robustness to
inset depicts the layered structure of the building blocks of other common influences, as plotted in Figure S7 (Supporting
the SSES, which exhibits mirror symmetry with respect to Information). These merits of the SSES ensure its broad appli-
the layer of the conductive fabric (polyester cloth, 110 µm cations in soft robotics.
in thickness). This layer also serves as a common ground
electrode, and two piezoelectric sensing units are placed
on both sides. Each unit consists of a PVDF (thickness of 2.2. SSES Fabrication and Shape Reconstruction
28 µm, polarized along the thickness) film sandwiched
between two Ag electrodes, and the entire device is pack- The perception of surface conformations plays an essential
aged by the outermost silicon layers. The robustness of the role in acquiring the real-time operating status of soft robots,
SSES is elaborated using a cross-sectional view of a building thereby providing opportunities for establishing reliable con-
block, as shown in Figure 1b. The conductive fabric is signifi- trol systems to realize realistic applications. Figure 2a and
cantly thicker than the adjacent PVDF films; therefore, the Figure S8 (Supporting Information) illustrate a facile strategy
neutral layer, a non-strained layer upon bending, is located for building an SSES-integrated soft robot. PVDF is selected to
within the conductive fabric, as indicated by the red dashed line. fabricate the SSES because it can directly convert mechanical
Hence, strains produced on the two PVDF films (PVDF A and agitation into electrical output. The main body of the SSES
B) are expected to be opposite when the building block is bent, was fabricated by laser cutting and magnetron sputtering, and
and identical when subjected to pressing or stretching; these the corresponding electrical connections were processed using
are the two common influences during bending sensing.[13] mature printed circuit board (PCB) technology. The ultrathin
This assumption is verified by the measured outputs thickness (<200 µm) and flexibility enable the conformal and
shown in Figure 1c–e; details of the measurement are available compact assembly of SSES in soft robots. Further, serpentine
in Note S1 (Supporting Information). The building block is electric wires in addition to island-structured PVDF films form
bent to form a convex curvature with an angle of 150°, and an “island bridge”, an electrically conductive network that
the acquired waveforms shown in Figure 1c are similar in provides a certain stretchability for SSES.
magnitude but opposite in polarity; this is consistent with the Figure 2b shows a scheme for reconstructing a curved
simulated results shown in Figure S1 (Supporting Informa- surface using the SSES-collected data. Specifically, an SSES
tion). However, the outputs of the two PVDF films are almost array was first integrated on the target surface, and curvatures
identical when the building block is stretched or pressed, at given spots were detected by the building blocks of the array.
as shown in Figure 1d,e, and Figure S2 (Supporting Infor- All these local curvatures were analyzed and processed using
mation). This feature inspired us to employ a differential computers to restore the conformation. Here, we consider a
approach to minimize interferences during sensing. typical soft crawl robot, 170 mm in length and 80 mm in width,

Adv. Mater. 2023, 35, 2211385 2211385 (2 of 9) © 2023 Wiley-VCH GmbH

 15214095, 2023, 18, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202211385 by Northeastern University, Wiley Online Library on [15/11/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Figure 1. Configuration and characterization of SSES. a) Illustration of the SSES-mediated soft robot assembly. b) Illustration of a sensing block under
bending. c) Open-circuit voltage output of the two units when bending under an angle of 150°. d,e) Output performance under stretching (0.2 mm)
(d) and pressing (6.6 N) (e). f) Voltage output under bending simultaneously with stretching. g) Linear relationship between measured angles and
voltage output, ranging from 24.6° to 172.8°.

as an example. The size of each block, a key parameter of the a close similarity was observed between the deformed soft
SSES, was optimized to 2.5 mm to achieve a balance between robot and the corresponding reconstructed shape, as shown in
signal magnitude and spatial density (see Figure S9a, Sup- Figure 2c and Movie S1 (Supporting Information).
porting Information). Thus, 28 blocks in total were integrated In addition to the recovery of static surfaces, we also demon-
into the SSES to reconstruct its overall shape; details can be strated the reconstruction of dynamic surfaces using an SSES.
found in Figure S9b (Supporting Information). As each block An Arduino-based gas control program was developed to enable
consists of two sensing units, a LabVIEW-based multichannel the programmable bending of the soft robot’s legs for crawling.
synchronous data-acquisition system was developed to simul- Real-time bending angles at the four specified spots (Figure S9e,
taneously acquire 56 sets of sensing signals. These 56 sets of Supporting Information) on the feet could be acquired by the
voltage signals were converted to corresponding curvatures SSES, as shown in Figure 2d. The crawling process and the
and displayed on a monitor using software developed based on reconstructed version are demonstrated in Movie S2 (Sup-
Unity (Figure S10, Supporting Information). With this strategy, porting Information). Note S5 (Supporting Information) further

Adv. Mater. 2023, 35, 2211385 2211385 (3 of 9) © 2023 Wiley-VCH GmbH

 15214095, 2023, 18, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202211385 by Northeastern University, Wiley Online Library on [15/11/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Figure 2. SSES fabrication and shape reconstruction. a) Craft flow diagram of the SSES. The laser cutting, magnetron sputtering, and organic soldera-
bility preservatives (OSP) technology facilitate the batch fabrication of SSES, and the flexible printed circuit boards increase the reliability of sensory
array interconnection. b) Schematic of the reconstruction based on the sensor array. c) Computational reconstruction of the soft robot’s bending. d)
Measured angle curves of four sensing blocks during typical crawling motions of a soft robot. Different colors are highlighted to mark phases.

establishes a kinetic model based on the SSES-acquired signals, their physical interactions with the ambient environment.
and various dynamic parameters, such as crawling velocity and This encouraged us to perceive the surrounding environment
distance, can be deduced. Hence, not only the perception of through the data collected by SSES. Figure 3a shows a sketch
surface conformation but also the awareness of dynamic fea- of a soft robot crawling over several representative terrains.
tures is realized for soft robots owing to SSES integration. These terrains can be discerned by the surface roughness,
which is normally related to the friction coefficient. Figure S11
(Supporting Information) depicts the measured normal force
2.3. Dynamic Behavior of the Robot on and friction coefficients when the soft robot crawls on these
Various Terrain Surfaces Revealed by SSES terrains; further details can be found in Note S6 (Supporting
Information).
The variation in the bending status of the soft robot’s limbs not The flow diagram in Figure 3b illustrates the principle of
only introduces a change in conformation but also indicates identifying different terrains via the statistical analysis of the

Adv. Mater. 2023, 35, 2211385 2211385 (4 of 9) © 2023 Wiley-VCH GmbH

 15214095, 2023, 18, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202211385 by Northeastern University, Wiley Online Library on [15/11/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Figure 3. Dynamic moving behavior analysis of a soft robot crawling on various terrain surfaces. a) Schematic of the soft robot crawling over various
terrains. The enlarged views are optical images of different terrain surfaces. b) Flow diagram for verifying the feasibility of sensing ambient environment
based on SSES-acquired data. c) Outlines of the bottom surface of the soft robot when it crawls over various terrains. d) The mean maximum bending
angle at the spot of the robot’s left forefoot, measured by SSES over 10 trials. Error bar denotes standard error of the mean. e) Average crawling speed
of the soft robot when it crawls over different terrains.

SSES-acquired data. First, the SSES-integrated soft robot was obtained on A4 paper, whereas the speed on soft rubber or
made to crawl over five different terrains (Figure S12, Sup- sandpaper is considerably slower. These detailed differences
porting Information), and its gestures during crawling were during the crawling process enable us to develop a rapid and
simultaneously captured by a high-frame camera. Subse- reliable strategy based on the SSES to identify different terrains.
quently, the SSES-collected signals were statistically analyzed
and compared with the differences in the optical images.
Although the five outlines of soft robots during crawling exhibit 2.4. Environment Awareness of SSES-Integrated
an overall similar shape, as shown in Figure 3c and Figure S13 Soft Crawling Robot
(Supporting Information), significantly small angular differ-
ences in the four limbs existed during crawling, as revealed by Analysis based on reconstructed conformation usually relies
the frame-by-frame optical images, as shown in Figure S14a on complicated image processing techniques, and the interac-
(Supporting Information), and the detected data from the tion between soft robots and their surrounding environment
attached commercial inclination sensors, as shown in Figure S14b can be statistically reflected by SSES-measured data. Thus,
(Supporting Information). This can be explained by the fact we attempted to employ machine learning (ML) technology to
that the larger the surface roughness, the higher the friction further exploit SSES-acquired data to realize environmental
that resists the bending deformation of the robot, and a smaller awareness. To establish an optimal ML training model, efforts
maximum bending angle (MBA) is expected (Figure S14c, have been made to examine the appropriate feature extrac-
Supporting Information). tion method and optimized algorithm (Figures S15–S17 and
To verify this assumption, the maximum bending angles Table S1, Supporting Information). The node-oriented decision
during crawling were extracted from the SSES-acquired data tree algorithm was preferred because the signals were inter-
after the statistical analysis. As shown in Figure 3d, the A4 leaved, and this method enabled us to build multiple decision
paper, which had the lowest roughness among the five terrains, trees and consider the average to fully exploit the advantages of
resulted in the highest MBA of the soft robot. In contrast, sand- various models. Moreover, the fact that only sensing units at cer-
paper or soft rubber, which demonstrated larger roughness, tain specific spots exhibit pronounced responses is expected to
resulted in a smaller MBA during crawling. Furthermore, this maximize the benefits of node classification.[16] The processing
phenomenon is supported by the differences in crawling speed, procedure is shown in Figure 4a. The features were extracted
as shown in Figure 3e, in which the fastest crawling speed is before the analysis. Voltage signals instead of derived angles

Adv. Mater. 2023, 35, 2211385 2211385 (5 of 9) © 2023 Wiley-VCH GmbH

 15214095, 2023, 18, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202211385 by Northeastern University, Wiley Online Library on [15/11/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Figure 4. Environment awareness of SSES-integrated soft crawling robot. a) Flow diagram of the machine learning process. The bagged tree algorithm
is chosen. b) Conceptual illustration of the soft robot for environment recognition. c,d) Optical images of the experiment scenarios. e) Terrain surface
training with five-fold cross-validation (five different trained pavements with varying roughness). Classification confusion matrix with an overall accuracy
of 98.0%. White text values are percentages of correct predictions, and cyan text values are percentages of correct predictions. The color bar represents
the amount of the predicted proportion. f) Identification of five new terrains that have not been trained to demonstrate the generalization ability of
the developed model. The new A4 paper is printed with ink; the new printed pattern is printed with smaller embossment and different materials; the
new metal is slightly polished; the new sandpaper’s grit size is 20% smaller (from 80# mesh to 100# mesh); the new soft rubber is contaminated with
dust. g) Mixed recognition of all the aforementioned environments, with an overall accuracy of 98.2%.

Adv. Mater. 2023, 35, 2211385 2211385 (6 of 9) © 2023 Wiley-VCH GmbH

 15214095, 2023, 18, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202211385 by Northeastern University, Wiley Online Library on [15/11/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

were employed as raw data because more information was con- resolution was 0.0025°, and the external interferences were
tained in the directly measured signals, which is beneficial for minimized owing to the differential piezoelectric matrix with
improving the accuracy of ML analysis. The standard deviation, a coplanar mirror sandwiched structure. This study proposed
median, and mean of the raw data were extracted as features. two processing principles regarding SSES-measured data.
After classification, these features were packaged into several Specifically, we first developed a general strategy for recon-
decision tree algorithms for training, and the results of the five- structing the surface conformations of soft robots according
fold cross-validation and new datasets test were obtained. to SSES-obtained data, thus enabling the proprioceptive abili-
Figure 4b depicts the expected conceptual illustration of an ties of soft robots. In addition, we introduced machine learning
SSES-integrated soft robot for environment recognition after (ML) technology to analyze SSES-generated data for identi-
its confluence with the ML technique. Figure S18 (Supporting fying different terrains, loading weights, and obstacles, thus
Information) shows the classification results of different grit endowing soft robots with exteroceptive abilities and primary
sizes of sandpapers, indicating that roughness is the only indi- intelligence. SSES and its integration with ML techniques lay
cator of identification. Figure 4c shows that five terrains with the foundation for soft robots to effectively acquire their own
different roughness values can be identified by a series of status and interact with the environment in a sophisticated
statistical data combined with ML with a high accuracy of 98.0%, manner. We expect that this paradigm will inspire more inno-
indicating the feasibility of the developed model. To examine vations that allow soft robots to swiftly cope with challenges in
the generalization ability of this model, the modified terrains real-world applications.
(Figure 4d; Figure S19, Supporting Information) were added to
the prediction set: the new A4 paper is printed with ink, which
actually decreases the surface roughness owing to the presence 4. Experimental Section
of carbon; the printed pattern is modified with smaller patterns
and different materials; the new metal is slightly polished; the Shape-Sensing Electronic Skin (SSES) Fabrication: The SSES was
grit size of the new sandpaper is reduced from 80# mesh to 100# composed of two pieces of piezoelectric film (polyvinylidene difluoride,
PVDF, 28 µm in thickness, MEAS) with silver electrodes (Epoxy
mesh; the new soft rubber is covered with dust. The integration conductive adhesive, 6 µm in thickness, LX-30 Yinxi) on both sides,
of ML analysis with the SSES-armed robot results in a satisfac- a thin flexible conductive fabric (Polyester cloth, 110 µm in thickness,
tory identification accuracy of 96.7% (Figure 4e). Ebenda 8869) with glue on both sides, a pair of flexible printed board
Similar to the roughness of terrains, different weight loads, (100 µm in thickness), and a thin layer of silicone (≈1 mm in thickness,
inclinations, and obstacles induce corresponding variations in Eco-flex 0030, Smooth-On) encapsulation. The fabrication of the sensor
sensing signals, implying that identifying these parameters pair began by preparing two films of PVDF with silk screen printed
silver glue on one side. Then the two PVDF films (the sides with silver
using ML is feasible. The training scenarios and real-time
glue) were symmetrically pasted on both sides of the double-sided
variations in the signals are shown in Figure 4f and Movie S3 adhesive conductive fabric (110 µm in thickness, Ben Yida). The size of
(Supporting Information). Here, fivefold cross-validation was the conductive fabric was larger than the PVDF film, which facilitated
conducted, using four sets of data for training and one set for subsequent fixation. Then flatly affix the raw sandwich structure on a
testing. Further details can be found in Experimental Section. Plexiglas substrate (2 mm in thickness, Creromem) of appropriate size
After training, a road with a slope of 10° in either the uphill that was cleaned with alcohol, removed air bubbles. After that, three
steps of laser cutting were required. First, the upper layer of PVDF was
or downhill directions could be recognized with an accuracy
cut using the laser-cutting machine (PLS6. 75) with appropriate focal
of 97.8% (Figure S20a, Supporting Information), the positions length (2 mm), power (18%), and cutting speed (70%), which required
of barriers on the road could be discerned (Figure S20b, Sup- dot positioning through the designed pattern. Second, flip to the other
porting Information), and the weights of the carried items side was flipped and the previous dots were used to position the new
could be recognized (Figure S20c, Supporting Information). face, and the laser parameter was the same as last step. Third, having
Furthermore, to explore the comprehensive perception of prepared the sensing matrix, the same method was used to define
boundaries (focal length (2 mm), power (80%), and cutting speed
the environment, we mixed all 20 terrain/barrier/load condi-
(50%)). As can be seen, accurate positioning is very important, especially
tions in the identification test (Figure S21 and Table S2, Sup- for the first time, which usually requires multiple points of repeated
porting Information). The results show an overall accuracy of positioning. Afterward, a single fluorinated polyethylene terephthalate
98.2%, as shown in Figure 4g. It is noteworthy that although (PET) film (50 µm in thickness) mask, whose pattern was slightly smaller
the employed decision tree algorithm is credited with a cer- than that of the PVDF sheets and defined by laser cutting, was placed on
tain degree of generalization, the accuracy is unsatisfactory the patterned PVDF film, fixing by previous adhesive conductive fabric.
when this model is confronted with some out-of-distribution The whole surface was kept clean and electroplate it with magnetron
sputtering apparatus (Cr/Ag, 10 nm/100 nm in thickness). Finally, in an
problems, especially for those involving multiple sensations alcohol environment, mechanically peeled off the defined structure from
simultaneously. This deficiency is attributed to the vulnerability the Plexiglas substrate. Here, PVDF is superior to other piezoelectric
of the decision tree algorithm. The use of deep neural networks material because PVDF is a polymer with lower thermal conductivity
or increasing data richness to obtain more information on the than other piezoelectric materials, which increases the precision of laser
environment is expected to further improve this model. processing by allowing heat to be concentrated more easily.[17]
As for the connecting wire, a soft, thin polyimide film covered with
copper (80 µm in thickness in total) is employed as a substrate, and
the serpentine patterned design enables its flexibility and stretchability,
3. Conclusion meeting the tiny displacement when it bends. On the polyimide
substrate, copper wires with width of 0.15 mm and spacing of 0.1 mm
A shape-sensing electronic skin (SSES) was developed to sense are etched as the electrodes (Figure S2, Supporting Information). The
the dynamic status of soft robots during operation. The limiting width and spacing of the electrode lines on the inside are consistent,

Adv. Mater. 2023, 35, 2211385 2211385 (7 of 9) © 2023 Wiley-VCH GmbH

 15214095, 2023, 18, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202211385 by Northeastern University, Wiley Online Library on [15/11/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

so that the interconnection electromagnetic interference is minimized. input lists to be more comprehensive, two more features, the mean
The resistance difference caused by the change in the length of the outer and median, were extracted from the raw data, and learning the three
side is also consistent by appropriately modifying the line width and kinds of feature data simultaneously could increase the final recognition
height. Organic Solderability Preservatives (OSP) technology was used accuracy by 1.5% (Figure 4g). After training and prediction, the results
to selectively overlay printed circuit board, so that the signal could be were further imported into origin for normalization, visualization, etc.
well exported. The flexible electrode board is only 0.1 mm thick and can During the experiment for terrains recognition, all these tested surfaces
be well attached to the conductive cloth substrate without affecting its were tightly adhered to a desktop while no visible move, wrinkle, or
bending. Besides, the areas where the board contacts the sensors are bulge would be produced when the soft robot was crawling on. Besides,
only 0.45 mm2, which is much smaller than the area of the sensors the thin thickness of these substrates (0.1-3 mm) could further reduce
themselves, so as to further reduce the influence of the electrode wires the influence from compliance.31 Hence, interferences from surface
on bending and ensures the output consistency. compliance are supposed to be very limited in surface roughness
The packaging of the SSES was carried out in the printed mold. experiments. As for the adhesion resulted from chemical properties,
The 1:1 (Eco-flex A: B) mixed eco-flex was spread on the bottom of the silicone rubbers, such as eco-flex and PDMS, are mainly consists of
mold, and then it was put into the sensor array, and then the mold was SiO2 which is chemically inert to a majority of chemicals in ambient
filled with eco-flex and the bubble was removed with pointed tweezers, conditions. And no chemical interactions have been reported among
waiting for 2 h at room temperature, and it was mechanically peeled off silicon rubber and five surfaces employed in our experiment. Thus, the
from the mold. influence caused by chemical adhesion could be reasonably excluded.
Soft Gripper and Robot Fabrication: The soft four-jaw gripper is made The reconstruction of soft crawling robots is validated by
of Eco-flex (0030, Smooth-on), nylon wires (Fashion Fish), rubber hose reconstruction system based on Unity and data acquisition system
(2 mm in inner diameter, Takuhiro), and silicone adhesive (Smooth-on based on LabVIEW. We designed and encode a new LabVIEW program
Sil-Poxy). Eco-flex A and B was mixed in a same ratio, stirred evenly, and that allows voltage information to be read directly from data acquisition
placed in a 3D-printed mold. A vacuum pump (Pritchett AP-01P) was cards (National Instrument 6356), and through the collaboration of
employed to remove bubbles. After 2 h of resting at room temperature, multiple collection cards, we have achieved data acquisition of arrays.
gripper was removed from the mold and nylon wire was wrapped The collected data is read by the Unity program (Version 2019.4.25f1c1),
around it. Then a thin layer of eco-flex was applied to the griper wrapped and after simple data processing such as filtering, differential, and
in nylon wire. The preparation method of the soft crawl robot body is proportional conversion, the voltage signal is converted into an angle
basically the same as that of gripper, yet nylon wire’s is unnecessary. The value, which gives dynamic changes to the joints of the model in Unity.
fabrication of the software robot has already been reported, and here It is worth noting that the joint bending system is based on a relative
is a focus on the process of bonding with the sensor array. Similarly, a coordinate system, and the movement of the limbs near the main body
thin layer of eco-flex is applied to the bottom of the SSES, then the soft is implicated.
body is placed in the appropriate position, the edges are further sealed Statistical Analysis: All raw statistical data were collected by NI
with eco-flex, and settled to cure. Once cured, the rubber hoses, which acquisition cards in conjunction with the LabView program, as described
use silicone adhesive to seal with the soft body, are used for connecting in Experimental Section 4.3. In the feature extraction process, the mean,
Arduino-controlled Solenoid valves and pumps. standard deviation, median, and maximum values are obtained by
Data Acquisition and Processing: The voltage output collection of the entering the original data into Origin and obtaining the corresponding
SSES was accomplished mainly by voltage preamplifier (Keithley 6517 statistics data with its statistical module. In the image recognition
System Electrometer). And only in the data acquisition process for ML feature extraction procedure, Origin is used to produce the waveform
is the multi-channel data acquisition system (National Instrument 4300, graph, and then the MATLAB program extracts the image gray difference
internal resistance is ≈140 MΩ) applied. Here, the customized interactive statistics, including mean, contrast, and image entropy. Additionally,
control software programed by LabVIEW completes the analog-to-digital Origin is used in the heat map to normalize the data from multiple
conversion and records the data. When studying the characteristics of channels in order to enable the observation of the rules between
a single pair, the signal is generated by the patch in series between the different types of data. Other simple data processing tasks, such as
linear motor (Linmot, E1100). Also, an ultra-high precision manual linear differentiation, angle conversion, filtering, and so on, are handled by
slide (NFP-x462, Zolix) is employed for accuracy calibration. The whole the Origin toolkit. Moreover, all error bars are presented in mean ± SD,
measurement process of the system was fixed on the optical platform. and 10 points from each set of data are taken to calculate the mean and
Moreover, the mechanical measurement of the sensor pair was done standard deviation. When conducting the recognition with the gripper,
by a high-precision commercial force sensor (Oruda, AT8301), and the each object is grasped ≈50 times. And except for objects such as blades
mechanical data were directly read and stored by its matching software. and bananas, which can only be grasped from a fixed direction, the
The soft crawling robot dynamical system is composed of four direction of each grasp is random and changeable, so that the total data
parts: pump (Jie Dong SP-36-9), solenoid valve group (Airtac 025-06), set (containing 43 500 sets) is obtained. In environment recognizing,
controller (Arduino), and amplifier. The pump provides stable air the total data (containing 25 125 sets) are acquired through crawling
pressure and leads to the solenoid valve group via a rubber hose, the on objects 30 times (Among them, the zero-weight and flat-road group
sequence switch of the solenoid valve group is controlled by an external are the same data). The training and testing datasets for objects and
amplifier of the Arduino controller, and the outlet of the solenoid valve is environment recognition are randomly selected from the total dataset.
connected to the soft robot porosity. The airtightness of the rubber hose
is guaranteed by a silicone adhesive.
In the recognition with ML, the data were directly collected and
recorded by the multi-channel data acquisition system and the Supporting Information
corresponding LabVIEW program, and further processed by Origin to
Supporting Information is available from the Wiley Online Library or
reduce data dimensions and extract the characteristic data. The learning
from the author.
algorithm was provided by the MATLAB toolbox. The standard deviation
of the 50 signals of the gripper and the 56 signals of the crawling robot
as a set of data were designed. More specifically, the multi-channel data
were imported into Origin, and Origin’s mathematical statistics were
used to calculate a series of parameters such as standard deviation,
Acknowledgements
variance, and mean, and the feasibility of these data after dimensionality The research was supported by the National Key R&D Project from the
reduction was compared to the corresponding algorithm. Meanwhile, Minister of Science and Technology (2021YFA1201604, 2021YFA1201601,
fivefold cross-validation was conducted here. To make the data of and 2021YFA1201603); Youth Innovation Promotion Association CAS,

Adv. Mater. 2023, 35, 2211385 2211385 (8 of 9) © 2023 Wiley-VCH GmbH

 15214095, 2023, 18, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/adma.202211385 by Northeastern University, Wiley Online Library on [15/11/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advmat.de

Beijing Municipal Science & Technology Commission (Z171100000317001, Acad. Sci. USA 2011, 108, 20400; b) Y. Wu, J. K. Yim, J. Liang, Z. Shao,
Z171100002017017, and Y3993113DF); and National Natural Science M. Qi, J. Zhong, Z. Luo, X. Yan, M. Zhang, X. Wang, R. S. Fearing,
Foundation of China (grant nos. 51605033, 51432005, 5151101243, and R. J. Full, L. Lin, Sci. Robot. 2019, 4, aax1594; c) X. Yang, L. Chang,
51561145021). N. O. Pérez-Arancibia, Sci. Robot. 2020, 5, aba0015.
[4] a) H. Lu, M. Zhang, Y. Yang, Q. Huang, T. Fukuda, Z. Wang, Y. Shen,
Nat. Commun. 2018, 9, 3944; b) A. Rafsanjani, Y. Zhang, B. Liu,
S. M. Rubinstein, K. Bertoldi, Sci. Robot. 2018, 3, aar7555; c) G. Gu,
Conflict of Interest J. Zou, R. Zhao, X. Zhao, X. Zhu, Sci. Robot. 2018, 3, aat2874;
The authors declare no conflict of interest. d) T. Jin, Z. Sun, L. Li, Q. Zhang, M. Zhu, Z. Zhang, G. Yuan,
T. Chen, Y. Tian, X. Hou, C. Lee, Nat. Commun. 2020, 11, 5381.
[5] a) Y. Kim, G. A. Parada, S. Liu, X. Zhao, Sci. Robot. 2019, 4, aax7326;
b) C. Li, G. C. Lau, H. Yuan, A. Aggarwal, V. L. Dominguez, S. Liu,
Authors’ Contribution H. Sai, L. C. Palmer, N. A. Sather, T. J. Pearson, Sci. Robot. 2020,
S.S. and Z.W. contributed equally to this work W.T. and Z.L.W. conceived 5, eabb9822; c) W. Hu, G. Z. Lum, M. Mastrangeli, M. Sitti,
the idea and designed the experiment. S.S. fabricated the device. S.S., Nature 2018, 554, 81.
Z.W., and P.C. performed the electrical measurements. S.S., Z.W., and [6] A. Koivikko, E. S. Raei, M. Mosallaei, M. Mäntysalo, V. Sariola,
W.T. analyzed the data. S.S. prepared the figures. J.Z. revised the data IEEE Sens. J. 2017, 18, 223.
and figures. S.S., Z.W., W.T., and Z.L.W. wrote the manuscript. All the [7] T. Li, L. Qiu, H. Ren, IEEE ASME Trans. Mechatron. 2020, 25, 406.
authors discussed the results and reviewed the manuscript. [8] B. Shih, D. Shah, J. Li, T. G. Thuruthel, Y. L. Park, F. Iida, Z. Bao,
R. Kramer-Bottiglio, M. T. Tolley, Sci Robot. 2020, 5, eaaz9239.
[9] C. Larson, J. Spjut, R. Knepper, R. Shepherd, Soft Rob. 2019, 6, 611.
[10] A. Moin, A. Zhou, A. Rahimi, A. Menon, S. Benatti, G. Alexandrov,
Data Availability Statement S. Tamakloe, J. Ting, N. Yamamoto, Y. Khan, F. Burghardt, L. Benini,
The data that support the findings of this study are available from the A. C. Arias, J. M. Rabaey, Nat. Electron. 2021, 4, 54.
corresponding author upon reasonable request. [11] C. Schlagenhauf, D. Bauer, K.-H. Chang, J. P. King, D. Moro,
S. Coros, N. S. Pollard, presented at 2018 IEEE-RAS 18th Int. Conf.
on Humanoid Robots (Humanoids), IEEE, Beijing, China 2018.
[12] a) Y. Liu, B. Chen, W. Li, L. Zu, W. Tang, Z. L. Wang, Adv. Funct.
Keywords Mater. 2021, 31, 2104770; b) M. S. Verma, A. Ainla, D. Yang,
D. Harburg, G. M. Whitesides, Soft Robot. 2018, 5, 133; c) J. Zhang,
curvature, intelligent soft robots, machine learning, shape-sensing skin
A. Jackson, N. Mentzer, R. Kramer, Front. Robot. AI 2017, 4.
[13] a) S. Timoshenko, J. M. Gere, in Theory of Elastic Stability, 2nd
Received: December 5, 2022
ed., McGraw-Hill, Inc, NY 1961, pp. 107–113; b) Z. Gao, J. Zhou,
Revised: February 1, 2023
Y. Gu, P. Fei, Y. Hao, G. Bao, Z. L. Wang, J. Appl. Phys. 2009, 105,
Published online: March 20, 2023
113707.
[14] Z. Zhou, K. Chen, X. Li, S. Zhang, Y. Wu, Y. Zhou, K. Meng, C. Sun,
Q. He, W. Fan, E. Fan, Z. Lin, X. Tan, W. Deng, J. Yang, J. Chen,
[1] D. Rus, M. T. Tolley, Nature 2015, 521, 467. Nat. Electron. 2020, 3, 571.
[2] a) G. Li, S. Liu, L. Wang, R. Zhu, Sci. Robot. 2020, 5, eabc8134; b) [15] a) Z. Wang, J. An, J. Nie, J. Luo, J. Shao, T. Jiang, B. Chen, W. Tang,
S. Terryn, J. Brancart, D. Lefeber, G. Van Assche, B. Vanderborght, Z. L. Wang, Adv. Mater. 2020, 32, 2001466; b) H. Wang, M. Totaro,
Sci. Robot. 2017, 2, aan4268; c) I. M. Van Meerbeek, C. M. De Sa, L. Beccai, Adv. Sci. 2018, 5, 1800541.
R. F. Shepherd, Sci. Robot. 2018, 3, aau2489; d) B. Ying, R. Z. Chen, [16] a) C. Kingsford, S. L. Salzberg, Nat. Biotechnol. 2008, 26, 1011;
R. Zuo, J. Li, X. Liu, Adv. Funct. Mater. 2021, 31, 2104665. b) B. Charbuty, A. Abdulazeez, J. Appl. Sci. Technol. Trends 2021, 2,
[3] a) R. F. Shepherd, F. Ilievski, W. Choi, S. A. Morin, A. A. Stokes, 20; c) S. B. Kotsiantis, Artif. Intell. Rev. 2011, 39, 261.
A. D. Mazzeo, X. Chen, M. Wang, G. M. Whitesides, Proc. Natl. [17] P. Saxena, P. Shukla, Adv. Compos. Hybrid Mater. 2021, 4, 8.

Adv. Mater. 2023, 35, 2211385 2211385 (9 of 9) © 2023 Wiley-VCH GmbH