Scalable sim-to-real transfer of soft robot designs

Sam Kriegman1 , Amir Mohammadi Nasab2 , Dylan Shah2 , Hannah Steele2 , Gabrielle Branin2 ,
Michael Levin3 , Josh Bongard1 , Rebecca Kramer-Bottiglio2
1
University of Vermont, 2 Yale University, 3 Tufts University
arXiv:1911.10290v1 [cs.RO] 23 Nov 2019

Fig. 1. The top 100 simulated 2-by-2-by-2 configurations of passive (cyan) and volumetrically-actuating (red) voxels (a) were manufactured in reality (b).

Abstract—The manual design of soft robots and their con- control. But, they are also, by definition, more permissive to
trollers is notoriously challenging, but it could be augmented—or, simulation inaccuracies, design flaws, and control precision: A
in some cases, entirely replaced—by automated design tools. soft gripper or foot will passively conform to complex objects
Machine learning algorithms can automatically propose, test,
and refine designs in simulation, and the most promising ones and terrain, reducing the burden on the simulator to perfectly
can then be manufactured in reality (sim2real). However, it is capture any single, “true” surface contact geometry.
currently not known how to guarantee that behavior generated Quantifying which soft robot designs, policies and behaviors
in simulation can be preserved when deployed in reality. Al- can be faithfully simulated is critical not only for robotics, but
though many previous studies have devised training protocols
also synthetic approaches to understand functional plasticity
that facilitate sim2real transfer of control polices, little to no
work has investigated the simulation-reality gap as a function of biological systems during development and regeneration.
of morphology. This is due in part to an overall lack of tools For both domains, testing candidate hypotheses in reality is
capable of systematically designing and rapidly manufactur- expensive, time consuming, and, in some cases, dangerous.
ing robots. Here we introduce a low cost, open source, and With the recent development of several high-space, many-
modular soft robot design and construction kit, and use it to
body, GPU-accelerated soft body simulators [10, 18], sim2real
simulate, fabricate, and measure the simulation-reality gap of
minimally complex yet soft, locomoting machines. We prove for soft robotics and synthetic biology has become more
the scalability of this approach by transferring an order of feasible. However, because these simulators have yet to be
magnitude more robot designs from simulation to reality than employed to design physical systems, their transferability is
any other method. The kit and its instructions can be found here: currently unknown.
github.com/skriegman/sim2real4designs
Previous work has demonstrated methods that promote
I. I NTRODUCTION successful sim2real transferal of soft object manipulation but
The simulation-reality gap1 for rigid-bodied robots is not soft robot behavior. For example, a rigid-bodied robot arm
steadily closing. Computational models of rigid body dy- was successfully trained in simulation to fold towels and drape
namics can now be regularized and tuned so that control pieces of cloth over a hanger [20]. However, the reality gap
policies optimized in simulation are just as successful when was not quantified beyond a binary success rate for each task.
tested on the physical system [2, 11]. The reality gap for Additionally, the robot’s geometry was fixed and controllers
soft robots, on the other hand, remains uncharted. It could were then optimized for it, whereas in the work reported here,
be wider than the gap for rigid bodies, or not. Soft bodies are the robot’s geometry is part of the solution space.
more challenging to accurately simulate, design, and precisely Hiller and Lipson [8] evolved the overall geometry and
distribution of hard and soft materials in simulation, and
1 Henceforth, “the reality gap”—as coined by Jakobi et al. [12]. transferred the structures and passive dynamics of various
 TABLE I
cantilever beams. In a separate experiment that included ac- S UMMARY OF PUBLISHED SIM 2 REAL TRANSFERENCE .
tuating materials, Hiller and Lipson evolved the morphology
and behavior of soft robots in simulation, and then built Author/citation Year Controllers Morphologies
one of the evolved designs physically. However, in order to Miglino et al. [21] 1994 1 1
Jakobi et al. [12] 1995 2 1
transfer the simulated behavior of this one design, the physical Harvey et al. [7] 1997 4 1
robot needed to be placed in a pressure and vacuum chamber, Lipson and Pollack [17] 2000 3 3
whereas the hundreds of soft robot designs built here can be Bongard et al. [2] 2006 34 2
Hiller and Lipson [8] 2011 1 5
internally pressurized and actuated. Koos et al. [13] 2012 2 2
More recently, Kriegman et al. [14] subjected a simulated Moeckel et al. [22] 2013 1 1
soft robot (composed of elastic voxels) to a series of damage Caluwaerts et al. [3] 2014 2 1
Cully et al. [5] 2015 10 10
scenarios that removed increasingly more of the robot’s struc- Cellucci et al. [4] 2017 1 3
ture. In each scenario, the robot was challenged to recover Tobin et al. [29] 2017 1 1
function (locomotion) by deforming its remnant structure, Rusu et al. [27] 2017 1 1
Peng et al. [24] 2018 1 1
without changing its predamage control policy. A pair of Pinto et al. [25] 2018 3 1
recovery strategies, automatically discovered by an evolution- Tan et al. [28] 2018 2 1
ary algorithm in simulation, were transferred to reality (using Golemo et al. [6] 2018 1 1
Matas et al. [20] 2018 3 1
silicone “voxels”), but function was not: The physical system Kwiatkowski and Lipson [15] 2019 2 2
could deform its resting structure as dictated by the recovery Hwangbo et al. [11] 2019 3 1
strategy, but it could not locomote, before or after damage. Kriegman et al. [14] 2019 1 5
Nachum et al. [23] 2019 3 1
The physical robot was heavy, had high friction feet, and was Akkaya et al. [1] 2019 1 1
symmetrically actuated in phase, so it just oscillated in place. Rosser et al. [26] 2019 1 16
To determine the particular challenges and opportunities of The results presented here 2019 1 108
soft robot transferal, it would be useful to greatly scale up
the number of design/policy pairs transferred. To this end,
B. The simulation.
we present a soft robot design and construction kit based on
the silicone voxel modules used in [14], but miniaturized to We used the soft-body physics engine Voxelyze [9] to
increase stability, simplified to improve reproducibility, and simulate robots composed of actuating and/or passive, elastic
arbitrarily actuated to permit the transferal of locomotion. voxels. The simulator models the distance between adjacent
Other modular yet rigid-bodied robot design and construc- voxels as Euler-Bernoulli beams (critically damped; ζ = 1).
tion kits exist, such as Molecubes [30]. However, our kit is Additionally, a collision detection system monitors the dis-
easier, faster, cheaper, and safer to use. In short, silicone is tance between the voxels on the surface of the robot at each
molded into hollow voxels, and tubing is attached to supply timestep. If a pair of surface voxels are detected to collide
low pressure actuation from a hand pump, causing volumetric (intersect), a temporary beam (underdamped; ζ = 0.8) is
changes in one or more of the voxels (Figs. 2 and 3). For constructed between the two until the collision is resolved.
simple behaviors robust to actuation noise, there is no need Designs were simulated with a gravitational acceleration of
to use a highly-pressurized air supply or program microcon- -9.81 m/s2 , and initialized on top of an infinite surface plane
trollers for open-loop control. There are also no expensive at z = 0. Coulomb friction is applied to voxels in contact with
motors or power supplies. the surface plane. Voxels were simulated to have 1 cm3 resting
Here, we employ the kit as an instrument to measure the volume (resting beam lengths), with Young’s modulus 107
reality gap as a function of morphology (Table I). To do Pa, Poisson’s ratio 0.35, and coefficients of static and kinetic
so, we fabricated 108 morphologies (transferal of structure) friction of 1 and 0.5, respectively. These hyperparameters were
and compared the behavior of nine simulated designs to their adopted from [14]. For more details about how the physics are
silicone equivalents (transferal of behavior). We hope that the actually modeled, see [9].
kit’s affordability, safety, speed, and simplicity will generate Volumetric actuation was implemented by varying the rest
increasingly more, and more reproducible, data about the length between voxels, in all three dimensions, when com-
automated design of increasingly competent soft machines. puting the elastic force between them. Volumetric expansion

II. M ETHODS
A. The design space.
Following [8] and [14], our kit uses elastic voxels as build-
ing blocks of structure. Here, we considered a 2-by-2-by-2
cartesian lattice workspace, within which voxels were con-
nected together to form a robot. At each x,y,z coordinate,
voxels could either be passive, volumetrically actuated, or
absent, yielding a total of 38 = 6561 different configurations. Fig. 2. A random morphology in the design space shown at atmospheric
We evaluated each configuration in simulation. (resting; a), positive (expanding; b), and negative (compressing; c) pressure.
 where xt , yt are the final coordinates of the design at the end
of the evaluation period.
C. Reality.
Following Kriegman et al. [14], simulated voxels were
realized physically as pneumatically-actuated, hollow silicone
voxels. The physical robot in [14] was constructed to transfer
symmetrical shape change, so its actuated voxels were dis-
tributed symmetrically and hooked into a single pressure inlet.
Thus, pressure oscillations occurred symmetrically in phase,
and the robot could only pulse in place. Moreover, due to
thin voxel walls relative to overall voxel size, and the tubing
and glue used to bond them together, the robot in [14] could
not fully support its own weight. The robot was lifted off the
ground by placing it on top of a small petri dish, positioned
underneath a segment of entirely passive voxels in the center
of the robot’s ventral surface. This permitted ventral (and more
extreme global) changes in surface curvature, yielding success-
ful sim2real transfer of shape change, but not locomotion.
The construction kit presented here rectifies the weight issue
by miniaturizing the voxels—voxel length was halved (from
Fig. 3. Manufacturing modular soft robots. Hollow, silicone voxels were 3cm to 1.5cm) and the wall thickness remained the same
created by partially filling an open-face mold with silicone (a), using a spatula
to spread it along the interior walls (b), and then securing the mold to a 1-axis (1mm), reducing voxel mass from 4.3g to 1.2g (including
rotational molding machine (c). This process allowed excess silicone to drip tubing but not pneumatic connectors). Further, the inter-voxel
out of the mold, while spreading the remaining silicone into a thin uniform tubing and glue was replaced with holes punched through the
layer. The cured, bottomless voxels were then appropriately arranged and
connected for each x,y slice of the design, and bonded with a shared bottom walls of adjacent active voxels in the same x,y slice, before
layer (d). Finally, tubing was attached (e), and the slices were stacked and attaching them with a shared bottom layer (Fig. 3d). Finally,
bonded to form the design (f). Video: youtu.be/jbQ2T7jIYRU. locomotion is now possible because separate contiguous sec-
tions of voxels in each slice can be arbitrarily actuated in or
in simulation and reality are both roughly spherical (Fig. 2b),
out of phase with other sections across the body.
but compression in reality is more complex and difficult to
simulate: the voxels buckle (Fig. 2c). So volumetric actuation D. The build protocol.
was here limited to expansion only (+90% rest volume). The The voxels were manufactured using a single-axis rotational
active voxels expand in phase with each other as dictated by molding machine.2 First, an open-face mold was fabricated by
a central pattern generator: a sine wave with frequency 4 Hz interlacing 26 acrylic strips into a flat base, to form a lattice
and amplitude 1.9 cm3 . When the sine wave is at or below of cubic concavities, resembling an ice-cube tray (Fig. 3a).
zero, the active voxels remain at their resting volume (1 cm3 ). Mold components were laser-cut (VLS2.30, Universal Laser
This produced quasistatic dynamics. System) from a flat acrylic sheet with a thickness of 0.025
Each design was simulated for 8831 timesteps, with a inch. Next, silicone (Dragon Skin 10 Fast; Smooth-On, Inc.)
stepsize of 0.000453 seconds, resulting in a total simulation was poured into the acrylic mold (Fig. 3a), and a spatula was
time of 4 seconds. During the first 552 times steps (0.25 sec), used to spread the silicone along the interior walls of each
the design was allowed to settle under gravity before actuation cavity (Fig. 3b). Colored pigment was added to each batch of
begins, ensuring that movement (if any) is a result of actuation, silicone to indicate whether the voxel was active or passive,
rather than passively falling forward. Just before actuation, the simplifying the assembly process. Here we used pink for active
design’s initial center of mass is recorded as (x0 , y0 , z0 ). The voxels and blue or yellow for passive voxels.
active voxels are then actuated for 3.75 sec at 4 Hz, or 15 The mold was then flipped upside down and secured to a
actuation cycles. 1-axis rotational molding machine. The machine was clamped
An exhaustive search of all 6561 designs (in batches of to a table with binder clips, angled 45◦ relative to horizontal,
50) took 58 CPU hours (1.8 wall-clock hours) on a single and set to rotate 90◦ every 45 seconds (Fig. 3c). This allowed
AMD Ryzen threadripper 1950X 16-core/32-thread processor. the silicone to flow and evenly coat the walls of the mold, as
Fitness was taken to be the net displacement (away from the excess silicone dripped out. After the voxels partially cured for
origin in any direction) of the design’s center of mass, in terms 25 minutes at room temperature, the mold was moved to an
of euclidean distance in the plane, where the origin is defined incubator, with a temperature of 60◦ C for another 20 minutes.
by the x, y components of the design’s initial center of mass (Without an incubator, the silicone will take 75 minutes to
(x0 , y0 ). Fitness is thus defined as: fully cure at room temperature.)
p
F = (xt − x0 )2 + (yt − y0 )2 , (1) 2 The required materials are listed at the end of the manuscript in Table II.
 The above steps were then repeated to add an additional quinary control space was mapped to 2D, by nesting pairs of
layer of silicone. Once the second layer cured, the bottomless dimensions within each other.
voxels were removed from the mold using an X-Acto knife, Here, the 8D ternary morphology space was reduced to
and excess silicone around their edges was trimmed. 2D by plotting pairs of dimensions nested within each other
In the next step, each x,y slice (or dorsal plane) of the (Fig. 5). The pixel in the exact center of Fig. 5, for instance,
design was assembled by using Sil-Poxy (Smooth-On, Inc.) to represents the configuration consisting entirely of passive
bond adjacent voxels and prevent the slice from shifting. Holes voxels, and thus cannot locomote (F = 0). Likewise, the
were then punched between adjacent active voxels so that pixel in the top right-hand corner of the heatmap represents
contiguous collections of voxels could be actuated together in the configuration of all active voxels (Fig. 6d), which actuated
phase. Each actuator group needed to contain at least one voxel symmetrically in phase, and thus (given its flat ventral surface)
on the surface of the design so that it could be controlled by an could not locomote across the flat ground plane (F = 0).
external pressure inlet. To create the bottom layer, two 1mm- Finally, the pixel in the bottom left-hand corner contains no
thick rulers were attached to an acrylic substrate using double- voxels at all, and thus F = 0.
sided tape and silicone was poured in the space between them. For locomotion, a good design obviously needs to have
Then, the slice of bottomless voxels was flipped, open-side a body, rather than none at all. With open-loop, in-phase
down, onto this uncured silicone layer (Fig. 3d). actuation, designs also need to have asymmetrical mass and/or
After the bottom layer cured, a thin layer of silicone was actuator distributions, or they will not generate any forward
applied with a popsicle stick along the outermost portions of movement. However it is not clear, even for this minimal
the interstices of the voxels, bonding adjacent voxels (without design space, exactly which asymmetrical designs will yield
gluing over inter-voxel holes). Then, the slice was cut from the highest fitness. Yet we can see small clusters and lines
the silicone sheet and a hole was poked into the side of one of similarly colored pixels in Fig. 5, representing morpho-
exterior voxel from each group of active voxels. Next, a 1/32” logically similar designs with similar fitness. This suggests
ID silicone tube was inserted into the hole, and glued in place that these configurations and substructures would be relatively
with Sil-Poxy, applied with a Q-tip (Fig. 3e). The end of this stable under random mutations or errors in fabrication.
tube was then connected to a straight pneumatic connector, Because fitness was measured by displacement in any
which was connected to 1/16” ID silicone tubing. direction away from the origin (Eq. 1), there are four con-
Occasional imperfections in alignment, silicone thickness, figurations—rotations, in the x,y plane, of a single geometry
or inter-voxel hole sizes would result in leaky structures. Leaks and distribution of passive and active voxels—with different
were detected by filling a beaker with water, submerging the
voxels, and inflating them. Bubbles would emanate from leaks,
which were repaired with Sil-Poxy. After repairing any leaks,
active

the slices were stacked on top of each other and bonded

together using a thin layer of silicone (Fig. 3e). Finally, these
layers were connected pneumatically with assorted pneumatic
connectors, attached to 1/16” ID silicone tubes.

III. R ESULTS
passive
vox 2

To test the effects of morphology on fitness and sim2real

transfer success, it is useful to first visualize the design space.
However, because there are
eight cartesian voxel coordi-
nates in the chosen workspace,
none

the design space here is eight

dimensional, which is difficult
to draw (let alone conceptu-
alize) without dimensionality
reduction. By nesting the di- none passive active
mensions of a search space vox 1
onto a single plot (Fig. 4), the
Fig. 5. Simulating modular soft robots. The design space is plotted as
entire space can be visualized a heatmap, containing one cell for each of the 6561 possible configurations.
Fig. 4. The 2D tessellation of 8D
as a 2D heatmap. This strategy ternary vector space used in Fig. 5. Lighter colored cells are fitter designs (Eq. 1). Each design is defined by a
was used by Cully et al. [5] to vector of eight ternary values, indicating what kind of voxel (none, passive,
or active) the design contains at the eight lattice points in the 2 × 2 × 2
neatly visualize the predicted fitness of a very large library workspace. The 8D ternary vector is reduced to a 2D heatmap by nesting
of control policies, as a function of the time a robot’s six pairs of dimensions within each other: four, nested 3 × 3 grids result in a
limbs were in contact with the simulated ground plane: 6D 34 × 34 = 81 × 81 overall heatmap.
Fig. 6. Measuring transferal from simulation to reality. Nine designs (a-i) were evaluated three times each in reality (green-to-blue gradient colored curves
in a'-i'). The behavioral trajectories start at the origin (green) and end at the robot’s final XY destination (blue) (in centimeters). The simulated movement
tracks (yellow-to-pink curves) are superimposed on top of the real ones. The relative volume (normalized by rest volume) was also recorded for each design
at four points during actuation under water (a''-i''). The simulated and real behavior of designs e and f can be observed here: youtu.be/UqjvmkYa9u4.

behaviors (they moved in different directions) but very similar and coefficient of friction of the surface on which their soft
(if not identical) fitness. There were also some configurations robot undulated determined the direction of locomotion. They
that, when rotated upward (in the x,z or y,z plane) fell into decomposed friction into load- and area controlled terms for
the same basic orientation and behavior but with a slightly point and surface contacts, respectively. On slippery surfaces
different heading. Thus, configurations with similar fitness with low interfacial shear resistance, the robot anchored about
(similarly colored pixels) are reflected across multiple, nested the point contact (expanded section) for locomotion and pulled
planes of symmetry in Fig. 5. These symmetries can also be its surface contact (passive segment). However, on surfaces
seen in the manufactured robots (Fig. 1b). The uniqueness of with high interfacial shear resistance, the robot anchored about
designs (i.e., the size of the search space of morphologies) is the surface contact and pulled the point contact toward it.
therefore a function of how behavior is measured. We hypothesize that such differences in tribological properties
Fig. 6 shows the behavior of nine different designs in simu- could have caused our designs to move in opposite directions
lation and reality. The real robot was actuated 90 times at 6 kPa in simulation and reality.
pressure on a surface covered with cornstarch (Argo®, ACH In an attempt to test this hypothesis and reduce the
Food Companies, Inc.) to reduce friction, and is compared to simulation-reality discrepancies that cause the virtual con-
23 simulated actuation cycles. Seven of the nine designs filled figurations in Fig. 6 to move differently than their physical
the cubic workspace with passive and active voxels, while the realizations, we performed a grid search of various simulation
other two share a more complex geometry: a single-voxel limb hyperparameters, including the coefficients of static and ki-
attached to the face of a 2-by-2 plane of voxels (Fig. 6e,f). netic friction. However, we could not identify a pair of friction
In one, the limb is active (Fig. 6e), in the other it is passive coefficients that resulted in correct movement heading for all
(Fig. 6f). These two designs achieved the two highest fitness nine of the behaving designs (Fig. 6a'-i'). This could be due to
scores (Eq. 1), in both simulation and reality. either low precision or low accuracy of the model. To isolate
By this measure, the reality gap appears small. However, and test the former possibility, we increased the resolution
these simulated designs move very differently from their man- of the simulated surface contact geometry by modeling each
ufactured equivalents. The simulated morphology in Fig. 6e silicone voxel as a 3-by-3-by-3 group of simulated “subvoxels”
pushes off its active limb, whereas in reality the design uses its (Fig. 7), and then re-ran the parameter sweep. Still, we could
limb to pull itself forward, in the opposite direction. Likewise, not find friction settings in which the simulated movement
the simulated morphology in Fig. 6f pushes off its active 2- direction matched the ground truth across all designs simulta-
by-2 torso, whereas in reality the design uses its torso to pull neously. This suggests that the accuracy of Coulomb friction
itself forward, in the opposite direction. model may be insufficient to model this type of movement.
Majidi et al. [19] showed that the interfacial shear strength The Coulomb approximation assumes that friction is simply
 of a design without investing the additional time and resources
required to fully build and examine the design itself. Sketches,
in other words, greatly increase the breadth of exploration in
design space. All sim2real methods embrace this utility of
simplifying sketches. Simulation, after all, is also a sketch.
However, there is a tacit assumption in robotics about Depth
First. A typical sim2real experiment begins by sending a
complicated robot design across the reality gap, and then
endeavors to learn transferable policies that control the mor-
phology in all its complexity. But this is not how most design
proceeds. An architect first roughly sketches a structure, say,
a bridge, on the back of a napkin. A diversity of designs
are then generated, tweaked, discarded or provisionally ac-
cepted—at this shallow level of napkin realism—before more
detailed blueprints are drawn under more stringent constraints.
Fig. 7. A higher resolution model in which each silicone voxel is approx- Blueprints, too, undergo this breadthwise evolution, before the
imated by a 3-by-3-by-3 group of simulated subvoxels: a high-res voxel.
The design in a and b are high-res instantiations of those in Figs. 6e and most promising are realized physically, first as scale models
6f, respectively. Spherical volumetric expansion in a high-res voxel (c) was (built from matchsticks and glue instead of concrete and steel),
approximated by increasing the rest length between the centermost subvoxel then, finally, at full scale and cost. This incrementally weeds
and the subvoxels at center of each face (green subvoxels in d).
out nontransferable features and adds mechanical complexity
only when and where it is necessary to do so, rather than
proportional to the vertical component N of the reaction force,
globally from the outset.
and independent of the contact area. However, friction is also
The assumption that the reality gap can be bridged by policy
a function of the surface area and interfacial shear strength
search alone, with a single robot design, is groundless. The
τ , a fixed constant which is mostly governed by adhesion
desired behavior of a robot is typically much more complicated
or mechanical interlocking between the contacting surfaces.
than that of architecture. This suggests the necessity of more,
A better model would thus consider friction as a function
not less, sketches. Soft robots are more complicated still. This
of both the normal force and the interfacial shear strength.
makes their automated design that much more appealing, but
However, before fundamentally changing the simulator, we
implies the need for even greater breadth of sketches, at more
plan to evaluate designs in noisy environments with imperfect
intermediate levels of realism. Though not every experiment
control over actuation characteristics to avoid ascribing high
will need to start from a blank slate. Instead, designers
fitness to designs that exploited unrealistic properties of the
(whether human or AI) could leverage prior knowledge to
simulation [12]. Additionally, data from reality could be used
reject truly awful designs before sketching them in the first
to automatically tune the geometry and resolution of the sim-
place. The designs transferred here add to a growing database
ulated finite elements [2], or to predict the kinds of behaviors
(prior probabilities) about which and how well different soft
that are more likely to successfully transfer [13], and which
robot designs and behaviors can be realized physically. Our
should be tested next [2]. Concurrently, we are investigating
construction kit has the potential to further increase this data
additional physical surfaces with varied tribological properties
by lowering not only cost and build times but also the barrier
in an attempt to match reality to simulation.
of entry to soft robotics for non-experts.
IV. D ISCUSSION The generality of such data beyond robotics is currently
not known, but it could also have important implications for
In this paper, we introduced a low cost, open source developmental biology and regenerative medicine. The bio-
platform for designing and rapidly building soft robots, and electric and genetic control policies that orchestrate adaptive
used it to transfer 108 different morphologies (voxels on a remodeling of growth and form in organisms are not yet under-
cartesian grid) from simulation to reality. We then measured stood, but could, in future, be reverse-engineered by machine
the reality gap as function of the robot’s design (geometry and learning, and then controlled externally to induce regenera-
distribution of passive and actuating voxels) by tracking the tion in otherwise non-regenerative organisms (such as adult
behavior of nine transferred morphologies. Under one measure humans), or to reprogram otherwise unbounded cancerous
(net displacement) the reality gap appeared rather small, but growth toward functional organogenesis [16]. However, such
under another (velocity) the gap was much wider, likely due advances in regenerative medicine and synthetic morphology
to oversimplified tribological contacts between the simulated will only be possible if hypotheses generated in simulation are
ground plane and the robot’s ventral surface [19]. transferable and testable in reality.
Although most of the transferred designs (99 out of the
108) were not actuated in reality, they nevertheless served an ACKNOWLEDGEMENTS
important function: they were sketches. Sketches let us think This work was supported by NSF award EFRI-1830870 and
more clearly about the behavior or properties (e.g., stability) DARPA contract HR0011-18-2-0022.
 TABLE II
T HE CONSTRUCTION KIT FOR MAKING FIFTY 2- BY-2- BY-2 DESIGNS .

No. Part Cost Notes

1 Dragon Skin 10 FAST, 2 lbs kit (Smooth-On, Inc.) $32.21 Created voxel bodies and connected x,y slices.
2 Sil-Poxy, 3 ounce tube (Smooth-On, Inc.) $30.72 Secured tubing and repaired air leaks in six-sided voxels.
3 1/32” ID silicone tubing, 52 ft $34.32 50A shore hardness; connected voxels to actuation system.
4 1/16” pneumatic plastic connectors, 200 pieces $100 Straight and T shaped.
5 1/16” ID silicone tubing, 65.6 ft $62.98 50A shore hardness; part of actuation system.
6 60 mL Plastic Syringe (McMasterCarr, Inc.) $3.13 Used to hand-actuate voxels while checking for leaks.
1000 mL beaker (PYREX VISTA Griffin, Fisher
7 $14 Voxels were inflated in this beaker filled with water to detect leaks.
Scientific International, Inc.)
8 Hole punch 1/4 rectangle (Fiskars) $10.35 Created holes between voxels within a slice so they actuated as a unit.
9 X-Acto Knife (McMasterCarr, Inc.) $4.11 Cut bottomless voxels out of acrylic mold, and trim the edges.
10 100 mL mixing cups (VWR international), pack of 100 $66 Where silicone was mixed.
11 Spatula (McMasterCarr, Inc.) $7.50 Spread the Dragon Skin on the edges of the acrylic mold.
12 2 Scrap acrylic sheets, 12”×12”×1/8” $10.58 Collected scrap Dragon Skin; the surface where voxel bottom were created.
13 2-1mm thick 30 cm metal rulers $6.99 Used to set a thickness for sixth voxel side (bottom layer).
14 Double-sided tape $4.99 Adhered the rulers to a scrap acrylic sheet.
15 Simple 30 cm metal ruler $3.99 Spread a thin sheet of Dragon Skin onto scrap acrylic sheet for sixth side.
16 Popsicle stick (11.3 cm × 1 cm), box of 1000 $13.49 Applied thin layer of silicone to bond adjacent voxels and x,y slices.
17 Cotton-tipped applicators, 6 inch, box of 1000 $8.99 (McKesson Corp.) Used to spread Sil-Poxy.
18 Disposable Gloves (Halyard Inc.), box of 100 $8.95 Wore when handling uncured silicone.
19 2 Acrylic sheets, 12”×12”×1/8” (McMasterCarr, Inc.) $18.30 Used to manufacture the open-face acrylic mold with laser-cut.
20 41 mm binder clips, pack of 12 $7.99 Held the acrylic mold onto the rotational molding machine.

The rotational molding machine:

21 Acrylic sheet, 12”×12”×1/4” (McMasterCarr, Inc.) $17.34 2× triangular plates, 3× motor mount - supports for rotational machine.
22 Acrylic sheet, 12”×12”×1/4” (McMasterCarr, Inc.) $17.34 Mounting plate; holes were cut to minimize weight.
23 Pololu 4756 DC rotational motor $39.95 Used for rotational molding machine.
24 Pololu 1999 mounting hum $7.95 Used to mount rotational molding machine.
25 Arduino Uno Microcontroller $22 Controlled rotation timing and degree.
25 Arduino Motor Shield $19.95 Controlled rotation motor.
26 AC charger $10.42 12mm×2.1×5.5mm barrel jack, 12V, Supplies power to the Arduino.
27 8020 T-Slotted Solid 1” beams (McMaster-Carr, Inc.) $12.31 2×10 cm, 2×40 cm, Supports for rotational molding machine.
28 8020 screws and T-nuts $7.92 Connected 8020 beams.
29 “M3×10” screws $1.24 Used to mount rotational molding machine.
30 Irwin QuickGrip 12”×2.75” Clamp $15.99 Held rotational molding machine to table.

Grand total: $622

R EFERENCES Baptiste Mouret. Robots that can adapt like animals.

Nature, 521:503–507, 2015. URL https://doi.org/10.
[1] Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, 1038/nature14422.
Mateusz Litwin, Bob McGrew, Arthur Petron, Alex [6] Florian Golemo, Adrien Ali Taiga, Aaron Courville,
Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, and Pierre-Yves Oudeyer. Sim-to-real transfer with
et al. Solving rubik’s cube with a robot hand. arXiv neural-augmented robot simulation. In Proceedings of
preprint, 2019. URL https://arxiv.org/abs/1910.07113. The 2nd Conference on Robot Learning, volume 87 of
[2] Josh Bongard, Victor Zykov, and Hod Lipson. Resilient Proceedings of Machine Learning Research, pages 817–
machines through continuous self-modeling. Science, 828. PMLR, 2018. URL http://proceedings.mlr.press/
314(5802):1118–1121, 2006. URL https://doi.org/10. v87/golemo18a.html.
1126/science.1133687. [7] I. Harvey, P. Husbands, D. Cliff, A. Thompson, and
[3] Ken Caluwaerts, Jérémie Despraz, Atıl Işçen, Andrew P N. Jakobi. Evolutionary robotics: the sussex approach.
Sabelhaus, Jonathan Bruce, Benjamin Schrauwen, and Robotics and Autonomous Systems, 20(2):205 – 224,
Vytas SunSpiral. Design and control of compliant tenseg- 1997. ISSN 0921-8890. URL https://doi.org/10.1016/
rity robots through simulation and hardware validation. S0921-8890(96)00067-X.
Journal of the royal society interface, 11(98):20140520, [8] Jonathan Hiller and Hod Lipson. Automatic design
2014. URL https://doi.org/10.1098/rsif.2014.0520. and manufacture of soft robots. IEEE Transactions on
[4] Daniel Cellucci, Robert MacCurdy, Hod Lipson, and Robotics, 28(2):457–466, 2011. URL https://doi.org/10.
Sebastian Risi. 1d printing of recyclable robots. IEEE 1109/TRO.2011.2172702.
Robotics and Automation Letters, 2(4):1964–1971, 2017. [9] Jonathan Hiller and Hod Lipson. Dynamic simulation of
URL https://doi.org/10.1109/LRA.2017.2716418. soft multimaterial 3D-printed objects. Soft Robotics, 1(1):
[5] Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-
 88–101, 2014. URL https://doi.org/10.1089/soro.2013. http://proceedings.mlr.press/v87/matas18a.html.
0010. [21] Orazio Miglino, Kourosh Nafasi, and Charles E. Taylor.
[10] Daniel Holden, Bang Chi Duong, Sayantan Datta, and Selection for wandering behavior in a small robot. Arti-
Derek Nowrouzezahrai. Subspace neural physics: fast ficial Life, 2(1):101–116, 1994. URL https://doi.org/10.
data-driven interactive simulation. In Proceedings of the 1162/artl.1994.2.1.101.
18th annual ACM SIGGRAPH/Eurographics Symposium [22] Rico Moeckel, Yura N Perov, Anh The Nguyen, Massimo
on Computer Animation, page 6. ACM, 2019. URL https: Vespignani, Stéphane Bonardi, Soha Pouya, Alexander
//doi.org/10.1145/3309486.3340245. Sproewitz, Jesse van den Kieboom, Frédéric Wilhelm,
[11] Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario and Auke Jan Ijspeert. Gait optimization for roombots
Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco modular robotsmatching simulation and reality. In The
Hutter. Learning agile and dynamic motor skills for IEEE/RSJ International Conference on Intelligent Robots
legged robots. Science Robotics, 4(26), 2019. URL and Systems (IROS), pages 3265–3272, 2013. URL https:
https://doi.org/10.1126/scirobotics.aau5872. //doi.org/10.1109/IROS.2013.6696820.
[12] Nick Jakobi, Phil Husbands, and Inman Harvey. Noise [23] Ofir Nachum, Michael Ahn, Hugo Ponte, Shixiang Gu,
and the reality gap: The use of simulation in evolutionary and Vikash Kumar. Multi-agent manipulation via loco-
robotics. In European Conference on Artificial Life, motion using hierarchical sim2real. In Proceedings of
pages 704–720. Springer, 1995. URL https://doi.org/10. The 3rd Conference on Robot Learning, 2019. URL
1007/3-540-59496-5 337. https://arxiv.org/abs/1908.05224.
[13] Sylvain Koos, Jean-Baptiste Mouret, and Stéphane Don- [24] Xue Bin Peng, Marcin Andrychowicz, Wojciech
cieux. The transferability approach: Crossing the reality Zaremba, and Pieter Abbeel. Sim-to-real transfer of
gap in evolutionary robotics. IEEE Transactions on robotic control with dynamics randomization. In The
Evolutionary Computation, 17(1):122–145, 2012. URL IEEE International Conference on Robotics and Automa-
https://doi.org/10.1109/TEVC.2012.2185849. tion (ICRA), 2018. URL https://doi.org/10.1109/ICRA.
[14] Sam Kriegman, Stephanie Walker, Dylan Shah, Michael 2018.8460528.
Levin, Rebecca Kramer-Bottiglio, and Josh Bongard. Au- [25] Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wo-
tomated shapeshifting for function recovery in damaged jciech Zaremba, and Pieter Abbeel. Asymmetric actor
robots. In Proceedings of Robotics: Science and Systems, critic for image-based robot learning. In Proceedings
2019. URL http://www.roboticsproceedings.org/rss15/ of Robotics: Science and Systems, 2018. URL https:
p28.html. //doi.org/10.15607/RSS.2018.XIV.008.
[15] Robert Kwiatkowski and Hod Lipson. Task-agnostic self- [26] Kent Rosser, Jia Kok, Javaan Chahl, and Josh Bongard.
modeling machines. Science Robotics, 4(26), 2019. URL Sim2real transfer degrades non-monotonically with mor-
http://doi.org/10.1126/scirobotics.aau9354. phological complexity for flapping wing design. arXiv
[16] Michael Levin. Reprogramming cells and tissue pattern- preprint, 2019. URL https://arxiv.org/abs/1910.13790.
ing via bioelectrical pathways: molecular mechanisms [27] Andrei A. Rusu, Matej Veerk, Thomas Rothrl, Nicolas
and biomedical opportunities. Wiley Interdisciplinary Heess, Razvan Pascanu, and Raia Hadsell. Sim-to-real
Reviews: Systems Biology and Medicine, 5(6):657–676, robot learning from pixels with progressive nets. In
2013. URL https://doi.org/10.1002/wsbm.1236. Proceedings of the 1st Annual Conference on Robot
[17] Hod Lipson and Jordan B Pollack. Automatic design and Learning, volume 78 of Proceedings of Machine Learn-
manufacture of robotic lifeforms. Nature, 406(6799):974, ing Research, pages 262–270. PMLR, 2017. URL
2000. URL https://doi.org/10.1038/35023115. http://proceedings.mlr.press/v78/rusu17a.html.
[18] Miles Macklin, Kenny Erleben, Matthias Müller, Nut- [28] Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen,
tapong Chentanez, Stefan Jeschke, and Viktor Makoviy- Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent
chuk. Non-smooth newton methods for deformable Vanhoucke. Sim-to-real: Learning agile locomotion for
multi-body dynamics. ACM Transactions on Graphics quadruped robots. In Proceedings of Robotics: Science
(TOG), 38(5):140, 2019. URL https://doi.org/10.1145/ and Systems, 2018. URL https://doi.org/10.15607/RSS.
3338695. 2018.XIV.010.
[19] Carmel Majidi, Robert F Shepherd, Rebecca K Kramer, [29] Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider,
George M Whitesides, and Robert J Wood. Influence of Wojciech Zaremba, and Pieter Abbeel. Domain random-
surface traction on soft robot undulation. The Interna- ization for transferring deep neural networks from simu-
tional Journal of Robotics Research, 32(13):1577–1584, lation to the real world. In The IEEE/RSJ International
2013. URL https://doi.org/10.1177/0278364913498432. Conference on Intelligent Robots and Systems (IROS),
[20] Jan Matas, Stephen James, and Andrew J Davison. Sim- 2017. URL https://doi.org/10.1109/IROS.2017.8202133.
to-real reinforcement learning for deformable object ma- [30] Victor Zykov, Andrew Chan, and Hod Lipson. Mole-
nipulation. In Proceedings of The 2nd Conference on cubes: An open-source modular robotics kit. In IROS
Robot Learning, volume 87 of Proceedings of Machine Self-Reconfigurable Robotics Workshop, 2007. URL
Learning Research, pages 734–743. PMLR, 2018. URL http://molecubes.org.