Skip to content
shaoming11Public

About

hand unscrews bottle in mujoco

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

41 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

hand-sim

Shadow Hand unscrewing a bottle cap by finger gaiting. Built from PRD.md, one milestone at a time; deviations, measured numbers and open questions live in NOTES.md.

the trained policy, 5x slow

The M8 policy at 5x slow, cap camera, fingertip sites and contact forces drawn -- one stroke-and-regrip cycle out of the ~93 it does per episode. Re-simulated on CPU at substep resolution rather than stretching the 20 Hz saves, and it tracks the recorded GPU trajectory to 0.14%. Full 60 s episode across all three cameras:

python3 scripts/eval.py logs/capturn-nominal-s1/checkpoints/.../000150732800 --save 4
python3 scripts/render.py artifacts/m8_s1_ep00.npz --camera all --sites --contacts

Where it got to

milestone result
M0–M4 scene, gait, MJX parity, benchmark, env done
PRD M5 nominal training pass — 3 seeds, 100.00% ± 0.00 success over 1024 episodes each
PRD M6 sensitivity sweep done — 8 of 9 DR parameter groups are free; bottle placement is fatal
PRD M7 DR training fail — best policy 63.6% against the >80% bar

The trained policy turns the cap +13,222 deg (36.7 turns) per episode at net/gross 0.991, opposed on 100% of steps, with the cleanest contact in the project at -0.170 mm of penetration. Its own trajectory replays CPU-vs-warp at 0.0%, so none of it is solver error.

It unscrews with forearm pronation, not finger-driven rotation: the index pad regrips ~90 times an episode and 74.7% of joint travel is fingers, but finger_carried_rotation reads 0.0% — no rotation is carried while the forearm is still. The reward pays either route by design, so this is the one it permits rather than the one it asks for.

M7 fails for a measurable reason: the hand's mount is fixed, so it cannot translate to follow a bottle that moves, and ±0.75 cm of placement takes success to 35.5% and 0.0%. DR training more than triples robustness (16.9% → 53.3%) without reaching the bar, and a from-scratch run at full range beats the PRD's prescribed curriculum by 10 points.

Every number above, how it was measured, and what went wrong on the way are in NOTES.md.

Setup (Mac, CPU)

pip install mujoco playground mediapy scipy pytest

Watching it

# the open-loop gait, re-simulated live in the viewer, looping
python3 scripts/squeeze_twist.py          # generate the trajectory first (~2 s)
python3 scripts/view_scene.py --play
python3 scripts/view_scene.py --play --speed 0.25     # slow motion

# the static pregrasp pose -- ctrl+drag the cap to feel the hinge friction
python3 scripts/view_scene.py

# an mp4 you can scrub or share
python3 scripts/render.py --camera all --sites
open artifacts/squeeze_twist.mp4

In the viewer: turn on site under the Rendering panel to see the fingertip and cap markers, and contact force to see where the grip is acting. [ and ] cycle the cameras (front, cap, back).

Checking it

python3 -m pytest tests/ -q               # 81 checks, ~25 s -- the M0-M6 done-criteria
HAND_SIM_SLOW=1 python3 -m pytest tests/ -q   # adds the MJX-Warp rollouts (minutes on CPU)
python3 scripts/view_scene.py --report    # scene summary, actuator limits, pregrasp geometry

# what the reward would pay for every trajectory in artifacts/, before spending a GPU hour
python3 scripts/score_reward.py
python3 scripts/score_reward.py artifacts/squeeze_twist.npz --per-step

# CPU MuJoCo vs MJX-Warp, plus buffer sizing for training
python3 scripts/check_parity.py
python3 scripts/check_parity.py --probe   # just naconmax / njmax

warp-lang ships a CPU build, so MJX-Warp runs on the Mac -- slowly, but it is the real backend. impl='jax' cannot run this scene (no cylinder/mesh collisions).

On the GPU box

export XLA_PYTHON_CLIENT_PREALLOCATE=false
unset MUJOCO_GL          # EGL broke `import mujoco` on the M3 image; re-check on a new box
python3 -c "import jax; print(jax.default_backend())"   # must print: gpu

python3 scripts/squeeze_twist.py     # regenerate the trajectory at the current timestep
python3 scripts/check_parity.py      # M2 on CUDA
python3 scripts/benchmark.py --num-envs 8192 --out artifacts/bench_h200.json

Run one num_envs per process. With XLA_PYTHON_CLIENT_PREALLOCATE=false JAX grows its pool and never releases it, so sweeping several env counts in one process starves Warp and the largest one fails to allocate. This is why the H100's num_envs = 16384 failure is not a real ceiling, and is worth one fresh-process retry on the H200's 141 GiB (NOTES.md, M3).

There is no git on the box -- sync with tar czf - . | ssh HOST 'tar xzf - -C ~/hand-sim'.

The training env

python3 -m pytest tests/test_env.py -q                    # layout, config, geometry
HAND_SIM_SLOW=1 python3 -m pytest tests/test_env.py -q    # + physics, ~7 min on the Mac
from capturn import CapTurn, default_config, ppo_config

env = CapTurn()                       # nominal phase: no observation noise, no action delay
env.observation_size                  # {'state': (210,), 'privileged_state': (283,)}

Asymmetric actor-critic: state is 3 stacked frames of what a real hand could measure, privileged_state adds velocities, cap rotation, fingertip geometry, actuator forces and a zeroed slot for the DR parameters. The actor never sees the cap angle -- the cap is rotationally symmetric, so it does not need it and could not measure it on hardware.

The action is a delta on the position targets, integrated out of data.ctrl rather than out of a copy in state.info. That is deliberate: Playground's auto-reset wrapper restores data at an episode boundary but leaves info alone, so anything cached there leaks into the next episode. ppo_config() asks for full_reset=True as well, because otherwise every env restarts from the one state cached at the first reset. Both are covered by tests and written up in NOTES.md, M4.

reach was replaced by grasp at M5, after the first trained policy turned the cap with the side of its index finger and no thumb at all -- the right contact surface, but a one-sided push rather than a grasp. reach watched three fingertip sites and could not see it. grasp asks for what a bottle-cap grasp actually is: the thumb and the index finger both on the cap, anywhere along their length, and on opposite sides. See NOTES.md, "What the flick policy was actually doing". action_rate at 0.01 is still a third of the reward signal for an untrained policy. Measured numbers in NOTES.md.

At M6 rotate became twist, which is grasp times the rotation the current grip's 60 deg budget still allows -- so rotation produced without a pinch pays nothing (and slip charges for it), and a grip that never releases is paid for one stroke however far it turns the cap. The reason it is a product rather than a second term is measured rather than argued: scripts/score_reward.py scores a saved trajectory with the live reward, and under the M5 weights the scripted reference gait came last of five. tests/test_reward_ranking.py pins the ordering so that cannot happen quietly again.

Training

# on the H200
export XLA_PYTHON_CLIENT_PREALLOCATE=false
unset MUJOCO_GL
tmux new -s train
python3 scripts/train.py --wandb                      # nominal, 150M steps, ~2 h
python3 scripts/train.py --num-timesteps 20_000_000   # a shorter shakeout first
python3 scripts/train.py --resume                     # after a preemption

The box is spot and its disk does not survive a stop, so --resume is the only thing standing between a preemption and starting over. It needs no arguments: the run name is derived from the phase and seed rather than a timestamp, so it finds the W&B run and its newest checkpoint artifact on its own, and spends only the step budget that is left. A checkpoint is written and uploaded roughly every 3.5 minutes (--checkpoint-every MIN to change that).

Nothing in train.py renders -- MUJOCO_GL=egl breaks import mujoco on the Daytona image.

# success rate and the exploit checks, on the box
python3 scripts/eval.py logs/capturn-nominal-s1/checkpoints --save 4 \
    --out artifacts/eval_nominal.json

# then, on the Mac
python3 scripts/eval.py --wandb-artifact ckpt-capturn-nominal-s1:latest --save 4
python3 scripts/render.py artifacts/eval_000150000000_ep00.npz --camera all --sites

eval.py answers both halves of the M5 criterion. The success rate is over 1024 episodes with the deterministic policy. The second half -- "real finger gaiting with no exploit" -- is four measurements instead of four videos: net/gross rotation (1 is one direction, 0 is shaking the cap), regrips per gaiting fingertip (a gait has to let go), the wrist's share of the joint travel (2 of 24 joints; more than that and the fingers are passengers), and the deepest contact penetration, replayed through CPU MuJoCo so it is comparable to M1's -0.566 mm.

python3 -m pytest tests/test_train.py -q                  # 32 checks, ~5 s
HAND_SIM_SLOW=1 python3 -m pytest tests/test_train.py -q  # + train -> resume -> eval, ~5 min
python3 scripts/train.py --smoke                          # just the training half, ~1 min

capturn/compat.py carries two shims, because Brax 0.14.2 and JAX 0.11.2 cannot train and then load what they trained: JAX removed jax.device_put_replicated, which ppo.train still calls, and Brax's checkpoint save writes a None kernel initialiser that its own load_config cannot read back. Both shims have tests that fail when the upstream bug is fixed, so they get deleted rather than inherited. Details in NOTES.md, M5.

Changing the scene

The hand placement and the pregrasp keyframe are solved numerically, not hand-tuned. After any change to the bottle or cap geometry, re-fit them:

python3 scripts/fit_pregrasp.py           # report only
python3 scripts/fit_pregrasp.py --write   # patch assets/right_hand_mjx.xml and assets/scene.xml

After any --write, re-run the gait before trusting the new pose. The fitter's objective is a proxy for gaiting quality, not a measure of it: one re-fit scored better on every term and gaited at 5 degrees instead of 88, because it left the thumb 0.7 mm off the cap wall.

python3 scripts/squeeze_twist.py --sweep
python3 scripts/squeeze_twist.py --fingers th,ff     # the PRD's two-finger version

Layout

assets/shadow_hand/        pristine mujoco_menagerie right hand (commit feadf76), LICENSE kept
assets/right_hand_mjx.xml  derived copy: fingertip sites, fitted mount, scene owns <option>
assets/scene.xml           hand + fixed bottle + hinged cap, sensors, pregrasp keyframe
capturn/scene.py           scene path and cap geometry, shared with the scripts
capturn/env.py             the MjxEnv subclass -- obs, reward, termination
capturn/config.py          env config and the Brax PPO config
capturn/compat.py          two shims for the installed Brax/JAX pair
scripts/                   fit_pregrasp, view_scene, squeeze_twist, check_parity,
                           benchmark, train, eval, render, score_reward
tests/                     M0 scene checks, M1 gait, M2 parity, M3 benchmark harness,
                           M4 env, M5 training and eval, M6 reward ranking
artifacts/                 generated trajectories and video (gitignored)
logs/                      training runs: checkpoints and configs (gitignored)

About

hand unscrews bottle in mujoco

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages