Roadmap to Sample-Efficient Real-World Reinforcement Learning Workshop (R2RL)

Talk abstract — DICE-RL

This talk presents DICE-RL, a framework for sample-efficient RL finetuning of pretrained robot policies on a real robot, guided by a distribution contractive view: using online feedback to sharpen the policy’s action distribution around successful behaviors. Building on this perspective, I’ll also talk about how to pretrain and select policies that preserve broad behavioral coverage for RL, supervised finetuning, and other forms of posttraining. Together, these works motivate a unified view of pretraining and posttraining: preserving coverage of successful behaviors for RL exploration and sharpening the policy distribution for reliable execution.

Call for Papers

We welcome submissions on sample-efficient real-world RL, especially work that identifies practical bottlenecks, reports lessons learned from real-robot experiments, or proposes methods that reduce the cost of learning on hardware. Topics of interest include, but are not limited to:

Example Topics

Supervision and Reward Cost: Methods for obtaining useful learning signals cheaply, including learned reward models, human feedback, human-in-the-loop interventions, and strategies for avoiding supervision as the bottleneck.
Exploration and Safety on Hardware: Algorithms and systems for efficient exploration under safety constraints, reset-free or autonomous RL, safe online adaptation, and approaches that improve wall-clock efficiency during real-world training.
Leveraging Priors for Real-World RL: Approaches that use pretrained policies, VLAs, offline datasets, world models, or sim-to-real transfer to reduce the number of required on-robot interactions.
Post-Deployment Adaptation: Methods for finetuning deployed policies, improving robustness to distribution shift, learning new behaviors from experience, and adapting generalist policies without catastrophic forgetting.
Long-Horizon and Contact-Rich RL: Techniques for improving sample efficiency in long-horizon manipulation, dexterous control, contact-rich tasks, and settings with delayed rewards, compounding errors, or difficult exploration.
Benchmarks, Metrics, and Lessons Learned: Benchmarks, evaluation protocols, shared platforms, negative results, system-level insights, and analysis of what worked, what failed, and why in real-world RL experiments.

Submission Guidelines

Submission portal: all papers must be submitted through our OpenReview portal.
Page limit: submissions should be 4–8 pages, excluding references and appendix.
Format: submissions must follow the official CoRL paper template and style, and must be properly anonymized for double-blind review.
Dual submission policy: we disallow work already accepted to the main CoRL 2026 conference. Accepted workshop papers will be listed on the website but are non-archival and will not appear in formal proceedings.

Spotlight Talks

All accepted papers will be presented at an in-person poster session. A small number of selected papers will additionally give a ~4-minute spotlight talk. Camera-ready versions of all accepted papers will be made available on the workshop website.

Accepted Papers

We received 37 submissions and accepted 28 papers: 7 oral presentations and 21 poster presentations. Oral presentations were selected by the organizing committee based on initial review scores, then filtered via a committee that excluded any conflicting author from selection. All accepted papers will be presented at the in-person poster session; oral papers will additionally give a ~4-minute spotlight talk. Accepted papers are non-archival. Camera-ready versions will be linked here once authors upload them.

Show accepted papers (28)

Oral Presentations

D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models Oral
Marius Memmel, Shawn Yin, Urvi Rutia, Gavin Angeloff, Eldon Li, Ankit Goyal, Jesse Zhang, Dieter Fox, Fabio Ramos, Anqi Li, Abhishek Gupta
ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control Oral
Xu Luo, Jiaying Yang, Hao Wu, Junlin Xie, Zijin Hong, Ji Zhang, Lianli Gao, Jingkuan Song
ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models Oral
Changyeon Kim, Kimin Lee, Yashraj Narang, Yijie Guo
Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning Oral
Sizhe Wei, Xulin Chen, Fengze Xie, Garrett Ethan Katz, Zhenyu Gan, Lu Gan
Post-Training, Finetuning, and Steering: A Field Guide for Curating Training Data Oral
Maria Attarian, Yifan Ruan, Claas A Voelcker, Chenyang Cao, Amy Zhang, Peter Stone, Nicholas Rhinehart, Igor Gilitschenski
Bi-Lipschitz-Regularized Pretraining for Efficient Policy Finetuning Oral
Shubham Mittal, Yunchu Zhang, Waleed Yagoub, William Huey, Zev Alvidrez, Kanta Hamasaki, Andrew Wagenmaker, Abhishek Gupta
VETO: Reward-Supervised World Models as Action Verifiers for Sample-Efficient RL Oral
Skand Peri, Matthew M Hong, Chaoxiang Zhang, Ross Worobel, Suraj Shivakumar, Dieter Fox, Jesse Zhang, Abhishek Gupta

Poster Presentations

PhySA: Decoupling Aesthetic Scoring from Physical Smoothness via Two-Channel Reward Engineering for Humanoid Locomotion
Yunwan Fu, Jiageng Mao, Sicheng He, Yue Wang
Value Sim-to-Real: Simulation Values for Efficient Real-World RL
Weijun Dong, Yilin Wang, Jiawei Fu, Yuchen Zhou, Henrik I Christensen, Zhao-Heng Yin, Hao Su
From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention
Sichang Su, Benjamin Yang, Zhiyun Deng, Boyuan Liang, Yip Fun Yeung, Zelin Wang, Lingfeng Sun
What Does It Take for a Generalist Robot Policy to Adapt from Minimal Data?
Shreyas Kowshik, Sreyas Venkataraman, Leo Wang, Niharika Pant, Max Simchowitz, Aviral Kumar
Agent-Guided Reinforcement Learning for Robot Policy Improvement
Weijun Dong, Carl Qi, Mingyuan Zhang, Manikantan Nambi, Lillian J. Ratliff, Jeroen Vanbaar, Dylan Randle, Jose Barreiros
Sample-Efficient RL for Policy Improvement via Flow-Reversed Action Anchors
Carl Qi, Weijun Dong, Xiaojie Wang, Peiyi Chen, Rahul Sajnani, Mingyuan Zhang, Amy Zhang, Jose Barreiros, Manikantan Nambi, Lillian J. Ratliff, Dylan Randle
PrO: Latent Proprioceptive Odometry for Goal-Reaching via Real-World Reinforcement Learning
Zhaoyang Cheng, Hongde Hao, Mengjie Zhao, Shaoting Zhu, Hang Zhao
You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vector
Omkar Patil, Ondrej Biza, Thomas Weng, Karl Schmeckpeper, Wil Thomason, Xiaohan Zhang, Kausik Sivakumar, Robin Walters, Nakul Gopalan, Sebastian Castro, Stephen Hart, Eric Rosen
Learning from Hindsight for VLA Reinforcement Learning
Iris Xu, Sunshine Jiang, John Marangola, Pulkit Agrawal, Zhang-Wei Hong
Multi-Fidelity Policy Gradients Stabilize Data-Scarce Reinforcement Learning
Xinjie Liu, Ruihan Zhao, Anirban Chaudhuri, Cyrus Neary, Ufuk Topcu, David Fridovich-Keil
Efficient Model-Based RL for Robot Control: An Online Optimization Approach
Fang Nan, Hao Ma, Qinghua Guan, Josie Hughes, Michael Muehlebach, Marco Hutter
BOOST: Bootstrapping Offline and On-policy Successes to Train Real-World Robot Policies
Anthony Liang, Jesse Zhang, Arjun Guru, Aditya Shah, Chaoxiang Zhang, Zhanyi Sun, Abhishek Gupta
Mulligan: Performance-Guided Data Collection for Efficient On-Robot Learning
Lars Ankile, Perry Dong, Rohan Tan Bhowmik, Aneesh Muppidi, David D. Yuan, Shuran Song, Chelsea Finn
A2A-FT: Sample-Efficient Lightweight Online Fine-Tuning for Vision-Language-Action Models via Action-to-Action Flow Matching
Jiahui Zhang, Muhammad Fadhil Ginting, Dong-Ki Kim, Abraham George, Max Olan Smith, Mohamed Kari, Qizhao Chen, Lihong Jin, Jesse Zhang, Yu Xiang, Andrzej Marek Reinke, Shayegan Omidshafiei, Ali-akbar Agha-mohammadi
Continual Robot Policy Learning via Variational Neural Dynamics
Jiaxu Xing, Zhiyuan Zhu, Yunfan Ren, Ismail Geles, Yifan Zhai, Rudolf Reiter, Davide Scaramuzza
STRIDE: Learning Humanoid Locomotion in Minutes with Acceleration-Based Rewards
Nico Bohlinger, Jan Peters
Prompt-Driven Exploration Language as an Exploration Space for VLA Reinforcement Learning
Sunshine Jiang, John Marangola, David Zhang, Raghuram Kowdeed, Ruiyang Luo, Nitish Dashora, Richard Li, Pulkit Agrawal, Zhang-Wei Hong
In-Context Exploration for Efficient Reinforcement Learning in Robotic Manipulation
Nikita Maurer, Rickmer Krohn, Vignesh Prasad, Gabriele Tiboni, Georgia Chalvatzaki
Learning Viability: Backup controller composed reward functions for filter-free Safe RL
Aditya Vats, Nadia Figueroa
FPO-ER: Adaptive Experience Replay for Sample-Efficient Flow Policy Gradients in Vision-Language-Action Models
Trong Khiem Nguyen, Sigmund Hennum Høeg, Marc Tuscher
Safety-Aware Reinforcement Learning through Adaptive Multi-Objective Preferences
Aditya Vats, Nadia Figueroa

8:30 - 8:40	Opening, Central Question, and Problem-board Framing
8:40 - 9:50	Keynote Talks
9:50 - 10:00	Problem-board Voting
10:00 - 10:30	Oral Spotlights of Key Contributed Papers
10:30 - 11:00	Poster Session + Coffee Break
11:00 - 11:50	Breakout Sessions on Top-voted Problems
11:50 - 12:20	Panel Discussion with Speakers and Audience Volunteers
12:20 - 12:30	Summary, Whitepaper Next Steps, (Tentative) Workshop Best Paper Prize

Roadmap to Sample-Efficient Real-World Reinforcement Learning

Overview

What makes this workshop unique?

Speakers

Breakout Session Leads

Workshop Schedule (Tentative)

Call for Papers

Submission Guidelines

Spotlight Talks

Accepted Papers

Oral Presentations

Poster Presentations

Organizers

Workshop Sponsor

Contact Us