Jekyll2026-10-04T17:42:41+00:00https://dscho1234.github.io/feed.xmlDeep Reinforcement Learning, Robotics researcherDaesol Cho2026-09-20T00:00:00+00:002026-09-20T00:00:00+00:00https://dscho1234.github.io/research/conference/AdaptManipAbstract: This paper presents Adaptive Whole-body LocoManipulation, AdaptManip, a fully autonomous framework for humanoid robots to perform integrated navigation, object lifting, and delivery. Unlike prior imitation learning-based approaches that rely on human demonstrations and are often brittle to disturbances, AdaptManip aims to train a robust loco-manipulation policy via reinforcement learning without human demonstrations or teleoperation data. The proposed framework consists of three coupled components: (1) a recurrent object state estimator that tracks the manipulated object in real time under limited fieldof-view and occlusions; (2) a whole-body base policy for robust locomotion with residual manipulation control for stable object lifting and delivery; and (3) a LiDAR-based robot global position estimator that provides drift-robust localization. All components are trained in simulation using reinforcement learning and deployed on real hardware in a zero-shot manner. Experimental results show that AdaptManip significantly outperforms baseline methods, including imitation learning-based approaches, in adaptability and overall success rate, while accurate object state estimation improves manipulation performance even under occlusion. We further demonstrate fully autonomous real-world navigation, object lifting, and delivery on a humanoid robot.

]]>Daesol Cho

2026-04-30T00:00:00+00:002026-04-30T00:00:00+00:00https://dscho1234.github.io/research/conference/DynaFLIPAbstract: Robot manipulation succeeds only when perception preserves the aspects of a scene that matter for action. Yet most robot learning pipelines still rely on visual encoders pre-trained for static recognition or vision-language alignment, leaving motion understanding to downstream policies. We introduce DynaFLIP, a dynamics-aware multimodal pre-training framework that pushes motion understanding upstream into perception. DynaFLIP uses image transitions, language, and 3D flow as training-time supervision to shape an image-only encoder, requiring no extra modalities at test time. Our key idea is to align these modalities jointly by minimizing the generalized simplex volume spanned by their embeddings, capturing higher-order structure beyond anchor-based pairwise objectives. To avoid geometrically ambiguous low-volume solutions and trivial collapse, we augment the objective with a cosine regularizer between selected modality pairs and embed the resulting cosine-augmented energy in an contrastive framework. We construct image-language-3D flow triplets from heterogeneous human and robot videos and evaluate the resulting representations across simulation and real-world manipulation. DynaFLIP consistently outperforms strong baselines on LIBERO and in a real-world VLA setup, with especially strong gains under visual, spatial, and semantic distribution shifts. Additional analyses confirm that the encoder attends to manipulation-relevant regions and preserve control-relevant information. Our results suggest that robot generalization improves when visual representations are trained to encode not just what is present, but how the world changes under action.

]]>Daesol Cho

2026-04-16T00:00:00+00:002026-04-16T00:00:00+00:00https://dscho1234.github.io/research/conference/EgoAVFlowAbstract: Egocentric human videos provide a scalable source of manipulation demonstrations; however, deploying them on robots requires active viewpoint control to maintain task-critical visibility, which human viewpoint imitation often fails to provide due to human-specific priors. We propose EgoAVFlow, which learns manipulation and active vision from egocentric videos through a shared 3D flow representation that supports geometric visibility reasoning and transfers without robot demonstrations. EgoAVFlow uses diffusion models to predict robot actions, future 3D flow, and camera trajectories, and refines viewpoints at test time with reward-maximizing denoising under a visibility-aware reward computed from predicted motion and scene geometry. Real-world experiments under actively changing viewpoints show that EgoAVFlow consistently outperforms prior human-demo-based baselines, demonstrating effective visibility maintenance and robust manipulation without robot demonstrations.

]]>Daesol Cho

2026-03-30T00:00:00+00:002026-03-30T00:00:00+00:00https://dscho1234.github.io/research/conference/FLAGAbstract: Maximum entropy reinforcement learning (MaxEnt-RL) enables robust exploration, yet practical implementations often restrict policies to simple Gaussians. While recent MaxEnt-RL approaches incorporate expressive generative policies via weighted supervised learning, they use importance sampling, which is prone to a collapse in effective sample size, limiting their scalability in high-dimensional action spaces. Our key insight is to mitigate this limitation by localizing the effective sampling region, avoiding the weight degeneracy induced by standard importance sampling over the entire action space. To instantiate this insight, we introduce FLAG (Flow policy MaxEnt-RL by Latent Augmented Guidance). FLAG leverages a composite policy consisting of a flow policy and a Gaussian head, optimized via a proxy MaxEnt-RL objective that we prove is consistent with the original MaxEnt-RL formulation. We empirically demonstrate that FLAG facilitates expressive policy learning even with limited importance samples and scales to high-dimensional control tasks. As a result, FLAG achieves state-of-the-art performance across challenging benchmarks.

]]>Daesol Cho

2026-03-16T00:00:00+00:002026-03-16T00:00:00+00:00https://dscho1234.github.io/research/conference/BooSTAbstract: Skill abstraction—the process of learning reusable and temporally extended behaviors—has emerged as a key paradigm for improving sample efficiency and generalization in robot learning. For efficient skill transfer to real robots, learned skills must generalize across tasks and domains, remain robust to visual and dynamic perturbations, and be efficient enough for practical deployment. However, existing methods typically satisfy only a subset of these properties, as they capture either high-level semantic intent (what) or low-level motion dynamics (how). This incomplete skill transfer yields weak priors for policy learning, thereby demanding substantial in-domain data for downstream adaptation. To address these challenges, we introduce BooST, a two-stage framework that explicitly bridges semantics and motions to satisfy all three desiderata. BooST first leverages a cross-modal VQ-VAE to capture both semantic intent and motion dynamics, yielding a unified skill representation. It then distills this representation into a lightweight policy for efficient downstream adaptation to new tasks. Extensive experiments across simulation and real-robot settings demonstrate that BooST achieves superior few-shot adaptation, cross-domain skill transfer, and robustness to dynamic visual distractors, while maintaining a lightweight yet expressive design suitable for real-world deployment.

]]>Daesol Cho

2026-02-02T00:00:00+00:002026-02-02T00:00:00+00:00https://dscho1234.github.io/research/conference/Temporal%20Action%20Representation%20Learning%20for%20Tactical%20Resource%20Control%20and%20Subsequent%20Maneuver%20GenerationAbstract: Autonomous robotic systems should reason about resource control and its impact on subsequent maneuvers, especially when operating with limited energy budgets or restricted sensing. Learning-based control is effective in handling complex dynamics and represents the problem as a hybrid action space unifying discrete resource usage and continuous maneuvers. However, prior works on hybrid action space have not sufficiently captured the causal dependencies between resource usage and maneuvers. They have also overlooked the multi-modal nature of tactical decisions, both of which are critical in fast-evolving scenarios. In this paper, we propose TART, a Temporal Action Representation learning framework for Tactical resource control and subsequent maneuver generation. TART leverages contrastive learning based on a mutual information objective, designed to capture inherent temporal dependencies in resource-maneuver interactions. These learned representations are quantized into discrete codebook entries that condition the policy, capturing recurring tactical patterns and enabling multi-modal and temporally coherent behaviors. We evaluate TART in two domains where resource deployment is critical: (i) a maze navigation task where a limited budget of discrete actions provides enhanced mobility, and (ii) a high-fidelity air combat simulator in which a F-16 agent operates weapons and defensive systems in coordination with flight maneuvers. Across both domains, TART consistently outperforms hybrid-action baselines, demonstrating its effectiveness in leveraging limited resources and producing context-aware subsequent maneuvers.

]]>Daesol Cho

2025-09-18T00:00:00+00:002025-09-18T00:00:00+00:00https://dscho1234.github.io/research/conference/Periodic%20Skill%20DiscoveryAbstract: Unsupervised skill discovery in reinforcement learning (RL) aims to learn diverse behaviors without relying on external rewards. However, current methods often overlook the periodic nature of learned skills, focusing instead on increasing the mutual dependency between states and skills or maximizing the distance traveled in latent space. Considering that many robotic tasks—particularly those involving locomotion—require periodic behaviors across varying timescales, the ability to discover diverse periodic skills is essential. Motivated by this, we propose Periodic Skill Discovery (PSD), a framework that discovers periodic behaviors in an unsupervised manner. The key idea of PSD is to train an encoder that maps states to a circular latent space, thereby naturally encoding periodicity in the latent representation. By capturing temporal distance, PSD can effectively learn skills with diverse periods in complex robotic tasks, even with pixel-based observations. We further show that these learned skills achieve high performance on downstream tasks such as hurdling. Moreover, integrating PSD with an existing skill discovery method offers more diverse behaviors, thus broadening the agent’s repertoire.

]]>Daesol Cho

2025-09-12T00:00:00+00:002025-09-12T00:00:00+00:00https://dscho1234.github.io/research/conference/Single-View%203D-Aware%20Representations%20for%20Reinforcement%20Learning%20by%20Cross-View%20Neural%20Radiance%20FieldsAbstract: Reinforcement learning (RL) has enabled robots to develop complex skills, but its success in image-based tasks often depends on effective representation learning. Prior works have primarily focused on 2D representations, often overlooking the inherent 3D geometric structure of the world, or have attempted to learn 3D representations that require extensive resources such as synchronized multi-view images even during deployment. To address these issues, we propose a novel RL framework that extracts 3D-aware representations from single-view RGB input, without requiring camera pose or synchronized multi-view images during the downstream RL. Our method employs an autoencoder architecture, using a masked Vision Transformer (ViT) as the encoder and a latent-conditioned Neural Radiance Fields (NeRF) as the decoder, trained with cross-view completion to implicitly capture fine-grained, 3D geometry-aware representations. Additionally, we utilize a time contrastive loss that further regularizes the learned representation for consistency across different viewpoints, which enables viewpoint-robust robot manipulations. Our method significantly enhances the RL agent’s performance both in simulation and real-world experiments, demonstrating superior effectiveness compared to prior 3D-aware representation-based methods, even when using only single-view RGB images during deployment.

]]>Daesol Cho

2025-09-01T00:00:00+00:002025-09-01T00:00:00+00:00https://dscho1234.github.io/research/conference/Unifying%20What%20and%20How%20Distilling%20a%20Pre-trained%20Unified%20Skill%20Representation%20for%20Efficient%20AdaptationAbstract: Skill abstraction, the process of learning reusable and temporally extended behaviors, has emerged as a key focus in robot learning for its potential to improve sample efficiency and generalization. However, existing methods exhibit complementary strengths and weaknesses, typically modeling either high-level semantic intent (what to do) while sacrificing motion fidelity, or fine-grained motion dynamics (how to do it) while lacking semantic context. To address these limitations, we introduce Unified Skill Representation (USR) that unifies both the semantic intent and motion dynamics into a single skill representation. USR employs a cross-modal VQ-VAE to learn a semantically grounded and dynamically aware skill codebook. Furthermore, we propose a decoupled training framework that reconciles large-scale skill pre-training with practical deployment. Our approach builds transferable skills from vast, diverse datasets and then trains a lightweight policy on small, in-domain data. Extensive experiments on the LIBERO benchmark demonstrate that USR not only achieves near expert-level performance on the training distribution but also substantially outperforms prior methods in few-shot transfer to unseen tasks. Our results highlight the importance of both unifying skill representations and decoupling the training pipeline, offering a step toward more generalizable and practical robotic agents.

]]>Daesol Cho

2025-06-16T00:00:00+00:002025-06-16T00:00:00+00:00https://dscho1234.github.io/research/conference/Leveraging%20Temporally%20Extended%20Behavior%20Sharing%20for%20Multi-task%20Reinforcement%20LearningAbstract: Multi-task reinforcement learning (MTRL) offers a promising approach to improve sample efficiency and generalization by training agents across multiple tasks, enabling knowledge sharing between them. However, applying MTRL to robotics remains challenging due to the high cost of collecting diverse task data. To address this, we propose MT-L´evy, a novel exploration strategy that enhances sample efficiency in MTRL environments by combining behavior sharing across tasks with temporally extended exploration inspired by L´evy flight. MT-L´evy leverages policies trained on related tasks to guide exploration towards key states, while dynamically adjusting exploration levels based on task success ratios. This approach enables more efficient state-space coverage, even in complex robotics environments. Empirical results demonstrate that MT-L´evy significantly improves exploration and sample efficiency, supported by quantitative and qualitative analyses. Ablation studies further highlight the contribution of each component, showing that combining behavior sharing with adaptive exploration strategies can significantly improve the practicality of MTRL in robotics applications.

]]>Daesol Cho