Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning
Abstract
Sequential fine-tuning with low-rank adaptation demonstrates superior performance in continual reinforcement learning for vision-language-action models, challenging conventional wisdom about catastrophic forgetting.
Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Fine-Tuning (Seq. FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of CRL for large pretrained VLAs across three models and five challenging lifelong RL benchmarks. We find that, contrary to established belief, simple Seq. FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between the large pretrained model, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability-plasticity trade-off, making continual adaptation both stable and scalable. Our results position Sequential Fine-Tuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era. Code is available at github.com/UT-Austin-RobIn/continual-vla-rl.
Community
CRL for VLAs
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning (2026)
- CRL-VLA: Continual Vision-Language-Action Learning (2026)
- Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning (2026)
- Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models (2026)
- CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion (2026)
- Reinforced Curriculum Pre-Alignment for Domain-Adaptive VLMs (2026)
- Continual Policy Distillation from Distributed Reinforcement Learning Teachers (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper