StarRankExplorer
Retour au classement

RLinf/RLinf

Pythonrlinf.readthedocs.io/en/latest/

RLinf: Reinforcement Learning Infrastructure for Embodied and Agentic AI

reinforcement-learningagentic-airlinfvla-rlembodied-airl-infra
Croissance des étoiles
Étoiles
4.2k
Forks
608
Croissance hebdomadaire
—
Issues
220
2k4k
sept. 25déc. 25avr. 26juil. 26
ArtefactsPyPIpip install rlinf
README
RLinf-logo
Hugging Face Ask DeepWiki

English 简体中文

RLinf: Reinforcement Learning Infrastructure for Embodied and Agentic AI

RLinf is a flexible and scalable open-source RL infrastructure designed for Embodied and Agentic AI. The 'inf' in RLinf stands for Infrastructure, highlighting its role as a robust backbone for next-generation training. It also stands for Infinite, symbolizing the system’s support for open-ended learning, continuous generalization, and limitless possibilities in intelligence development.

RLinf-overview

What's NEW!

  • [2026/07] 🎉 RLinf v0.3 is released with major upgrades in the real-world RL full pipeline (data collection → SFT → RL → deployment), more simulators and SOTA models, and system-level optimizations. Release notes: RLinf v0.3.
  • [2026/07] 🔥 RLinf supports RLT for online RL fine-tuning of VLA policies. Doc: RLT.
  • [2026/06] 🔥 RLinf supports STEAM for offline advantage estimation and policy optimization. Doc: STEAM.
  • [2026/06] 🔥 RLinf supports reinforcement learning fine-tuning for GR00T-N1.7. Doc: RL on GR00T-N1.7.
  • [2026/06] 🔥 RLinf supports reinforcement learning fine-tuning with the Polaris simulator. Doc: RL on Polaris.
  • [2026/06] 🔥 RLinf supports reinforcement learning fine-tuning for GR00T-N1.6. Doc: RL on GR00T-N1.6.
  • [2026/06] 🔥 RLinf supports reinforcement learning fine-tuning for Genesis. Doc: RL on Genesis.
  • [2026/05] 🔥 RLinf achieves 25× end-to-end speedup for the BEHAVIOR simulator through system-level optimizations (slimming, on-demand observation, hybrid pipeline parallelism), reducing rollout latency from 1028.7 ms/step to 41.2 ms/step. Blog: BEHAVIOR System Optimization
  • [2026/05] 🔥 RLinf supports reinforcement learning fine-tuning for ABot-M0. Doc: RL on ABot-M0 Model.
  • [2026/05] 🔥 RLinf supports RL training and SFT with Megatron-Bridge actor beckend. Doc: Megatron-Bridge.
  • [2026/05] 🔥 RLinf supports AgentLightning for single-agent RL training. Doc: AgentLightning Calc-X.
  • [2026/05] 🔥 RLinf supports DreamZero SFT with a refactored training pipeline, achieving nearly 4× throughput improvement over the official baseline and better convergence. Doc: DreamZero
  • [2026/05] 🔥 RLinf supports GimArm. Doc: GimArm
  • [2026/05] 🔥 RLinf supports real-world reinforcement learning with a dexterous hand. Doc: Franka + Dexterous Hand
  • [2026/05] 🔥 RLinf supports DM0. In particular, RLinf and Dexbotic enable Lego-style SFT-RL integration. Link: Dexbotic project link
  • [2026/04] 🔥 RLinf supports Dexmal DOS-W1 for real-world reinforcement learning. Doc: Real-World RL on Dexmal DOS-W1.
  • [2026/04] 🔥 RLinf supports RECAP (RL with Experience and Corrections via Advantage-conditioned Policies) for offline advantage-based policy optimization. Doc: RECAP.
  • [2026/04] 🔥 RLinf now supports offline IQL training on D4RL benchmarks. Doc: IQL on D4RL, paper: Offline Reinforcement Learning with Implicit Q-Learning.
  • [2026/04] 🔥 RLinf supports EmbodiChain as an embodied environment for RL, with a reference MLP + PPO CartPole recipe. Doc: EmbodiChain.
  • [2026/04] 🔥 RLinf supports reinforcement learning fine-tuning for RoboVerse. Doc: RL on RoboVerse.
  • [2026/04] 🔥 RLinf supports reinforcement learning fine-tuning for StarVLA. Doc: StarVLA.
  • [2026/04] 🔥 RLinf now supports HG-DAgger (Human-Gated DAgger) for real-world online training. Doc: HG-DAgger for Real-World Franka.
  • [2026/03] 🔥 RLinf now supports Stereolabs ZED cameras and Robotiq 2F-85 / 2F-140 grippers for Franka real-world RL. Doc: Franka with ZED & Robotiq.
More updates

Key Features

RLinf has high flexibility to support diverse RL training workflows (PPO, GRPO, SAC and so on), while hiding the complexity of distributed programming. Users can easily scale RL training to a large number of GPU nodes without modifying code, meeting the increasing demand of computation for RL training.

The high flexibility allows RLinf to explore more efficient scheduling and execution. The hybrid execution mode for embodied RL achieves up to 2.434× throughput compared to existing frameworks.

Multiple Backend Integrations

  • FSDP + HuggingFace/SGLang/vLLM: rapid adaptation to new models and algorithms, ideal for beginners and fast prototyping.
  • Megatron + SGLang/vLLM: optimized for large-scale training, delivering maximum efficiency for expert users with demanding workloads.

Examples

Embodied AI

RLinf supports SFT, simulation RL, and real-world RL for World Action Models (WAM) and Vision-Language-Action Models (VLA). The current support list is as follows:

Simulators Models Algorithms
Real-world Robotics Data Collection

Agentic AI

Single-Agent Multi-Agent

Quick Start

Installation: Users can refer to our installation guide to install RLinf. We recommend users to use our provided docker image (i.e., Installation Method 1), as the environment and dependencies of embodied RL are complex.

Run a simple example: After setting up the environment, users can run a simple example of embodied RL with ManiSkill3 simulator following this document.

SOTA RL Training Reproduction: RLinf provides end-to-end recipes that reproduce or match state-of-the-art (SOTA) RL results out of the box—users can directly run our configs and scripts to obtain SOTA performance without custom engineering. Check out our example gallery for more details.

Awesome Community Projects with RLinf

We are excited to see a growing ecosystem of projects building on top of or integrate with RLinf, spanning embodied AI, robotics, and long-horizon agentic systems. Here are some awesome community projects:

  • i4h-workflows: NVIDIA team open sourced RL-based workflow built on Isaac ecosystem, integrating RLinf for healthcare-oriented embodied intelligence.
  • pi-StepNFT: Extends RLinf for step-level training and optimization of π-series VLA models.
  • Dexbotic: A robotics + RL system integrating RLinf for scalable training and deployment of embodied agents.
  • RoboTwin: A digital twin + robotics platform leveraging RLinf for large-scale embodied RL training.
  • IsaacLab: Official integration of RLinf within IsaacLab, enabling seamless reinforcement learning workflows on top of NVIDIA Isaac Sim based robotics environments.
  • RISE: A robot reinforcement learning framework based on a compositional world model, using RLinf for online reinforcement learning.

💡 Want to feature your project here? Open a PR and we’ll be happy to include it!

Adoption

RLinf is a production-grade, open-source reinforcement learning framework for embodied AI. It is being adopted by leading companies and startups across AI infrastructure and robotics, including AgiBot, X Square Robot, PsiBot, Dexmal, Moore Threads, D-Robotics, DexForce, YinWang, Robbyant and GigaAI.

adoption

✨ If your organization is using RLinf, feel free to reach out or submit a PR to be listed here.

CI Test Status

RLinf has comprehensive CI tests for both the core components (via unit tests) and end-to-end RL training workflows of embodied, agent, and reasoning scenarios. Below is the summary of the CI test status of the main branch:

Test Name Status
unit-tests GitHub Actions Workflow Status
agent-reason-e2e-tests GitHub Actions Workflow Status
embodied-e2e-tests GitHub Actions Workflow Status
scheduler-tests GitHub Actions Workflow Status

Contribution Guidelines

We welcome contributions to RLinf. Please read contribution guide before taking action. Thank the following contributors and welcome more developers to join us on this open source project.

Citation and Acknowledgement

If you find RLinf helpful, please cite the paper:

@article{yu2025rlinf,
  title={RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformation},
  author={Yu, Chao and Wang, Yuanqing and Guo, Zhen and Lin, Hao and Xu, Si and Zang, Hongzhi and Zhang, Quanlu and Wu, Yongji and Zhu, Chunyang and Hu, Junhao and others},
  journal={arXiv preprint arXiv:2509.15965},
  year={2025}
}

If you use RL+VLA in RLinf, you can also cite our technical report and empirical study paper:

@article{zang2025rlinf,
  title={RLinf-VLA: A Unified and Efficient Framework for VLA+ RL Training},
  author={Zang, Hongzhi and Wei, Mingjie and Xu, Si and Wu, Yongji and Guo, Zhen and Wang, Yuanqing and Lin, Hao and Shi, Liangzhi and Xie, Yuqing and Xu, Zhexuan and others},
  journal={arXiv preprint arXiv:2510.06710},
  year={2025}
}

@article{liu2025can,
  title={What can rl bring to vla generalization? an empirical study},
  author={Liu, Jijia and Gao, Feng and Wei, Bingwen and Chen, Xinlei and Liao, Qingmin and Wu, Yi and Yu, Chao and Wang, Yu},
  journal={arXiv preprint arXiv:2505.19789},
  year={2025}
}

@article{chen2025pi_,
  title={$$\backslash$pi\_$\backslash$texttt $\{$RL$\}$ $: Online RL Fine-tuning for Flow-based Vision-Language-Action Models},
  author={Chen, Kang and Liu, Zhihao and Zhang, Tonghe and Guo, Zhen and Xu, Si and Lin, Hao and Zang, Hongzhi and Zhang, Quanlu and Yu, Zhaofei and Fan, Guoliang and others},
  journal={arXiv preprint arXiv:2510.25889},
  year={2025}
}

If you train your policies in physical world with RLinf, you can cite our paper:

@article{zang2026rlinfuser,
  title={RLinf-USER: A Unified and Extensible System for Real-World Online Policy Learning in Embodied AI}, 
  author={Hongzhi Zang and Shu'ang Yu and Hao Lin and Tianxing Zhou and Zefang Huang and Zhen Guo and Xin Xu and Jiakai Zhou and Yuze Sheng and Shizhe Zhang and Feng Gao and Wenhao Tang and Yufeng Yue and Quanlu Zhang and Xinlei Chen and Chao Yu and Yu Wang},
  year={2026},
  journal={arXiv preprint arXiv:2602.07837},
  url={https://arxiv.org/abs/2602.07837}, 
}

If you use World Model + VLA + RL in RLinf, you can cite our paper:

@article{jiang2026wovr,
  title={WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL}, 
  author={Zhennan Jiang and Shangqing Zhou and Yutong Jiang and Zefang Huang and Mingjie Wei and Yuhui Chen and Tianxing Zhou and Zhen Guo and Hao Lin and Quanlu Zhang and Yu Wang and Haoran Li and Chao Yu and Dongbin Zhao},
  year={2026},
  journal={arXiv preprint arXiv:2602.13977},
  url={https://arxiv.org/abs/2602.13977}, 
}

If you use RL-based sim-real co-training in RLinf, you can cite our paper:

@article{shi2026rlinf,
  title={Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models},
  author={Shi, Liangzhi and Chen, Shuaihang and Gao, Feng and Chen, Yinuo and Chen, Kang and Zhang, Tonghe and Zhang, Hongzhi and Zhang, Weinan and Yu, Chao and Wang, Yu},
  journal={arXiv preprint arXiv:2602.12628},
  year={2026},
  url={https://arxiv.org/abs/2602.12628},
}

If you use WideSeek-R1 in RLinf, you can cite our paper:

@article{xu2026wideseek,
  title={WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning},
  author={Xu, Zelai and Xu, Zhexuan and Zhang, Ruize and Zhu, Chunyang and Yu, Shi and Liu, Weilin and Zhang, Quanlu and Ding, Wenbo and Yu, Chao and Wang, Yu},
  journal={arXiv preprint arXiv:2602.04634},
  year={2026},
}

Acknowledgements RLinf has been inspired by, and benefits from, the ideas and tooling of the broader open-source community. In particular, we would like to thank the teams and contributors behind VeRL, AReaL, Megatron-LM, SGLang, and PyTorch Fully Sharded Data Parallel (FSDP), and if we have inadvertently missed your project or contribution, please open an issue or a pull request so we can properly credit you.

Contact: We welcome applications from Postdocs, PhD/Master's students, and interns. Join us in shaping the future of RL infrastructure and embodied AI!

Dépôts similaires
Developer-Y/cs-video-courses

List of Computer Science courses with video lectures.

computer-sciencealgorithms
82.6k11.4k
unslothai/unsloth

Unsloth is a local UI for training and running Gemma 4, Qwen3.6, DeepSeek, Kimi, GLM and other models.

PythonPyPIApache License 2.0fine-tuningllama
unsloth.ai/docs
68.7k6.2k
labmlai/annotated_deep_learning_paper_implementations

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

PythonPyPIMIT Licensedeep-learningdeep-learning-tutorial
nn.labml.ai
67.2k6.7k
ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

PythonPyPIApache License 2.0raydistributed
ray.io
43.3k7.8k
rohitg00/ai-engineering-from-scratch

Learn it. Build it. Ship it for others.

PythonPyPIMIT Licenseagentsai
aiengineeringfromscratch.com
41.6k6.9k
sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

PythonPyPIApache License 2.0cudainference
sglang.io
30.6k7.3k
eugeneyan/applied-ml

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

MIT Licenseapplied-machine-learningproduction
29.9k4k
d2l-ai/d2l-en

Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 500 universities from 70 countries including Stanford, MIT, Harvard, and Cambridge.

PythonPyPIOtherdeep-learningmachine-learning
d2l.ai
29.2k5.1k
AI4Finance-Foundation/FinGPT

FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.

Jupyter NotebookMIT Licensechatgptfinance
fingpt.io
20.9k3k
stefan-jansen/machine-learning-for-trading

Code for Machine Learning for Trading, 3rd edition — from data sourcing to live execution.

Jupyter NotebookMIT Licensemachine-learningtrading
ml4trading.io
20k5.4k
Unity-Technologies/ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.

C#Otherreinforcement-learningunity3d
unity.com/products/machine-learning-agents
19.6k4.5k
tensorflow/tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

PythonPyPIApache License 2.0machine-learningmachine-translation
17.4k3.8k