Robot learning @LeRobotHF @huggingface. Building Physical AI @codecopenflow
Paris, France
Joined September 2020
- Tweets1.1K
- Following2.3K
- Followers669
- Likes12.2K
Pinned Tweet
Cool robot demo. More interesting: adding the next RL algorithm to LeRobot is now one file + one config.
SAC was first. Who's next?
The best is yet to come. 🤖
🤖Adding new RL algorithms to LeRobot just got much easier.
Demo: HIL-SERL training with a SAC-based RL algorithm on an SO-100 for a hole-in-hand peg-in-hole task.
Sparse reward, only 30 offline demos mixed with live robot experience, and ~1 hour of online training with human interventions only when the policy fails.
The bottom graph tracks intervention rate: high at the start, steadily dropping as the policy improves.
The refactor separates algorithm logic from training infrastructure:
• RLAlgorithm owns learning logic
• RLTrainer handles orchestration
• DataMixer combines rollouts, demos, interventions, and future data sources
Adding an RL algorithm now looks much closer to adding a policy: one algorithm file, one config, one registry entry.
SAC is first. RLT, RECAP, ConRFT, QC-FQL, DSRL, and VLA RL fine-tuning next!
@Thom_Wolf @ClementDelangue
lil’km retweeted
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
images.nvidia.com/pdf/Open-W…
lil’km retweeted
Jensen Huang resigned.
Hopefully, he will be able to have it as a vacation. Recharge, recover and come back.
Open source has many strong fighters and will prevail.
lil’km retweeted
Five new VLAs in LeRobot 0.7. The zoo keeps growing 🤖
- G0.5 from @GalaxeaDynamics: a 2B VLA that writes its subtask, then its actions, in one stream. Works with /subtask and /autosteer
- LingBot-VLA 2.0 from @robbyant_brain: one model across embodiments (55-D action space, ~60,000 h of pretraining). Its sparse MoE runs an action chunk in 120 ms in LeRobot, down from 756 ms
- DM0.5 from @Dexmal_AI: a Gemma3-4B VLM + a 680M flow-matching action expert, on standard LeRobot datasets
- Wall-OSS 0.5 from @XSquareRobot: a 4B VLA that also writes text. It answers /vqa and plans its own subtasks
- FineART-VLA: π0.5 with a language head, trained on FineART (more in our language post)
Docs in the comments 👇
lil’km retweeted
10,000 hours of open egocentric data with SOTA 3D head and hand tracking from @GroundedSI are now available 🤗
RoboCap-10k spans 21 industries in 5 continents with 619 real, paid work tasks all using 6-camera/dual IMU RoboCap hardware.
Explore the dataset and research ↓
Introducing RoboCap-10k. 10,000 hours of open egocentric data with metric hand / head pose on every frame.
Captured on RoboCap across 11 countries and 21 industries.
GroundedAPI produces 3D enrichment giving robot policies high-fidelity action grounding.
with @BitRobotNetwork @huggingface @rerundotio ↓
lil’km retweeted
Praxis-1: an open-weight world action model for real robots 🤖
New from @runwayml and now in LeRobot.
Praxis-1 learns how objects behave and how tasks unfold from large-scale video, which gives it a head start on a new robot. One checkpoint adapts with light fine-tuning or LoRA: fine-tuned on Google's RT-1 robot, which it never saw in training, it reaches 80.0% on SimplerEnv. Weights, inference and fine-tuning code are open under Apache 2.0, commercial use included, and it runs on NVIDIA from H100 to Jetson Thor.
Model, blog and docs in the comments 👇
lil’km retweeted
Robots that imagine before they act 🔮🤖
World action models predict what happens next, then act. LeRobot 0.7 adds four of them:
- Cosmos3 from @NVIDIARobotics: 16.5B Nano and 4.6B Edge, 97.9% LIBERO average. Edge runs on one GPU
- FLUX 3 Action from @bfl_ai: one open 7B diffusion transformer predicts future video and actions together, with an SO-101 checkpoint
- LaWAM from RLinf: imagines the future in latent space (DINOv3 features), 98.5% on LIBERO measured in LeRobot
- Long-WAM from NVIDIA: looks back over 19.2 s of video history, 99.0% on LIBERO measured in LeRobot
Docs in the comments 👇
lil’km retweeted
We just crossed:
- 20M AI builders on @huggingface
- 300PB of models, datasets, agent traces, and RL environments hosted on the @huggingface hub (9PB added by users just last week)
Also October was our biggest month ever in terms of DAUs, MAUs and revenue.
Let's go open-source AI!
lil’km retweeted
Hugging Face is the best place to host and share 3D assets - drop in .glb, .ply, .stl, or splat files and get a 3D gallery automatically.
Congrats @nvidiaomniverse on achieving 1M downloads of SimReady 3D assets!
To publish your own 3D datasets with an interactive gallery, check out the docs: huggingface.co/docs/hub/data…
1 million downloads of SimReady assets on @HuggingFace. 🎉
SimReady assets help teams use 3D content across supported tools and simulation workflows, from robot workcells to AI factory digital twins.
Thank you to everyone experimenting, sharing feedback and building with SimReady assets.
Find assets for your next simulation in SimReady Central: huggingface.co/spaces/nvidia…
lil’km retweeted
It took me ~21 hours to build @asimovinc from @menloresearch 🤖
Ever wondered what it takes to go from a pile of parts to a walking humanoid? Watch it !
Full timelapse in the reply 👇
Built at @huggingface with heart 🤗
lil’km retweeted
Train on terabytes. Don't download them. 📦
LeRobot 0.7 trains straight from the Hub:
- Streaming: each worker fetches only the video bytes it needs. 0.52% of step time spent waiting for data (π0.5, 4× H100, 10.7 TB dataset)
- LanceDB, built with @lancedb: frames are read in place. 1.7% waiting for data, vs 37.4% from local disk after a 384 GB download
- GPU augmentations: 2.8× faster ACT training on an H100, with 8 new robotics augmentations
Plus a training stack for big models: FSDP2 + HSDP, checkpoints saved 3.81× faster and resumable on any GPU count, FP16, EMA and Trackio.
--dataset.streaming=true
Docs in the comments 👇
lil’km retweeted
Talk to your robot while it works 🗣️🤖
New in LeRobot 0.7: interactive rollouts. Start a policy, then steer it mid-run, without a reset:
- /subtask: switch what it's doing
- /vqa: ask what it sees
- /autosteer: give it a goal and let a planner pick the steps
- record episodes while you steer
lerobot-rollout --interactive=true
No text-generating policy? Bring any VLM as the planner: transformers serve, vLLM or Inference Providers.
And FineART-VLA, trained on FineART from @scale_AI and Hugging Face (1,718 h, 533,913 subtask labels), says its next subtask out loud, then does it.
Docs in the comments 👇
lil’km retweeted
Infinite Factory 🏭
An endless robot factory, generated live in your browser with Wave Function Collapse.
Every robot, conveyor and box is a SimReady OpenUSD asset streamed from the HF Hub, rendered with huggingface/usd-web-viewer
Try it: hf.co/spaces/mishig/simready…
lil’km retweeted
LeRobot v0.7.0 is out, and this is a very big one 🦾
You can now talk/steer a robot while it’s running; change its task:
`/subtask pick up the red cube`
Ask what it sees:
`/vqa how many cubes are left?`
Or give it a higher-level goal:
`/autosteer clear the table`
and let the policy or an external VLM planner figure out the subtasks!
And this just scratches the surface of v0.7.0! Other banger features include 👇
Training directly on huge datasets from the Hub, Storage Buckets or @lancedb without downloading them first; particularly relevant when the Hub is already approaching !!300 TB of open Physical AI data!!
The training stack has also been rebuilt for 5-20B parameter world action models: FSDP2, HSDP, distributed checkpoints, FP16, EMA and more.
The model zoo keeps growing:
- 4 new world action models, including Cosmos 3 and FLUX 3 Action
- 4 new VLAs
- remote inference, so the robot can stay lightweight while the policy runs on a GPU server
- a common simulation stack that works much more like real hardware
LeRobot is becoming less about individual robot policies and more about having one open stack from TBs of demonstrations, all the way to a robot you can interact with in real time!
lil’km retweeted
Open-source is critical for robotics AI. It means a student, a researcher or a startup can build on years of progress and contribute something the whole community can use rather than concentrating everything in a few companies building a black-box robot.
That’s why we’re building @huggingface into the platform for open-source robotics AI! Amazing to see the progress we've made in the six months and it's accelerating!
- Almost 100,000 public robotics datasets on hf.co/datasets, up 48%
- Over 30,000 public robotics models, up 41%
- Microduck and Reachy mini, our open-source robots became the most popular robot of 2026 (probably ending the year with 50,000+ sold and 20,000+ shipped)
- We’re a founding partner of @RoboColosseum, evaluating robots across 140+ real-world scenes. Every run is published publicly on HF, including failures, so everyone can learn from what actually happens.
And @LeRobotHF got downloaded over 1M times and v0.7.0 just landed with new world action models, interactive robot control and remote GPU inference: huggingface.co/blog/lerobot/…!
Huge thanks to everyone making this happen 🤗🦾
lil’km retweeted
We integrated SimArena into @machinedotfun, so a project can show more than a pitch.
A simulation on the project page is a robot doing a task you can watch, before the hardware exists.
lil’km retweeted
How long does it take to build @asimovinc ?
Took me 22hrs, what surprised me the most is that it worked out-of-assembly, check belowvc.
Should I unveil the full timelapse of assembly? 🛠️