Pinned Tweet
๐ค What if robots could reason not just in words, but through the dynamics of the physical world?
๐ Introducing ๐ง๐ต๐ถ๐ป๐ธ๐ช๐ผ๐ฟ๐น๐ฑ from @NVIDIAAI !
๐ก By reasoning through shared physical dynamics, robots could learn from human experience and transfer motion intelligence across embodiments.
๐๐ผ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐๐ต๐ฎ๐ถ๐ป-๐ผ๐ณ-๐๐๐ป๐ฎ๐บ๐ถ๐ฐ๐ (๐๐ผ๐): A shared motion reasoning space across human and robot embodiments.
โ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Imagines future world representations conditioned on CoD.
โ ๐ฅ๐ ๐ณ๐ฟ๐ผ๐บ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Reinforces reasoning through alignment between imagined and observed futures, enabling learning from human and robot videos.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Robust real-world manipulation under unseen objects and environments.
โ Human-guided skill composition for novel multi-step robot tasks.
โ Robust generalization under task and environment shifts in simulation.
๐ ๐ข๐๐ฟ ๐๐ถ๐๐ถ๐ผ๐ป: Scale generalizable motion intelligence through human videos toward a unified System-1 reasoning layer for Physical AI.
Paper, robot demos & more below! ๐
Fu-En (Fred) Yang retweeted
Robots can now perform an increasingly wide range of manipulation tasks. But how far are we from robot assistants that can operate autonomously in the open world?
We introduce RobotAgentBench (RAB), a comprehensive benchmark for evaluating open-world robot autonomy in realistic household environments.
We evaluate 8 VLA models, 3 WAMs, and diverse robot agent harness designs across five dimensions: task discovery, context reasoning, manipulation, failure recovery, and robustness. Four open-ended household scenarios test how these capabilities work together to fulfill high-level user needs.
What we find:
- Manipulation โ autonomy. Standalone VLA/WAM policies struggle with task discovery and open-ended scenarios. Agent harnesses substantially improve performance, but reliable open-world autonomy remains challenging.
- Hybrid Code-VLA/WAM Policy performs best. Combining learned policies with code-based control achieves the strongest performance among evaluated configurations.
- RSI further improves robot agents. Refining reusable tools and accumulating experience lead to additional gains across iterations.
- Best couple under our default agent setting: Antigravity CLI (Gemini 4 Argon) + InternVLA-A1.5. This pairing performs best among the planner-executor combinations we evaluated, achieving an open-ended score of 10.4, compared with 8.4 for fully code-based Codex (GPT-6 Astra) and 1.1 for standalone ฯ0.5.
Thanks @_akhaliq for sharing our work!
Paper: arxiv.org/abs/2610.37666
Webpage: thinkworld-project.github.ioโฆ
ThinkWorld
World Imagination Reinforces Chain-of-Dynamics Reasoning
paper: huggingface.co/papers/2610.3โฆ
Fu-En (Fred) Yang retweeted
ThinkWorld
World Imagination Reinforces Chain-of-Dynamics Reasoning
paper: huggingface.co/papers/2610.3โฆ
Fu-En (Fred) Yang retweeted
ThinkWorld is a vision-language-action model that reasons about robot motion in a space shared by human and robot bodies, then imagines future world states to check that reasoning.
Per the creators, reinforcement learning aligns the imagined futures with what videos actually show, so it can learn from both human and robot videos, and they report robust manipulation with unseen objects and environments.
Why it matters: learning from human videos could be a scalable way to teach robots new skills.
Paper: arxiv.org/abs/2610.37666
Credit: @FuEnYang1
For a deeper dive, check out Chaehyunโs posts below:
xitter.gq/ChaehyunK87776/status/โฆ
Introducing ThinkWorld ๐
Chain-of-Dynamics (CoD) reasoning gives a shared motion language. We reinforce this reasoning by comparing resulting imagined futures with observed outcomes.
๐ thinkworld-project.github.ioโฆ
๐งต The ideas, training, and results, step by step โ
Huge thanks to our amazing team for making ThinkWorld possible! ๐
Chaehyun Kim (@ChaehyunK87776 ), Chi-Pin Huang (@chipinhxyz ), Jisoo Kim, Min-Hung Chen (@CMHungSteven ), Yu-Chiang Frank Wang, Seungryong Kim, Fu-En Yang (@FuEnYang1 ) Grateful for this collaboration between @NVIDIAAI and @KAIST_AI ! ๐
Also accepted to NeurIPS 2026 RoboWM & PTA workshops. See you in Sydney! ๐ฆ๐บ
@nvidia
@NVIDIARobotics
Fu-En (Fred) Yang retweeted
Introducing ThinkWorld ๐
Chain-of-Dynamics (CoD) reasoning gives a shared motion language. We reinforce this reasoning by comparing resulting imagined futures with observed outcomes.
๐ thinkworld-project.github.ioโฆ
๐งต The ideas, training, and results, step by step โ
Fu-En (Fred) Yang retweeted
Internship Opportunity ๐ฃ
Weโre hiring Research Scientist Interns at NVIDIAโs newly established NVIDIA Research Lab in Singapore, Agentic Robot Intelligence Laboratory (ARIEL)!
Our focus is agentic robotics: building the next generation of agentic frameworks for robots.
nvidia.wd5.myworkdayjobs.comโฆ
Fu-En (Fred) Yang retweeted
After working on Dream-1 for the last 4 months, here's my take:
LLMs vs WAMs isn't the right framing. A model like Dream-1 will soon be a tool call for LLMs, and that gets more feasible as LLMs inevitably get faster and cheaper.
Two things we didn't put in the blog post๐งต๐
Fu-En (Fred) Yang retweeted
Thanks @_akhaliq for sharing our work!
๐ Try our interactive demo with your own LLM!
๐ฌ Demo: huggingface.co/spaces/uclanlโฆ
๐ Website: openvlharness.github.io
๐ Paper: arxiv.org/abs/2610.26860
๐ป Code: github.com/OpenVLHarness/Opeโฆ
OpenVLHarness - a Hugging Face Space by uclanlp
Visual tool-use agent on your own model
huggingface.coOpenVLHarness
Real World Visual Agentic Tool-Use with Multimodal Memory
paper: huggingface.co/papers/2610.2โฆ
Fu-En (Fred) Yang retweeted
๐ Introducing OpenVLHarness:
Turn any LLM into an evidence-based reasoner that thinks and codes in visual space.
๐ GPT-6 Astra gains +9.9 points on average over 22 datasets.
โก Beats Codex and other VL harnesses by 7.7โ8.5 points, at โค37% of their cost.
๐งต More details โ
Fu-En (Fred) Yang retweeted
NVIDIA Cosmos world models have come to life with real-time interactivity! ๐๐ค๐
We are releasing our post-training framework for real-time interactive world simulation for Physical AI. The framework builds upon Cosmos 3 through causal training, distillation, and optimized streaming inference. The post-trained models stream visual observations via autoregressive generation in response to incoming actions and scene controls.
We are also releasing example model checkpoints for robotics, autonomous vehicles, and sim-to-real transfer. The release includes optimized streaming inference code to power the models.
Cosmos world foundation models provide broad visual knowledge. Each simulation setting adds its own controls, data, and evaluation requirements. Our shared recipe and example models give researchers and developers a practical starting point for building specialized interactive world simulators.
Huge shoutout to the team for the dedication, creativity, and hard work poured into this release! I am incredibly proud of this work and feel fortunate to lead such a talented team. A special thank-you to the broader Cosmos group for building the world foundation models together. Team green ๐
I am excited to see how you build your own interactive world simulator with Cosmos!
๐ Project website: research.nvidia.com/labs/cosโฆ
๐ค Model checkpoints: huggingface.co/collections/nโฆ
๐ป Code: github.com/NVIDIA/cosmos
Fu-En (Fred) Yang retweeted
Recently, thereโs been some "P(doom)" in the robot learning community around the idea that LLMs will solve robotics. I believe LLM agents are great for planning, but handling dynamics and dexterous manipulation still requires a fast System 1โor System 0.
Inspired by @typesafeai JEVโs release of a fast System 1 model, we hope this result offers another perspective on why System 1 still mattersโand why thereโs still so much worth pursuing in robotics research.
We partnered with @robocurve @chooi_jeq , an independent evaluation company, to benchmark Dream-1. It achieved the highest score among the models evaluated on RoboDojo-RC Tier 1: 59, while being faster and cheaper. โก
Regardless of VLA, WAM, LLM, etc, they can all be complementary. Ultimately, Iโm glad to see the community keeping an open mind and exploring different ways to solve robotics. Ultimately, the objective matters more than any particular approach: making robots more accessible.
Dream-1 by @dreamlabs_ai is the first model to complete "Stand Up Bottles", a difficult robotics task that frontier LLMs struggle with.
Fu-En (Fred) Yang retweeted
Introducing Flora, an open model suite for scaling robotics data engines.
It watches robot and human video and annotate what happened (timestamped instructions) and how well it went (rewards and failure reasons).
๐ย research.nvidia.com/labs/geaโฆ
We use it to train NVIDIA GR00T. ๐งต
Fu-En (Fred) Yang retweeted
Today, Dream Labs is coming out of stealth with Dream-1, a robot foundation model that controls bimanual robots it has never seen, on tasks it was never shown, in places it has never been, from $3/hour.
Dream API is live for everyone, today. ๐งต
Fu-En (Fred) Yang retweeted
Today, Dream Labs is out of stealth.
In February 2026, while we were at NVIDIA, @scott_e_reed @youliangtan @GeYunhao and I started calling a new kind of robot model a World Action Model, one that learns how the world moves from video, then uses that to decide how to act.
Today, weโre releasing Dream-1 through a public API that anyone in the world can use.
When you rent a car youโve never driven, you donโt learn to drive all over again. Within a few minutes, it feels like your own. Robots should work the same way: a good model should be able to pick up a new robot and get to work, without being retrained from scratch.
Thatโs what weโve spent the last 6 months on, and itโs what Dream-1 does. It controls bimanual robots with parallel grippers it has never seen before, zero-shot, on unseen tasks in unseen environments.
Generalization has been my North Star in robotics: first by distilling video world models into VLAs (DreamGen), then by learning to act from those world models directly (DreamZero), and now by transferring across robots (Dream-1). Iโve come to believe itโs the only direction that lasts. Pushing a narrow model from 90% to 99.9% on a single task is real work, but much of it gets washed away once a model that generalizes arrives.
Since the start of the company weโve stayed single-minded about generalization.That focus is what got Dream-1 to control robots it was never trained on, on tasks it was never shown. And itโs what makes a public API possible at all: a model that only does one task on one robot canโt serve millions of users.
Dream-1 is still a research beta, and it will make mistakes. But it already drives cars itโs never seen.
Generalization wins in the end, and the end is closer than it looks.
Hereโs Dream-1 taking the wheel of a robot itโs never seen. Full details ๐
Fu-En (Fred) Yang retweeted
Gemini 4 Argon taught a robot to play cornhole just by watching a video.
It mapped the physics, coded its own simulation, and taught @Apptronik's Apollo humanoid to sink 7 throws in a row.
Fu-En (Fred) Yang retweeted
We are open-sourcing V1.0 of our agentic robotics framework, ASPIRE, with full real robot skill discovery and self improvement capabilities! On the day Astra arrived, our lab switched the backend model and saturated all the benchmarks to ~99% over night.
And the frontier is already on the move:
- Opus 5.5 slightly edges out Astra on our robot simulation bench.
- Sonnet 5.5 hits the Pareto frontier of cost vs. robot capability, $12/task.
We first introduced ASPIRE and /robot skill library in June, way before the big models could reliably do robotics. A rising frontier tide lifts all boats, but Astra was a 0->1 tsunami on many tasks.
Now you can also feel the Physical Bitter Lesson at home like we did on that fateful day ;)
links in thread