
Intern @ Amazon FAR | CS PhD student @UofIllinois | Previously @Tsinghua_IIIS and XJTU | robotics & ai
San Francisco, CA
Joined April 2020
- Tweets179
- Following1.6K
- Followers616
- Likes1.7K
Pinned Tweet
Thrilled to announce that HERO has been accepted to @corl_conf 2026🤠! Congrats to every contributor!
I will be attending CoRL in person, and see you guys at Texas and let's talk about humanoids!
Want to ask your humanoid to get novel objects you want from a novel table? We introduce HERO, first achieving open-vocabulary visual loco-manipulation using human language queries! Now we can ask the humanoid to see and grasp the target object (eg, a carrot instead of a tissue box) with whole-body coordination.
How do we do it? Key designs and findings in this work:
We propose a residual-aware end-effector tracking policy that tracks the target accurately in a closed loop.
We find that the robot's forward kinematics are quite inaccurate, and thus we propose residual neural forward models that correct the end-effector FK and base leg odometry.
We design a modular system powered by visual foundation models that achieves generalizable grasping capabilities with 83% success rate on novel daily objects and daily scenes.
Check out our new project:
Learning Humanoid End-Effector Control for Open-Vocabulary Visual Loco-Manipulation
Project page: hero-humanoid.github.io/
Paper: arxiv.org/pdf/2602.16705
Runpei Dong retweeted
Day 1 of #CoRL2026 and the reading list is already huge!
What if you can connect different works and have easy summaries.
Introducing Orbis 🌌
Read, take notes, discuss with your own AI agent, and turn your library into a universe of ideas.
Open source. Your research with your agents!
🧵 1/8
Zero-shot robot learning is always what we want but challenging to solve. This is a great work showing that self-improvement of VLAs with test-time rollouts can achieve promising results!
Ever faced embodiment gaps deploying robotic foundation models zero-shot on your hardware? As soon as we fine-tune them, they start to forget generalist priors, while pretraining data is proprietary.
So how do we get a multi-task manipulation policy on our robots without recollecting each task? Super excited to shared "self-demonstrated generative control"!
Just rollout the policy on your robot, fine-tune with its self-supervised rollouts to (1) distill pretraining tasks, (2) boost expert task performance, (3) preserve held-out tasks! 🧵
3
7
449
We just released HALO — Perceptive Humanoid Motion from Simulation.
Trained entirely on synthetic data from 1,000 procedurally generated homes. Zero-shot deployment in 6 unseen real environments. 91% success rate across 360 physical trials. Fully onboard.
Paper, code, and project page are all live.
apple-aiml-research.github.i…
Runpei Dong retweeted
🤖 A reference motion tells a humanoid where to move, but leaves the interaction underspecified. We argue force is a missing interface.
💡 Introducing Ctrl-F: Force as an Interface for Humanoid Loco-Manipulation. [1/6]
🌐 forcectrl.github.io
📄 arxiv.org/abs/2611.05507
When a humanoid interacts with the world, force is one of the most important information that drives the physics. It’s interesting to see this incredibly effective work that enables force-sensitive humanoid loco-manipulation!
💪 We teach humanoids where to move and how hard to apply force, using human demonstrations collected without a robot.
Introducing 🌺 FLoRe: Learning Force-Reactive Humanoid Loco-Manipulation.
Our policy learns to predict actions and forces for each hand. A hybrid compliance whole-body controller tracks these commands while the robot walks and balances.
🧵 (1/6)
Very cool
8 hours on the production line. 🤖
Xiaomi’s full-size humanoid robot takes on an 8-hour task at the Center Console Side-Cover Sorting Workstation at Xiaomi Auto Factory.
Beyond sustained operation, it can autonomously detect and correct anomalies while coordinating with the production line system — another step toward real-world industrial deployment.
#XiaomiRobotics #EmbodiedAI
4
299
Runpei Dong retweeted
Can you believe these are open-loop rollouts?
Excited to introduce CARD 🃏 — Contact-Anchored Retargeting for Dexterous Manipulation.
Almost every method today is limited to kinematics alone as the 𝐢𝐧𝐭𝐞𝐫𝐟𝐚𝐜𝐞 between the human and the robot. With CARD, we track real world contact and forces alongside kinematics to perform unprecedented dexterous tasks through tactile reactive replay!
CARD is scalable:
📹 Multiple input sources
🤖 Multiple robot hands
⚡ needs no object models, no simulator, 10 Hz
Runpei Dong retweeted
Introducing DreamFAR, a world model trained from scratch using only robot data, without internet pretraining. It supports:
🎮 Real-time teleoperation
⏱️ Minutes of long-horizon rollout
🧩 Generalization across 1,000+ tasks
yimingdou.com/DreamFAR
🧵 (1/8)
Runpei Dong retweeted
Collecting data for dexterous hands is hard. Teleop needs a full setup. Bringing UMI-like devices to 20-dof hands is challenging.
Introducing DexPort 🖐️: a hand-held interface w/ a real robot hand. No embodiment gap, recorded joint state AND actions, easy to use.
Runpei Dong retweeted
RPL code is released at github.com/amazon-far/RPL, including experts training, multi-depth distillation and sim2sim. If you have an Xbox controller lying around, it’s a lot of fun playing it in Mujoco.
Its perceptive features were adopted by some cocurrent work like LadderMan (ladderman-robot.github.io/) and PHP (php-parkour.github.io/) with promising results.
RPL was also accepted to CoRL2026. See you in Austin.
Robust humanoid perceptive locomotion is still underexplored. Especially when different cameras see different terrains, paths get narrow, and payloads disturb balance...
Introduce RPL, tackling this with one unified policy:
• Challenging terrains (slopes, stairs and stepping stones);
• Multiple directions;
• Payloads;
Trained in sim. Validated long-horizon in the real world.
Watch the robot walk it all🦿
Details below👇
Runpei Dong retweeted
Your navigation model already found the goal. It just doesn't know when to stop.
Introducing StopWatch, a navigation agent that grounds the goal explicitly before every action. It sets a new state of the art on six object-goal navigation benchmarks.
Watch the flower vase episode below. Both agents reach the goal. StopWatch stops there. LightNav-0 turns away and keeps going.
Code, models and data are open source 👉 stopwatch-nav.github.io/
Work done at Amazon FAR w/ @yunshengtian @QiuYuhengQiu @pabbeel @peterxichen @GuanyaShi @simonforrest
🧵 [1/n]
Runpei Dong retweeted
Can we make dexterous robots compliant without sacrificing precise control?
Excited to introduce FOMO-Dex: Force-Modulated Dexterity! 🖐️
FOMO-Dex brings force-aware compliance to dexterous teleoperation, enabling robots to pick up thin cards and chips 🃏, cut paper 🔪, turn nuts 🔩, and use screwdrivers 🪛.
These force-aware demonstrations also train better autonomous policies!
🎥 Videos & paper: yingyuan0414.github.io/fomo-…
🧵👇 (1/8)
Runpei Dong retweeted
My new video on ai and design is now live on @Microsoft’s official website!
I believe designing intelligence means shaping humanity.
In the video, I talk about:
- Building @copilot into a thinking partner
- Designing beyond interfaces: models, memory, evaluation...
And a question I keep coming back to: as ai reshapes our world, how can we live fully, creatively, and meaningfully?
Runpei Dong retweeted
Perceptive Humanoid Parkour (PHP) is now OPEN-SOURCE! 🔥
Try training your own parkour robot! 🤖
🔗 Project: php-parkour.github.io/
💻 Code: github.com/amazon-far/php_pa…
🎮 Try it in your browser: php-parkour.github.io/demo.h…
Runpei Dong retweeted
🤖 For parallel-jaw grippers, high-level plans can already be executed with mature primitives such as grasping. What about dexterous hands?
Introducing GOTT (General Object Trajectory Tracking): object trajectories to dexterous execution with a reusable cross-embodiment contact primitive.
The same primitive supports diverse real-world tasks including sweeping, pouring, whiteboard erasing, and hammering.
With one checkpoint, GOTT reaches 90-96% grasp-and-lift success across multiple objects on three real-world arm-hand platforms.
Check videos 👇
Runpei Dong retweeted
From 2D tracks to 3D tracks... what's next for video motion estimation? 🎬
Check out MoSE3, our new framework for dense, world-space SE(3) motion estimation from monocular video!
Beyond where points move, MoSE3 predicts how they move: full 6DoF transformations (rotation + translation) plus motion grouping, for rigid, articulated, and deformable objects.
The result is a structured, actionable motion representation for dynamic scenes. It could be useful for robot learning and manipulation.
Excited to share that MoSE3 was accepted as #NeurIPS2026 Spotlight!
MoSE3 estimates an SE(3) transform at every pixel from monocular video, all in a shared world coordinate frame and in a single forward pass. It captures rigid, articulated, and deformable motion.
1/
Runpei Dong retweeted
Practice makes perfect. A truly agentic robot shouldn’t just know what to do. It should also figure out how to get better.
What if a robot could watch a demonstration, figure out what it needs to practice, and learn how to get better through practice?
Introducing RPG: Guided Self-Improvement for Embodied Agents, a real-to-sim-to-real pipeline that turns real-world demonstrations into targeted practice.
RPG reconstructs practice tasks from demonstrations, uses demo videos to guide skill improvement, and continually updates a shared skill system from failures.
On real robots, RPG achieves 30/30 successes across 3 tasks. In simulation, it reaches 95.0% success across 22 manipulation tasks after 15 rounds of self-improvement.
Demo → Reconstruct → Guided Practice → Self-Improve → Go Real
Okay, now mathematical research scales.
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
424























