PHD at Shanghai Institude of Technology (SII) & HIT Research Intern at Roboparty Lab AI & Robotics
China
Joined February 2023
- Tweets45
- Following315
- Followers123
- Likes107
Pinned Tweet
Can humanoid motion generation and control be unified in a single policy?
PredActor uses a single end-to-end diffusion policy rather than separate motion generation and tracking. With proprioceptive feedback, it predicts internal future states and directly outputs actions at 50 Hz onboard a Unitree G1. Text-conditioned motion, joystick steering, disturbance response, and semantic interpolation are supported in the single policy.
Project Page: masteryip.github.io/predacto…
Lei Ye (Raymon Yip) retweeted
With @JayHe748646 @KehanWen170077 @leggedrobotics , we push humanoid mobility to next frontier.
Achieved by a generalist contact planner + a generalist tracker + a generalized neural mapping, warmstarted from few human data.
Website: leggedrobotics.github.io/ACP…
Promo Video 👇
Lei Ye (Raymon Yip) retweeted
We’re excited to announce the official release of NVIDIA Isaac Lab 3.0. 🎉
Isaac Lab 3.0 brings a new modular and lightweight architecture along with major new capabilities across the robot learning workflow: multi-physics and multi-rendering support, training at scale, and streamlined deployment on real hardware.
With Newton integration, the new OVRTX renderer, RL post-training, and LEAPP deployment, there’s a lot to explore.
Here’s a look at what’s new 🧵
(1/7)
Lei Ye (Raymon Yip) retweeted
How can we represent whole-body interaction for humanoid robots—compactly enough for low-frequency planning, yet richly enough to preserve high-frequency feedback control?
Introducing FIBER! 🤖
FIBER learns a unified latent interface between high-level planning and whole-body control, enabling a humanoid to acquire diverse interaction behaviors from heterogeneous experts through a pretrained, frozen Behavior Foundation Model (BFM).
With a single planner and BFM, we demonstrate heavy-door pushing, box carrying, object grasping, controllable navigation, and more!
🌍 Website & Live Demo: fiberaa.github.io
📄 Paper: arxiv.org/abs/2610.24675
A thread on the ideas behind FIBER 🧵👇
Lei Ye (Raymon Yip) retweeted
The @Tesla Roadster stunts, simulated in MuJoCo! Thrusters are site actuators, the ramp suction uses adhesion actuators, and the builtin box QP decides how hard to fire each jet.
The intent representation and S0's responsiveness are crucial for a general humanoid control interface.
In PredActor, unified diffusion for motion and action generation proves responsive, reminding me that the MPC-like paradigm remains worthwhile in the Astra era.
My blog on System 0 in the Astra era:
As spatial grounding (S1/2) improves, we need dynamics-grounded controllers (S0) and expressive interfaces that let Astra steer physical interaction.
Hope this resonates or sparks disagreement. I’d love your thoughts!
haoyangweng.pages.dev/blog/2…
Like the C550 metioned in the wandering earth II: Self-organizing, self-adaptive, self-sensing, and self-programming.
cool work.
Robotics has spent years scaling demonstration data and model size. What happens when we scale EXPLORATION for robot agents?
We're incredibly excited to share RAX, an agentic robotics system that learns by scaling online exploration.
A coding agent connects directly to the robot’s perception and control APIs, learns from feedback, and builds programs that perceive, plan, and act. The model stays frozen. As exploration scales, the agent improves its programs and builds up reusable skills.
The results:
• ZERO teleoperation or action demonstrations on real robots—from insertion with 0.1 mm clearance and air-fryer manipulation to deformable objects and long-horizon mobile whole-body control.
• +37 percentage points on average over leading learned policies across five simulation benchmarks: RoboDojo, BEHAVIOR-1K, HumanoidArena, DexJoCo’s bimanual tasks, and LIBERO-Pro.
• One framework spanning single-arm, bimanual, dexterous, humanoid, and mobile robots, with new embodiments connected through APIs.
The craziest part: on each real-world embodiment, going from scratch to mastering the tasks took roughly two hours. No teleop. No model finetuning.
We’re open-sourcing the full framework AND every experiment session. We can’t wait to see what people build with it!
Project + sessions: robot-agent-exploration.com/
Paper: assets.robot-agent-explorati…
GitHub: N
Project fully led by @RunhanH and @xczhangxx who are absolutely awesome.
How it works, experiments, and robot videos below 🧵
(1/N)
Lei Ye (Raymon Yip) retweeted
Robotics has spent years scaling demonstration data and model size. What happens when we scale EXPLORATION for robot agents?
We're incredibly excited to share RAX, an agentic robotics system that learns by scaling online exploration.
A coding agent connects directly to the robot’s perception and control APIs, learns from feedback, and builds programs that perceive, plan, and act. The model stays frozen. As exploration scales, the agent improves its programs and builds up reusable skills.
The results:
• ZERO teleoperation or action demonstrations on real robots—from insertion with 0.1 mm clearance and air-fryer manipulation to deformable objects and long-horizon mobile whole-body control.
• +37 percentage points on average over leading learned policies across five simulation benchmarks: RoboDojo, BEHAVIOR-1K, HumanoidArena, DexJoCo’s bimanual tasks, and LIBERO-Pro.
• One framework spanning single-arm, bimanual, dexterous, humanoid, and mobile robots, with new embodiments connected through APIs.
The craziest part: on each real-world embodiment, going from scratch to mastering the tasks took roughly two hours. No teleop. No model finetuning.
We’re open-sourcing the full framework AND every experiment session. We can’t wait to see what people build with it!
Project + sessions: robot-agent-exploration.com/
Paper: assets.robot-agent-explorati…
GitHub: N
Project fully led by @RunhanH and @xczhangxx who are absolutely awesome.
How it works, experiments, and robot videos below 🧵
(1/N)
Lei Ye (Raymon Yip) retweeted
Can humanoids adapt to new terrain using only flat-ground references? 🤖
Meet P-BFM! 🥳
We learn a terrain-agnostic intent space with unsupervised RL and forward-backward representations, paired with a terrain-aware policy that adapts across diverse terrains—without terrain-specific demonstrations.
📄🎬💻 Check out our paper, video, and code here!
🌐 roboparty.github.io/P-BFM/
Lei Ye (Raymon Yip) retweeted
Excited to share 𝗕𝗥𝗔𝗖𝗘: a whole-body tracker for humanoids that can exert forces from diverse poses and adapt to varied terrains.
Humanoid whole-body trackers are typically trained to follow references recorded on flat ground, carrying nothing. As a result, they know 𝗻𝗲𝗶𝘁𝗵𝗲𝗿 𝘁𝗵𝗲 𝗳𝗼𝗿𝗰𝗲𝘀 𝘁𝗵𝗲 𝗿𝗼𝗯𝗼𝘁 𝗺𝘂𝘀𝘁 𝗲𝘅𝗰𝗵𝗮𝗻𝗴𝗲 𝘄𝗶𝘁𝗵 𝗼𝗯𝗷𝗲𝗰𝘁𝘀 𝗻𝗼𝗿 𝗰𝗮𝗻 𝘁𝗵𝗲𝘆 𝗮𝗱𝗮𝗽𝘁 𝘁𝗼 𝗱𝗶𝘃𝗲𝗿𝘀𝗲 𝘁𝗲𝗿𝗿𝗮𝗶𝗻𝘀. BRACE closes both gaps in a single tracker.
Our key innovation is a 𝗿𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲-𝗮𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻 𝗳𝗼𝗿𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻. Given humanoid reference trajectories, we adapt them to the terrain and the commanded force, then train the controller on these adapted references. Tracking them naturally produces the stance and bracing needed to handle the terrain and exert the force (or compensate for external loads). 𝗧𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁𝗶𝗻𝗴 𝗰𝗼𝗻𝘁𝗿𝗼𝗹𝗹𝗲𝗿 𝗱𝗲𝗽𝗹𝗼𝘆𝘀 𝗱𝗶𝗿𝗲𝗰𝘁𝗹𝘆 𝗼𝗻 𝘁𝗵𝗲 𝗿𝗲𝗮𝗹 𝗿𝗼𝗯𝗼𝘁.
To know more, please visit our:
Project page: multiplylabs.github.io/brace…
Interactive demo: multiplylabs.github.io/robog…
More details in this thread🧵
1/n
This is what i really want to do before. the way they collect data is eye-catching.
super.
Perceiving the environment, understanding spatial constraints, and coordinating whole-body motion are essential for humanoid robots to integrate into human society—and remain major challenges in robotics.
To address this, the Galbot team introduces PASSAGE, a framework that combines environmental perception, whole-body motion planning, and closed-loop control, enabling humanoid robots to autonomously plan and execute flexible traversal motions based on their surroundings.
Built on 100 hours of human data across 1,500 scenes, it achieves 98.67% success on the Clutter Scene Traversal benchmark.
#PASSAGE #Galbot #embodiedAI #humanoidrobot
Lei Ye (Raymon Yip) retweeted
Love how easy it is to swap the hands and head, the remote emergency stop, and the ability to change the battery without turning the robot off. 🦾🔋
Best of all, the design files are fully open source—a real sign of trust in the developer community. It’s been amazing to watch this team build it from the ground up, step by step! 🚀
🚀RP1 is officially here.
🌍Putting control of robotics back in developers’ hands.
Introducing RP1 (ROBOTO 01) by RoboParty—a high-performance, fully open-source humanoid robotics platform for frontier robotics research and embodied AI model development.
🤖 Open across the full hardware and software stack:
• Robot platform: mechanical CAD, bill of materials, and assembly documentation
• Actuators: in-house ROMOMO joint designs and related technical resources
• Control: electrical systems, firmware, low-level drivers, and motion control implementations
• Development: simulation models, SDKs, data processing tools, and training tools
• Engineering foundations: technical documentation and research materials
Open the black box.
Understand the system.
Make it your own.
A deviation in motion can originate in the model, control policy, actuator response, or mechanical structure.
RP1 opens up the design and implementation of these layers so developers can trace problems, modify the system, and validate changes on real hardware.
Open-source algorithms for advanced motion control
RP1 has demonstrated continuous walking, recovery from disturbances, getting back up after a fall, and human motion tracking.
UFO and MimicLite connect motion learning, control policies, and execution on real hardware.
The open toolchain connects data collection, simulation training, hardware deployment, and experimental validation—supporting joint iteration across models, control, and hardware.
From RPO to RP1, we’re turning one team’s engineering experience into a starting point for many more.
Opening the future of embodied AI to everyone ready to explore it.
🔗RP1 on GitHub:
github.com/Roboparty/RP1
#RoboParty #RP1 #OpenSourceRobotics #EmbodiedAI
Lei Ye (Raymon Yip) retweeted
🚀RP1 is officially here.
🌍Putting control of robotics back in developers’ hands.
Introducing RP1 (ROBOTO 01) by RoboParty—a high-performance, fully open-source humanoid robotics platform for frontier robotics research and embodied AI model development.
🤖 Open across the full hardware and software stack:
• Robot platform: mechanical CAD, bill of materials, and assembly documentation
• Actuators: in-house ROMOMO joint designs and related technical resources
• Control: electrical systems, firmware, low-level drivers, and motion control implementations
• Development: simulation models, SDKs, data processing tools, and training tools
• Engineering foundations: technical documentation and research materials
Open the black box.
Understand the system.
Make it your own.
A deviation in motion can originate in the model, control policy, actuator response, or mechanical structure.
RP1 opens up the design and implementation of these layers so developers can trace problems, modify the system, and validate changes on real hardware.
Open-source algorithms for advanced motion control
RP1 has demonstrated continuous walking, recovery from disturbances, getting back up after a fall, and human motion tracking.
UFO and MimicLite connect motion learning, control policies, and execution on real hardware.
The open toolchain connects data collection, simulation training, hardware deployment, and experimental validation—supporting joint iteration across models, control, and hardware.
From RPO to RP1, we’re turning one team’s engineering experience into a starting point for many more.
Opening the future of embodied AI to everyone ready to explore it.
🔗RP1 on GitHub:
github.com/Roboparty/RP1
#RoboParty #RP1 #OpenSourceRobotics #EmbodiedAI
Lei Ye (Raymon Yip) retweeted
I saw many people say this is trivial (at least on the idea level).
But if you have ever tried any adaptive curriculum thing before (like simple monotonic succ-rate scheduling, or progress-based ones), you will be amazed that how SGS is way more stable, with such a weight curve.
(3/5) So how does pure RL find these behaviors? Scale, but scale alone isn’t enough.
With uniform task sampling, adding more parallel environments often doesn’t help: most experience goes to tasks the robot has already mastered or tasks still far beyond its current capabilities.
Key Idea: Feed the policy a balanced data diet. SGS tracks the success rate of every task configuration and resets more often to the ones the policy is just starting to solve. The result is RL that keeps improving as we scale up to a million parallel environments.
Lei Ye (Raymon Yip) retweeted
Google’s Fruit Fly Brain Map Becomes an #AIAgent That Controls Robots
by @ZentrixHQ
#ArtificialIntelligence #AI #Neuroscience #BrainComputerInterface
Lei Ye (Raymon Yip) retweeted
JEPAs & BFMs folks, Remember INTACT in July?
From JEPA into RL: meet LOGO in I-BFM 🤖
Local intent. Long-horizon purpose. Two horizons, one actor.
G1 MuJoCo carry success: 51.0% → 94.3% with LOGO.
🎬 No retraining. Just use next reward, next action.
How it works + paper & code 👇
Lei Ye (Raymon Yip) retweeted
🤖 Coming soon. 🚀
RP1 — The fully open-source humanoid robot.
Built for developers. Born for intelligence.
#RoboParty #RP1 #OpenSource