I am planting acorns one at a time with policy gradient.
London
Joined November 2011
- Tweets2.3K
- Following2.4K
- Followers3K
- Likes3.9K
Pinned Tweet
What can unsupervised representation learning learn from deep RL? As it turned out, we can learn representations (unsupervised) by making plans!
Check out our new work at geyang.github.io/plan2vec
People are underestimating the importance of sensorimotor learning.
We loved Eka’s demo, and want more people to be able to do the same.
so we figured that we should show what a few MIT undergrads are able to do in a week. This video was our first test.
@EkaRobotics has raised a $150M Series A.
Generality v/s Performance is the fundamental tension at the frontier of robotics. Generality without performance is just a prototype. Performance without generality is just a single-purpose tool.
Customers shouldn’t have to choose. Eka breaks the tradeoff. Powered by our Vision-Force-Action (VFA) model, we deliver the performance and versatility customers need to put robots to work every day.
Eka is built on three convictions:
⚡️Force is the language of manipulation.
🚀 Reinforcement learning is how dexterous manipulation scales.
🎂 System 1—fast, reliable physical execution—is the bottleneck to solve.
With this funding, we will scale our model and accelerate customer deployments.
Grateful to be building alongside a dream team and investors who share our ambition.
You got to take a look at the cut where the arm flew off together with the ball :P
A perfect example of how massive closed-loop training can do to physical systems.
2/9 To train it, we reset the simulator to a recorded hardware state, apply the same recorded commands, and learn command corrections so the simulated rollout matches the recorded hardware trajectory.
We previously used UAN on a quadruped's arm (uan.csail.mit.edu).
But motion with contact was tricky.
such insight, delivered in earnest.
Hi, I'm Felix. I'm an eng lead working on Claude. I love watching people take flight and do things they never thought possible.
We often describe that as "being more ambitious" when using AI, which is tricky advice to follow. I wrote a tutorial:
felixrieseberg.com/how-to-be…
Ge Yang retweeted
Introducing LPM 1.0 — a video-based character performance model that speaks, sings, listens, reacts, and emotes in real time.
- Generating full-duplex conversation, identity-consistent infinite-length generation, and nuanced human-like performance.
- Building across a co-designed data pipeline, Base model, Online model, and streaming inference optimization.
- Key advantages over other video generation models: performance quality, emotional conversation, precise lip-sync, identity preservation, and lifelike naturalness.
Turning an image into a performance video, LPM 1.0 serves as a visual engine for conversational agents, live streaming characters, and game NPCs.
Page: large-performance-model.gith…
Arxiv: huggingface.co/papers/2604.0…
This is one of those iconic designs that will go down in history, that is destined to be remembered.
Today, we present a step-change in robotic AI @sundayrobotics.
Introducing ACT-1: A frontier robot foundation model trained on zero robot data.
- Ultra long-horizon tasks
- Zero-shot generalization
- Advanced dexterity
🧵->
for the love of robots ❤️
Today, we present a step-change in robotic AI @sundayrobotics.
Introducing ACT-1: A frontier robot foundation model trained on zero robot data.
- Ultra long-horizon tasks
- Zero-shot generalization
- Advanced dexterity
🧵->
Ge Yang retweeted
Spent just over an hour collecting UMI data in Stanford, sent it to @harshgupta in CMU to run drone I've never seen before, and it just works 🤯 I was expecting to iterate on the data a few more times but embodiment guidance is so robust!
umi-on-air.github.io/ to learn more!
Ge Yang retweeted
I'm super excited to announce mjlab today!
mjlab = Isaac Lab's APIs + best-in-class MuJoCo physics + massively parallel GPU acceleration
Built directly on MuJoCo Warp with the abstractions you love.
Ge Yang retweeted
Excited to share our latest work MaskedManipulator (proc. @SIGGRAPHAsia 2025)!
With: Yifeng Jiang, @erwincoumans, @zhengyiluo, @GalChechik, and @xbpeng4
Really cool to see the g1 do this! Was expecting some punching involved 😆
This quoted post is unavailable.
Glad to play a role in driving VisionPro sells ☺️
Learning about active perception with @Haoyu_Xiong_ -- your robot needs a head, and to be able to control where it's looking, in order to perform complex tasks!
It is so cool to see vuer.ai in action
Many of today's data collection systems do not capture human perceptual behaviors. The observation mismatch—between what the human sees and what the robot learns from—hinders the learning of effective manipulation policies.
To see what the robot sees, we developed a VR interface—Async Teleop, which introduces decoupled view rendering to reduce motion sickness 😵💫🥴🤢 — an issue that is insufficiently addressed in prior VR teleop systems.
3/7
Check out this awesome demo!
We have been focusing on policy learning for robotics for a while. But can hardware be learned as well? Check out @yswhynot ‘s recent co-design work that learns what a soft gripper should be if we want to do better manipulation.
Ge Yang retweeted
Meet 𝐀𝐌𝐎 — our universal whole‑body controller that unleashes the 𝐟𝐮𝐥𝐥 kinematic workspace of humanoid robots to the physical world.
AMO is a single policy trained with RL + Hybrid Mocap & Trajectory‑Opt.
Accepted to #RSS2025.
Try our open models & more 👉 amo-humanoid.github.io
Congratulations! Incredible team and incredible traction. – at Somerville, MA
Excited to launch @DynaRobotics with a team of incredible researchers, engineers and company builders!
At Dyna, our mission is to bring affordable general-purpose AI robots to real production environments.
