Rutav (@rutavms) profile picture

🤖🧠 Ph.D. @UTCompSci

Joined June 2019
Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)Photo by Rutav (@rutavms)
Intelligent humanoids should have the ability to quickly adapt to new tasks by observing humans Why is such adaptability important? 🌍 Real-world diversity is hard to fully capture in advance 🧠 Adaptability is central to natural intelligence We present MimicDroid 👇 🌐 ut-austin-rpl.github.io/Mimi…
7
38
15
126
43,634
I’m hiring the first PhD students to establish the Zurich Embodied AI Lab at ETH, starting August 2027! We’ll work on general-purpose embodied intelligence for humanoid robots and dexterous manipulation. If you’d like to help shape the lab from day one, I’d love to hear from you. Application link below ⬇️
6
72
10
598
32,364
Subtle changes in geometry or friction can drastically change contact dynamics. Can robots generalize to contact conditions never experienced? FACE generalizes across unseen surfaces and shapes with the right representation. 🌐 rcilab.khu.ac.kr/face 👇 See more🧵(1/4)
2
12
4
65
6,133
Excited for #CoRL! I’ll present PanoVine🌱at today’s poster session 5-6:30pm, give spotlights on my latest loco-manipulation work at the contact-rich loco-manipulation & umi arena workshops, plus a HoMMI device exhibition! I’m also on the faculty job market—let’s connect 🙌
🌱 How do you control a robot whose body is constantly growing, buckling, and reshaping? 📷 Put 19 cameras on it. Meet PanoVine, the first autonomous vine robot system. We distribute 19 cameras along a 6 meters, 7-DoF soft growing vine robot, giving it whole-body visual feedback of both itself and its surroundings. Using these observations, we train an end-to-end visuomotor policy that autonomously navigates branches, slopes, gaps, turns, and obstacles. 🔗 panovine-bot.github.io
5
3
57
2,120
Can you believe these are open-loop rollouts? Excited to introduce CARD 🃏 — Contact-Anchored Retargeting for Dexterous Manipulation. Almost every method today is limited to kinematics alone as the 𝐢𝐧𝐭𝐞𝐫𝐟𝐚𝐜𝐞 between the human and the robot. With CARD, we track real world contact and forces alongside kinematics to perform unprecedented dexterous tasks through tactile reactive replay! CARD is scalable: 📹 Multiple input sources 🤖 Multiple robot hands ⚡ needs no object models, no simulator, 10 Hz
11
59
27
280
48,432
Thrilled to share CARD: Contact-Anchored Retargeting for Dexterous manipulation. So far, collecting robot data, teleop, UMI, kinesthetic teaching, has run into the same two problems: 1) the mapping is largely kinematics only, and 2) the human has to act through the robot and hope the interaction works out. CARD instead treats contact and forces as the interface, from recording to replay, which is what lets it handle dexterous tasks that motion-only methods struggle with. 🧵 below 1/
Can you believe these are open-loop rollouts? Excited to introduce CARD 🃏 — Contact-Anchored Retargeting for Dexterous Manipulation. Almost every method today is limited to kinematics alone as the 𝐢𝐧𝐭𝐞𝐫𝐟𝐚𝐜𝐞 between the human and the robot. With CARD, we track real world contact and forces alongside kinematics to perform unprecedented dexterous tasks through tactile reactive replay! CARD is scalable: 📹 Multiple input sources 🤖 Multiple robot hands ⚡ needs no object models, no simulator, 10 Hz
6
15
5
53
5,853
Rutav retweeted
Common belief: Robot policies need continuous action heads; discrete actions are 𝐥𝐞𝐬𝐬 𝐧𝐚𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐥𝐞𝐬𝐬 𝐩𝐫𝐞𝐜𝐢𝐬𝐞. ❗We find policies w/ the simplest discrete representations 𝐦𝐚𝐭𝐜𝐡 𝐨𝐫 𝐛𝐞𝐚𝐭 continuous ones—in sim & real, in single-step & diffusion/flow settings. Here's why 🧵
13
49
12
288
31,243
Another way to think about our work: learn to act (pre-training) and compress past experiences separately, then bring the two together. It's a simple way to give pretrained models memory. Otherwise, learning both together can lead to shortcuts Amazing work led by @yilinwu!
What to remember in the past to act—spatial, temporal or motion details? We often assume what’s useful, but that’s hard to know in advance. Meet IMPRINT 🧠: retain rich memory first, then learn to use it for control. Compact latents keep inference fast as history grows. 🧵 1/10
1
18
766
🤖 For parallel-jaw grippers, high-level plans can already be executed with mature primitives such as grasping. What about dexterous hands? Introducing GOTT (General Object Trajectory Tracking): object trajectories to dexterous execution with a reusable cross-embodiment contact primitive. The same primitive supports diverse real-world tasks including sweeping, pouring, whiteboard erasing, and hammering. With one checkpoint, GOTT reaches 90-96% grasp-and-lift success across multiple objects on three real-world arm-hand platforms. Check videos 👇
12
36
7
144
14,655
What is the role of simulation in the era of Astra? Traditionally, in sim2real transfer, we transfer data or policies, which requires relatively accurate simulation. What excites me about “agentic sim2real” is that we can also transfer tools, knowledge, and procedures. A roughly correct simulator can give agents a place to test their physical intuition, run experiments, and build reusable capabilities. That’s the idea behind SimEX, led by @JiahengHu1: develop capabilities through simulated experimentation, then use a few real-world trials to refine both the tools and the simulator.
We’ve all seen the cool “Astra for robotics” demos. But behind the scenes? Slow rollouts, hand-engineered APIs, and/or simulated behaviors that struggle on real robots. How do we make coding agent actually work in the physical world? 🦾 Introducing SimEX: Simulation-Integrated Robotics AutoResearch 🧵 arxiv.org/abs/2609.38982
5
12
1
131
11,959
We’ve all seen the cool “Astra for robotics” demos. But behind the scenes? Slow rollouts, hand-engineered APIs, and/or simulated behaviors that struggle on real robots. How do we make coding agent actually work in the physical world? 🦾 Introducing SimEX: Simulation-Integrated Robotics AutoResearch 🧵 arxiv.org/abs/2609.38982
17
60
8
299
33,439
Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids. - learned directly from human manipulation data - no teleop/robot data - close to human-level dexterity and efficiency - multi-robot collab
156
478
292
3,509
1,788,257
Who Moved The Coke?
2
4
36
3,511
GPT-6 Astra scores 46% vs 12% for MolmoAct2, a state-of-the-art robotics VLA, across 200 trials on five bimanual tasks. That's 3.9x higher. 🧵
65
86
47
998
217,527
We have a huge news to share today! Today we are unveiling the first truly accessible RL robot - welcome Microduck A 25 cm tiny open-source biped with 15 actuators and packed with sensors (camera, speaker, LiDAR, NFC, bluetooth, wifi, etc) that you train yourself with reinforcement learning. It's also playable out of the box with more than half a dozen fun and playful pre-trained policies to have it walk, sit, crouch, roller-skate, pick up objects with its articulated beak, and recover on its own. And all for less than $400. See all the details, play with the simulator and order it at: pollen-robotics.com/microduc… (video with sound on 🔊)
589
914
749
8,176
2,927,083
"The generalization you want in your VLA has to be in the expert demos!" folks say. "Didn't we train VLM backbones on trillions of tokens already for it?", we ask. We set out to reclaim some of this (free) OOD generalization in VLAs. Just need to think beyond BC! 🥲 anchoralignvla.github.io Two edits on top of BC. Same demos. No extra data.
6
28
8
122
20,638
We are proud to launch are agent harness. We have been experimenting with multiple internal use cases using our harness and the framework is opensource now!! Please go checkout and let us know the feedback
1
2
183
Very excited to share that our CRL-VLA paper has won the outstanding paper award at the Reinforcement Learning Conference (also best paper at the ICRA RL4IL workshop)! The full list can be found at rl-conference.cc/RLC2026Awar…. This is my first time attending @RL_Conference, and RLC has already became my favorite conference (alongside CoRL). The community, the great review process, the hot takes... Everything is amazing! Congratulations to all co-authors: @jayjshim @ChenTangMark @yoonchangsung @cranialxix @PeterStone_TX @RobobertoMM
VLA models are capable generalists. But can they continually self-improve? Such Continual Reinforcement Learning (CRL) problems are traditionally considered very challenging. Surprisingly, we found that with the right setup, the simplest CRL recipe can work really well! arxiv.org/abs/2603.11653
3
10
2
72
5,698
one thing that has really stood out to me over the last couple of days covering this story is how many conversations eventually came back to the same thing. nirmal “nimsdai” purja was not just one of the worlds greatest mountaineers. he helped change who the world recognized when it looked at the highest mountains. before climbing became his life, nims spent 16 years in the british military, including service with the gurkhas and uk special forces. after leaving the military, he climbed all 14 of the worlds 8,000 meter peaks in just six months and six days. helped lead the first successful winter ascent of k2 and became one of the most recognizable climbers in the world. before this story, i think a lot of people outside the climbing community knew the word sherpa without really knowing the people behind it. nims became one of the people who helped change that. not just through what he climbed but through who he was. i think that is part of why this loss has reached so far beyond the climbing community. rest in peace, nims.
for many people outside the climbing community, 14 peaks: nothing is impossible was likely their first introduction to nirmal “nimsdai” purja. it was mine. i have always been fascinated by mountaineering and the people willing to push into places most of us will never experience. the documentary introduced me to one of the sports most extraordinary climbers and gave me a much deeper appreciation for what he and so many others do in the worlds highest mountains. if you have never seen it — it is well worth your time. rest in peace, nims.
6
45
3
435
102,921
Really excited for this initiative! Failure data is becoming very useful, and the best part, every robot lab collects them; they're just rarely shared. OopsieData aims to change that by pooling real-world robot failures across labs, making a collective dataset for all to use! 👇
This is a robot failing to grasp a ball. Almost every robot lab produces clips like this daily… and almost all of them get thrown away. This is the most abundant but underused resource in robot learning. We’re collecting all of it now as ✨OopsieData✨, please join us!
7
2
20
5,075
We scaled robot policies to 8K timesteps of visuomotor context, orders of magnitude beyond current SoTAs, at constant inference latency. Introducing RoboTTT 🤖 With minutes of experience in context, our robots: 🎥 one-shot imitate human video demos 📈 improve themselves during deployment 🛡️ recover from perturbations ⚙️ complete a 5-minute, 10-stage assembly end to end 🌐 research.nvidia.com/labs/gea… Dive in 🧵
7
48
21
256
268,127