student researcher @deepmind | phd @uwcse | prev. @meta_fair | undergrad @iitkgp | building robots and agents that are curious
Seattle, WA
Joined June 2020
- Tweets104
- Following748
- Followers361
- Likes427
Pinned Tweet
Excited to present this at #CoRL2026 next week.
Two takeaways: real human data should shape (1) how we benchmark intervention learning, and (2) then build algorithms that learn from corrections.
Our work shows that, unlike standard assumptions, humans don't react instantaneously when supervising robots and are often suboptimal. We improve benchmarking to generate better correlations across sim and real-world rankings.
Using these data-driven insights, we propose Q-OIL, a method that uses interventions to provide optimistic rewards using corrections. With important design choices, we can train policies across multiple tasks in 30-60 min, including post-training pi-0.5 for interesting bimanual tasks.
Check out Pranav’s thread for more details!
P.S.: he spent hours running human experiments for this work, but the insights pay off 💪
Excited to share the first paper of my undergrad, Q-OIL, which we’ll be presenting at #CoRL2026 in Austin!
Interventions provide a natural source of feedback for robots to learn from in the real world. But, humans intervene suboptimally, which needs to be accounted for in both simulation benchmarking and algorithms.
We analyze human intervention data and measure intervention suboptimality in both timing and execution. Thus, rather than using interventions as supervision, we use them as soft exploration guidance.
(1/8)
Sriyash Poddar retweeted
If you want to learn from humans, you should probably know something about how humans work. We found that prior work on robot learning from interventions relied on flawed assumptions about how humans give feedback. We ran human studies, analyzed the issues, and built a much more accurate human simulation model. Using this, we developed a state-of-the-art method for learning from humans interventions during deployment, allowing robots to adapt online to complex tasks like unbuttoning a shirt!
Excited to share the first paper of my undergrad, Q-OIL, which we’ll be presenting at #CoRL2026 in Austin!
Interventions provide a natural source of feedback for robots to learn from in the real world. But, humans intervene suboptimally, which needs to be accounted for in both simulation benchmarking and algorithms.
We analyze human intervention data and measure intervention suboptimality in both timing and execution. Thus, rather than using interventions as supervision, we use them as soft exploration guidance.
(1/8)
Sriyash Poddar retweeted
Turns out that humans are far from optimal when intervening on robot behavior. This makes many algorithms for learning from human interventions either too conservative or incorrectly reliant on human optimality.
@_pranavnt, @sriyash__ and team first analyze how people actually intervene on robots. Then they use this to a propose a simple RL algorithm - Q-OIL that uses these interventions to learn very quickly.
Q-OIL proposes a simple idea - be optimistic on state-action pairs where people intervene. Think of it as - "even if the human isn't perfectly right, it's probably worth checking that out". This leads to fast improvement and a lot fewer human interventions for robots learning with RL in the real world.
If you are doing RL in the real world using interventions, this should be a simple drop in for things like HiL-SERL etc. We've got lots of robot results + a fun interactive demo.
Website: weirdlabuw.github.io/qoil/
Paper: weirdlabuw.github.io/qoil/pa…
Excited to share the first paper of my undergrad, Q-OIL, which we’ll be presenting at #CoRL2026 in Austin!
Interventions provide a natural source of feedback for robots to learn from in the real world. But, humans intervene suboptimally, which needs to be accounted for in both simulation benchmarking and algorithms.
We analyze human intervention data and measure intervention suboptimality in both timing and execution. Thus, rather than using interventions as supervision, we use them as soft exploration guidance.
(1/8)
Sriyash Poddar retweeted
Excited to share the first paper of my undergrad, Q-OIL, which we’ll be presenting at #CoRL2026 in Austin!
Interventions provide a natural source of feedback for robots to learn from in the real world. But, humans intervene suboptimally, which needs to be accounted for in both simulation benchmarking and algorithms.
We analyze human intervention data and measure intervention suboptimality in both timing and execution. Thus, rather than using interventions as supervision, we use them as soft exploration guidance.
(1/8)
Sriyash Poddar retweeted
What if a swarm of AI agents could learn to work together on its own? 🐝
No leader. No roles. No turn order. Just one shared reward.
Our new paper s³: multi-agent RL lets LLM swarms self-organize, beat best-of-N, and scale past their training size!
scalingswarms.com/ 🧵
Sriyash Poddar retweeted
Want to turn any VLM into a better reward model for robotics? (yes, even Astra)
Introducing D&A: decoupling description and assessment improves any VLM family for zero-shot robot rewards.
D&A scales with improvements across GPT, Gemini, Claude, Qwen, ...
It enables zero-shot RL and filtered IL, improves code-as-policies via rich language feedback, and achieves SOTA on three benchmarks.
Website: weirdlabuw.github.io/dna
Sriyash Poddar retweeted
Finding feasible robot states for goal-conditioned RL is one thing. Sampling as diverse states as possible is another.
Diverse resets and goals help GCRL, but contact and collision constraints make truly uniform sampling hard.
Our NeurIPS ’26 work introduces MASEM. 🧵👇.
Sriyash Poddar retweeted
In LLM post-training, policy gradient is "the RL loss".
When it fails, we blame the RL: exploration, credit assignment, sampling noise.
Image classification has none of these, and the policy gradient is exact. Following it is terrible...
ImageNet, 4% vs 62% for cross-entropy.
Sriyash Poddar retweeted
What emerges is richer than sampled static states: scooping, whole-body contacts, recovery, reorientation, edge/corner balancing. Optimization finds where interaction is possible. RL learns how to move between those states.
📄 Paper: arxiv.org/pdf/2602.08557
🌐 Project: user.tu-berlin.de/mtoussai/2…
Project lead by @Marc__Toussaint and collaboration w @Denis_Shcherba , @tupryk , Tilman Burghoff, Sayantan Auddy, Armand Jordana and Justin Carptentier.
Sriyash Poddar retweeted
Really cool to see OpenAI is using self-play for red-teaming now! Our group was the first to show the promise of this approach arxiv.org/abs/2506.07468, and I really believe it's a more viable pathway for robustness than single-agent safety fine-tuning. Great to see it adopted at scale!
Introducing GPT-Red
An internal automated red teamer on a mission to find our models’ prompt injection vulnerabilities at scale, helping us build stronger defenses before wider deployment.
openai.com/index/unlocking-s…
Sriyash Poddar retweeted
Can we get robots to think before they act? In Latent Memory Palace, we achieve this through a classic principle: variational inference. (1/8)
Sriyash Poddar retweeted
Latent reasoning for robotics == variational inference
We want models that are able to allocate variable amounts of compute at test time, but we need it to operate in latent space to capture the fine-grained details of robotics. @chuning_zhu shows that this problem can be cast as variational inference on a variable length latent variable model! Simple ELBO objective, optimize in imagination with RL to get policies that can reason :)
Why does this matter -
1) Latent reasoning shows adaptive allocation of compute, directly improving downstream policy performance on robots.
2) Turns out it can give you a pretty great tokenizer! Reasoning paths through the latent space directly correspond to action tokens. Does great compared to other tokenizers!
Check it out, it's a pretty cool framework to think about reasoning!
Lots of great details in the paper:
Website: weirdlabuw.github.io/lmp/
Paper: arxiv.org/abs/2607.08724
Sriyash Poddar retweeted
RL can't teach an LLM to solve problems it never solves. Robot policies suffer from exactly the same limitation.
The fix turned out to be almost embarrassingly simple: train the policy with diffusion noise during pre-training.
That's it.
The policy covers a much wider action distribution, RL finally has somewhere to search, and we fine-tune VLAs on real robots in under an hour.
Introducing TMRL.
🧵(1/9)
Sriyash Poddar retweeted
Here’s a pretty weird and surprising result - retrieval-augmented generation works unreasonably well for robot learning – but only when parameterized using difference vectors!
We introduce Difference-Aware Retrieval Policies for Imitation Learning (DARP), a simple, semi-parametric RAG architecture for imitation learning that achieves gains of up to 200% over standard behavior cloning. No additional assumptions beyond BC, just a little architecture switch! The theory backing it up is pretty cool too and it works on real robots! :)
Play with our website to understand better: weirdlabuw.github.io/darp-si…
🧵(1/7)
Sriyash Poddar retweeted
Real-world RL is still too brittle and data-hungry for long-horizon, contact-rich tasks.
We introduce Simulation Distillation (SimDist), which turns large-scale simulated experience into reusable world-model priors for rapid real-world adaptation.
By combining online planning with dynamics adaptation, SimDist achieves high success rates on tasks requiring precision, force, and reactivity.
Play with our interactive visualization to see for yourself: sim-dist.github.io
(1/n)
Sriyash Poddar retweeted
We present Compute Optimal Tokenization! 🔡
Common in LLM scaling works stick to one tokenizer, sweeping data/model size.
But what happens when we control the tokenizer’s compression rate (bytes/token)?
Here we sweep tokenizers, params, and data across compute budgets: [1/N]
Extremely simple and scalable way from @patrickhyin @TylerW24089 to generate dextrous behavior that transfers sim2real!!
Have been personally using it to generate expert policies that work from literally any reset position.
We’re releasing OmniReset, a framework for training robot policies using large-scale RL and diverse resets for contact-rich, dexterous manipulation.
OmniReset pushes the frontier of robustness and dexterity, without any reward engineering or demonstrations.
Try the policies yourself in our interactive simulator! weirdlabuw.github.io/omnires…
(1/N 🧵)
Sriyash Poddar retweeted
Animals can’t learn by being tele-operated. But, they do learn by observing and interacting with the world around them. So, why don’t robots learn this way?
Excited to release, “Planning from Observation and Interaction”, for real-world observational learning on robots! 🧵(1/12)
Sriyash Poddar retweeted
A reward model that works, zero-shot, across robots, tasks, and scenes?
Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories.
Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more!
🧵 (1/12)
