MIT PhD student in Computer Science. @MIT @MIT_CSAIL @MITEECS robots, RL, embodied reasoning, video-language models
MIT
Joined November 2025
- Tweets138
- Following2.3K
- Followers2.5K
- Likes11K
Pinned Tweet
Excited to share SOLE-R1 is accepted to NeurIPS 2026!
General-purpose reward models are fundamental to unlocking real-world RL.
We show SOLE-R1's zero-shot reward prediction can serve as the SOLE signal for learning over 20 unseen manipulation tasks through online RL (i.e., randomly initialized policies, 0% starting success rate).
This work provides a strong proof of concept: RL with zero-shot dense reward prediction can do more than just increase success/speed for existing tasks; it can enable robots to learn entirely new tasks from scratch, without any demonstrations or ground-truth rewards, starting from zero competence.
Very proud of this project!
Webpage: philip-mit.github.io/sole-r1…
Thank you to my coauthors @thomas_weng, @_ericrosen, @BizaOndrej.
#NeurIPS2026
Philip Schroeder ✈️ CoRL 2026 retweeted
Kicking off CoRL 2026 🤠 with an Atlas video!
Atlas has continuous joints, so can we just replace the ankles with wheels? 🤔
Philip Schroeder ✈️ CoRL 2026 retweeted
Philip Schroeder ✈️ CoRL 2026 retweeted
What I care about isn't the learned policy, it's - how to get a robot into a site and start producing as fast, as efficiently, and as cheaply as possible. That is fundamentally different from clinging onto learning.
Philip Schroeder ✈️ CoRL 2026 retweeted
Want to turn any VLM into a better reward model for robotics? (yes, even Astra)
Introducing D&A: decoupling description and assessment improves any VLM family for zero-shot robot rewards.
D&A scales with improvements across GPT, Gemini, Claude, Qwen, ...
It enables zero-shot RL and filtered IL, improves code-as-policies via rich language feedback, and achieves SOTA on three benchmarks.
Website: weirdlabuw.github.io/dna
Philip Schroeder ✈️ CoRL 2026 retweeted
SOLE-R1 points to a useful route to generalization: transfer how to assess progress, then learn how to act through interaction. I'd track how those judgments transfer across tasks and embodiments, and how many new skills they support within a fixed interaction budget
Excited to share SOLE-R1 is accepted to NeurIPS 2026!
General-purpose reward models are fundamental to unlocking real-world RL.
We show SOLE-R1's zero-shot reward prediction can serve as the SOLE signal for learning over 20 unseen manipulation tasks through online RL (i.e., randomly initialized policies, 0% starting success rate).
This work provides a strong proof of concept: RL with zero-shot dense reward prediction can do more than just increase success/speed for existing tasks; it can enable robots to learn entirely new tasks from scratch, without any demonstrations or ground-truth rewards, starting from zero competence.
Very proud of this project!
Webpage: philip-mit.github.io/sole-r1…
Thank you to my coauthors @thomas_weng, @_ericrosen, @BizaOndrej.
#NeurIPS2026
Philip Schroeder ✈️ CoRL 2026 retweeted
The Boston Dynamics Spot legs have always intrigued me, specifically how the knees are actuated!
There is a great patent on this, “US20180172121A1: Screw Actuator for a Legged Robot”. The abstract captures the design quite succinctly (included below).
I got Claude to generate a rough diagram to see how the linear screw actuator can be used to control the lower leg link! I’m also including an image from the patent to compare to the real thing!
“An example robot includes: a leg having an upper leg member and a lower leg member coupled to the upper leg member at a knee joint; a screw actuator disposed within the upper leg member, where the screw actuator has a screw shaft and a nut mounted coaxial to the screw shaft such that the screw shaft is rotatable within the nut; a motor mounted at an upper portion of the upper leg member and coupled to the screw shaft; a carrier coupled and mounted coaxial to the nut such that the nut is disposed at a proximal end of the carrier; and a linkage coupled to the carrier, where the linkage is coupled to the lower leg member at the knee joint.”
Philip Schroeder ✈️ CoRL 2026 retweeted
Great materials for learning about AI fundamentals that balance technical facts and digestability:
1. How LLM works (in 20min) open.spotify.com/episode/53t…
This whole podcast (Fragmented -- AI developers podcast) is a goldmine for whoever wants to learn about essential concepts for AI models/agents without touching the hardcore science behind them.
1. How always-on agents work youtube.com/watch?v=L1OBRtfP…
2. DeepSeek R1 Explained to your grandma youtube.com/watch?v=kv8frWeK…
4.. Direct Preference Optimization (DPO) paper explained youtube.com/watch?v=XZLc09hk…
fine-tuning our reward models on @dm_robots task. Still early, but cool to see our model show near-perfect progress prediction on this unseen trajectory with interweaving errors and recoveries.
the task has a nice combination of factors that make it a good challenge for the reward model (and policy):
- bimanual + multi-view: requires coordinating both arms and reasoning across three camera views
- on-robot cameras only: no external cameras providing a global view of the scene
- precision: demands precise insertion of the actuator into the hole where the visual difference between near-miss and success is very subtle
- start state and goal state change every episode: different actuator to pick up, different hole to insert it into
the task is right on the edge of hard, but not too hard to get to >99% success rate
exciting to see @DominiqueCAPaul and @aurel_arnold make so much progress with this task in a very short period of time
Philip Schroeder ✈️ CoRL 2026 retweeted
First time being on a podcast, and it's RoboPapers! 🤩
I had a great time discussing SOLE-R1 (philip-mit.github.io/sole-r1…) with @micoolcho, @DJiafei, and @philip_mit
Reinforcement learning is often seen as key to achieving scalable, highly reliable real-world robot performance, yet it requires good reward functions that can supervise the wide range of tasks that need to be performed. The obvious thing to try would be a large vision-language model (VLM), but these models often fail under common real-world conditions like partial observability and occlusion.
As a solution @Philip_MIT and @thomas_weng propose SOLE-R1, a video-language reasoning model trained to serve as a reward signal for online RL. Given real video, SOLE-R1 will reason over input video frames, generates reasoning traces, and predicts a reward signal. This allows online RL from random initialization on new manipulation tasks.
To learn more, watch Episode 111 of RoboPapers, with @micoolcho and @DJiafei!
the @RoboPapers episode on SOLE-R1 is out now! we break down our recipe for training a reward model that stays robust to reward hacking as the policy actively optimizes against its zero-shot reward prediction during online RL.
check it out if you're interested in real-world reinforcement learning or general-purpose reward reasoning models!
Reinforcement learning is often seen as key to achieving scalable, highly reliable real-world robot performance, yet it requires good reward functions that can supervise the wide range of tasks that need to be performed. The obvious thing to try would be a large vision-language model (VLM), but these models often fail under common real-world conditions like partial observability and occlusion.
As a solution @Philip_MIT and @thomas_weng propose SOLE-R1, a video-language reasoning model trained to serve as a reward signal for online RL. Given real video, SOLE-R1 will reason over input video frames, generates reasoning traces, and predicts a reward signal. This allows online RL from random initialization on new manipulation tasks.
To learn more, watch Episode 111 of RoboPapers, with @micoolcho and @DJiafei!
Philip Schroeder ✈️ CoRL 2026 retweeted
Full episode dropping soon!
Geeking out with @Philip_MIT @thomas_weng on SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL philip-mit.github.io/sole-r1…
Co-hosted by @micoolcho @DJiafei
very fun time talking about how to train general-purpose reward models for real-world RL on @RoboPapers with @thomas_weng, @micoolcho, and @DJiafei
full episode coming soon!
Full episode dropping soon!
Geeking out with @Philip_MIT @thomas_weng on SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL philip-mit.github.io/sole-r1…
Co-hosted by @micoolcho @DJiafei
2
25
2,004
Philip Schroeder ✈️ CoRL 2026 retweeted
Full episode dropping soon!
Geeking out with @Philip_MIT @thomas_weng on SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL philip-mit.github.io/sole-r1…
Co-hosted by @micoolcho @DJiafei
Philip Schroeder ✈️ CoRL 2026 retweeted
Designing reward functions for real-world RL is a serious challenge in learning for manipulation.
Zero/few-shot reward models that “just work” is going to be a huge unlock for the entire field to go beyond behavior cloning!
Excited to share SOLE-R1 is accepted to NeurIPS 2026!
General-purpose reward models are fundamental to unlocking real-world RL.
We show SOLE-R1's zero-shot reward prediction can serve as the SOLE signal for learning over 20 unseen manipulation tasks through online RL (i.e., randomly initialized policies, 0% starting success rate).
This work provides a strong proof of concept: RL with zero-shot dense reward prediction can do more than just increase success/speed for existing tasks; it can enable robots to learn entirely new tasks from scratch, without any demonstrations or ground-truth rewards, starting from zero competence.
Very proud of this project!
Webpage: philip-mit.github.io/sole-r1…
Thank you to my coauthors @thomas_weng, @_ericrosen, @BizaOndrej.
#NeurIPS2026
Philip Schroeder ✈️ CoRL 2026 retweeted
Currently looking at new research directions and SOLE-R1 keeps coming up!
Excited to share SOLE-R1 is accepted to NeurIPS 2026!
General-purpose reward models are fundamental to unlocking real-world RL.
We show SOLE-R1's zero-shot reward prediction can serve as the SOLE signal for learning over 20 unseen manipulation tasks through online RL (i.e., randomly initialized policies, 0% starting success rate).
This work provides a strong proof of concept: RL with zero-shot dense reward prediction can do more than just increase success/speed for existing tasks; it can enable robots to learn entirely new tasks from scratch, without any demonstrations or ground-truth rewards, starting from zero competence.
Very proud of this project!
Webpage: philip-mit.github.io/sole-r1…
Thank you to my coauthors @thomas_weng, @_ericrosen, @BizaOndrej.
#NeurIPS2026
Philip Schroeder ✈️ CoRL 2026 retweeted
Working on first real-world RL token experiments today.
> I’m only activating it for the critical part (insertion) as in paper
> first rollouts with RLT work decent
> After ~10 rollouts with updates the policy gets very jerky - maybe because I’m injecting noise for exploration or the residual policy isn’t regularised strongly enough. Noise injection should show up as jerkiness is first rollouts as well though?
In the video you can see the jerkiness. Also, once I turn the RLT network off and unedited VLA takes over everything runs smooth and actuator is inserted cleanly.
Philip Schroeder ✈️ CoRL 2026 retweeted
Two months of ablating π0.5 finetunes on a real manufacturing task. The policy now succeeds 98% of the time.
We're publishing all results, all the data, and every run, including the ones that went nowhere.
What we learned:
1/ 5x more data took us from 63% to 76%. Pure scaling was the weakest lever we found.
2/ Diversity got more out of the same hours. 4h spread across five scenes beat 4h in the eval scene by 30pp.
3/ Quality did even better. One clean, slower hour on top of the 21h: 76% → 90%. That single hour added more than the 17 hours before it.
4/ So we tried quality on its own: 1h of clean data plus 240 rollouts with human interventions. 1.7h in total, 28% → 88%. That's 12 points above the 21h model with a twelfth of the data.
5/ Inference settings add more on top, with no retraining. They took the 21h model from 76% to 93%, and the 21h+1h model from 90% to 98%.
Just as useful: what didn't matter. Learning rate, batch size, relative vs. absolute joints, image augmentation. All less important than we expected.
Full write-up linked below.
Philip Schroeder ✈️ CoRL 2026 retweeted
Blog post out for our RL post-training method, Q2RL!
Looking to improve your deployed robot with reinforcement learning in the real world?
Presented at RSS 2026, Q-Estimation and Q-Gating from Behavior Cloning for Reinforcement Learning (Q2RL) improves robot policies in a few hours of real world interaction, avoiding catastrophic forgetting when fine-tuning a Behavior Cloning (BC) policy via Reinforcement Learning (RL) post-training.
Q2RL estimates a Q-function for the BC policy, and uses it to guide online RL. Learn more about the method here: rai-inst.com/resources/blog/…
