CMU, ex-TikTok AI https://xitter.gq/t.co/8BWbkmDWJJ
Joined August 2021
- Tweets18
- Following1.4K
- Followers203
- Likes147
Pinned Tweet
[1/6] Ego-centric World Models
We introduce EgoWM — a video world model that simulates EVE-1X humanoid interactions from a single ego-view image + full-body joint angle trajectories.
Moreover it effortlessly generalizes to extreme OOD domains, including paintings !
Anurag Bagchi retweeted
[1/7] We trained a world model on 30,000 hours of first-person video.
It learned exactly where the hands go. What happens to the things they touch, much less so.
Going from 300 to 30,000 hours did not narrow the gap, and a 4× larger model barely did either.
Anurag Bagchi retweeted
Excited to share our project - Sim2Reason!
Key Insight: Simulators are an untapped source of cheap supervision for scientific reasoning. LLMs can learn physical reasoning from simulation to improve on real world benchmarks such as the International Physics Olympiad!
Anurag Bagchi retweeted
In my recent blog post, I argue that "vision" is only well-defined as part of perception-action loops, and that the conventional view of computer vision - mapping imagery to intermediate representations (3D, flow, segmentation...) is about to go away.
vincentsitzmann.com/blog/bit…
[5/6] Temporal compression
Unlike prior works, we preserve full-sequence diffusion and compress actions to the latent temporal resolution. EgoWM achieves +42% better action alignment at +4s horizon vs. frame-wise autoregressive NWMs
even with 4× temporal compression (Cosmos-2B).
[6/6] Fine-grained humanoid manipulation
EgoWM enables precise 25-DoF joint-angle manipulation with the EVE-1X humanoid,
even at 4× temporal compression (Cosmos-2B).
📷 Learn more:
Project page: egowm.github.io
Paper: arxiv.org/pdf/2601.15284
Happening now! ICCV 25 poster#323
Drop by to chat and see some cool results!
[ICCV 25] Refer Everything Model (REM)
(1/6) We leverage Text-to-Video Generation models to zero-shot segment any concept in a video using text. REM generalises to dynamic concepts like smoke, light-beam and more without ever having seen segmentation masks for these entities.
(5/6) REM demonstrates how Text-to-Video generation can serve as a powerful pre-training paradigm for downstream video understanding. The days of large-scale, labor-intensive video annotation may soon be behind us — pre-train to generate, fine-tune lightly to understand.
(6/6) We’re at the start of the internet-scale "video" era, and the possibilities are exciting. Learn more at refereverything.github.io — our code & model weights are available. Visiting ICCV? Come see our poster on Oct 23 to chat and see results in action!