NLP Group at @MIT_CSAIL! PIs: @yoonrkim @jacobandreas @lateinteraction @pliang279 @david_sontag, Jim Glass, @roger_p_levy
Cambridge, MA
Joined March 2025
- Tweets146
- Following73
- Followers4.7K
- Likes140
MIT NLP retweeted
1/
Presenting CultureScore tomorrow at #EMNLP2026 🇭🇺
📍Hall H, Session 2 · Sun, Oct 25, 11:00–12:30
📄 arxiv.org/abs/2606.07311
Come say hi 👋
MIT NLP retweeted
We let agent swarms collaborate for 10K hours in simulated agent-owned GitHubs.
Gemini-3.8-flash collaborate the best while gpt-6-astra scores the worst among frontier models tested⁉️
What are the elements of successful agent collaboration at scale? Check World of SWECraft 🧵
MIT NLP retweeted
🚀T-WLA: Visual memory for fast touch
🖐️Memory mechanism connects vision and touch
⚡6× T-Rex’s decision rate (A6000 Pro)
📉30% lower finger-joint error
🤖2× T-ReX’s average success rate across 4 real-robot tasks
🎬 arxiv.org/pdf/2610.28814
@MIT @MIT_CSAIL @medialab @Harvard
check out @akarshkumar0101’s awesome interview on @MLStreetTalk!!!
"Evolution is anything but random."
This is INSANE...
@akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.
MIT NLP retweeted
I've been getting a lot of DMs about Jev and how it relates to our RLCR (RL with Calibration Rewards) paper. Whatever the connection, anyone who's talked research with me in the last year knows I can go on and on about calibration as one of the most overlooked parts of post-training, so I'm really happy to see it finally getting attention.
If we want models that are actually useful in the real world, in healthcare, law, or scientific discovery, they need to know what they don't know. That's why I've spent so long thinking about better paradigms for uncertainty-aware decision making.
There's still so much to do: training more calibrated models, building systems that interact well with users in uncertain settings (life🙂), and figuring out how to evaluate all of it in a way that reflects real workflows instead of completion based rewards. DMs open if you want to chat!
(congrats to @CompleteSkeptic on the launch, and thanks to @rasbt for the great writeup!)
📄 arxiv.org/pdf/2507.16806
MIT NLP retweeted
VISTA is now on arXiv! 🎉 We show that VISTA generalizes to interactive visual environments beyond ARC-AGI-3. Check out the paper for more results and ablations!
Our 🔭VISTA🔭 paper is now on arXiv, with more ablations and results!
We introduced VISTA in our Aug 5 blog post. The paper further evaluates it on visual games and reasoning tasks beyond ARC-AGI-3 🧵
Paper: arxiv.org/abs/2610.02200
Code: github.com/joshhhhhan/VISTA
Previous blog post: vista-research.github.io/
let’s get linear attention to be more expressive!
MIT NLP retweeted
Is Jev secretly learning a Value function?
A model can always choose “A” without being certain that “A” is correct. That distinction matters when an agent must decide whether to act fast or switch to deeper reasoning.
We spent some time exploring:
1. How Jev fits into System 1/2 decision-making and why calibration matters for knowing when to switch
2. Why maximising expected reward in RL does not guarantee calibrated probabilities.
3. How predicting eventual task success in agentic tasks connects to a familiar RL quantity: the Q-function.
Could value prediction help build a calibrated System 1—and could that be part of the story behind Jev?
Link: souradip-chakraborty.github.…
#Jev #RLCD #System1
MIT NLP retweeted
(1/7) Humans can acquire many capabilities without (catastrophically) forgetting previous ones. Why can't neural networks do the same?
Local Support Learning (LSL) takes us a step closer towards this goal.
In LLMs of up to 7B parameters, LSL augments gradient-based training to retain prior capabilities while learning new tasks at full capacity. Notably, it does so without access to prior data.
Key idea: casting forgetting as a geometric problem, and optimizing for the worst case.
Paper: arxiv.org/abs/2610.02126
Website and code: assafbk.github.io/lsl
MIT NLP retweeted
🚀 Honeycomb: constant-size scene memory for video world models
🐝 Six fixed-size planes with recurrent read/write.
🔁 Better revisit consistency
💾 Constant storage
⚡ ~13 ms updates
📃 arxiv.org/pdf/2609.37690
@Harvard @NUSingapore @MIT
#AI #ComputerVision #VideoGen
MIT NLP retweeted
new preprint: language models are “insecure” reporters.
researchers and engineers have become increasingly reliant on LLM-written reports---second-hand accounts of code and experiments---to understand agent-generated work.
in this study, we ask: “how transparent are these LLM-generated reports?” 🔍
MIT NLP retweeted
Your agent can now grow new abilities, just by asking.
Reef now supports personalized harness evolution with /reefine: describe what you want your agent to do. Reef builds the change, checks it, and ships a new version.
Examples:
💬 "> /reefine add a /chat mode for faster responses"
🔊 "> /reefine tell me out loud when you're done"
⚡ "> /reefine add jev as a tool": 120 papers screened in 9 s
Works with any agent harness through a Reef adapter: Codex, Hermes, OpenCode, Pi and Terminus 2 today.
Try it here: github.com/Human-Agent-Socie…
#agent #harness #rsi #llm
We do teacher forcing on input tokens, why not do it for RNN memory states as well?
Pretraining Recurrent Networks without Recurrence by @akarshkumar0101 & @phillip_isola is a great paper!
It makes an end-run around the problems of RNNs via a transformer teacher to learn good predictive state representations & supervised learning of a memory transition function
new paper! A continual learner must build on an ever-growing history of experience, making adaptive compression of the past a fundamental part of intelligence
In-context continual learning requires models to accumulate experience and reuse it later in the same sequence. But an RNN compresses an ever-growing history into a fixed-size state, where each token gets a single write into memory.
We study dynamic compression: letting the model revisit the past and reorganize its state as it discovers what needs to be reused.
MIT NLP retweeted
🚀 CFP: 1st Workshop on #PhysWorldAI at #NeurIPS2026!
Explore physical geometry, physical properties, multimodal sensing & embodied AI.
🎤 Keynotes: Anandkumar, Snavely, Finn & Freeman.
📍Atlanta, Dec 12/13 | 📅 Due Sept 9
physworld-org.github.io/phys…
MIT NLP retweeted
Last fall, I spent a while trying to get a better understanding of the information retrieval (IR) landscape. This mostly involved a lot of research conversations, some half-baked ideas, and, for a while, not much tangible research progress.
With OBLIQ-Bench being a few months old, I wanted to write a short blog post about what led to its creation, what it says, and what I think should come next!
check it out: dianetc.github.io/musings/ob…
MIT NLP retweeted
🚀 Stream3D is accepted to CVPR-W 2026!
The first framework for long-horizon 3D asset generation from monocular streams 🌊
— turning frozen 3D generators into streaming generators via evidential memory, no retraining
📈 Better consistency across long streams
💾 Constant memory footprint
📃 stream-3d.github.io/stream3d…
@MIT @Harvard @medialab @hkust
#AI #ComputerVision #3D
training LMs to be right in the right way: 👉
MIT NLP retweeted
📢 New paper: SynIB, an informational bottleneck for maximizing synergy in multimodal learning.
Train a multimodal model and you'll often find it doesn't really use both modalities. It leans on the easy one and falls apart exactly when both are needed. 🧵
MIT NLP retweeted
Our new paper on training user simulators! Excited to share this work and think about the many future directions of Turing-RL.
How should we train LLMs for user simulation? A good user simulator shouldn't merely imitate what someone said. It should be indistinguishable from what users could have said.
Introducing 🎭Turing-RL: training user simulators with a Turing-Test reward. The model gets rewarded when an LLM judge can't tell its response from the real users’.
