PhD student @jhuclsp, 2x research scientist intern @AIatMeta FAIR. Previously @nyuniversity. I work on making LMs intelligent and interesting.
Baltimore, MD
Joined November 2022
- Tweets338
- Following792
- Followers453
- Likes2.3K
Pinned Tweet
So excited that this finally came out!! Studying reward hacking at scale in autoresearch agents is hard, and we ended up injecting hacks into long agent trajectories, so that we can do a controlled study on how monitors catch them.
HackBench serves as a stress test of agent monitor’s long context recall, understanding of the research task, and reasoning over multiple turns.
Introducing *HackBench* 🤡💀⏱️📊 ⏲️ 💪🔧 🔥
- Detecting & localizing reward hacking in autoresearch agents
- Frontier models fail
- Dataset on HuggingFace
- Recipe to collect natural & synthetic hacks
Blog post: facebookresearch.github.io/R…
🧵1/5
Tianjian Li retweeted
We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment. Right after it responds, Claude gives a preliminary safety assessment of the conversation.
Since we began using self-assessment in June, it has surfaced misuse that our other safeguards weren’t designed to catch.
Because Claude has already read the conversation, self-assessment is very efficient. And unlike classifiers, which target a discrete set of harms, it draws on Claude’s own judgment, so it gets more accurate as our models improve.
Read more on our new Safeguards blog:
safeguards.anthropic.com/202…
Tianjian Li retweeted
Claim: Humans & AI are better together if we want to recursively (co-)improve to a brighter future for *humanity*
Instead of racing towards methods for AI replacing us, work on how to make AI work *with* us!
New paper that makes 4 contributions here:
1) The role of humans in directing research agents: Autobenchmark
2) Reward hacking in autoresearch agents: HackBench
3) Training agents to interact with humans: AutoInteract
4) Training agents to communicate with humans: unslopping AI
Full Paper: arxiv.org/abs/2610.38882
Blog posts for each section:
1) facebookresearch.github.io/R…
2) facebookresearch.github.io/R…
3) facebookresearch.github.io/R…
4) facebookresearch.github.io/R…
Tianjian Li retweeted
LLMs might be superhuman at math, but can they act in real time?
Not even close!
Presenting Gym-Anything-RT, a framework for generating real-time minigames in which the game keeps moving while the model thinks.
Even the best frontier model, GPT-6 Astra, scores just 9%!
🌐: gym-anything.github.io/gym-a…
🧵
Tianjian Li retweeted
I will present at EMNLP on ✨Oct 27 (9 AM) ✨about our work "Exploring Sparse Autoencoders in Text-Based Causal Confounding Adjustment". Many causal questions hinge on texts. For example, does the type of media coverage of a bill affect the number of legislators who vote for it?
Tianjian Li retweeted
Introducing *HackBench* 🤡💀⏱️📊 ⏲️ 💪🔧 🔥
- Detecting & localizing reward hacking in autoresearch agents
- Frontier models fail
- Dataset on HuggingFace
- Recipe to collect natural & synthetic hacks
Blog post: facebookresearch.github.io/R…
🧵1/5
Tianjian Li retweeted
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?
We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.
In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.
📝 xingjianbai.com/reactive-opt…
💻 Try it on your task: github.com/xingjian-bai/opal
🧵
Each time I see Seungone in the summer, I saw him had a Physics paper printed and reading it carefully. Always amazed by that he can dig so deep into fields that he has few prior exposure, and understands them so fast. Congrats!
🧪Can AI agents take a scientific paper and write a meaningful follow-up to it?
We introduce Into the Unknown, an AI agent benchmark of 15 open scientific research problems. Each problem comes from a scientist currently or formerly at US DOE National Labs:
@LosAlamosNatLab @SandiaLabs @BerkeleyLab @AmesNatLab @PPPLab @Livermore_Lab
Tianjian Li retweeted
This new benchmark evaluates nine frontier agents on 15 open scientific follow-up problems developed with domain scientists, producing 135 research papers.
Most of the papers fail integrity checks, including discrepancies between claimed work and supporting execution evidence.
Tianjian Li retweeted
Excited to share ResearchTrails, a dataset of human research decision trajectories extracted from Git commit histories.
Final papers show the outcome, but they often do not capture the full path: abandoned methods, omitted experiments, and ablations. ResearchTrails recovers these intermediate decisions, and we show that learning from them improves LLMs’ ability to assist with research exploration!
Blog, code, data, and the trajectory visualizer are at researchtrails.org
🧵
Tianjian Li retweeted
Can LLMs generate novel research ideas? We propose Priced Guidance to quantify this via compression!
We lower bound P(LLM forecasts a future paper’s idea X). Instead of sampling ideas until X is matched, the LLM plays “20 Questions” to recover X, charging each hint by surprisal.
Tianjian Li retweeted
‼️Have you ever used a chess engine and wished it could explain its moves and evaluations?‼️
Introducing QUEEN, a 4B-parameter chess-language model that plays at the level approaching a typical Grandmaster and explains its moves in natural language.
Tianjian Li retweeted
New post: Capabilities research expands the safety-usefulness Pareto frontier too.
It’s tempting to define safety research as research that enables developers to deploy an AI system more safely without making the deployment much more expensive or much less useful.
You can visualize this definition of safety research as pushing out the safety-usefulness Pareto frontier.
At any given level of usefulness, there's greater safety available.
Awkwardly, this definition counts basically all capabilities research as safety research. For example, consider performance optimization for inference. By making inference more efficient you can use weaker, safer models more extensively than you would otherwise be able to, pushing out the Pareto frontier. Likewise, any successful research whatsoever pushes out this Pareto frontier because research can only ever create more options.
It seems like something has gone wrong with our definition of safety research if it includes seemingly all capabilities research.
Here, I spell out one reason why enabling improved safety without hurting usefulness is an insufficient standard for safety research. The core observation is that developers have to choose a particular point on the Pareto frontier, and some technological improvements incentivize them to sacrifice safety. Safety research typically reshapes the Pareto frontier in a way that causes developers to choose greater safety, while capabilities research typically does the opposite.
However, I also argue there is an important and plausible future regime involving much higher political will than we have right now, in which certain kinds of capabilities research would be an effective way to improve safety. I don't write this post to argue that more people should be doing capabilities research, and in fact I think that currently capabilities research at any frontier AI company is probably bad. Instead, I'm treating this as an interesting puzzle to sharpen our models of when and why safety and capabilities research are good.
Link: lesswrong.com/posts/nwrx9DHp…
Nice work! I really liked the part where they used suffix KV cache
There are so many works on RSI/autoresearch for methods, and we show that agents can also iteratively refine a benchmark! However, human oversight is still very important in generating a high quality benchmark. Led by the amazing @seungonekim
⏱️⚙️Introducing *AutoBenchmark* 📊🏁
- Creating benchmarks automatically
- Benchmarking benchmark creation
- Studying the role & impact of humans in the loop
Blog post: facebookresearch.github.io/R…
Key takeaways:
1) We find human-agent collaboration gives big wins over agents alone
- fine-grained feedback in ideation stage crucial
- autobench can be used to measure this in the future with stronger agents
2) We show that it's possible to make *autoresearch benchmarks* for AI research using this recipe – full recursive improvement loop!
3) Important Ingredients: Autobenchmark creation works best with feedback from two sources: benchmark solvers + external verifiers (human+AI).
🧵1/5
Tianjian Li retweeted
𝐒𝐞𝐥𝐟𝐂𝐨𝐦𝐩𝐚𝐜𝐭 is accepted to NeurIPS 2026 🚀 Check out our harness that enables agents to effectively manage their own context!
arxiv.org/abs/2606.23525
great work! congrats to Zhihui and Qwen
As hybrid computer use—the ability to dynamically interleave CLI and GUI actions for complex work—becomes essential for universal digital agents, we need tasks and environments to measure and advance it.
Introducing RecreationWorld: scalable, verifiable environments grounded in executable references across Ubuntu, macOS, Windows, Android and Web. 🧵
Website: recreation-bench.cc/
Code: github.com/QwenLM/Recreation…
Tianjian Li retweeted
We've long known that AI agent benchmarks are broken. Check out our work back from July 2025 on establishing best practices for making rigorous AI agent benchmarks!
(links below)
Tianjian Li retweeted
Does your Claude model really need Claude Code…? 🤔
We evaluate 7 models on Claude Code, Codex, and Pi. Three surprising findings emerge:
1️⃣Harness choice has little effect on task success rate, but can significantly affect the cost
2️⃣A simple harness can be competitive
3️⃣The native harness isn’t always the best.
Millions of people are using coding agents, but the impact of harness choice remains unclear.
(1/n) More details in the thread. 🧵
Tianjian Li retweeted
🧵 With unlimited compute, how fast can agents surpass humans? We introduce Elo-per-token analysis to profile agent performance curves across multiple open-ended tasks.
• Agents initially scale faster than repeated sampling, but over long horizons converge toward their theoretical log-linear scaling curve. Humans, in contrast, improve superlinearly.
• These curves also tell us how to spend test-time compute: the scaling inflection point gives a simple rule for splitting a fixed budget across agent sessions. Split a long run in a principled way, and you can get significant gains over a single run.
• Fitting human-time and agent-token curves also gives us a fun way to translate AI compute into human time. Taking OpenAI’s ~130B-token Navier–Stokes run as input and extrapolating across the two curves gives an equivalent of ~41 years of work by a mathematician at 8 hours/day 😮.