Bingsen (Bale) Chen (@BaleCcchen) profile picture
Joined June 2022
Photo by Bingsen (Bale) Chen (@BaleCcchen)Photo by Bingsen (Bale) Chen (@BaleCcchen)Photo by Bingsen (Bale) Chen (@BaleCcchen)Photo by Bingsen (Bale) Chen (@BaleCcchen)Photo by Bingsen (Bale) Chen (@BaleCcchen)
🧐 Human researchers rarely write a report in a single shot as we iteratively draft and revise from peer feedback or self-reflection. Can Deep Research Agents also reliably revise their reports in response to user feedback? Unfortunately, no. Even though they follow your feedback, they may silently break 16–27% of the content you didn't ask them to touch! 🧵
1
6
4
44
11,218
Bingsen (Bale) Chen retweeted
Could long-context architectures “find all contradictions” in a science literature? Not yet! 🧵 We study a new class of "high-complexity” tasks whose difficulty scales quadratically with corpus size (as opposed to linearly), reversing common LCLM decisions! (block-sparse attention, hybrid models…)
15
70
8
340
61,456
Bingsen (Bale) Chen retweeted
New 📜 on the interplay between OPD and RLVR 💡Don’t skip RL after OPD We find that after using OPD for reasoning, adding an RL phase consistently improves performance. Across reasoning tasks, OPD→RL outperforms pure OPD, pure RLVR, and many joint OPD+RL methods.
5
42
5
306
21,545
Bingsen (Bale) Chen retweeted
Some Theoretical and Practical Thoughts on Diffusion Language Models jzhao2024.github.io/notes/20…
4
33
9
265
71,133
Just landed in SD🌊 for ACL! MR DRE will be in Oral Session G @ Harbor A on July 7th, 12:10PM🥳 Recently I’m thinking about the {post-training, eval, controllability, personalization} of {long-horizon, interactive} agents. Let’s connect and hang!
🧐 Human researchers rarely write a report in a single shot as we iteratively draft and revise from peer feedback or self-reflection. Can Deep Research Agents also reliably revise their reports in response to user feedback? Unfortunately, no. Even though they follow your feedback, they may silently break 16–27% of the content you didn't ask them to touch! 🧵
1
5
351
Bingsen (Bale) Chen retweeted
Can LLMs generate diverse outputs for open-ended questions? Is it helpful if we ensemble outputs from multiple models? We study 18 LLMs on 4 datasets and find that no single model is best at generating diverse outputs 👇/ 🧵
3
35
5
182
27,925
Glad to see that OpenAI's Deep Research can now take follow-up requests from users, which is a crucial missing piece in many current Deep Research Agents' training and system design! We built an evaluation suite for such multi-turn report revision scenarios. Check it out at arxiv.org/abs/2601.13217 !
Replying to @OpenAI
Now in deep research you can: - Connect to apps in ChatGPT and search specific sites - Track real-time progress and interrupt with follow-ups or new sources - View fullscreen reports
1
2
171
🧐 Human researchers rarely write a report in a single shot as we iteratively draft and revise from peer feedback or self-reflection. Can Deep Research Agents also reliably revise their reports in response to user feedback? Unfortunately, no. Even though they follow your feedback, they may silently break 16–27% of the content you didn't ask them to touch! 🧵
74
Bingsen (Bale) Chen retweeted
What if an LLM could EDIT its own tokens in real-time, not just generate them? 🤯 Introducing LLaDA2.1 — a diffusion model that breaks from autoregressive dominance. It drafts fast, then fixes its own mistakes on the fly with Token-to-Token editing. The result? 892 tokens/sec on a 100B model. 🔥 ⚡ 892 TPS on HumanEval+ (coding) ⚡ 801 TPS on BigCodeBench 🧠 Real-time self-correction via T2T editing ✅ @lmsysorg SGLang Day 0 support — production-ready now A "non-consensus" architecture now challenging the mainstream. Open-sourced TODAY. 👇 #LLaDA #TokenEditing #OpenSource #LLM #dLLM
48
83
65
368
371,867
[5/n] We then proposed two inference-time fixes to see whether such issues can be simply resolved through prompt engineering or by adding a sub-agent specifically for report revision. Although both fixes show some improvements, the high break rate and citation degradation problems remain unsolved.
1
1
2
248
[6/n] We view multi-turn report revision as a critical missing piece in developing truly useful DRAs, and our Mr Dre aims to track and drive progress toward agents that can both conduct complex research and reliably adapt to users' evolving needs. Check out our paper and codebase for more details! 📝Paper: arxiv.org/abs/2601.13217 💻Code: github.com/BaleChen/Mr-Dre (Note that we designed the unified and lean evaluation protocol for any Deep Research benchmark with question-specific checklist evaluation, so give it a try if you are also working on DRAs!) My awesome collaborators: @biji13429782393, Ping Nie, Yuyu Zhang, @xiye_nlp, @henryzhao4321.
1
1
266
Bingsen (Bale) Chen retweeted
We will have a pre-EMNLP workshop about LLMs next Monday at @nyushanghai campus! Speakers are working on diverse and fantastic problems, really looking forward to it! We also provide a zoom link for those who cannot join in person :) (see poster)
3
11
43
14,532
Bingsen (Bale) Chen retweeted
Check out our new work on making reasoning models think broadly! 🤔 We find a minimalist, surprisingly effective recipe to THINK for CHAT: RLVR + a strong reward model, trained on real-world prompts. This project was fun and surprised me in a few ways 👇 📌 We can run RL directly on a base model (no SFT), showing base models might already chat well. Llama-3.1-8B-Base with only 7K prompts ends up chatting well, matching Llama-3.1-8B-Instruct. This is interesting since Instruct was trained with a complex multi-stage pipeline. Also nice to see this working on Llama, while most RLVR papers only show success on Qwen. 📌 Interesting findings about rewards. Leaderboard scores of reward models aren’t always the best indicator of downstream performance. We also tested checklist-based rewards, which helps on synthetic instruction-following tasks (IFEval) but didn’t generalize well to chat. I still believe in this direction, and would love to see more open-source efforts. 📌 Real user prompts (shout out to WildChat @wzhao_nlp ) were the most effective. These prompts often require “thinking before answering,” which makes them fit for teaching models general thinking. The recipe is simple, we need good ingredients to cook better. 📌 Algorithms, like GRPO vs PPO, has a bigger impact when training directly from base models, but once warm-started with SFT, models are less sensitive to the choice. Overall, my feeling is: if we start with a strong base LM, and put it in the right “chat environment” (good prompts + good rewards), simple RL training goes a long way. Thus we are quite excited to explore more on pretraining and reward design!
Language models that think, chat better. We used longCoT (w/ reward model) for RLHF instead of math, and it just works. Llama-3.1-8B-Instruct + 14K ex beats GPT-4o (!) on chat & creative writing, & even Claude-3.7-Sonnet (thinking) on AlpacaEval2 and WildBench! Read on. 🧵 1/8
23
98
18,617
Bingsen (Bale) Chen retweeted
[1/8] How embeddings from Text Diffusion Models ✨compare to those from LLMs 🦙? Check out our work “Diffusion vs. AR Language Models: A Text Embedding Perspective”! We introduce a new diffusion-based embedding model that excels in long-document and reasoning-intensive retrieval.
1
6
3
25
6,912