Chenghao Yang (@chrome1996) profile picture

Senior AS @Microsoft. Ph.D. @UChicago Ex-SR @google Ex-Scientist @AWS. Ex-RA @jhuCLSP @columbianlp @TsinghuaNLP. Ex-Intern @IBM @AWS. Opinions are my own.

Redmond, WA
Joined March 2017
Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)Photo by Chenghao Yang (@chrome1996)
Chenghao Yang retweeted
Excited to share that I’ll be presenting our #ICML2026 paper in Seoul: ExVerus: Verus Proof Repair via Counterexample Reasoning 📍 Poster: Thu, Jul 9, 5:00–6:45 PM KST, Hall A #2004 💡 DL4C Workshop: Fri, Jul 10, Session 2, 3:20–4:50 PM KST, Hall A (Board 100-119) Paper: openreview.net/forum?id=FNDk… Happy to discuss! This work focuses on LLM-based Verus proof repair. Verus is a practical Rust-based deductive verification framework that has been used to verify large-scale critical systems, such as cluster management controllers (Anvil) and virtual machine security modules (VeriSMo). In this work, we propose a new paradigm for LLM-based proof repair: repairing a proof via synthesizing, validating, and blocking counterexamples. 1/n
1
7
15
1,164
Chenghao Yang retweeted
[🛫ICML] We will be presenting our paper about *base-aligned model collaboration* on Tuesday morning and also Saturday (HAI Co-Creativity workshop). Happy to chat! PS: We've updated our paper, code, and reading list to include the latest results and resources. Check it out!
Lack of diversity in your LLM generation? (also noted by Artificial Hivemind, best paper @NeurIPSConf) Time to bring your base model back! An inference-time, token-level collaboration between a base and an aligned model can optimize and control diversity and quality!
9
23
2,774
Chenghao Yang retweeted
Introducing VaSE: Value-Aware Stochastic KV Cache Eviction. Reasoning models think in CoT, bloating the KV cache. Eviction caps memory but suffers capability drop. VaSE is a training-free recipe that cuts that cost: keep large-magnitude value states, evict stochastically.
1
5
3
35
58,723
Chenghao Yang retweeted
On-policy RL has driven the biggest leaps in training coding agents. Extending it to machine learning engineering agents should be a natural next step. But it almost never works. What I mean is, the recipe is right there — standard trajectory-wise GRPO, the same that worked for SWE. However, the problem is that one rollout step on an MLE task may take hours because the agent has to actually train a model on a real dataset at every step (preprocessing, fitting, inference, scoring). So even with the N rollouts in a group running in parallel, a single GRPO run may still take days. Every MLE agent paper I've read has retreated to SFT or offline proxy rewards for exactly this reason, giving up the exploration benefits of on-policy learning. That's why I'm excited about our new paper, SandMLE, which fixes this with a move that sounds almost too reckless to work. The instinct when on-policy RL is too slow is to engineer around it — async rollouts so the trainer doesn't sit idle waiting for slow environments, off-policy or step-wise proxies to avoid running full trajectories at all. But when we profiled where the time was going, the bottleneck had nothing to do with the algorithm. Unlike SWE where execution latency comes from compilation and test logic, MLE latency is overwhelmingly driven by the size of the dataset the ML pipeline has to chew through. Therefore, rather than downsampling existing data (which corrupts evaluation), we built a multi-agent pipeline that procedurally generates diverse synthetic MLE environments from a small seed set. Specifically, we extract the structural DNA of seed tasks (modality, label cardinality, distribution shape), mutate them into new domains (e.g., repurposing animal classification into road damage detection), inject realistic noise, embed deterministic hidden rules connecting features to labels, and construct full evaluation sandboxes with progressive milestone thresholds. Each task is constrained to only 50–200 training samples. The execution speedup is dramatic — average per-step latency drops over 13×, which makes trajectory-wise GRPO go from infeasible to routine. We also designed a dense, milestone-based reward to address the sparse credit assignment problem in long-horizon MLE. The ablation shows this matters — under a sparse reward, the 30B model's medal rate drops from 27.3% to 13.6% and valid submission collapses from 100% to 86.4%. Results across Qwen3-8B, 14B, and 30B-A3B on MLE-bench are consistently strong — 66.9% better performance in medal rate over SFT baselines. It is worth noting that the SFT baselines are not weak— we trained them on high-quality Claude-4.5-Sonnet trajectories. But SandMLE still delivers much larger gains, suggesting that direct environment interaction does teach capabilities that imitation alone does not (as expected). The most convincing evidence to me that the model's intrinsic performance gets improved is the framework-agnostic generalization. We trained exclusively with ReAct but the gains transfer to AIDE, AIRA, and MLE-Agent scaffolds at evaluation time — up to 32.4% better performance in HumanRank on MLE-Dojo. The SFT models, by contrast, are brittle when moved to unfamiliar scaffolds. The 30B SFT model collapses to 17.7% valid submission rate on MLE-Dojo with MLE-Agent, while the 30B SandMLE model achieves 83.9%. SandMLE is teaching genuine engineering reasoning, not scaffold-specific patterns. What I find most interesting beyond the specific result is that none of the hard parts of RL changed here. The algorithm is the same. The reward is conventional. We just shrunk the environment until on-policy learning became affordable. The field has largely treated environment design and RL algorithm design as separate concerns. SandMLE is a concrete case that the environment is itself the lever. When training is too expensive, the instinct is to build cleverer algorithms to tolerate it. However, often the better move is to reshape the environment so the simple algorithm just works. Paper: arxiv.org/pdf/2604.04872
5
42
3
285
34,923
We are starting in 15 mins!! Join us for the #NewInML workshop happening now at @NeurIPSConf! Location: Upper Ballroom 31ABC, San Diego Convention Centre. We have an amazing speaker lineup that you don’t want to miss!
2
2
3
1,587
📢Our @NewInML workshop will be happening today at the Room Upper 31ABC, San Diego Convention Center, starting at 12 PM! If you're at @NeurIPSConf, it's a great opportunity for you to join us! We also have amazing speakers! We're looking forward to welcoming you! #NeurIPS2025
🚨 New in ML Workshop at @NeurIPSConf We're so excited to invite you to the New In ML Workshop (@NewInML), taking place on Tuesday, December 2nd, 2025, at the San Diego Convention Center! Great opportunity, specifically for people who are new in machine learning! Details🧵
5
10
958
🚨 New in ML Workshop at @NeurIPSConf We're so excited to invite you to the New In ML Workshop (@NewInML), taking place on Tuesday, December 2nd, 2025, at the San Diego Convention Center! Great opportunity, specifically for people who are new in machine learning! Details🧵
5
19
3
80
19,160
Chenghao Yang retweeted
Mitigating racial bias from LLMs is a lot easier than removing it from humans! Can’t believe this happened at the best AI conference @NeurIPSConf We have ethical reviews for authors, but missed it for invited speakers? 😡
172
772
337
3,687
2,229,193
Chenghao Yang retweeted
🧐Can we create a navigational agent that can handle thousands of new objects across a wide range of scenes? 🚀 We introduce DivScene bench and NatVLM. DivScene contains houses of 81 types and thousands of target objects. NatVLM is an end-to-end agent based on a Large Vision Language Model (LVLM) for generalizable navigation. Homepage: zhaowei-wang-nlp.github.io/d… Arxiv: arxiv.org/abs/2410.02730 Code and Data: github.com/zhaowei-wang-nlp/…
2
22
4
36
6,906
Chenghao Yang retweeted
Excited to share our #NAACL2024 paper! We discovered that self-reflective prompting in LLMs shows mixed results. 👀Key insight: self-reflection is most effective when models are uncertain, and it works by reducing the model majority-following tendencies over initial responses.
1
1
1
12
1,444
Incredibly excited to announce that Chenhao, Mina, and I are taking on the challenge of investigation the interplay of communication and intelligence in this new era of non-human language users!
Announcing the Communication & Intelligence (C&I) group at UChicago! Comprised of @universeinanegg, @MinaLee__, @ChenhaoTan—C&I will tackle how AI and communication co-evolve as LLMs break long-held assumptions. We're recruiting PhDs & Postdocs for 2024! uchicagoci.github.io/
1
7
55
7,765
Chenghao Yang retweeted
It was my honor to lead this collaborative effort. It'd never be possible without co-organizers @chrome1996 @saujasv @hyunw__kim @jennifer @nlpxuhui @alsuhr @_jennhu, advisory board @dan_fried @gneubig @ybisk @aidanematzadeh. And huge thanks to reviewers!
❤️ We thank everyone who made this workshop possible! It was a fantastic experience to meet all the amazing participants, speakers, and panelists! 🌟 We KNOW we will meet again at the next ToM workshop! 😉
2
11
1
51
10,194
Chenghao Yang retweeted
FINAL UPDATE: On June 24th, Armando Solar-Lezama (Professor in EECS and COO/Associate Director of CSAIL, MIT), Tonio Buonassisi (Professor of Mechanical Engineering, MIT), and Yoon Kim (Assistant Professor in EECS and CSAIL, MIT) released a public statement regarding the paper.
A recent work from @iddo claimed GPT4 can score 100% on MIT's EECS curriculum with the right prompting. My friends and I were excited to read the analysis behind such a feat, but after digging deeper, what we found left us surprised and disappointed. dub.sh/gptsucksatmit 🧵
16
96
61
580
774,619
Chenghao Yang retweeted
Do you have an idea/paper related to theory-of-mind that you want everyone to know?😎 You know that we encourage all 🌟ICML/ACL papers & rejections and NeurIPS/EMNLP submissions🌟 to our workshop, right? We know everyone is going to love your paper here😍 openreview.net/group?id=ICML…
10
1
28
7,053
Chenghao Yang retweeted
1. 🔔**𝘾𝙖𝙡𝙡 𝙛𝙤𝙧 𝙋𝙖𝙥𝙚𝙧𝙨 𝙛𝙤𝙧 𝙏𝙝𝙚𝙤𝙧𝙮-𝙤𝙛-𝙈𝙞𝙣𝙙 𝙒𝙤𝙧𝙠𝙨𝙝𝙤𝙥**🔔 The First Workshop on Theory of Mind in Communicating Agents (ToM 2023) will be hosted at @icmlconf in July'23 in Honolulu 🌺 CfP: tomworkshop.github.io 🧵 #ICML2023 #ToM2023 #ML #NLProc
3
35
10
88
45,652
Chenghao Yang retweeted
Thrilled and honored to be selected as a Sloan Research Fellow this year! This would have not been possible without the support from students and collaborators @ChicagoHAI. Looking forward to further advance human-centered AI! #SloanFellow
We are delighted to announce the winners of this year’s Sloan Research Fellowship! These outstanding researchers are shining examples of innovation and impact—and we are thrilled to support them. Meet the winners here: sloan.org/fellowships/2023-F… 🎉 #SloanFellow #STEM #ScienceTwitter
36
12
250
41,514
Chenghao Yang retweeted
We release LLM.int8(), the first 8-bit inference method that saves 2x memory and does not degrade performance for 175B models by exploiting emergent properties. Read More: Paper: arxiv.org/abs/2208.07339 Software: huggingface.co/blog/hf-bitsa… Emergence: timdettmers.com/2022/08/17/l…
16
249
43
1,201
Chenghao Yang retweeted
(1/n) I am excited to share our paper "machine explanations and human understanding", which I will be presenting at the ICML workshop on Human-Machine Collaboration and Teaming today 4:30-4:40 ET! Our paper was selected as the best paper at the workshop.
2
10
1
70
Chenghao Yang retweeted
Got a new explainable model or a post-hoc explanation algorithm? Does it work? Check out our #NAACL2022 tutorial on human-centered evaluation of explanations next Sunday at 2 PST, and tutorial content and video on our website: xai-hcee.github.io! 🧵
1
27
4
86