@gu_yuling

PhD-ing at NYU in NYC 🚕🍎 | Previously @nyuniversity ➡️ @UW ➡️ @allen_ai @yuling_gu@sigmoid.social

yulinggu-cs.github.io/
Joined September 2019
  • Tweets142
  • Following828
  • Followers898
LLMs may not be natural explorers out of the box… 🫤 🚀 But we can make them better explorers! 🚀 Introducing 💡 SCOOP (Strategy COordination OPtimization) 🌟, a simple framework that unlocks effective exploration in LLMs. Blog post: yulinggu-cs.github.io/blogs/… 🧵1/8
4
18
3
82
11,604
🌍 Can these learned principles transfer to OOD competitions? The HMMT champion strategist successfully transfers to the out-of-distribution BRUMO 2025 competition! This suggests that SCOOP learns generalizable principles for strategized exploration, rather than benchmark-specific tricks. 🧵7/8
1
1
2
193
LLMs may not be natural explorers out of the box. But with SCOOP, they can learn to explore more effectively, navigating complex problem spaces across challenging tasks. 🗺️ 🚀 We hope this is a step toward AI systems that help humanity unlock discoveries at the hardest frontiers of science and beyond. 🧵8/8
1
3
160
Check out SimpleToM at #ICLR2026 where we reveal a critical fragility in LLMs’ social reasoning — the explicit vs. applied ToM gap. 🗓️Fri, Apr 24, 2026 3:15 PM – 5:45 PM BRT 📍Pavilion 3 P3-#1407
Yuling Gu
@gu_yulingMar 6
🎉 SimpleToM has been accepted to #ICLR2026! LLMs can tell you what someone knows (explicit ToM). But when asked to apply it to predict behavior or judge actions (applied ToM), even frontier LLMs still fail. 🤯 The gap between knowing and applying is real… and huge. 👀 1/
1
1
18
2,264
I can’t be there in person due to visa issues, but please meet my amazing co-authors! My DM (and email) are open if you’d like to connect!
1
3
525
🎉 SimpleToM has been accepted to #ICLR2026! LLMs can tell you what someone knows (explicit ToM). But when asked to apply it to predict behavior or judge actions (applied ToM), even frontier LLMs still fail. 🤯 The gap between knowing and applying is real… and huge. 👀 1/
2
17
2
121
15,920
more replies
SimpleToM exposes this gap 🔎 and provides a benchmark to diagnose, improve, and push LLMs toward robust social reasoning 🚀 Try SimpleToM on any model : huggingface.co/datasets/alle… 5/

allenai/SimpleToM · Datasets at Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co
1
2
5
455
Work done during my time at @allen_ai with wonderful collaborators Oyvind Tafjord, @hyunw_kim, @jaredlcm, @Ronan_LeBras, Peter Clark, @YejinChoinka. 📜 Paper: arxiv.org/abs/2410.13648 💻 Code: github.com/yulinggu-cs/Simpl… 6/

SimpleToM: Exposing the Gap between Explicit ToM Inference and...

Large language models (LLMs) are increasingly tested for a "Theory of Mind" (ToM) - the ability to attribute mental states to oneself and others. Yet most evaluations stop at explicit belief...

arxiv.org
2
9
408
i gave a keynote talk at NeurIPS'25 just last week. here's the slide deck (link below) i've used to share my thoughts on who we are and what we do.
3
28
245
20,664
Super proud of the amazing work that my Ai2 friends have been doing! 🤩 Check this out! ✨
Ai2
@allen_ai20 Nov 2025
Announcing Olmo 3, a leading fully open LM suite built for reasoning, chat, & tool use, and an open model flow—not just the final weights, but the entire training journey. Best fully open 32B reasoning model & best 32B base model. 🧵
11
1,241
🌍 Introducing WorldValuesBench! A benchmark to evaluate how well LLMs reflect cultural differences in human values. Built from 94k+ participants in the World Values Survey → 20M examples of (demographics, value question → answer). 🧵
1
2
5
666
Evaluating language models is tricky, how do we know if our results are real, or due to random chance? We find an answer with two simple metrics: signal, a benchmark’s ability to separate models, and noise, a benchmark’s random variability between training steps 🧵
Ai2
@allen_ai19 Aug 2025
📢 New paper from Ai2: Signal & Noise asks a simple question—can language model benchmarks detect a true difference in model performance? 🧵
4
54
9
239
47,870
Load more