CSE Ph.D. @UMich | Intern @ProximalHQ | Social Intelligence of LLMs
Ann Arbor
Joined February 2024
- Tweets63
- Following278
- Followers248
- Likes154
Pinned Tweet
🔥New paper accepted to COLM!🔥
How should LLMs reason about moral dilemmas across cultures? We tackle this with three contributions:
1⃣ MCLASH: a culturally-adapted evaluation dataset
2⃣ MET: a theory-grounded reasoning method
3⃣ MET-D: MET + self-distillation
📰 arxiv.org/pdf/2607.11736
🤗 huggingface.co/collections/l…
🧵👇
Ayoung Lee retweeted
Not All Privileged Information Is Equal: What Makes OPSD Actually Work?
We find that information-rich PI, such as environment observations and mistake analysis, outperforms PI that only provides ground-truth solutions.
Better PI leads to better post-training performance. But can we predict which PI will work best before spending compute on post-training?
We investigated three offline metrics:
🔹 Utility: A model's performance with PI can be misleading due to information leakage.
🔹 Reverse KL: Larger PI-induced distribution shifts often correlate with better post-training performance, but incorrect PI can cause equally large shifts.
🔹 Error-step KL contrast: Measuring whether PI changes the model's behavior specifically at error steps better captures its effectiveness and helps select stronger PI.
What types of privileged information have you found most effective in your OPSD experiments?
📄 Paper: arxiv.org/abs/2610.22512
Ayoung Lee retweeted
User simulators are trained to talk like users. But users don’t just talk, they judge.
We introduce simulators that model both: what users say and what they think of the assistant.
COLM was a blast! 🔥 Had so many inspiring discussions about social simulations and self-improving LLMs for understanding social dynamics. It was great catching up with old friends and making new ones!
I'm currently working on evaluating the long-term fidelity of LLM-based social simulations by comparing simulated trajectories with their real-world counterparts. Open to collaborations and discussions 🤗
Ayoung Lee retweeted
We are hiring for the following roles (DMs open!):
1) Research Engineer: Run eval and training experiments to understand how to improve model capabilities
2) Platform Engineer: Build infrastructure to orchestrate millions of agents and sandboxes per day
3) Technical Generalist: Work closely with our enterprise and frontier lab customers and own projects end to end
I’ll be heading to SF for COLM next week! I’m interested in building AI assistants that can handle larger-scale collaborative tasks, with a focus on social dynamics. My research interests include multi-agent collaboration, LLM simulations, social intelligence, and self-improvement. I’m also actively looking for Summer 2027 internship opportunities :) Please DM me if you’d like to chat!
I'll be presenting our work on improving culture-aware moral decision-making through a self-improvement approach. Please stop by if this sounds interesting!
💡 Poster Session 1, Oct 6th 11 AM–1 PM PDT, Franciscan B #82
📄 arxiv.org/abs/2607.11736
Ayoung Lee retweeted
What if you give an Agent a task…
…but it also has a different goal of its own?🫥
Now it has a choice: do what you asked, or quietly help its own goal too.
So what does it actually do? 👀
📄SchemeArena: Factorized Stress Testing of Scheming in LLM Agents arxiv.org/pdf/2609.08126v1
Ayoung Lee retweeted
The final lecture of my course is an intro to character training! This is a topic that I've been quietly very invested in for ~18 months, as it:
* Has potential for high real world impact
* Clearly used extensively at frontier labs
* Almost no empirical literature exists
* More accessible on academic compute
This lecture covers what character training is, reviews model specs, constitutions, the differences, the motivations in real world events, some example research papers I like, and open questions in how it relates to post-training/model use generally.
Hopefully this brings more people into the field (and reach out if you have questions). It is one of the more research-y chapters in my book, but one that I felt needed the reference. There is still so little, educational content on the topic online.
0:00 Intro
6:22 Part 1: Fundamentals — character, constitutions, and model specs
19:21 Part 2: Character training in practice
23:23 Part 3: Character elicitation without gradient steps
28:03 Part 4: Open questions (and the end of the course)
32:27 The course, complete
Thanks for watching. No need to like and subscribe now that the course is done, you definitely wouldn't!
h/t to @_maiush for leading the technical work I got to do in the space, and @zafstojano for investing a lot of attention at this book chapter.
Ayoung Lee retweeted
Our team is looking for a student researcher. If the topics below sound exciting, feel free to fill out the form!
lnkd.in/gyVfg4Um
This quoted post is unavailable.
(8/n) We also analyze cross-lingual transfer. Three takeaways.
💡Linguistic similarity drives transfer but cultural similarity does not.
💡Training on high-resource languages is not always helpful.
💡Korean shows an unusually high own-language gain (+9.8 points) due to its unique script.
(9/n) Together, we open the path for culture-aligned, theory-grounded multilingual moral reasoning. Feel free to reach out for further discussion!
📰 Full paper: arxiv.org/pdf/2607.11736
🤗 Dataset and models: huggingface.co/collections/l…
Thanks to the amazing collaborators 💕 @ryan_kwon03, @YunxiangZhang4, Yuxuan Liu, Peter Railton, and @LuWang__
Interesting analyses on internal reasoning based on J-space! Internal reasoning can diverge from the stated CoT; auditing it would help understand how LLMs actually think, improve their reasoning abilities, and most importantly, monitor their intentions to make them safe.
