MTS @AnthropicAI. Prev: RS @GoogleDeepMind. Postdoc @Meta. PhD @UCBerkeley. ML security, privacy, robustness.
United States
Joined March 2015
- Tweets81
- Following695
- Followers355
- Likes346
Chawin Sitawarin retweeted
The first autonomous agent cyberattack is an unprecedented event that deserves unprecedented transparency. Today we’re sharing everything we can: a full technical timeline, an interactive replay, and how we used an open model to defend ourselves, so defenders everywhere can learn from it and prepare for what’s next.
huggingface.co/blog/agent-in…
I'm in Seoul to enjoy Korean food and help present some papers @icmlconf (while thinking about Korean food).
Please don't hesitate to reach out. Happy to chat about research and stuff.
You can find a pic of soy-marinated crabs I had today below along with the paper info ⬇️
Rapid Poison: Practical Poisoning Attacks Against the Rapid Response Framework: arxiv.org/abs/2606.16242 (spotlight). Special shout out to @davidhuang33176, Jaewon Chang, @Avidan_Shah56.
📍Poster session 5 (Wed evening)
Extracting alignment data in open models: arxiv.org/abs/2510.18554 w/ @fedzbar et al.
📍Poster session 8 (Thu evening)
How much do language models memorize?: arxiv.org/abs/2505.24832 (oral, best paper honorable mention)
📍Poster session 8 (Thu evening)
Chawin Sitawarin retweeted
Honored that our paper on memorization with @jxmnop, @csitawarin @srush_nlp Chuan Guo, Narine Kokhlikyan, Edward Suh and Saeed Mahloujifar got an Outstanding Paper Honorable mention at ICML this year! Most of the credit goes to @jxmnop and Saeed.
Paper: arxiv.org/abs/2505.24832
🏆Announcing the #ICML2026 Awards! 🏆
Including Outstanding Papers (research paper & position paper, winner & honorable mentions) and the Test of Time Award!
Check out the blog post for all winners (or read on), laudatio, & description of the processes.
blog.icml.cc/?p=1347
🇧🇷 Going to ICLR!
Will present a poster on data poisoning attack at Agents in the Wild workshop.
Great work by @davidhuang33176 Jaewon Chang, @Avidan_Shah56, @prateekmittal_
📄 openreview.net/forum?id=tI0o… (full paper, website, code soon). But please come visit and chat!
Chawin Sitawarin retweeted
To be honest, I was initially confused and reserved about AI alignment. It's not that I was against the research direction, quite the opposite. For 15 years, I'd been developing the foundations of what had been rebranded as alignment. But, I've changed my mind. 1/6
Chawin Sitawarin retweeted
It is notoriously hard to defend LLMs against prompt injections. Most defenses show good performance on static benchmarks but fall apart against stronger adaptive attackers.
In our latest work, we present an almost embarrassingly simple defense that delivers ~3× better robustness against the strongest adaptive prompt injection attacks to date - while keeping utility degradation acceptable.
Joint work with @csitawarin, Jamie Hayes, @davidstutz92, @iliaishacked.
Chawin Sitawarin retweeted
Feel free to check out the paper here :)
arxiv.org/abs/2510.18554
Special thanks to the amazing co-authors!
w/ @gu_xiangming @Chris_Choquette @csitawarin Matthew Jagielski @itay__yona @PetarV_93 @iliaishacked Jamie Hayes
Chawin Sitawarin retweeted
5 years ago, I wrote a paper with @wielandbr @aleks_madry and Nicholas Carlini that showed that most published defenses in adversarial ML (for adversarial examples at the time) failed against properly designed attacks.
Has anything changed?
Nope...
Chawin Sitawarin retweeted
🚨 Got a great idea for an AI + Security competition?
@satml_conf is now accepting proposals for its Competition Track! Showcase your challenge and engage the community.
👉 satml.org/call-for-competiti…
🗓️ Deadline: Aug 6
Very cool thought-provoking piece! In practice, computation units are much more nuanced than what theories capture. But just trying to identify classes of problems that benefit from sequential computation (or is unsolvable without it) seems very useful!
Some problems can’t be rushed—they can only be done step by step, no matter how many people or processors you throw at them.
We’ve scaled AI by making everything bigger and more parallel: Our models are parallel. Our scaling is parallel. Our GPUs are parallel.
But what if the real bottleneck isn’t size—but depth?What if the model just didn’t have enough serial steps to get it right? Some problems need depth, not width.
This is the Serial Scaling Hypothesis.
This is not the same as recent studies in scaling test-time compute, which focus on train vs. test and are agnostic to parallel vs. serial.
For example: test-time majority voting increases compute by running models in parallel — but doesn’t help when the task itself is serial.
We argue: what really matters is how the compute is structured. And for many real-world problems, it must be serial.
Read more at: arxiv.org/abs/2507.12549 or 🧵.
(In collaboration with: @layer07_yuxi , Kananart Kuwaranancharoen and @YutongBAI1002 )
Chawin Sitawarin retweeted
Some problems can’t be rushed—they can only be done step by step, no matter how many people or processors you throw at them.
We’ve scaled AI by making everything bigger and more parallel: Our models are parallel. Our scaling is parallel. Our GPUs are parallel.
But what if the real bottleneck isn’t size—but depth?What if the model just didn’t have enough serial steps to get it right? Some problems need depth, not width.
This is the Serial Scaling Hypothesis.
This is not the same as recent studies in scaling test-time compute, which focus on train vs. test and are agnostic to parallel vs. serial.
For example: test-time majority voting increases compute by running models in parallel — but doesn’t help when the task itself is serial.
We argue: what really matters is how the compute is structured. And for many real-world problems, it must be serial.
Read more at: arxiv.org/abs/2507.12549 or 🧵.
(In collaboration with: @layer07_yuxi , Kananart Kuwaranancharoen and @YutongBAI1002 )
I will be at ICML this year after a full long year of not attending any conference :)
Happy to chat, and please don’t hesitate to reach out here, email, on Whova, or in person 🥳
Chawin Sitawarin retweeted
We are starting our journey on making Gemini robust to prompt injections and in this paper we present the steps we have taken so far.
A collective effort by the GDM Security & Privacy Research team spanning over > 1 year.
Our new @GoogleDeepMind paper, "Lessons from Defending Gemini Against Indirect Prompt Injections," details our framework for evaluating and improving robustness to prompt injection attacks.
Chawin Sitawarin retweeted
new paper from our work at Meta!
**GPT-style language models memorize 3.6 bits per param**
we compute capacity by measuring total bits memorized, using some theory from Shannon (1953)
shockingly, the memorization-datasize curves look like this:
___________
/
/
(🧵)
Chawin Sitawarin retweeted
1/🔒Worried about giving your agent advanced capabilities due to prompt injection risks and rogue actions? Worry no more! Here's CaMeL: a robust defense against prompt injection attacks in LLM agents that provides formal security guarantees without modifying the underlying model!
Chawin Sitawarin retweeted
🧵 Announcing @open_phil's Technical AI Safety RFP!
We're seeking proposals across 21 research areas to help make AI systems more trustworthy, rule-following, and aligned, even as they become more capable.
