Chacha Chen (@chachaachen) profile picture

MTS @MicrosoftAI / PhD @ChicagoHAI @UChicagoCS, human-centered ML.

Mountain View, CA
Joined October 2016
Photo by Chacha Chen (@chachaachen)Photo by Chacha Chen (@chachaachen)Photo by Chacha Chen (@chachaachen)Photo by Chacha Chen (@chachaachen)Photo by Chacha Chen (@chachaachen)Photo by Chacha Chen (@chachaachen)
✨ New paper at #FAccT2023 next week ✨ Machine Explanations and Human Understanding arxiv.org/abs/2202.04092 Explanations are hypothesized to improve human understanding of machine learning models. However, empirical studies have found mixed and even negative results.
2
46
3
181
30,739
Chacha Chen retweeted
🚀 Recruiting Student Researchers (SRs) interested in Multilingual RL! Current student working on RL, LLM post-training, or multilingual LLMs? We'd love to hear from you! Apply: google.com/about/careers/app… CV + interests: docs.google.com/forms/d/e/1F… Reposts appreciated!
3
25
5
158
36,708
Chacha Chen retweeted
Brilliant idea! Next up: Apple randomly reboots your Mac if you're building competing tech, Gmail silently edits your email if you mention rival platforms, and Tesla Autopilot swerves if it detects you're working on self-driving cars. All in the name of safety, of course. Because malicious actors controlling the world’s operating systems, inboxes and cars would be extremely dangerous!
mythos will be bad ON PURPOSE on ai "frontier llm research" tasks, this is very very sad for the research community also the fact that this is un purpose not visible to the user is crazy
94
744
51
6,656
369,259
🚨Why can’t Transformers learn multiplication?🧮 Even with billions of params, models struggle with multi-digit multiplication. In our new work, we reverse-engineer two models: a standard fine-tuned (SFT), and an implicit chain-of-thought (ICoT) model to see why. Read on! 1/n 🧵
11
101
7
657
55,473
1/n You may know that large language models (LLMs) can be biased in their decision-making, but ever wondered how those biases are encoded internally and whether we can surgically remove them?
1
9
1
13
2,719
🚨 New paper alert 🚨 Ever asked an LLM-as-Marilyn Monroe who the US president was in 2000? 🤔 Should the LLM answer at all? We call these clashes Concept Incongruence. Read on! ⬇️ 1/n 🧵
2
10
2
19
1,957
I’m going to present at the poster session on Friday morning. Come find me! 😃😃
Just got accepted to NAACL!
1
11
1,577
CaseSumm is publicly available on HuggingFace! We hope this dataset enables: - Better evaluation of long-context summarization - Research on legal language understanding - Development of more accurate & reliable legal AI tools Dataset: huggingface.co/datasets/Chic… I'll be presenting this work at 2pm and will be around until Sunday. Please reach out if you're interested in this line of work - would love to connect in person or virtually!
1
1
1
229
🚀🚀🚀Excited to share our latest work: HypoBench, a systematic benchmark for evaluating LLM-based hypothesis generation methods!
1
11
1
15
2,286
Just got accepted to NAACL!
1/ ✨ Can Multimodal LLMs generate radiology reports from chest X-ray images? In our recent evaluation paper, we show: not yet.
1
1
29
6,782
Chacha Chen retweeted
Even ChatGPT knows. It is very hard to think of a positive spin for Rosalind here. A shocking display of ignorance and arrogance.
Mitigating racial bias from LLMs is a lot easier than removing it from humans! Can’t believe this happened at the best AI conference @NeurIPSConf We have ethical reviews for authors, but missed it for invited speakers? 😡
2
11
143
10,832
1/ ✨ Can Multimodal LLMs generate radiology reports from chest X-ray images? In our recent evaluation paper, we show: not yet.
5
11
1
17
7,460
and I especially like the human evaluation we did. The ground truth reports are 100% clinically usable, although they slightly fall behind the fine-tuned Llama in terms of clarity and readability.
1
292
the last row shows GPT-4v's gap with SOTA models
1
267
Joint work with @ChicagoHAI and @bmervakMD
1
160
We’re excited to present this work at @SymposiumML4H next month in Vancouver. Happy to chat and feel free to reach out 😊.
160
4/ Our Evaluation Overview: We started with direct report generation using different prompting strategies, but found the model failed terribly. To understand the low performance, we decomposed the task into two steps: image reasoning and report synthesis.
1
228
5/ We also conducted a human reader study to compare the quality and readability of AI-generated and human written reports. Paper: arxiv.org/abs/2407.12176 Code:github.com/ChicagoHAI/cxr-ev… Altogether, our findings cast doubt on the viability of using GPT-4V in a radiology workflow.
181
I will be in Rio for #FAccT2024 next week. Hit me up if you will also be there in person and would like to chat!🏖️🍻😎
1
22
2,859