Senior Technical Staff Member, NLP at IBM Research AI. Opinions are my own
Israel
Joined January 2013
- Tweets25
- Following36
- Followers43
- Likes34
Roy Bar Haim retweeted
🚨 Benchmarks tell us which model is better — but not why it fails.
For developers, this means tedious, manual error analysis. We're bridging that gap.
Meet CLEAR: an open-source tool for actionable error analysis of LLMs.
🧵👇
Roy Bar Haim retweeted
🔔 New Paper!
We propose a challenging new benchmark for LLM judges: Evaluating debate speeches.
Are they comparable to humans? Well... it’s debatable. 🤔
noy-sternlicht.github.io/Deb…
👇 Here are our findings:
Roy Bar Haim retweeted
Interested in Agent Evaluation? 🤖
We’re excited to launch our new repo:
“Evaluation of LLM-based Agents: A Reading List” 📚
Browse benchmarks, methods, and frameworks from our recent survey.
👉 Explore & Contribute: github.com/Asaf-Yehudai/LLM-…
#LLMAgents #AgentEvaluation
Roy Bar Haim retweeted
Survey on Evaluation of LLM-based Agents 🤖
Our paper is the first to provide a comprehensive overview of LLM-based agent evaluation 📜
Paper: arxiv.org/pdf/2503.16416
Roy Bar Haim retweeted
New preprint! ✨
Interested in LLM-as-a-Judge?
Want to get the best judge for ranking your system?
our new work is just for you:
"JuStRank: Benchmarking LLM Judges for System Ranking"
🕺💃
arxiv.org/abs/2412.09569
Roy Bar Haim retweeted
Say I want to compare system qualities - pick between 2 configurations, or rank a whole bunch of models. I'll use LLM-as-a-judge, right? 🧑🏻⚖️
But how do I know the LLM judge is up to the task? Who is a good judge for ranking systems?
Enter our new paper!✨🧵
arxiv.org/abs/2412.09569
Roy Bar Haim retweeted
ArgMining 2024 ended with a great photo of its wonderful community. Kudos to all of your great ideas, contributions, and help in organizing.
Roy Bar Haim retweeted
The call for papers for the 11th Workshop on Argument Mining #argminig_2024 is now out:
argmining-org.github.io/2024…
Roy Bar Haim retweeted
We are excited to announce that the Argument Mining workshop will take place at #ACL2024 in Bangkok, Thailand. For more info see our website at argmining-org.github.io/2024…
Roy Bar Haim retweeted
We are happy to announce two shared tasks for ArgMining 2024: 1) Perspective Argument Retrieval organized by Neele Falk and Andreas Waldis. 2) DialAM-2024 organized by Ramon Ruiz-Dolz, John Lawrence, Ella Schad, and Chris Reed.
Roy Bar Haim retweeted
Curious to see how can we summarize opinions beyond plain text summaries?
Check out our #ACL2023 paper: From Key Points to Key Point Hierarchy: Structured and Expressive Opinion Summarization
with Lilach Eden, @yoavkantor @RoyBarHaim from @IBMResearch @IBM @biunlp
>>
Roy Bar Haim retweeted
Want to build a text classifier in a few hours?
Even if you don’t have any:
labeled data
#machineLearning knowledge
programing skills
Label Sleuth ibm.biz/label-sleuth a new open-source no-code system for annotations 🧵 @IBMResearch @NotreDame @StanfordHCI UT Dallas #NLProc
Roy Bar Haim retweeted
Interested in TARGETED #SentimentAnalysis beyond restaurant reviews? In #NAACL2022 we suggest a robust multi-domain model relying on self-training, with no extra annotation -- arxiv.org/abs/2205.03804
@OrithToledo @MatanOrbach @YoavKatz73 @noamslonim
#NLProc #IBMResearch
(1/5)
Roy Bar Haim retweeted
Welcome PrimeQA at #NAACL2022! Replicate the state-of-the-art on multilingual open QA quickly! Here’s a new open-source repo in collab with with @stanfordnlp, @huggingface, @Uni_Stuttgart @NLPIllinois1. Link: github.com/primeqa/primeqa Talk to me or read:
research.ibm.com/blog/primeq… 🧵
NAACL 2022 is starting on Sunday! Visit our website ibm.biz/naacl2022 to learn about the exciting NLP work from IBM Research that will be presented at this conference. @IBMResearch @naaclmeeting #NAACL2022
NAACL'21 main conference is starting today!
Meet our researchers and recruiting team at the @IBMResearch virtual booth: underline.io/events/122/expo…, and learn more about IBM Research's presence at @NAACLHLT, careers and booth schedule at ibm.biz/naacl2021