@NegarEmpr
Account based inUnited States
About this account
- Account based in
- United States
- Connected via
- United States App Store
Account-level information from X, not a live location or the device used for a specific post.
Postdoc @UCBerkeley @BerkeleySky |๐ฉ๐ปโ๐ปPrev @google, @MSFTResearch, @SpotifyResearch | ๐@UWaterloo | Interested in Information Retrieval
Berkeley, USA
Joined April 2017
- Tweets549
- Following1.2K
- Followers1.7K
- Likes6K
Pinned Tweet
So excited to share that I received the SIGIR Early Career Research Award! ๐ฅน
I still remember attending my first SIGIR, watching someone receive this award, and thinking, โThereโs no way that could ever be me.โ And now, here I am!๐ต
This truly means so much to me. Thank you to the @ACMSIGIR community for recognizing my contributions. #SIGIR2026
P.S. Get yourself a PhD supervisor who matches your award-celebration energy ๐ So cool to celebrate this moment together! @claclarke
Negar Arabzadeh retweeted
๐ข ๐๐๐๐ ๐ฎ๐ฌ๐ฎ๐ณ ๐ช๐ผ๐ฟ๐ธ๐๐ต๐ผ๐ฝ ๐๐ฎ๐น๐น ๐ณ๐ผ๐ฟ ๐ฃ๐ฎ๐ฝ๐ฒ๐ฟ๐
๐ง๐ฟ๐๐๐๐๐ผ๐ฟ๐๐ต๐ ๐๐๐ผ๐น๐๐๐ถ๐ผ๐ป ๐ผ๐ณ ๐๐ ๐๐ด๐ฒ๐ป๐๐: ๐ผ๐ฃ ๐ผ๐๐๐ฃ๐ฉ ๐๐ฅ๐๐ง๐๐ฉ๐๐ฃ๐ ๐๐ฎ๐จ๐ฉ๐๐ข ๐๐๐ง๐จ๐ฅ๐๐๐ฉ๐๐ซ๐
๐ ๐ฃ๐ฎ๐น๐ฎ๐ถ๐ ๐ฑ๐ฒ๐ ๐๐ผ๐ป๐ด๐ฟ๐ฒฬ๐, ๐ ๐ผ๐ป๐๐ฟ๐ฒฬ๐ฎ๐น, ๐๐ฎ๐ป๐ฎ๐ฑ๐ฎ ๐จ๐ฆ
๐๏ธ ๐๐ฒ๐ฏ๐ฟ๐๐ฎ๐ฟ๐ ๐ฎ๐ฎโ๐ฎ๐ฏ, ๐ฎ๐ฌ๐ฎ๐ณ (one-day workshop, exact day TBA)
AI agents now continuously evolve. How do we keep them trustworthy?
๐ก ๐ช๐ต๐ ๐ฎ ๐๐๐๐๐ฒ๐บ๐ ๐ฝ๐ฟ๐ผ๐ฏ๐น๐ฒ๐บ?
Many failures come not from a broken model, but from a bad memory, an over-privileged tool, or an update that cannot be undone. Operating systems ๐ฅ๏ธ solved this long ago. We bring that lens to agents, making their evolution ๐๐ฟ๐ฎ๐ฐ๐ฒ๐ฎ๐ฏ๐น๐ฒ, ๐ฏ๐ผ๐๐ป๐ฑ๐ฒ๐ฑ, ๐ฎ๐ป๐ฑ ๐ฟ๐ฒ๐๐ฒ๐ฟ๐๐ถ๐ฏ๐น๐ฒ ๐ก๏ธ
๐ ๐ง๐ผ๐ฝ๐ถ๐ฐ๐
๐ง ๐ ๐ฒ๐บ๐ผ๐ฟ๐ (File systems): provenance, isolation, rollback
๐ ๏ธ ๐ง๐ผ๐ผ๐น๐ & ๐ฆ๐ธ๐ถ๐น๐น๐ (Permissions): least privilege, sandboxing, revocation
๐ ๐ ๐ผ๐ฑ๐ฒ๐น๐ & ๐ฃ๐ผ๐น๐ถ๐ฐ๐ถ๐ฒ๐ (System updates): versioning, no silent alignment drift
๐ค ๐ ๐๐น๐๐ถ-๐๐ด๐ฒ๐ป๐ (Process isolation): stop one failure from spreading
๐ ๐๐๐ฎ๐น๐๐ฎ๐๐ถ๐ผ๐ป (Logging): trace every failure to the change behind it
๐ค ๐ฆ๐ฝ๐ฒ๐ฎ๐ธ๐ฒ๐ฟ๐ & ๐ฃ๐ฎ๐ป๐ฒ๐น๐ถ๐๐๐: @Yoshua_Bengio (UdeM, LawZero, Mila), @MengdiWang10 (Princeton), @du_yilun (Harvard), @hugo_larochelle (Mila), @ZheqingZhu (Pokee AI), and @yzhao010 (USC).
โฐ ๐๐ฒ๐ ๐๐ฎ๐๐ฒ๐ & ๐ฆ๐๐ฏ๐บ๐ถ๐๐๐ถ๐ผ๐ป
๐ Submission deadline: Nov 20, 2026 (AoE)
๐ฌ Author notification: Dec 2, 2026 (AoE)
- Full papers: 8 pages; Short / position papers: 4 pages
- Double-blind review, AAAI format
๐ ๐ช๐ผ๐ฟ๐ธ๐๐ต๐ผ๐ฝ ๐ช๐ฒ๐ฏ๐๐ถ๐๐ฒ: trust4rsi-agent.github.io/AAโฆ
๐ ๐ฃ๐ฎ๐ฝ๐ฒ๐ฟ ๐ฆ๐๐ฏ๐บ๐ถ๐๐๐ถ๐ผ๐ป: openreview.net/group?id=AAAIโฆ
๐ข ๐๐ป๐๐ฒ๐ฟ๐ฒ๐๐๐ฒ๐ฑ ๐ถ๐ป ๐ฏ๐ฒ๐ฐ๐ผ๐บ๐ถ๐ป๐ด ๐ฎ ๐๐ผ๐ฟ๐ธ๐๐ต๐ผ๐ฝ ๐๐ฝ๐ผ๐ป๐๐ผ๐ฟ? Contact us at [email protected]
Happy to organize with @NegarEmpr, @HuaWenyue31539, @BangL93, and @lcharlin.
Big thanks to the support and advisory from @StevenYuan666, Yankai Chen, Tianyu Shi, @BokwaiHo, @preslav_nakov, Steve Liu, @alex_pentland
We look forward to seeing you in Montrรฉal! ๐จ๐ฆ
#AAAI2027 #AIAgents #TrustworthyAI #AISafety #LLM #CallForPapers
Excited to be co-organizing our Trust4RSI-Agent Workshop at #AAAI2027 ! A Workshop on Trustworthy Evolution of AI Agents! ๐
๐ฅณWe have an amazing lineup of speakers and panelists: @Yoshua_Bengio , @MengdiWang10 , @du_yilun , @hugo_larochelle , @ZheqingZhu , and @__yuezhao__ !
๐ Submission deadline: November 20, 2026
๐ฌ Author notification: December 2, 2026
๐ February 22โ23, 2027 ยท Montrรฉal, Canada
๐ trust4rsi-agent.github.io/AAโฆ
Please share with colleagues working in this space and looking forward to your submissions!
Happy to be organizing this alongside @Haolun_Wu0203 @HuaWenyue31539 @BangL93 @lcharlin! #AAAI2027 #AIAgents
Had a great time presenting DeepScholar-Bench at #COLM2026 today! Lots of interesting conversations and interactions! ๐คฉ
If youโre building a deep research system, give our benchmark a try!
@COLM_conf has been such an engaging and fun conference so far! ๐
Today at #COLM2026 I will present DeepScholar-Bench and how we can evaluate AI-generated research synthesis!
๐ Imperial Ballroom, Poster #72
๐ Today, 11 AMโ1 PM
Come stop by! Would love to chat!
I'll be BARTing over to #COLM2026 this week ๐๐
Around all week and happy to chat about search agents & deep research systems!
๐ Thu: DeepScholar-Bench
arxiv.org/abs/2508.20033
๐ Fri: Multi-Agent Transactive Memory @ Lifelong Agents Workshop
arxiv.org/abs/2606.19911
Negar Arabzadeh retweeted
Agent Swarms are scaling to 1000+ agents, and your message board gets congested. Who builds search for them? #COLM2026
๐Lifelong Agent workshop (Friday)
I'll be BARTing over to #COLM2026 this week ๐๐
Around all week and happy to chat about search agents & deep research systems!
๐ Thu: DeepScholar-Bench
arxiv.org/abs/2508.20033
๐ Fri: Multi-Agent Transactive Memory @ Lifelong Agents Workshop
arxiv.org/abs/2606.19911
Negar Arabzadeh retweeted
For IR folks: @xueguang_ma and I quickly tested Jev as a pointwise/pairwise/setwise/listwise reranker on the classic DL19 and DL20, reranking the top 100 BM25 results. It is pretty good and cheap.
Negar Arabzadeh retweeted
Quick intro video on ๐ฐHarnessTax๐ฐ is out!!
I also shared some thoughts on what it means for our community to be aware of such cost in coding agents at the end!
Claude Code vs Codex vs Pi: which coding agent wins?
@melissapan, PhD candidate at UC Berkeleyโs Sky Computing Lab and previous @arena intern, explored the hidden โharness taxโ: how the system surrounding an AI model affects its cost and performance.
She reports three surprising findings. One: harness choice impacts cost more than accuracy.
To learn about all three findings, and what they mean for building useful coding agents on realistic budgets, check out the full video here: youtu.be/4R8qZGtEnAE
Negar Arabzadeh retweeted
Could long-context architectures โfind all contradictionsโ in a science literature? Not yet! ๐งต
We study a new class of "high-complexityโ tasks whose difficulty scales quadratically with corpus size (as opposed to linearly), reversing common LCLM decisions! (block-sparse attention, hybrid modelsโฆ)
Negar Arabzadeh retweeted
Impressive paper showing how much the first retrieval step matters for deep research agents.
It helps to improve GPT-5.5 from 83.1% to 90.5% on BrowseComp-Plus with the same retriever and the same agent loop.
It seems that the gain comes from the opening context.
The authors propose Question's Gambit which runs once, before the agent starts searching.
It splits the question into clues, turns each clue into complementary searches, pools the results, and reranks them.
The agent then starts its loop with that ranked set already in context.
The same change lifts GPT-5.4-mini from 68.1% to 79.0% and DeepSeek-v4-pro from 71.4% to 76.9%, and roughly halves calibration error for GPT-5.5. It costs between 2.3 and 5.3 extra tool calls per question.
In an error analysis, only 3 of the 79 remaining GPT-5.5 errors come from the gold document never being retrieved. The other 76 happen later, when the agent previews, opens or uses the evidence.
Paper: arxiv.org/abs/2609.14412
Chat with Paper: academy.dair.ai/papers/questโฆ
Thanks @omarsar0 for sharing our paper before we even got the chance to! ๐
Loved your framing so much we borrowed it for our thread
"same retriever, same agent loop, just a better opening move. โ "
Impressive paper showing how much the first retrieval step matters for deep research agents.
It helps to improve GPT-5.5 from 83.1% to 90.5% on BrowseComp-Plus with the same retriever and the same agent loop.
It seems that the gain comes from the opening context.
The authors propose Question's Gambit which runs once, before the agent starts searching.
It splits the question into clues, turns each clue into complementary searches, pools the results, and reranks them.
The agent then starts its loop with that ranked set already in context.
The same change lifts GPT-5.4-mini from 68.1% to 79.0% and DeepSeek-v4-pro from 71.4% to 76.9%, and roughly halves calibration error for GPT-5.5. It costs between 2.3 and 5.3 extra tool calls per question.
In an error analysis, only 3 of the 79 remaining GPT-5.5 errors come from the gold document never being retrieved. The other 76 happen later, when the agent previews, opens or uses the evidence.
Paper: arxiv.org/abs/2609.14412
Chat with Paper: academy.dair.ai/papers/questโฆ
1/ Weโre excited to share Questionโs Gambit module for search agents. โ
๐ฅQuestion Gambit got first on the BrowseComp-Plus leaderboard for recall with BM25 as our only retriever!
๐ It focuses on the first retrieval move in search agents and got 96.6% recall with about 6 times fewer agent search calls.
Same BM25 index. Same agent loop. Just a better opening move.
๐ Paper:
arxiv.org/abs/2609.14412
3/ High recall isnโt enough. The agent has to use what it finds.
With GPT-5.5, Pi-Serini surfaces 94.4% of gold documents, but opens or cites only 56.1%.
Questionโs Gambit raises these to 98.1% and 75.9%. A much larger gain in evidence use than in retrieval.
4/ Finally, we're proud that Question's Gambit is the top academic submission on BrowseComp-Plus, at 90.5% answer accuracy.
And we're honored to share the top of the leaderboard with @allen_ai @ai2_allennlp , @sailresearchco , and @mixedbreadai . ๐
And of course, thanks to the whole BrowseComp-Plus team for maintaining such an interesting benchmark! ๐@xueguang_ma @zijian42chen @luyu_gao @haora @ShengyaoZhuang @crystina_z @beirmug @lintool @WenhuChen
๐: arxiv.org/pdf/2609.14412
๐ป : github.com/radinhamidi/Questโฆ
From amazing team: @radin_hrad @amin_bigdelii @sadjadeb @claclarke @ebrahim_bagheri @bfung123
Negar Arabzadeh retweeted
jev for semantic operators is a great example of how LLM data processing at scale requires rethinking across the stack, model architecture and learning algs included
new work and many more thoughts on this coming soon ๐
in the meantime, check out lotus, star and stay tuned (link below)
I borrowed a ton of concepts from the lotus paper. The idea was well ahead of its time. Credit to the core contributors: @lianapatel_ @sid_jha1 @pgasawa @melissapan Harshit Gupta
Negar Arabzadeh retweeted
Two papers you should read from the information retrieval community about scaling up agent swarms:
Negar Arabzadeh retweeted
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs.
No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase.
Meet Rigel ๐งต
Negar Arabzadeh retweeted
Cool blogpost from Melissa and team on how much the harness matters for coding agents. On the two benchmarks tested, switching harnesses often changed costs much more than success rates, and the default harness of model providers do not necessarily give results that are pareto optimal!
Does your Claude model really need Claude Codeโฆ? ๐ค
We evaluate 7 models on Claude Code, Codex, and Pi. Three surprising findings emerge:
1๏ธโฃHarness choice has little effect on task success rate, but can significantly affect the cost
2๏ธโฃA simple harness can be competitive
3๏ธโฃThe native harness isnโt always the best.
Millions of people are using coding agents, but the impact of harness choice remains unclear.
(1/n) More details in the thread. ๐งต
So we found out that your Claude model might not neccesarily need Claude Code ๐
Finding 1: Harness affects cost more than correctness.
Fable 5 can cost twice as much for a 1.1-point gain in success rate. On SWE-bench Lite:
- Claude Code: 97.8% accuracy, $1.33/rollout
- Pi: 96.7% accuracy, $0.67/rollout
While Claude Code reaches the highest success rate on the SWE-bench Lite frontier, Pi and Codex often achieve similar success rates at lower cost across the models we test.
So you may be paying a hidden โharness taxโ if you pick your harness based only on the success rateโฆ ๐ธ
(2/n)