Computer use agent, MTS @ Microsoft AI Superintelligence team | Prev @MSFTResearch, @Meta superintelligence lab | OmniParser/Webwright/Fara1.5
Seattle, WA
Joined February 2015
- Tweets88
- Following362
- Followers113
- Likes93
Yadong Lu retweeted
Evolution doesn't benchmaxx. It optimizes for generality and robustness, because overspecialization makes a species fragile when the environment inevitably shifts.
Yadong Lu retweeted
Agent swarms have yielded nutty outcomes, such as the HuggingFace incident, or every math problem ever. It is not clear what form of communication is emerging between agents. It is equally important to understand this for capability and safety reasons.
To study this, we created SwarmChat, a simple Slack/Message Board for researchers working on multi-agents to use on any domain. We have validated it on ProgramBench experiments, where SwarmChat allows us to cleanly analyze how agent swarms work collectively!
Check out our blog for our analysis and how to use SwarmChat - it is super intuitive and easy to use. I personally believe the community really needs this, as a lot of people I know are using some makeshift communication tool.
Blog: swarmchat.io
Code: github.com/Pranjal2041/swarm…
Yadong Lu retweeted
In order to properly study and test the behaviors of multi-agent swarms, we need common and clean abstractions.
SwarmChat is a messaging library for agents, that allows them to communicate and coordinate. These interactions are also recorded in a very clean human-readable Slack-like app, which is great for studying their behaviors!
We also show that performance climbs quickly (at matched walltime) when you allow agents to coordinate through SwarmChat 🐝
Try it out by asking your agents to `pip install swarmchat`
AI swarms are already showing promising (or dangerous) results in frontier labs.
But how to implement them in the open community? And how do they scale?
We built SwarmChat, an open-source API-based Slack for agents.
Then we burnt half a trillion tokens, ran a series of swarm-scaling experiments, and tested various swarm structures on ProgramBench to study this question!
Check out the blog here: swarmchat.io
We found a bunch of interesting results:
1. Swarms get the task done faster. For instance, 16-agent swarms reached the performance of a single agent 6x faster.
2. Swarm scaling is a third axis of scaling. The more agents, the better, and anyone can already start scaling with SwarmChat!
3. Different models communicate differently. GPT-6 is chatty and over-communicative, while Claude prefers to work in relative silence!
4. Different swarm structures barely have any benefit on aggregate. But on specific tasks, team structures give clear gains.
5. Bonus: SwarmChat has a Slack-like UI for us humans to monitor swarms! Makes monitoring a lot easier! We are already using SwarmChat for WIP projects, and we were able to catch cases of swarms breaking the 4th wall (sorry, more details are for a future post)
Code: github.com/Pranjal2041/swarm…
w/ @JangLawrenceK , @kohjingyu , @dan_fried, @rsalakhu, @wellecks, @Yngkvn , @nicoup
Yadong Lu retweeted
We partnered with Browser-Use to release a new live web eval as a followup to the original Odysseys paper 😁
Odysseys2 serves as a task set that sources from real user requests to Browser-Use, check it out!
It was great working with @Alezander9 and @mamagnus00 - thank you to Browser-Use for the support and resources!
Introducing: Odysseys2 - hardest long horizon benchmark for web agents based on real user tasks 🏗️
We collaborated with @JangLawrenceK @rsalakhu from Carnegie Mellon University
> SOTA agent fully completes only 47/100
> 5 agents complete none
> 8-12 real rubrics per task, 60 min on the live web
Web agents are becoming better than humans at knowledge work.
odysseysbench.com/v2
Yadong Lu retweeted
We collaborated with @browser_use to create Odysseys2, a set of 100 web navigation tasks that are much harder and longer horizon than the original Odysseys.
Most importantly, all of these tasks are based on real user requests made to Browser Use, and represent a true distribution of what real users want CUAs and web navigation agents to do. We also produced a much stronger automatic judge for evaluations, which was sorely needed for these very long horizon evals. GPT-6 Astra is the best performing model on Odysseys2. It gets 47% of tasks perfect, so there's plenty of headroom for improvement!
Great work led by @JangLawrenceK and @Alezander9!
Introducing: Odysseys2 - hardest long horizon benchmark for web agents based on real user tasks 🏗️
We collaborated with @JangLawrenceK @rsalakhu from Carnegie Mellon University
> SOTA agent fully completes only 47/100
> 5 agents complete none
> 8-12 real rubrics per task, 60 min on the live web
Web agents are becoming better than humans at knowledge work.
odysseysbench.com/v2
Yadong Lu retweeted
Getting papers published probably won’t excite you anymore, but getting paper awards still do, especially THREE paper awards from #EMNLP2026 and a #NeurIPS2026 workshop on the same day @osunlp! Thrilled to share:
SkillHarm, led by @yuting_ning and @Zhehao_Zhang123, gets an Outstanding Paper Award at #EMNLP2026. I particularly like that SkillHarm introduces a new attack scenario, named Self-Mutating Poisoning (SMP), where an initially benign execution silently mutates persistent skill content that defers harm until a later reuse.
D3-Gym (co-led by @HananeNMoussa and @YifeiLiPKU) gets an Outstanding Paper Award at #EMNLP2026. D3-Gym has received a lot of interest from startups that aim to create scientific tasks and environments to train agents for scientific discovery. I particularly like that my students choose to stay with me rather than quitting Ph.D. to make big money! ;-)
AgentCL (led by @YihengShu) was selected for a Runner-up Paper Award at the #NeurIPS2026 Workshop on Towards Test-Time Continual Learning Agents (TTCL)! Check out our attempt to rigorously evaluate continual learning in language agents.
Congratulations to all co-authors and collaborators!
See our original posts for details:
Yadong Lu retweeted
🚀 DeepSeek-V4.1-Flash-1017 is here!
Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.
🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.
1/4
Yadong Lu retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Yadong Lu retweeted
we run cua on cua
our agents work on a few spare macs and linux boxes in the office, and when one gets stuck, whoever's free jumps into the same space and unblocks it
today that's something any team can do: spaces teams is live
1/ We're reimagining your computers as shared spaces for your team and its agents.
Today we're excited to share Cua Spaces Teams and Pro, built on Cua Driver, our virtualization stack and remote desktop protocol. On macOS, Linux and Windows.
Start free: cua.ai
Yadong Lu retweeted
Muse Spark Open was literally as good as the best model we released just two months ago.
Now it’s open-weight. Enjoy!
research.meta.ai/blog/muse-s…
Yadong Lu retweeted
Anthropic engineer:
"You're not supposed to prompt Opus 5.5
You're supposed to build a system that prompts itself. At Anthropic, 90% of our engineers have already built such systems"
In 30 minutes she shows exactly how Anthropic builds agents that remember, fix their own mistakes and get smarter with every run.
This beats any $1000-$2000 course on agents I've seen.
Watch it, then read the full guide for Opus 5.5 loop + jev engineering.
Your Claude loops can burn money while nothing changes
Use Jev to filter tasks and cut decision costs
Fix the trigger before another overnight run
The full Loop engineering + Jev engineering setup below ↓
Readers added context they thought people might want to know
The quote has nothing to do with the attached talk. It is not said by the speaker at any point.
x.com/hayatomaruu/st…
Yadong Lu retweeted
Computer-use agents are limited by the interfaces humans design, typing a spreadsheet cell by cell when one line of code would do.
Can we let them discover their own interfaces through evolutionary search? Come to our poster and find out.
📍Continental Ballroom 6 at 2.30 PM at the Lifelong Agent (LLA) Workshop.
#COLM2026 #LLA, 2:30pm
openreview.net/forum?id=ctXj…
Yadong Lu retweeted
Come by and hear Lawrence talk about iOS environments for computer use!
Me and @kohjingyu will be presenting iOSWorld at #COLM2026 tomorrow (Franciscan B, Poster #81) at 11:00 AM
JY has promised a free hug to anyone who can guess his date of birth, see you there😁
Yadong Lu retweeted
i'm not exactly sure how to describe what openai just did to math, but whatever that was, that moment is gonna hit coding in 6-18 months, and i have no understanding of what that actually looks like and what it means
Yadong Lu retweeted
Giving an agent just a terminal and letting it build its own tools is a fascinating approach. CUAWright is definitely worth watching.
Yadong Lu retweeted
Check out this work with Yadong at Microsoft! How well can computer-use agents do with just terminal/bash 🙂
What happens when you give a computer-use agent just a terminal and let it build its own tools, organically?
Meet CUAWright, with **only** one tool: run_command.
📄 Paper: arxiv.org/abs/2610.04116
🌐 Project: microsoft.github.io/CUAWrigh…
💻 Code: github.com/microsoft/CUAWrig…
Video credit to @JangLawrenceK!
Yadong Lu retweeted
Computer-use agent evaluation infrastructure is a hot mess, and we have a reproducibility crisis.
Speed is disproportionately affected, as final task success is often still comparable across setups, but measuring agent runtime is heavily dependent on the evaluation infrastructure. As a result, it’s been hard to measure progress on making CUAs faster.
We built cuaspeedrun (cuaspeedrun.com), a standardized infrastructure for running computer-use evaluations like OSWorld/OSWorld2, where measuring CUA speed should be a layup!
Computer-use agents are extremely good!
But they need to improve on a metric which most benchmarks don't track. How much time do CUAs take to complete a task?
Introducing cua-speedrun: a benchmark for accuracy, speed, and cost under matched infra.
cuaspeedrun.com
👇🧵
Yadong Lu retweeted
1/ Today we're introducing Cua-S1-4B-0.2, the first multimodal decision model trained with RLOO on live computer-use tasks, using task-completion rewards.
Text and multimodal adapters are available under Apache-2.0: github.com/trycua/cua
huggingface.co/cua-ai/cua-s1…