Senior Research Scientist at Google DeepMind. Former intern at @MSFTResearch and @GoogleAI. Opinions are my own.
Joined August 2020
- Tweets33
- Following181
- Followers589
- Likes767
Da Yu retweeted
🌳Excited to introduce Orchard! 🚀
🛠️ Orchard-SWE: 67.5% on SWE-bench Verified (30B-A3B, ~3B active)
🖥️ Orchard-GUI: 68.4% avg on WebVoyager / Online-Mind2Web / DeepShop (4B!)
📬 Orchard-Claw: 73.9% pass@3 on Claw-Eval
🌳 Introducing Orchard — an open-source agentic modeling framework! 🎉
One thin & cheap sandbox infra powers training recipes across SWE / GUI / personal-assistant agents:
⚙️ Orchard Env: 0.28s exec latency; 100% success @ 1,000 parallel sandboxes 💪
🛠️ Orchard-SWE: 67.5% on SWE-bench Verified (30B-A3B, ~3B active)
🖥️ Orchard-GUI: 68.4% avg on WebVoyager / Online-Mind2Web / DeepShop (4B!)
📬 Orchard-Claw: 73.9% pass@3 on Claw-Eval
🔗 arxiv.org/abs/2605.15040
📦 Code and data are coming soon!
Let's accelerate open agentic AI! 🚀
Da Yu retweeted
Excited to introduce WebHarbor! 🌟
⛑️Mirrors real sites into local Docker environments that are stable and RL-ready.
✅Ship with all 15 WebVoyager sites for reproducible and CAPTCHA-free evaluation.
📢Scaling to 100+ sites next, call for contributors!
Introducing WebHarbor ⚓ — an open community effort to dock real websites into local, deterministic, and evolving environments for web agent research. 🌐 Come help us build it. 🤝 Contribute new web environments or fix existing ones — will be included in the author list! ✍️
🎉 First release: 15 multimodal, high-fidelity environments covering all 643 WebVoyager tasks — full frontend, backend, database, and auth, all in one lightweight Docker image.
Why? Web agent eval today is broken😦: reCAPTCHA, geo-blocks, content drift, flaky networks, and login-gated deep features (e.g., account and checkout) that benchmarks can't touch. Live sites can't be reset either — making online agent RL impractical. Again, the bottleneck isn't the agent. It's the environment.
WebHarbor: dock real websites into stable, reproducible local mirrors with sub-second reset. But here's the key 🌱 — you can't clone the entire web upfront, and you don't need to. WebHarbor evolves with the agent: as harder tasks arrive, environments grow to support them. Coding agents (e.g., Claude Code/CodeX) build mirrors fast; human reviewers catch what coding agent hacks (shortcuts, leaks, fake completions).
We need you. 🙌 Help us scale to 100+ and beyond:
🔨 Contribute a new web environment
🐛 Fix or improve existing mirrors
🔍 Audit task fidelity & interaction realism
See more details and join the effort:
- 🏠 Project Page: aiming-lab.github.io/webharb…
- 💻 GitHub Repo: github.com/aiming-lab/WebHar…
- 📝 Contribution Form: forms.gle/ngcD1rzAfUEphNmRA
Let's build the open-source environment infrastructure for GUI web agents! ⚓
Initiating institutions: UNC-Chapel Hill ✖️Microsoft
#AIAgents #WebAgents #LLM #OpenSource #AgenticAI
Da Yu retweeted
Congrats to the LightMem team! 👏Great to see the continued exploration of topic-based segmentation and lightweight compression for building efficient memory systems for LLMs. Glad that our findings in SeCom and LLMLingua-2 have been useful building blocks for the community. 😀
We’re thrilled to share that our team’s work LightMem has been accepted to ICLR 2026 🎉
Paper: arxiv.org/abs/2510.18866
Code: github.com/zjunlp/LightMem
LightMem is a lightweight, modular memory system for LLM agents that enables scalable long-context reasoning and structured memory management across tasks and environments.
Recent updates:
1️⃣ Introduced a comprehensive baseline evaluation framework for benchmarking memory layers (Mem0, A-MEM, LangMem) across datasets like LoCoMo and LongMemEval
2️⃣ Released a demo video showcasing long-context handling, along with tutorial notebooks covering multiple usage scenarios
3️⃣ Enabled multi-tool invocation via MCP Server integration
4️⃣ Added full LoCoMo dataset support and integrated GLM-4.6, achieving strong performance and efficiency with reproducible scripts
5️⃣ Supported local deployment through Ollama, vLLM, and Transformers with automatic model loading
#ICLR2026 #LLM #Agents #MemorySystems #LightMem
Da Yu retweeted
We've released the full package for GUI-Libra! 🌟
📂 Data/Model: huggingface.co/GUI-Libra
📄 Paper: arxiv.org/abs/2602.22190
🌐 Project: gui-libra.github.io
Happy to hear feedback from the community!
Collecting high-quality GUI trajectories for agent training is expensive. But are we fully leveraging the open-source data we already have? 🤔
✨Introducing GUI-Libra (gui-libra.github.io): 81K high-quality, action-aligned reasoning dataset curated from open-source corpora, plus a tailored training recipe that combines action-aware SFT with step-wise RLVR-style training (⚠️partially verifiable rather than fully verifiable!).
Result: stronger native GUI agents on both offline step-wise evaluation and online environments across mobile and web domains.
Take away: With careful data curation + tailored post-training recipe, a small subset of open-source trajectories can still go a long way for training native GUI agents.
Check out our paper (arxiv.org/abs/2602.22190) and code/dataset/model (github.com/GUI-Libra/GUI-Lib…) for more details. #GUI #agent #VLM
Da Yu retweeted
🔊2026 Summer Internship @MSFTResearch Deep Learning Group🔊
We’re looking for a self-motivated intern with strong background on ⛑️building GUI agent environments and/or 🏗️reinforcement learning.
📩Interested? Send your CV + a short intro to [email protected]!
I’ll be at NeurIPS this Friday! You can catch me at the Google booth from 9:30–10:30 AM for the Q&A session with the Gemini team. Later in the afternoon, I’ll be presenting our SCONE paper (arxiv.org/abs/2502.01637) from 4:30–6:00 PM at Exhibit Hall C,D,E #5315. I'm looking forward to seeing familiar faces and meeting some new ones!
We'll be doing two different Q&A sessions at #NeurIPS2025 w/members of the Gemini team (including me). One is on Thurs., 2:30 to 3:30 PM and the other is Fri., 9:30 to 10:30 AM. Both at the @Google booth. Looking forward to talking about Gemini 3 ♊, Nano Banana 🍌, and more!
Da Yu retweeted
I’m really excited about our release of Gemini 3 today, the result of hard work by many, many people in the Gemini team and all across Google! 🎊
We’ve built many exciting new product experiences with it, as you’ll see today and in the coming weeks and months.
You can find it today on @GeminiApp and AI Mode in Search. For developers, you can build with it now in @GoogleAIStudio and Vertex AI.
blog.google/products/gemini/…
The model performs quite well on a wide range of benchmarks.
Da Yu retweeted
We’ve been intensely cooking Gemini 3 for a while now, and we’re so excited and proud to share the results with you all. Of course it tops the leaderboards, including @arena, HLE, GPQA etc, but beyond the benchmarks it’s been by far my favourite model to use for its style and depth, and what it can do to help with everyday tasks.
✨ Internship Opportunity @ Google Research ✨
We are seeking a self-motivated student researcher to join our team at Google Research starting around January 2026. 🚀 In this role, you will contribute to research projects advancing agentic LLMs through tool use and RL, with the goal of enabling breakthrough applications.
We are particularly interested in PhD students with a strong background in these areas. If interested, please send a brief self-introduction and your CV to [email protected]. Looking forward to connecting with talented researchers in this exciting space!
Da Yu retweeted
🚀 Excited to share GUI-Actor—a new approach for GUI grounding! Big thanks to @_akhaliq for featuring our work!
🌐 Project page: microsoft.github.io/GUI-Acto…
📜 Paper: arxiv.org/pdf/2506.03143
🤔 What's limiting coordinate generation-based GUI grounding?
1️⃣ Weak spatial-semantic alignment
2️⃣ Ambiguous supervision signals
3️⃣ Vision–action granularity mismatch
👀 But think about it: humans don’t calculate precise screen coordinates—we perceive elements and then act directly.
💡 Meet GUI-Actor: a VLM with an attention-based action head that:
✅ Addresses above limitations
✅ Proposes multiple candidate regions in one pass, enabling flexible downstream strategies.
✅ Performs coordinate-free grounding that better mirrors human behavior
➕ We also introduce a grounding verifier to select the most plausible action region — and it can boost other grounding methods too.
🎯 Results? GUI-Actor achieves SOTA on several benchmarks, even GUI-Actor-7B outperforms UI-TARS-72B on ScreenSpot-Pro, all using the same Qwen2-VL backbone.
Da Yu retweeted
Check out our SeCom and other amazing works!
In this issue of Research Focus, we examine a new conversation segmentation method that delivers more coherent and personalized agent conversation, and we review efforts to improve MLLMs’ understanding of geologic maps. Check out the latest research: msft.it/6019q9k33
Da Yu retweeted
🚀 Introducing SECOM🚀
How can conversational agents **better retain and retrieve past interactions** for more **coherent and personalized** experiences? Our latest work on **Memory Construction & Retrieval** tackles this challenge head-on!
🔍 Key Takeaways:
✅ **Granularity matters** – Turn-level, session-level & summarization-based memory struggle with retrieval accuracy and the semantic integrity/relevance of the context.
✅ **Prompt compression (e.g., LLMLingua-2) can denoise memory retrieval**, boosting both retrieval accuracy and response quality.
💡 Meet **SeCom** – an approach that **segments conversations topically** for memory construction and performs memory retrieval based on compressed memory units.
📊 Result? Superior performance on long-term conversation benchmarks such as LOCOMO and Long-MT-Bench+!
📖 Dive into the details: arxiv.org/abs/2502.05589
Da Yu retweeted
Google presents:
Scaling Embedding Layers in Language Models
Outperforms a 1.9B parameter baseline across diverse corpora, while using only half the inferencetime FLOPS
Da Yu retweeted
It is just so sad that the #NeurIPS2024 main conference ended with such a racist remark by a faculty when talking about ethics. How ironic!
I also want to commend the Chinese student who spoke up right on spot. She was respectful, decent, and courageous. Her response was exemplary: she began by acknowledging the speaker’s efforts, then gave the speaker an opportunity to clarify (though, regrettably, the speaker’s reply only reinforced her bias), and ultimately called attention to the inappropriate racial bias and offered constructive suggestions. Thank you for speaking out!
Da Yu retweeted
🚀 2025 Summer Internship @MSFTResearch Deep Learning Group 🚀
We’re looking for a self-motivated intern to work with us on building intelligent agents that reason, plan, and interact with complex UIs!
📩 Interested? Send your CV and a brief self-intro to [email protected]!
#Internship #AIAgents #UINavigation
Da Yu retweeted
[𝗜𝗻𝘁𝗲𝗿𝗻 𝗛𝗶𝗿𝗶𝗻𝗴]We are hiring a [𝐒𝐩𝐫𝐢𝐧𝐠 𝟐𝟎𝟐𝟓] [𝐟𝐮𝐥𝐥-𝐭𝐢𝐦𝐞] intern working on 𝗣𝗿𝗶𝘃𝗮𝘁𝗲 𝗘𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻. If you are interested, please apply here jobs.careers.microsoft.com/g… and send me an email: zinanlin at microsoft dot com
#ICML2024 Spotlight
<Differentially Private Synthetic Data via Foundation Model APIs 2: Text> was accepted at ICML 2024 as Spotlight! Unfortunately, we are not attending ICML in person. But feel free to reach out to us if you are interested!
Paper: arxiv.org/abs/2403.01749