Xingang Guo (@Xingang20) profile picture

ScaleAI Researcher

San Francisco, CA
Joined May 2020
Photo by Xingang Guo (@Xingang20)Photo by Xingang Guo (@Xingang20)Photo by Xingang Guo (@Xingang20)Photo by Xingang Guo (@Xingang20)Photo by Xingang Guo (@Xingang20)Photo by Xingang Guo (@Xingang20)
Amazing scores on HSS, congrats on Gemini 4 models!
🚨 JUST IN: The Gemini 4 family sweeps the top three spots on Humanity's Sixth Sense. 🥇 Gemini 4 Neon 🥈 Gemini 4 Carbon 🥉 Gemini 4 Hydrogen
9
223
Xingang Guo retweeted
In collaboration with @HuggingFace, we’ve released the largest open robot dataset, alongside their first open System 1-System 2 model that can reason over step-by-step actions. FineART is built with 40K+ demonstrations across 151 tasks and 500K subtask labels. Together, the dataset and model were designed for each other, and both are available on @LeRobotHF where robotics developers work today.
29
36
23
331
46,322
Xingang Guo retweeted
🚀 Excited to share what I have been working on: Glimpse, a new approach to visual thinking from Elorian. Understanding a scene means keeping track of what moves, how perspectives change, and what happens next. Glimpse brings visual representations into its reasoning process, using an internal “visual scratch space” to examine details and revisit the evidence. Here’s what we built: 🧠 Reasoning in visual space, with the ability to follow objects over time, through shifting viewpoints and complex motion. ⚡ Performance that matches or exceeds GPT-6 Astra and Claude Opus 5.5 on key multimodal benchmarks. 💰 Pricing up to 94% below competing frontier models: $0.60 / 1M input tokens and $3.00 / 1M output tokens. 🛠️ End-to-end training to spawn subagents and work with multimodal tools, including code interpreters and video processors. elorian.ai/blog/introducing-…
5
5
35
1,724
Xingang Guo retweeted
Congrats to @GoogleAI's Gemini 4 Argon for taking the top spot of our Humanity's Sixth Sense (HSS) benchmark with a score of 55.4%, edging out GPT-6-Astra at 53.6%. 🥇 HSS tests AI’s ability to reason beyond what’s visible, from hidden objects to predicting what happens next and understanding human behavior. We took a closer look at performance across categories, and that’s where things get interesting. 👀🧵
39
129
41
1,166
163,220
Xingang Guo retweeted
The human–AI gap is striking—93% vs. 54%! Had a lot of fun to explore the examples where humans win!! 😇 The part I found most fascinating is the causal intuition gap: humans can infer hidden physical mechanisms, what likely happened, and what may happen next from subtle visual cues. Models often miss those cues or struggle to connect them to the underlying causal dynamics. More reasoning steps alone don't necessarily close this gap. An exciting frontier for multimodal intelligence! Great partnership with the @ElorianAI team! Congratulations to everyone involved on this fantastic work! @AndrewDai @_yunzhong @yinfeiy @danielyuez
AI is usually measured on problems that are hard for people. Humanity’s Sixth Sense, our newest benchmark with @ElorianAI, tests the everyday visual judgements we make without thinking. Humans scored 93.1%. The strongest AI model scored 53.6%. labs.scale.com/blog/introduc…
6
23
702
Xingang Guo retweeted
AI is usually measured on problems that are hard for people. Humanity’s Sixth Sense, our newest benchmark with @ElorianAI, tests the everyday visual judgements we make without thinking. Humans scored 93.1%. The strongest AI model scored 53.6%. labs.scale.com/blog/introduc…
10
18
5
88
172,444
Excited to share Humanity's Sixth Sense (HSS), a new benchmark for intuitive visual reasoning, built at @ScaleAI with @ElorianAI. Humans: 93.1% Best model (GPT-6-Astra): 53.6% Even the strongest models still fail at intuitive tasks that are easy for human eyes.
We’re introducing Humanity’s Sixth Sense, a new benchmark testing the intuitive visual reasoning people rely on every day. Across 522 open-ended tasks spanning images and video, HSS tests everything from spatial and causal reasoning to social understanding. The gap is significant. Humans score 93.1%, while the strongest model, GPT-6-astra, reaches 53.6%. The median model scores just 30.9%.
2
4
27
1,641
Xingang Guo retweeted
We’re introducing Humanity’s Sixth Sense, a new benchmark testing the intuitive visual reasoning people rely on every day. Across 522 open-ended tasks spanning images and video, HSS tests everything from spatial and causal reasoning to social understanding. The gap is significant. Humans score 93.1%, while the strongest model, GPT-6-astra, reaches 53.6%. The median model scores just 30.9%.
84
146
93
2,227
209,446
Xingang Guo retweeted
Full-duplex voice models can listen and speak at the same time, but can they change how they speak when you ask them to? Introducing SteerDuplex + SteerBench: the first benchmark that measures steerable full-duplex dialogue. 65.1% audio-steering APR vs 20.6% for the best open baseline. 🧵
3
15
3
118
15,496
Xingang Guo retweeted
An AI agent can top a benchmark and still be undeployable. Introducing READY (Reliable Enterprise Agent Deployment), the first evaluation framework for qualifying AI agents for deployment on real enterprise workflows. We’re inviting the research community to build it with us. The testbed is open for contributions today (link below).
2
7
1
47
2,526
New models this week are much more codebase-aware: they check and break their own tests, and like senior/experienced engineers, they even care about code hygiene. As part of that, I want to shout out some major gains we saw on our SWE Atlas leaderboard @ScaleAILabs. @AnthropicAI is tied for 1st on all three SWE Atlas leaderboards, with the latest Opus and Fable models clustered at the top. But Fable 5.1 feels like a qualitatively different model: more consistent, more efficient, and noticeably more codebase-aware. In Test Writing, it's the first model we've seen that actively tries to mutate code and break its own tests to check for robustness. In Refactoring, it consistently cleans up old leftover code without being asked. On SWE Atlas, no model has cracked 70 percent yet. There's a lot of work these agents still can't do.
1
8
36
1,677
Xingang Guo retweeted
🎉 Our work from @ScaleAILabs, Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning, has been accepted to EMNLP 2026 Main Conference! One of the first works to systematically benchmark vision tool use in multimodal LLMs - testing whether models can go beyond simply seeing an image to actively manipulate, inspect, and reason over it with tools. 📄 arxiv.org/abs/2510.12712 🏆 labs.scale.com/leaderboard/v…
4
11
1
60
2,849
Xingang Guo retweeted
📣Call for contributions + co-authorship! RSI Bench is our ongoing effort to evaluate whether AI agents can develop the capabilities required to advance AI R&D. Got a frontier AI research problem on your mind? Submit it to RSI Bench, and set the standard the whole industry uses to measure progress! Contributors receive: → Co-authorship on the RSI Bench research paper → $2,000 per accepted task for the initial 50 tasks → Modal compute credits to build + iterate → Access to the RSI Bench research community Register below for full requirements.
Launching rsi-benchmark.com: The work of AI R&D has always belonged to humans. For the first time, though, it no longer seems certain that it always will. Recursive self-improvement is within a line of sight. It may still be far, but it is close enough that we should start measuring it.
13
30
11
297
73,333
Xingang Guo retweeted
Today we're releasing the complete EnigmaEval dataset, our benchmark built from sophisticated puzzle-hunt problems in partnership with @CAIS. We believe open access to EnigmaEval will help push reasoning and evaluation research forward. So far, Claude Fable 5 and GPT-5.6 Sol take the lead. Full dataset available below.
5
10
1
86
4,976
Xingang Guo retweeted
We're back from Seoul! Great sharing our research on agent training, self-optimization, and evaluation through our paper presentations, lightning talks, and reception at @ICMLConf. The momentum toward bringing AI agents into real professional domains is massive, and we're thrilled to drive that conversation. Next stop: @NeurIPSConf 🇳🇿
1
9
2
41
4,318
Xingang Guo retweeted
📝 New research from @scale_AI Frontier SWE benchmarks are usually single-turn, one-shot tasks: the agent gets a detailed spec upfront, then implements autonomously. That is not how most real coding-agent workflows feel. Introducing SWE-Interact. 🧵
5
28
6
131
24,138
New @ScaleAILabs benchmark: 𝐃𝐫𝐮𝐠𝐃𝐢𝐬𝐜𝐨𝐯𝐞𝐫𝐲𝐁𝐞𝐧𝐜𝐡! 💊🧬👩‍🔬 We built it with @phylo_bio to evaluate agents on early drug discovery workflows. Bringing a drug to patients can take decades and billions of dollars. In a typical campaign, thousands of compounds are tested, and early decisions shape everything downstream.
1
17
1
53
3,312
Xingang Guo retweeted
1/ New from @ScaleAILabs: Rubrics (a.k.a. checklists) have become the default reward interface for RL on open-ended tasks without final verifiable answers. But most rubric RL still relies on static aggregation: fixed human weights over criteria, summed into one scalar reward. We show that this conflates what should matter in the final answer with what can actually teach the current policy. arxiv.org/abs/2605.20164
2
21
2
77
10,189
Excited to share that our paper Toward Engineering AGI: Benchmarking the Engineering Design Capabilities of LLMs has been accepted to NeurIPS 2025 (Datasets & Benchmarks Track)! 🚀🚀🚀 📍 Poster: Thu, Dec 4, 2025 • 11 AM–2 PM PST Exhibit Hall C,D,E • #1912 Would love to chat and connect, please drop by our poster tomorrow!
2
7
1
11
3,506
Modern LLMs shine at Q&A, but real engineering design demands synthesis, constraints, simulation, and trade-off reasoning. So we built EngDesign: the first benchmark that tests whether LLMs can actually solve engineering design problems, not just answer exam-style questions. 🔧 EngDesign includes: • 101 real-world engineering design tasks • 9 engineering domains • 473 rubric-based evaluation items • MATLAB / SPICE / MQSim simulation pipelines
1
1
4
209
📉 Frontier models score <35% overall. ⚡ Analog IC design → 0% across all models. Yet with iterative, feedback-driven refinement, strong reasoning models reach ~60% on solvable tasks—showing real promise for AI-assisted engineering workflows.
3
101