
Research @Meta Superintelligence Labs • Co-creating Muse models • MIT PhD • Opinions are my own NOT from affiliation
Bay Area, USA
Joined December 2023
- Tweets824
- Following582
- Followers6K
- Likes596
Zengyi Qin retweeted
We asked both models to arrange 0, 7, 8 and 2 into the largest possible number.GPT-6 Astra gets 8, 7 and 2 onto the first three pads, then moves 2 off again.
It was preferred over Gemini 4 Argon in this annotated comparison.
Zengyi Qin retweeted
We gave both the same simulated tower building task. GPT-6 Astra stacks 2 blocks on a raised board. Claude Opus 5.5 is still handling boards when its recording ends.
The visible difference is construction progress, not a confirmed completed tower.
Zengyi Qin retweeted
GPT-6 Sol leads Gemini 3.8 Flash on our Physical Agent Capability Index, 1,012 to 972 Elo.
In this simulated red mug task, Flash turns the mug upright in one attempt. Sol leaves it on its side.
Let's see the video👇
Flora inference code is public on GitHub! 🌸
github.com/NVIDIA/Flora
Weights are on HF. Now clone the repo and label your own robot or human videos into what happened (Captioner) and how well it went (Rewarder).
Introducing Flora, an open model suite for scaling robotics data engines.
It watches robot and human video and annotate what happened (timestamped instructions) and how well it went (rewards and failure reasons).
🔗 research.nvidia.com/labs/gea…
We use it to train NVIDIA GR00T. 🧵
3. More on Muse Spark's multimodal intelligence: research.meta.ai/blog/multim…
4. But capabilities like these create a harder evaluation problem.
How do you measure an agent when:
• tasks are open-ended
• outputs can be websites, games, files, or other artifacts
• there may be no single ground-truth answer?
We’re previewing WildArtifactBench to tackle this:
research.meta.ai/wild-artifa…
Today we release Muse Wonder, a strong model from Meta Superintelligence Labs.
It beats GPT 6 Astra with 1/10 the price.
Key features of Muse Wonder:
➤ Strong personal and professional agent performance:
Muse Wonder scores 59.1 on the Personal Superintelligence Index, leading Fable 5.1 (49.9), GPT-6 Astra (54.5), and Opus 5.5 (55.2). It also ranks first on Harvey LAB-AA v1.1 (13.1), AutomationBench-AA (45.4%), and MemoryBench (85.48), demonstrating strengths in legal work, end-to-end business automation, and long-term context retention.
➤ Stronger long-horizon coding:
Muse Wonder scores 79.0 on DeepSWE, ahead of GPT-6 Astra (74.1) and Opus 5.5 (74.2). On SWE-Marathon, which evaluates long-horizon software projects, Muse Wonder reaches 57.5, compared with 46.3 for Opus 5.5, 45.6 for Fable 5.1, and 42.5 for GPT-6 Astra. These results highlight Muse Wonder's advantage on complex coding tasks that require sustained execution across multiple steps.
➤ Competitive across a broad range of evaluations:
Muse Wonder achieves 55.8 on the Artificial Analysis Intelligence Index, ahead of Fable 5.1 and GPT-6 Astra (both 53.0). While Opus 5.5 leads on several individual benchmarks, Muse Wonder delivers a strong combination of professional-agent capabilities, memory, and long-horizon coding performance.
Coinbase CEO @brian_armstrong has poor knowledge on frontier model eval for 110% which causes such ridiculously wrong and scientifically unrigorous conclusion.
Best Paper in IROS 2026
Five hours of imperfect human motion data helped a humanoid robot learn to sustain tennis rallies with a human. LATENT, presented in the IROS 2026 Best Paper, turns fragments of strokes and footwork into working tennis skills on a Unitree G1.
Five amateur players supplied the demonstrations: forehands, backhands, and steps, captured without editing or annotation. LATENT learns to combine those movements and correct the racket wrist during a swing.
The real-world evaluation covered 20 human–robot rally matches. Forehand return success reached 90.90%, compared with 16.67% for a version trained without randomizing ball physics. Backhand success reached 77.78%. Success here means landing inside the opponent’s court, so these results measure valid returns rather than precise placement.
That comparison makes a useful point: learning human movements was only part of the task. Training for variation in how the ball behaves helped those skills work on hardware.
The robot still uses motion capture during deployment, and training focuses on returning incoming balls. Which next step is harder: replacing motion capture with vision, or learning to compete in a full tennis match?
Best Paper in IROS 2026
Feeding 3D scene features directly into a vision-language model raised navigation success from 66.7% to 74.2% versus a text interface on HM3D-OVON’s seen split. Both tests used the same 3D encoder and vision-language model.
On the unseen split, success and route efficiency diverged. SoftNav reached more goals than the Text-Hint baseline: 66.7% versus 63.3%. But its score for success weighted by path length was lower: 25.7% versus 28.3%. On episodes both solved, SoftNav took longer routes. A richer text baseline also matched SoftNav’s success rate statistically.
SoftNav, the IROS 2026 Best Paper, passes learned 3D features directly into the model as “soft tokens” to guide navigation. Training used 1,187 examples and about 17 million trainable parameters, with the base models frozen.
For researchers passing spatial data to models as text: have you compared text with direct features while keeping the base models fixed? Did the difference show up in reaching the goal, taking a shorter route, or both?
A robot’s task knowledge can transfer to a different kind of hand without new task demonstrations. KITE shows this across grippers, dexterous hands, and a gripper-plus-stick setup by separating task decisions from motor control.
The shared policy predicts where to engage an object and in what direction. A decoder for each robot turns that intent into motor commands. The policy learns from source demonstrations; the decoders learn from synthetic data generated using the robots’ geometry and joint models.
In simulation, KITE achieved 100% cube-picking success when transferring from a Robotiq gripper to Barrett, Allegro, and Wuji hands. Each target was tested on 100 unseen episodes. The paper also evaluates keyboard pressing and bottle pumping, where the robot must lift the bottle and press its pump.
On a physical Wuji hand, it succeeded in 7/10 cube-picking trials using a gripper source, 8/10 keyboard-pressing trials using a human-hand source, and 7/10 bottle-pumping trials using a human-hand source. These deployments used no target-task demonstrations or real-world fine-tuning.
“Zero-shot” here still includes training with the target robot’s kinematic model. Transfer also depends on whether that robot can achieve the requested contacts. The useful result is that task demonstrations can be reused across different hardware.
Claude Opus 5.5 tops the Artificial Analysis Intelligence Index at 58. But the cost story deserves a closer look: token prices fell 20%, while cost per task stayed level with Opus 5.
The reason is output volume. At max effort, Opus 5.5 uses about 119k output tokens per Index task, versus about 73k for Opus 5. Its lower prices, $4 per million input tokens and $20 per million output tokens, come alongside substantially more generated text.
The gains span several kinds of work. Opus 5.5 leads on six of the Index’s ten evaluations. On Terminal-Bench 4.0, it scores 59.6%, matching GPT-6 Astra at xhigh effort. On AA-Briefcase, which tests professional work, it reaches an Elo of 1822 and beats Claude Fable 5.1 on both analytical quality and presentation.
Effort settings also matter. Four of Opus 5.5’s five settings sit on the Index’s intelligence-versus-cost frontier. That makes the settings worth comparing alongside the headline max-effort score. The lead still has gaps: Opus 5.5 remains slightly behind on CritPt, AA-LCR and GDP.pdf.
Adding recovery skills raised Recova’s mean success from 77.5% to 87.5% across four real-robot tasks. This gain came after training on collected robot experience had already improved task execution.
One recovery means undoing progress: removing a ring stacked in the wrong order so stacking can resume. Recovery added 15 percentage points each on ring stacking and pencil-box packing, where a misplaced object can block the remaining steps.
Recova trains recovery separately from task execution. An agent develops corrections in simulation, invokes them on the robot, and verifies that the scene is workable before execution resumes. Human demonstrations of unresolved failures supply additional training data.
The evaluation used 20 trials per task and configuration, covering pencil-box packing, ring stacking, and drawing and discarding mahjong tiles. These results make a case for collecting demonstrations of scene repair alongside successful task runs: the robot needs examples of how to undo mistakes, too.
For teams collecting robot demonstrations: which recurring failures in your runs need a dedicated recovery skill before another task attempt can work?
Muse Gadgets provides open source ESP32 firmware and a Linux SDK for making hardware that works with Muse. To start building, get an API token and point your favorite coding agent at the GitHub repo.
This also raises a specific design question: what information from Muse would earn a permanent place on your desk, and what would make a dedicated display useful in your daily routine?
A robot learning a new task from 25 human video demonstrations, with its model weights frozen: that’s one result from Cornell’s Unified Motion-Action (UMA) model. Adaptation updates only a small set of task tokens that represent what to do.
UMA connects human videos to robot control through 3D object motion: how points on objects move over time. It learns motion from videos without robot action labels, and learns the connection between motion and actions from robot data. Pretraining combines human videos, real robot demonstrations, and simulation.
In video-only adaptation, UMA’s success rate was approximately 25 percentage points higher than UVA’s on each task: utensil insertion, sweeping, and folding jeans. Each task had 20 evaluation trials. The human demonstrations came from the evaluation scene and viewpoint.
The practical takeaway is that a new task can be taught through human demonstrations without collecting robot action labels for that task. Prior robot training still supplies the action knowledge; deploying on a new robot embodiment still requires compatible robot data.
How far does video-only adaptation transfer when the demonstrations come from a different scene or viewpoint? That’s the next test I’d want to see.
Zengyi Qin retweeted
Muse Spark 1.3 ranks #2 on Agents' Last Exam (ALE) leaderboard, ranked #1 at the time of its release.
Kudos to the team! Somehow this didn't get enough buzz.
Leaderboard: agents-last-exam.org/leaderb…















