Pinned Tweet
Great work with @NunchuxAI. Thanks to all collaborators and mentors!
Introducing VC-Attention: fast and accurate low-bit attention without retraining.
On MiniMax-H3, VC-Attention speeds up attention by 1.6× on B200 and 1.5× on B300 over FlashAttention-4, with better fidelity than SageAttention2. It also works with existing sparse attention methods.
Two key innovations:
• V-Smooth reduces value quantization error.
• ExpCast-FP8 speeds up softmax.
Nunchux Attention, our proprietary extension, pushes the speedup to 1.9× on B200 and 1.8× on B300.
Blog: nunchux.ai/blog/attention-is…
Technical Report: arxiv.org/pdf/2609.15810
Joint work by researchers at MIT, CMU, UC Berkeley, Stanford, and NVIDIA.
3
10
829
Xingyang Li retweeted
With the success of KDA (kernel design agent), we release QDA (quantization design agent):
Agent loops need fast feedback, and LLM quantization doesn't have it: evaluations of reasoning models are slow, and PTQ methods are hard to port and combine.
Meet QDA. A coding agent designs the quantization recipe for each model and precision, with a KL score that takes minutes and a library of 20+ stackable PTQ methods.
📈 Beats the best public recipe on 11 of 12 model-precision pairs: +0.9 / +1.4 / +2.2 pts at W4A16 / W4A8 / W3A16
🔥 GLM-5.3-Flash (320B) at NVFP4: 98.3% of FP8 accuracy, vs 96.9% for NVIDIA's release
💰 Only $19–75 of Claude Code tokens per search
⚡ Runs on stock vLLM, SGLang and llama.cpp, zero inference overhead
Blog, paper, code and checkpoints below 🧵
Agent loops need fast feedback, and LLM quantization doesn't have it: evaluations of reasoning models are slow, and PTQ methods are hard to port and combine.
Meet QDA. A coding agent designs the quantization recipe for each model and precision, with a KL score that takes minutes and a library of 20+ stackable PTQ methods.
📈 Beats the best public recipe on 11 of 12 model-precision pairs: +0.9 / +1.4 / +2.2 pts at W4A16 / W4A8 / W3A16
🔥 GLM-5.3-Flash (320B) at NVFP4: 98.3% of FP8 accuracy, vs 96.9% for NVIDIA's release
💰 Only $19–75 of Claude Code tokens per search
⚡ Runs on stock vLLM, SGLang and llama.cpp, zero inference overhead
Blog, paper, code and checkpoints below 🧵
Across 4 models × 3 precisions, one search per pair improves on the best public recipe in 11 of 12 cases. On the 320B GLM-5.3-Flash at NVFP4, a $69 search keeps 98.3% of FP8 accuracy, vs 96.9% and 96.3% for the NVIDIA and Red Hat AI checkpoints.
What's next: agents that make their own tokens cheaper. Day-one quantization for every new open model, a library that grows with the steps agents invent, and the same loop for KV caches, sparsity and new low-bit formats.
📝 Blog: mit-han-lab.github.io/qda-bl…
📄 Paper: arxiv.org/abs/2611.23388
💻 Code: github.com/humanfia/qda
🤗 Models: huggingface.co/qda-lab
🤖 Agent harness: github.com/humanfia/humanize
Joint work with Tianlang Zhao, @songhan_mit and @LigengZhu. Thanks to @hancai_hm, @conlesspan, @HaochengXiUCB, @syn7xavier, @ZiyiXu_2004, @mark_sf, @EnricoShippole and @JJitsev, and to @laion_ai, SK Telecom, MGAIC and JUPITER for support.
Xingyang Li retweeted
4 new models just landed on Nunchux Modelverse! ⚡
Seedance 2.5 Flash @seedance
Kling Image 4.0 @Kling_ai
Kling v4 Flash @Kling_ai
MiniMax-H3-Image @MiniMax_AI
Try them now with free credits: nunchux.ai/sign-up
Xingyang Li retweeted
What would it take for AI assistants to know when to step in, rather than always waiting to be asked?
Deep Personalization explores this direction by learning from work patterns and implicit feedback, with a local Observer Agent on NVIDIA DGX Spark making persistent observation economically viable.
Exciting collaboration between @NVIDIAAI, @stanfordnlp, and @MIT! 🚀
AI is moving from on-demand to always-on. But can it learn when you actually want help?
We introduce Deep Personalization: an Observer Agent that continually learns from your work patterns and implicit feedback to decide when to step in.
It improves proactive-timing F1 by 2.4–3.0× across three model backbones.
We co-design the agent and its training and inference engines to run the Observer locally on systems like NVIDIA DGX Spark, keeping your screen observations on-device.
The next PC won’t wait for your prompt. It will learn when to help! Check out our blog at: nvlabs.github.io/deep-person…
Xingyang Li retweeted
Training with fewer bits is faster, but quantization error can interfere with learning.
Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout.
thinkingmachines.ai/blog/end…
Xingyang Li retweeted
the NVFP4 recipe behind Inkling 2 Small: 4-bit forward from the first pretraining step to the final RL rollout, MXFP8 + stochastic rounding for backward
and still more to come 🪜
Training with fewer bits is faster, but quantization error can interfere with learning.
Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout.
thinkingmachines.ai/blog/end…
The A-Team
You’ve seen Astra draw. This time, it’s an open-source 27B reasoning model we trained! RL entirely in simulation. No demonstrations. No agent harness. Native robot use.
I still can‘t believe what we achieved, from an open-source foundation model with almost no native robot use capability to perform well on the real robot. This is really an exciting moment for me as it means a fundamentally different path from existing robot foundation models, directly addressing their biggest limitations.
Robot foundation models have largely relied on learning manipulation from robot data, which is painfully scarce. Robot-use agent inherits the LLM’s native world knowledge and reasoning capability, and we just unleash and put them to work on robot. With robot demonstrations so scarce, being able to draw on that existing intelligence matters enormously. The exceptional instruction following and generalization have been striking to watch. Capabilities we’ve come to expect from LLMs suddenly feel remarkable when they show up in a physical robot.
In our experiments, more compute and more simulation environments keep translating into stronger capabilities. We also see transfer across tasks, across robots, and from simulation to the real world. That makes the direction feel tangible: scalable simulation compute can translate into transferable robot capabilities. There’s something deeply encouraging about seeing a path where investing more resources keeps moving the needle.
We’re sharing our observations and results, and open-sourcing the model, training infrastructure, and deployment stack so everyone can build on them. We’ve also laid out the limitations in detail. Each one points to an open research question, and together they capture much of what we believe are promising directions in the future.
Excited to see what others discover—and build.
Enjoy! mit-han-lab.github.io/pear-b…
4
289
Xingyang Li retweeted
Everyone's been showing Fable and GPT-6 Astra driving robot arms.
We trained an open 27B Qwen model to do it, purely from its own attempts in simulation, with zero demonstrations.
And it can run locally at 220+ tok/s on an RTX 5090.
On a real robot it had never touched, it now succeeds in 18 of 30 tests, up from 0 before training. It also copies a painting and designs its own chemistry experiment.
It generalizes and follows instructions better than π0.5, and on LIBERO it comes close to Astra max.
All without losing its general capabilities, like math and coding.
Built on Qwen3.8-27B. We've open-sourced the model weights, code, and full training infra! Check out our blog at mit-han-lab.github.io/pear-b….
Amazing work!
So fortunate to have watched this grow from a neat observation into a full paper.
Congrats to the whole team!
Replying to @conlesspan
Amazing collaboration with @HaochengXiUCB @KanZhu854772 @XYLi_Bruce @BetaGamma20 @MayankMish98 @zhangho72245693 @WilliamXinze @songhan_mit @KurtKeutzer @rishiyer @istoica05 !
Huge thanks to @shulynnliu @yhfchitanda @YichuanM @melissapan for insightful discussions and helpful feedback!
Feel free to check out the paper, code, and demo. We'd love to hear your feedback! 🙌
(7/7)
Xingyang Li retweeted
Thanks to @AMDServer for featuring our work! We’re excited to keep working together to make multimodal generative AI faster and more interactive on @AMD chips.
Building with image, video, or world models? Get in touch to see what Nunchux AI can do for your workloads.
The next frontier for AI video isn't just faster generation. It's creating experiences people can shape in real time.
See Nunchux AI's work with AMD Instinct MI355X GPUs and AMD ROCm software: bit.ly/4yRTJd5
Xingyang Li retweeted
Nunchux now leads 10 @ArtificialAnlys leaderboards for image and video generation speed, covering MiniMax-H3, LTX-2.3, Qwen-Image, and FLUX series. 🚀
These results come from building the Nunchux Model Optimizer and Nunchux Inference Engine together. We want to make image and video generation interactive and affordable, so you can try an idea and refine it as you go.
The waitlist is gone. Nunchux Modelverse is open to everyone, with 80+ models. More models and categories are on the way. Sign up today for free credits and generate faster for less.
Full results and comparisons on our blog: nunchux.ai/blog/nunchux-top-…
Special thanks to @ArtificialAnlys team for onboarding us to their benchmarks.
#Nunchux #AIInference #MultimodalAI #ImageGeneration #VideoGeneration
Xingyang Li retweeted
I'll be attending the @GoogleStartups Gemini Startup Forum this week!
If you're building in multimodal AI, happy to connect!
Xingyang Li retweeted
🚀 Can making an LLM weaker actually make it smarter?
Excited to share our latest work: Weaken to Strengthen — Weakening-Induced On-Policy Self-Distillation (WISD)!
💡 Instead of relying on a stronger teacher, we deliberately weaken the model through quantization or pruning, then use the contrast to improve its reasoning.
No external teacher. No reference solutions. No reward signals. Just the model itself and unlabeled prompts!
🔥 Key results:
📈 +4.0 / +5.5 percentage points average accuracy gains with quantization / pruning.
🧮 HMMT 2025: 57.7% → 70.5% (Qwen3.5-4B).
💻 LiveCodeBench v6: 39.1% → 56.7% (Qwen3.5-4B).
Sometimes, getting stronger starts with getting weaker!
💻 Code: github.com/jet-ai-projects/W…
🌐 Website: jet-ai-projects.github.io/WI…
#LLM #Reasoning #SelfDistillation #EfficientAI
Xingyang Li retweeted
Come join us! 🧡
Inco is going to #SGLangSummit 2026 🧡
Our open-model endpoints run several times faster than most providers on @ArtificialAnlys. Our co-founder @ZhijianLiu will lead a breakout session on #DFlash and what it takes to build inference for the agentic era, at scale.
🎙️ DFlash and Beyond: How Inco Runs the Fastest Open-Model Endpoints
📍 Nov 12–13, SF
🎟️ luma.com/sglang-summit
Coming to the summit? Stop by, say hi, and let's talk about your inference stack.
We entered First Proof Batch 3 as the only open-weight model: Kimi K3, running on the open-source Humanize harness.
No full solves yet, but real partial progress on a few problems and plenty of lessons. Here's what we learned along the way.
markkim.bearblog.dev/first-p…






















