Fellow at Tenstorrent; believes in dynamic typing, first-class functions, the immortal essence of the human soul and tea. Tweets are my own.
Lübeck, Germany
Joined January 2010
- Tweets4.4K
- Following394
- Followers1.2K
- Likes7K
Mark retweeted
🤯逆天了！Qwen 3.8 27B注意力层被彻底击穿了！10M超长上下文下，Prefill注意力快72.4倍、Decode快42.5倍！端到端首token延迟砍到约1/9！
闲置GPU的朋友注意了：把这个新方法丢给Claude/Codex，让它直接移植到Qwen 27B上！注意力算法“移植”已经被验证可行。
惊人之处：
🔹10M token（约1000万）超长上下文下，传统稀疏注意力有98%时间都在“找该看哪些token”，真正算注意力反而没那么贵
🔹新方法Latent Sparse Attention (LSA)：先把Key按“意思相近”聚类成层级结构，Query只看聚类中心，再挑最相关的几个桶去算。从“全扫”变成“树搜索”
🔹实测（Qwen-3.8-27B）：单层注意力Prefill快72.4倍，Decode快42.5倍！端到端首token延迟也能降到约1/9
🔹精度几乎不掉，甚至在RULER等长上下文任务上还略好一点
🔹只需要短蒸馏训练Indexer（基座模型冻结），就能套到现成模型上。Native训练版本也能跟上全注意力精度，还更快
🔹对GPU紧缺党意义巨大：超长上下文（百万级）不再需要堆满H100/H200才能跑，单卡/少量卡也能玩得起真正长上下文Agent
适配步骤：
1️⃣git clone 仓库后用 uv sync 安装（Python 3.12）
2️⃣官方已放出Qwen3.8-27B的蒸馏权重（d-lsa-weights-v1 release），直接下载即可
3️⃣用他们的SGLang fork镜像启动服务，挂载仓库+权重
4️⃣启动时加 --json-model-override-args '{"lsa": {"projections": "/maps/qwen38_27b.pt", "query_heads": "all"}}'（记得加 --disable-prefill-cuda-graph）
5️⃣想自己训练的话，跑 lsa distill targets → lsa distill train → export 即可（基座冻结，只训Indexer）
代码已开源，论文也在，谁有空闲卡先冲一波？
论文：arxiv.org/abs/2610.42147
GitHub：github.com/Prefrontal-AI/lat…
#Qwen #LSA #LongContext #AI #GPU #LocalLLM #注意力优化 #Qwen38 #阿里巴巴
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.
Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster.
With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
Mark retweeted
i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models.
Mark retweeted
This is simultaneously very cool but also a bit exaggerated. Reading the prompt you gave, the prompt veeery much hints that looping is the issue and to find neurons that fix this. And as you say (screenshot below), it couldn't otherwise improve math.
So your project and website are great, but the tweet is a bit of a bait. But it worked, i read the full page including prompts 😄
Mark retweeted
I think what this shows most of all is that Opus 5.5 and Gemini Carbon likely bought their svg envs non-exclusively from the same vendor.
Which is good news for Carbon i guess because Opus5.5 is pretty good.
(Or ⚗️)
Mark retweeted
This is so much more cool and impressive than like 98% of what I’ve seen from most major FPS studios the last like 5 years.
Mark retweeted
Hi folks, I wrote half the materials science benchmark on this one. It tests the whole arc of the matsci workflow from candidate proposal to characterisation, and I'm bewildered by how performant the models already are at some of my most complex tasks, especially Astra.
Mark retweeted
It’s worth emphasizing that certain models like Opus 5.5 and GPT-6 Astra have sandbagged overall scores due to their safety classifiers. If we exclude cyber, biology, and alignment—the three domains where safety classifiers sometimes kick in—you get the following results instead.
We were pleasantly surprised by how well this ranking lines up with our subjective impression of the models. (4/7)
Mark retweeted
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains.
GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
Mark retweeted
Weekend hack: HackerTube. I was annoyed with YouTube's recommendations so I had Claude filter down 43M comments over 20 years of HN to 6,454 videos that keep being recommended. *Significantly* better viewing experience, ~2 hours with Opus 5.5:
hackertube.org/
- Excellent recs without any logins (state is kept in local browser session)
- Thumbnail is a representative frame from middle of the video instead of the super duper A/B test hyper-optimized user uploads
- Underlying data from the comment analysis as one zip file, point your Claude at it and ask for your own version
- HackerTube Longs ✊
A bit more on the process:
I use Claude Code (desktop app for Mac), Opus 5.5 on Medium effort as my default. I only talk to Claude and have it do everything, the only "manual work" I helped with in tandem was in the browser (reminiscent of MenuGen): 1) I had to create some kind of API key from Google Cloud UI to query YouTube for video metadata, 2) I booted up a VPS box for serving and 3) I bought the domain name. Once the box was up I gave Claude the IP and told it update, secure, harden, deploy and optimize the code, and it did the whole thing.
Many awesome projects that would have normally taken weeks/months even can now be few-shotted for breakfast. What else are we not thinking of.
Mark retweeted
This is from the Roblox Q3 2026 Shareholder Letter. The top chart is daily users of their MCP server, the bottom is assets uploaded. Opus 5.5 was released September 22nd. Over the past month, more than 80% of the top 100 games used AI. Things changed in September.
Mark retweeted
Julius convinced me to move from extra high to high on Opus 5.5 a week or two ago. He was right. This is better almost all of the time.
Mark retweeted
I mostly use frontier models in day to day work. I tried out deepseek v4.1 flash again recently, and it is just so much less precise than i'm used to. It gets more confused about things that happened earlier and make trivial mistakes because of it. I don't know if it's the compressed attention, lower training budgets, different RL focus, or what, but whatever the benchmarks say, there is this consistent meaningful gap here and it's one I personally find quite frustrating to work with.
I'm sure you can get a lot done with these models, but that kind of detail matters a lot more to me in day to day use than some math benchmark. It's more of an ergonomic concern than a peak capability concern, perhaps.
I don't believe that a few more points on the AA index would indicate this problem is fixed This is still a thing benchmarks don't effectively capture. It seems like it should show up in some long context benchmark, but for whatever reason they are still not quite capturing it.
Mark retweeted
So… i might have do a thing… it actually fucking works ITS FUCKING DRIVING ON UNSEEN TRACKS
Note that the acceleration data from the game is of course a bit "noisy" due to euler being euler or something, but the model really doesn't miss, even when you hit the snow, it still gets it essentially correct exactly how fast you just decelerated in meters/second^2.
Considering people cannot reliably tell how fast they are driving from visual cues (it knows *exactly*), this is superhuman many times over. And yet, the hard part isn't yet solved, this is still just a fancy proof of concept and a dead-end if all you can do is *recover* these values. Copy-cat policies are a real pain...
Mark retweeted
I used Divyansh's approach to build an entire Nintendo into ~1GB of Qwen3.5 weights.
Qwenputer, meet Nesformer: github.com/dbreunig/nesforme…
Mark retweeted
In lieu of flowers, memorial contributions may be made to:
Johns Hopkins Alzheimer’s Disease Research Center Fund
alzresearch.org/giving/
Alzheimer’s Foundation of America Caregiver Resources
alzfdn.org/donate/
Photo Credit: Tommy Garcia
Mark retweeted
Day 25/
We are doubling all usage limits for all Pro 100/200/500 tiers and Business Premium, effective today.
We heard your kind feedback and won’t be reducing usage limits for Pro 200, so your Work and Codex usage will stay the same. We’re also going to continue offering extended limits for dots for now, and usage of your dot will still continue to be separate from your default usage.
And we'll of course also keep shipping better models and other efficiency improvements that increase your effective usage over time.
