Ologan retweeted
Kimi K3.1 recalls instructions it was never given on Terminal Bench. They match the hidden tests. K3.1 while solving: "Instructions likely distilled pitfalls from previous agents." 🤯
When we edit a task to contradict them, K3.1 refuses the new instructions and fails. Other models follow the edit.
Is it terminal(ly) benchmaxxed? 🧵
Ologan retweeted
We cut through the mystery around interpretability in training.
A single, mathematically rigorous framework transforms model internals into training feedback and works across domains, problems, and modalities.
We call this *mechanistic shaping*, and we’ve used it to mitigate reward hacking, remove shortcuts in chest x-rays, and reduce sycophancy and harmfulness. We believe this is a big step forward for technical alignment.
But it carries a known risk: training against a model’s internals can make it harder to monitor. We explore both the technique and this concern below. 🧵
Ologan retweeted
PTX 9.5 just landed with some interesting new additions -
sin/cos .2pi - Angles can now be supplied in cycles
tensormap.replace .alloc_class - TMA descriptors can explicitly track small/large tensor allocations and query that state
trace.mark + .metadata_section - Kernel code can emit timestamped performance
fabric publish and sync instructions like that of TMA commit_group and wait_group
cp.async.bulk.wait_group.sync_restrict::proxy::async - A narrower async copy wait, that orders prior async proxy accesses against subsequent async proxy without unnecessarily ordering the generic proxy. A weaker ordering in a sense (so it could be more async)?
fence.alias.sync_restrict::read.release.sys - A system scope release fence specifically for prior reads through aliased generic addresses.
%locality_domain_id and %locality_domain_supplemental_sm - Lets a kernel discover which locality domain its cluster was launched on and lets persistent kernels/schedulers distinguish guaranteed locality, domain SMs from supplemental SMs.
This means we can now query the number of domains and SMs per domain, allocate memory inside a particular locality domain, and create a green context whose SMs come from that same domain. I remeber what @pranjalssh is grouping SMs into those two sets, then assigning an A-row to one requester side so its repeated reads build useful residency in only one L2 half. CUDA locality domains now let you request something closer to -
allocate memory in locality domain X
run the kernel on SMs from locality domain X
docs.nvidia.com/cuda/paralle…
Ologan retweeted
Something a middle-schooler emailed us about:
“A ‘combination lock’ should *technically* be called a ‘permutation lock,’ because the order of the numbers matters in permutations, but in combinations the order doesn’t matter.”
10/10, no notes.
Ologan retweeted
(1/3) - Here's how my September "speedrun the backward pass" project (decoder-rtx) played out.
Halfway through the month, a swarm of AIs run by rekursiv.ai set a new record on almost exactly this task, and porting their techniques is really what got us down to 12 minutes.
Their model uses giant n-gram tables, and needs a 141GB H200. We got it all to fit on the 96GB RTX 6000, though, by inlining the table optimizations into the backward pass so we could forgo 22.5GB in gradient buffers.
I've come to really like the architecture, and wrote it up as a blog post, published as the project README: github.com/chrisjmccormick/s…
A couple highlights are in this thread.
Ologan retweeted
sm_120 is just weird it gets CLC but no tcgen05 or 2-CTA MMA or TMA multicast. cutlass defaults sm_120 GEMM scheduler to PersistentTileSchedulerSm100, running on try_cancel.
CCCL's for_each_canceled_block does 2 __syncthreads per steal. CUB's look-ahead scan is off on sm_120 below nvcc 13.4 because of a codegen bug (#8528), so DeviceScan on a 5090 still does look-back for me (i'm on 13.2), well time to rebuild some stuff on 13.4.
Ologan retweeted
Convincing evidence that sparse attention should be organized semantically rather than the more common positional approach. This makes me wonder if some of the online neighborhood tree algorithms&datastructures are relevant.
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.
Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster.
With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
Ologan retweeted
Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.
ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵
Ologan retweeted
You can't make this up: Our latest investigation into how expensive high-bandwidth memory chips get smuggled to #China from #SouthKorea.
The trick? Pair them w/ cheap server chips to avoid export detection; solder the memory chips off the cheap server, viola. via @theinformation
Ologan retweeted
meh, the more you dig into each experiment, the less magical it gets; it feels like a loose collection of cherry-picked demonstrations with a grand narrative built around them
Exp 1-2 are tiny (~20K params) with unusually easy to interpret architecture; Exp 1 is an MLP, where nearly all params are used in the first layer, so you can easily tell what kind of objects the network is trying identify
Exp 5 is about designing a game strategy (we already know that Astra is extremely good at this) and then converting it to a tiny (~20K params) MLP; entirely doable by a human, but of course would require a lot of effort
Exp 3 & 8 are closer to automated mechinterp using python and established techniques, rather than direct weight interpretation (at least according to chat)
to be clear, those are genuinely cool results, but I think people are reading too much into them
I don't think any of this should meaningfully change anyone's intuition about what models are capable of
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Ologan retweeted
Opus 5.5 is terrifying.
In our study, only 3% of humans beat it so far at predicting which video people will click on (and that number has barely moved in two days).
I hope you understand the implications.
If you'd like to try, the study is still open: lab.clickhunt.io
If your income relies heavily on the attention economy (whether directly or indirectly) don't ignore this, this should seriously concern many of you.
Take a look at the picture below.
This study (to me at least) starts to prove there will be a before and an after Opus 5.5.
Since many of you don't understand market dynamics, I'll explain it for you:
> before AI, the competition for attention was purely humans vs humans and the ones who were good at catching attention had a huge gap over everyone else (the median human being around ~83 so far)
> now an AI ANYONE can use (Claude today) predicts what people click better than the LARGE majority of humans who took the test so far
ANYONE. CAN. USE. IT. Do you understand what it means? Economically speaking it's called a commodity (cheap and available to everyone) so:
> everyone below it will have to use it (or die), which means its level becomes the NEW BOTTOM anyone can reach in the attention market (~117 so far, study still going).
> if you score above it, you still have a competitive advantage (for now at least)
> if you score below it, you're competing BELOW a commodity everyone has access to.
The numbers are what the data shows so far. I can already see the replies coming, "this is just an extrapolation". Yes, definitely, it is. But when I smell smoke, I don't wait to see the flames, I'd rather be wrong than wake up in a year to find out that what I spent years learning is now worth nothing.
You can argue with it if you want, but those who know me should seriously not ignore this, friendly advice.
You might think I'm doing this just to collect data or whatever (which, full disclosure, I do, I'm trying to understand a paradigm shift at its heart, but you literally benefit from it since I share the discoveries with you).
And believe it or not this data is worth far more to you (as a group) than to me. The more humans take the test, the more accurate the final report of the study will be and the clearer the direction it will give.
Trust me or don't trust me, in 6 months/a year when the market has become more mature with these models as commodities, you'll remember this post if you ignored my warnings.
Get your score NOW, follow this study's results and figure out a strategy according to your score once you get them.
-> lab.clickhunt.io/
It's just a few minutes (I suggest at least 4-5 sessions for a more accurate score), worst case I'm wrong and you lost a few minutes, but if I'm right, you'll look back on this day as one of the moments you're most grateful for.
Ologan retweeted
.@N8Programs was ahead of their time with this tweet
(w.r.t. rikiyat.github.io/blog/weigh… going mega-viral today)
Beat it by having Codex hand-craft weights:
gist.github.com/N8python/02e…
100% accuracy on 10 million random test cases w/ only 343 parameters. As a bonus, it uses the vanilla Qwen3 architecture, just with the right weights.
Ologan retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
