Pinned Tweet
There are only two choicès for us(sentients)...
-either explore the possibilities beyond the horizon to unravel the true Nature of reality and the universe is just one part of it...
- or just live your life in complete ignorance and Dogma and never have a curious mind about exploring what can be out there among the stars and beyond that darkness there maybe a way out to truly find out the solution to nature's creation...
And seen from a civilizational standpoint, the average population of the entire planet is just steering their mindset into the second choice...
Remember, This path demands courage, intellectual curiosity, and a willingness to challenge established paradigms...
Mostly seen is Seek comfort in the confines of pre-defined beliefs and doctrines, opting for a life devoid of existential exploration.
This path offers the illusion of stability, built upon a foundation of unquestioning acceptance. Yet, it condemns us to a myopic existence, forever blind to the wonders that lie beyond the veil of our immediate perception...
[The future of our civilization hinges on the choice we make. Will we remain content with the shadows, or will we dare to step into the light?
Will we choose stagnation, or will we embrace the endless possibilities that lie beyond the horizon?]
This is not merely a personal decision;
it is a collective choice that will shape the destiny of our species. By choosing to explore the unknown, we embark on a journey that promises not only to unravel the mysteries of the universe but also to unlock the full potential of our own being.
The choice is clear:
embrace the unknown and embark on the grand adventure of existence, or remain tethered to the shores of ignorance. The future of our civilization, and the very essence of our sentience, hangs in the balance.
So yeah, here we are at this point in what has been happening with this project, with a limited amount of compute(cause I am GPU-poor) :(
StreamAttn update: native attention kernels across A100, H100, H200, and B200, with the code and benchmarks open for anyone to use.
The goal is to make long-context LLM inference faster and cheaper, from the attention kernel through the cache and execution plan. Basically, to push the frontier of inference engineering that has ever been done before
On the measured exact-decode configurations:
A100: 1.07–1.48× speedups with the contiguous D64 path.
H100: 1.21–2.07× with paged D64/G8.
H200: 1.03–1.58× with paged D128/G4.
B200: 1.16–1.44× across six promoted paged D128/G8 configurations.
These are full-context attention-call speedups against FlashInfer. The graphs show the workload, baseline, and measurement method for each GPU.
I started StreamAttn as a research project. It has grown into an attention engine with native backends, paged KV-cache support, reusable plans, model integration, and public APIs for decode, prefill, and training.
The larger research aim is to avoid attention work when its contribution can be bounded, while making the decision cheap enough to actually improve inference...
I’ve been building this with limited compute, and I’d love to see more people use it and help push it further. Try the code, open issues, share reproductions, or send a PR.
A few details on the kernels, what the engine supports, and where contributions would help are below.
Now the time has come for some updates about this project... This started as a research project by feb 2024 in terms of doing or creating some worthful data/contribution in inference engineering
Many things have happened within the project till this point that have truly accelerated the progress of what can possibly happen within a research question!!!
And also there's a paper(which also won the best paper at MLsys2026) called BLASST with the same concept/idea just recently posted by @jiayiy (P. S he didn't copy my idea or something, it's just that he thought about the same thing :) )...
Will be posting soon the main updates what has happened since the start of this project!!!
3
2
106
The work also extends beyond exact decode.
For grouped prefill, 11 of 30 tested H100 configurations passed the performance gate at 1.02–1.18× over graph-captured PyTorch Flash SDPA. Three of four H200 canary configurations were faster, reaching 1.14× against that baseline. These prefill kernels are still experimental.
B200’s four strict prefill comparisons came out at 0.64–0.99× against the fastest correct graph-replay baseline. Improving those kernels and connecting routing decisions to equivalent measurements are useful next contributions.
The adaptive work asks a different question, which is really interesting to see is that "can we skip K/V reads or matrix products while controlling the change in the attention output?"
The H200 experiments verified physical PV-work removal, but haven’t produced a repeatable complete-path adaptive speedup. Selection, metadata, and execution overhead still matter.
That’s the challenge I want to keep pushing on; a cumulative error budget, a cheap decision, and savings that survive when the entire path is timed. The exact-decode results above are the foundation for that work... :)
1
11
StreamAttn is open source, and contributions are welcome at every level.
Use it on your own workloads. If a shape is slow, a result looks wrong, the routing is surprising, or an example is difficult to follow, pls open an issue or a feature request, or contribute to it to push the envelope here.
Here are some useful places to initially contribute:
Reproduce the GPU results and extend coverage to more shapes and layouts.
Improve prefill kernels and bring routing in line with the benchmark evidence.
Add correctness tests, clearer examples, and integration fixes.
Explore cheaper selection rules and better bounds for adaptive attention.
For a performance or correctness issue, include a small reproduction, your GPU and software versions, the tensor shapes, and the backend that ran.
You don’t need a large GPU setup to help. CPU tests, documentation, reproducible plots, and focused issues are valuable too, as much as traceable evidence can be gained.
I want more people to be able to use this, question it, and build on it. Clean, robust, efficient, well-explained PRs are very much welcome.
Code: github.com/MagellaX/StreamAt…
7
This is going to be hella crazy if they truly manage to do it in the first try...
Now that's gotta be something really insane to pull off
if SpaceX achieves a double catch, it will be one of the single most important moments in the history of rocketry and mark a new era of humanity’s access to space. tune the fuck in! don’t miss it
1
54
this barely made it onto my timeline & honestly i would probably have missed it if @ns123abc hadn’t sent me Scott Aaronson’s blog. so i went through the post and followed the exposition it links to, and the part that stayed with me was how specific the mathematical progress actually is.
PSI’s proof of the Aaronson–Ambainis conjecture, using an internal Claude model, deserves a serious look.
for anyone who hasn’t followed this problem, imagine a function that takes a long string of bits and returns a number between 0 and 1. suppose it’s a polynomial of low degree, and its output varies appreciably across inputs. the conjecture says there must be some individual bit with a measurable influence on that output. crucially, the guarantee depends on the degree and the amount of variation, however many bits there are. for constant variance, the published result gives influence at least a constant times 1/d⁶.
the quantum connection is what makes this especially interesting. a quantum algorithm’s probability of answering yes can be represented by one of these polynomials. the theorem lets a classical algorithm approximate that probability with only polynomially many input queries, for most uniformly random inputs. those qualifications matter: this concerns queries and yes-or-no questions. it doesn’t give an efficient classical simulation of every quantum computation, and it doesn’t undermine Shor’s factoring algorithm.
what caught my attention was the obstruction inside the proof. an earlier approach by Shachar Lovett and Jiapeng Zhang depended on a conjectured inequality. the new exposition actually gives a counterexample to that inequality. so simply finishing that proposed argument was never going to work.
the successful proof replaces that step with a different inequality. it takes information from both yes-inputs and no-inputs into account, and crucially uses averaging where a worst-case bound would be insufficient. a strengthened version of David Reimer’s butterfly lemma supplies a key part of the machinery. you can identify the failed step and explain what the new mathematics changes. that’s a substantial contribution.
there’s also a Lean formalization, and Guy Blanc, Jordan Docter, Carmen Strassle and Li-Yang Tan have worked through the proof and written an exposition. Aaronson is candid that he hasn’t yet understood the whole proof himself; the formal certificate and the experts who have understood it are why he finds it convincing. i’m taking that evidence seriously, while being clear that reading the explanation is different from independently checking the entire formalization.
the attribution deserves the same care. David Turturean developed PSI’s research system, scoped the problem and ran it. Jim Halverson audited the proof, and Phiala Shanahan commented on the formalization. the internal Anthropic model was part of that research process. those contributions should remain visible when we talk about what “Claude solved.”
and there are still questions i’d like answered. the model isn’t public, and Aaronson explicitly says he doesn’t know how much to attribute to the model versus PSI’s research harness. the proof gives us something we can examine; it doesn’t, by itself, tell us how reproducible the whole discovery process is.
this brings me back to OpenAI’s Navier–Stokes announcement. my criticism is that the company’s framing puts the capability of its internal model at the center, while understanding the achievement requires a much fuller account of the research process. its own writeup describes human steering and roughly 10,000 concurrent agents in the successful group. OpenAI does disclose that, and deserves credit for doing so. i think those details should shape the headline and the allocation of credit just as much as the final theorem does.
but the mathematical criticism has to be accurate too. dismissing OpenAI’s result simply because it uses external forcing is wrong: the Clay formulation explicitly permits suitable smooth forcing in its breakdown alternatives. the unforced question remains separate. we can insist on that distinction without pretending the forced result is automatically invalid or insignificant.
i want the same standard applied to PSI and Anthropic. explain the exact result, show the new idea, credit the people and earlier mathematics it depends on, and make the evidence available for scrutiny. a result this interesting deserves more than becoming ammunition in an OpenAI-versus-Claude argument.
so this is what i think; the evidence here gives PSI and its Claude-based system substantial credit for resolving a longstanding mathematical problem. the new inequality is worth understanding, and so is the work of the mathematicians turning the proof into something other people can learn from.
i want to know which obstacle an AI-assisted proof gets around, why the replacement argument works, and what somebody else can now build with it. that’s what made this worth reading for me. i wish more of the discussion on my timeline got that far.
scottaaronson.blog/
God, I love this technical Elon so much!!!
pretty neat
Replying to @cradrrat
We have dumped JAX/SGLang and rewritten the entire RL and inference stack from scratch in C/C++, which took some extra time.
Training (aka pre-training) is done. Lot of cleanup was needed on the SFT data to get that mostly right. Now, 4.8 is in RL and the evals are starting to climb. We will do a soft rollout to Bot users later this week and see how it goes.
Grok 4.8 will bear no resemblance to 4.7 (thankfully). It is not yet incredible, but, this time, the foundations are built right, so I expect it to evolve to be something special.
1
85
Now the time has come for some updates about this project... This started as a research project by feb 2024 in terms of doing or creating some worthful data/contribution in inference engineering
Many things have happened within the project till this point that have truly accelerated the progress of what can possibly happen within a research question!!!
And also there's a paper(which also won the best paper at MLsys2026) called BLASST with the same concept/idea just recently posted by @jiayiy (P. S he didn't copy my idea or something, it's just that he thought about the same thing :) )...
Will be posting soon the main updates what has happened since the start of this project!!!
"StreamAttn" is a fused on-the-fly softmax attention that achieves O(N) memory use and stable normalization in one GPU kernel pass, yielding dramatic gains on long sequences (~97% less memory at 16K tokens where baseline attention OOMs). It's Triton kernel streams queries across key/value tiles to avoid building any full NxN attention matrix, and it even scales across GPUs via NCCL. Under active development as a research prototype, the repo already includes experimental “Ring” & “Star” modules for near-infinite context and multi-host scaling. The docs wryly note GPUs sometimes need “a little extra coaxing”– it’s powerful but finicky, a cutting-edge attention design bridging research and production.
github.com/MagellaX/StreamAt…
Every company now will posttrain models to be SOTA in their own specific domains.
...The only thing you'll need is data... Like proper stream of domain specific rich data/traces through different environments...
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
89
A new trend has started
literal hardest role to hire for rn and every ai startup wants someone
- young and smart
- exceptional at keeping time
- strong musical foundation
- knows how to drum
- willing to put personal ambitions on hold
- willing to commit for 10 years
- reasonable comp expectations, would take 10-15 mil or something like that
65

















