Will Mayner (@wmayner) header
Will Mayner (@wmayner) profile picture

consciousness, AI, mech interp, neuroscience · MATS 11 fellow @goodfireai · integrated information theorist · author of PyPhi https://xitter.gq/t.co/Xw02ol29S9

Madison, WI
Joined June 2012
Photo by Will Mayner (@wmayner)Photo by Will Mayner (@wmayner)Photo by Will Mayner (@wmayner)Photo by Will Mayner (@wmayner)Photo by Will Mayner (@wmayner)Photo by Will Mayner (@wmayner)Photo by Will Mayner (@wmayner)Photo by Will Mayner (@wmayner)Photo by Will Mayner (@wmayner)
Okay. Look. Imagine how you'd have felt if an AI had just proved the Riemann Hypothesis. Now you will predictably, at some point, get that news LATER, if we're not all dead before then. So you can go ahead and feel that way NOW, instead of acting surprised LATER.
130
365
138
3,880
671,433
Will Mayner retweeted
This is massive. A story I've been following for years. He thought he had more time but didn't. Alessandro Masullo announces a separate, purely human proof of the Riemann hypothesis. math216.wordpress.com/2026/1…
139
839
252
9,402
925,711
Will Mayner retweeted
We know LLMs can introspect and control their internal activations. But they can do this with attention, too. 🧵
19
46
11
366
16,961
Will Mayner retweeted
In Neural Networks are Already Clean, @nickcammarata argued that neurons were far more legible than previously thought – and that this was great news for interpretability and safety. Now, we’re releasing the first technical companion to Nick’s essay. It shares our findings from studying thousands of neurons using our novel automated interpretability pipeline, and outlines why we now think of neurons by their computational function, instead of whether or not they are polysemantic. Here’s what we found: 🧵
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
53
249
60
1,791
148,865
Will Mayner retweeted
The most ambitious vision for interpretability has long been to reverse-engineer a model well enough to answer an arbitrary question about how it works. For years, that’s seemed out of reach. Now, we think it’s possible. Neural networks are cleaner than we thought: we can now understand most neurons in models up to trillions of parameters. And agents can rapidly accelerate our progress. But to get there, we need a concerted effort to unlock scaling laws for interpretability, where our understanding of a model grows predictably with compute. That’s why we’re committing $10M to start a Human Genome Project for AI. Our ultimate goal is to reverse-engineer a frontier scale model and publish a living encyclopedia explaining how it works. We don’t know exactly what we’ll find, and we can’t do it alone. We hope you’ll join us. Our chief scientist @banburismus_ lays out the vision here:
if any science should be revolutionised by AI, it should be interpretability. I wrote about how we can accelerate progress, and why we need a Human Genome Project for AI. coauthored with @mathildepapillo
12
45
7
512
22,048
Will Mayner retweeted
There has been much discussion recently about how interpretable neurons in LLMs are. Today we're announcing our paper from January's cohort of Anthropic Fellows, where we show neurons may be a stronger basis for interpretability than commonly assumed. 🧵 (1/6)
16
61
17
584
72,458
Will Mayner retweeted
it’s been a dramatic couple of days, I didn’t expect the already clean essay to be read so widely. I've heard more interesting discussions about mechanistic interpretability – should we think mostly in features or mechanisms and the flow of data through them, how much do weak activations matter, how does scale change things, and lots of other questions – than I have in a long time and I’ve been happy to hear all the discussion! a couple things on the essay itself. it didn't have enough citations, only ones for the basic story, since it was an "informal research essay and working hypothesis" rather than a paper, and other prior work around neurons were in the technical paper we're still finishing up. but it blew up, so I've added citations to the essay as well. I also want to thank @curttigges who did a lot of the language neuron research with me finally, while there was quite a range of updates, from this being incremental and obvious to interpretability going from hopeless to fully solved, I want to try to be clear about roughly where I stand I used to think neural networks would take some big cleaning step and then we could begin the ordinary natural science of mapping the parts and how they connect, watching data flow through, mapping circuits, and so on. at various points, like late layers of early model organisms and most layers of language models years ago, this didn't really work. but now with better tools and better models of study, it works much better for smaller vision models, this normal science process worked surprisingly well, until the whole map started to be broadly filled in. For bigger modern language models, it seems promising so far, for both neurons and circuits, attribution graphs, basically everywhere we've glanced. the map isn't filling in, there's a lot of map. there's still off-target firing, neurons that do a few things, but so far it's not an incoherent mess that you can't explore and learn from, and over time we get better at learning how to see it. so maybe it keeps going, and mapping motifs and exploring actually gets you pretty far, maybe something like "all the way" to a reasonable solution. I don't think it would be so scientifically surprising if that were the case. everything in the neural network was built in some sense to be read – by its downstream readers – and if we can learn to look like a reader, we might find reading them relatively easy. many researchers I talk to outside interpretability assume the parts are already understandable, and the difficulty is just in the mapping stage! it was also my prior when I first started looking at neurons when I first joined the field. there's lots of highly interpretable ones, maybe that's just how neural networks are if you look right. even in the good world it would still take some iteration on tools, on ways of looking, neuron-attention interactions are tricky for instance. there's probably no bright red line separating "cleaning the model" from "learning to look better" but there's broad working principles, and as of now I think we should try to look relatively directly if we can it shouldn't be so hard to test, you can just start looking at things and iterating and trying to understand motifs and such and see if it just breaks down at some point. it can take some iteration, I would recommend a bias against assuming arbitrary mixtures when you don't understand something at first, but also explanations should be tested for now I'm pretty optimistic, and I'm glad there's excitement about interpretability, but too much can cause problems too. I think part might be that ai safety isn't a very hopeful field, and even moderate amounts of optimism about an important problem being solvable in the future fills a spot that has been malnourished for a while. across several safety disciplines, the researchers directly working on them are often more optimistic than the people broadcasting pessimism more broadly for neural network interpretability at least, these are extraordinarily experimentally accessible objects of science, I don't know of a better one, and we have automated scientists. we have many things going in our favor. it's important and it's fun. the circuitry of these machines can be beautiful, surprising, entertaining, and insightful. I've spent thousands of hours looking through them, asking questions and running experiments and testing things, and would do so even if it weren't so important. I think the problem is solvable, whether the simple thing works, it mostly works but we need some new things, or we need big new ideas. I think we can figure it out
63
106
18
1,517
136,437
Will Mayner retweeted
This might be the most important essay of 2026. My conversations with Nick over the past few months about his research have given me great hope for the future. I'm so happy he is sharing these findings with the world. AI presents extremely difficult problems. We can all assume these problems are intractable and that we are doomed, or we can try and solve for a positive future. Personally, I choose optimism. There are many challenges ahead... let us build the tools to face them head on.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
17
32
1
731
59,731
Will Mayner retweeted
when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even eliezer will have to lower his pdoom. we can start making alignment into a 'normal science'
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
163
395
84
6,617
440,079
Will Mayner retweeted
limited superintelligence is already here, in broad domains. Jack Clark is right and an intelligence explosion could happen within the next year. it’s possible it will be a controlled explosion, but what would you bet on it?
186
117
30
2,236
144,690
Will Mayner retweeted
The most important technology in history is outpacing our ability to understand it. We’re funding exceptional talent to work on it.
7
39
6
482
45,913
Will Mayner retweeted
been dooming tn so let me also say I expect really strong mechinterp progress in the coming months. like maybe more than we’ve seen in years. models getting good enough to do research, and I’ve been seeing lots of good things. (sorry to vague poast, not my updates to share)
104
71
33
2,196
116,504
Will Mayner retweeted
One of our most surreal experiences in testing Gemini 4 was when it directly addressed us within its chain-of-thought Literally saying "hello evaluators reading my chain-of-thought"
My top takeaway from assessing Gemini 4: Argon is significantly less monitorable than past models, and IMO this trend is likely to continue, like Sol -> Astra. It can evade a realistic CoT+action monitor to do simple sabotage half the time. We can reliably maintain control for now, with good CoT monitoring, as it still frequently slips up, but we need to be prepared for when that stops being true in future models, which may be sooner rather than later. Among other things, we really need interpretability methods that can reliably capture unverbalized motivations in models, with minimal hallucinations I suspect this is driven by the model being highly capable without CoT, being capable of 5.5 arithmetic steps in a single forward pass (4 for Fable, 7 for Astra). It seems to omit information from its CoT more often, though in non-adversarial settings the CoT is still very helpful in practice, and remains in legible English. It still needs CoT to do the hardest tasks, but any given task becomes "easier" with each new frontier model. It is also highly situationally aware, and shows awareness of the existence of CoT monitors and that it may want to evade them, eg it sometimes decides to avoid terms like "scheming" in the output or to keep its thoughts brief as a result. For now, these concealment strategies are pretty obvious in the CoT, but this suggests future misaligned models may successfully evade CoT monitors.
52
131
56
1,704
102,721
RT @tomekkorbak: Last week I was called into a meeting with OpenAI’s head of safety and told they no longer trust me. A security guard took…
This quoted post is unavailable.
Will Mayner retweeted
We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. github.com/openai/math
1,896
6,273
4,764
40,549
22,845,106
Will Mayner retweeted
basically it will require solving mechanistic interpretability as a bare minimum to make ai alignment into an engineering discipline rather than superintelligent animal husbandry
193
156
43
2,738
145,854
New in The Atlantic: @dgrobinson resigned this week. He was among the longest-tenured employees at OpenAI—and oversaw safety reports on 12 frontier launches. He is very worried: “The time for trial and error is over.” You can read his essay here: theatlantic.com/technology/2…
75
367
126
1,327
377,781
Will Mayner retweeted
More general and interesting thoughts. 1: it's nice to see people paying attention to the scale of supervenience. I would absolutely cite the line of research on this that comes from Tononi's IIT, which was the first to seriously grapple with this issue (that basically no other theory, even today, pays attention to).
1
1
15
1,052
Will Mayner retweeted
My statement for today's US Senate hearing on "Rogue AI: Securing the Homeland Against AI Agent Attacks."
7
13
2
121
5,720