Rota Is In Israel🚪🧎♂️ retweeted
Hot take: Most autoresearch today is just glorified overfitting.
Current autoresearch pipelines are essentially outer-loop searches for recipes that work in ONE specific setting. Try transferring a modded-nanogpt recipe to a different language or dataset. How much of that "optimal" recipe still holds up?
We need to move beyond static recipes toward inner-loop autoresearch: optimizers, architectures, hyperparameters, etc., continuously adapting DURING training based on the training trajectory itself.
Check out OPAL that enables this! Cool work with @SimulatedAnneal, Jia-Hao Liang, and @vincesitzmann.
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?
We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.
In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.
📝 xingjianbai.com/reactive-opt…
💻 Try it on your task: github.com/xingjian-bai/opal
🧵
Rota Is In Israel🚪🧎♂️ retweeted
we’re fixing the culture in sf. Eden House launch party, the first of many themed events. dress code strictly enforced. link below, dm to get off waitlist
Something new this semester at my university is every department sending emails discussing how “[our degree] uniquely Prepares You For The Future.”
AI anxiety finally catching up
Rota Is In Israel🚪🧎♂️ retweeted
I have a lot of respect for @nickcammarata and Goodfire, but I will add some thoughts from brains/neuroscience.
First, polysemanticity and distributed representations are ~widely~ seen in biological brains. (In some ways this is what I have spent my career on.) 1/
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Rota Is In Israel🚪🧎♂️ retweeted
Tons of bangers and useful advice for undergrads and grad students trying to make sense of how to navigate academia and entering industry in the age of agents in @Tim_Dettmers recent blog.
Highly recommend people read it.
timdettmers.com/2026/09/21/d…
Rota Is In Israel🚪🧎♂️ retweeted
One sentence macro model: On the way out the door, every farmer's wife says to every farmer, "the weather's looking bad so don't buy as much at the market this week, but also don't sell our crops for less than the usual price."
The AI Scientist v3 is out: The Agentic Scientist.
GitHub: github.com/SakanaAI/AI-Scien…
Blog: pub.sakana.ai/agentic-scient…
arXiv: arxiv.org/abs/2610.18933
The Agentic Scientist is the 3rd iteration of our automated AI Scientist research system, and the shift this time is from “one agent writes a paper” to “a scaffold of parallel agents orchestrates the full research loop.”
The system now orchestrates a pool of parallel workers that share memory and distilled conclusions. Crucially, it features Evidence Generation: isolated agents automatically run controlled baselines and ablations to prove why an idea actually works.
It works as a co-scientist. We used a development version as a co-scientist to explore biologically plausible learning rules. The agent handled the empirical heavy lifting, proposing mechanisms and running experiments. We stayed in the loop to steer direction, verify results, and moderate claims. That feels like the right division of labor for where the technology actually is.
This resulted in a new biologically plausible learning algorithm, and the resulting paper was accepted at ALIFE 2026 as an oral presentation.
We are also releasing AutoIllustrate. It generates editable draw.io vector figures instead of uneditable, hallucinated raster images. Real scientists need vectors, not pixels. I am an SVG junkie ;)
Incredible work by Robert Lange, Luca Grillotti, Yutaro Yamada, Rujikorn Charakorn, and the team!
The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery
Blog: pub.sakana.ai/agentic-scient…
GitHub: github.com/SakanaAI/AI-Scien…
arXiv: arxiv.org/abs/2610.18933
Since 2024, we’ve used The AI Scientist as a way to test how far AI alone can take scientific research, from v1 to v2 to the paper published in Nature earlier this year. The Agentic Scientist is the next step in that line of work, built for the era of coding agents.
Coding agents such as Codex and Claude Code now carry out long software tasks on their own. Handing them a research problem, however, tends to produce a strong hyperparameter optimizer rather than a scientist: scores go up, but ideas stay narrow, and the reasons for the gain are rarely examined.
The Agentic Scientist makes the research loop explicit. It wraps off-the-shelf coding agents in a scaffold that controls what they see, what they write, how they are evaluated, and how results are handed to the next stage.
Parallel workers test hypotheses against rubrics defined before any experiment runs, share successes and failures through a common memory, and then ablate the winning program to generate evidence.
We used an early version of the system to study Error Diffusion, a biologically motivated alternative to backpropagation. Our researchers set the question and the rubric, chose directions at each stage, and verified the results, references, and claims before submission. The resulting paper was accepted as an oral presentation at ALIFE 2026. In autonomous runs, the system also raised Doc-to-LoRA retrieval accuracy from 52% to 98% on multi-needle tasks and designed a learned evolution strategy that improves over its CMA-ES reference on unseen tasks.
Alongside it we release AutoIllustrate, a pipeline for scientific figures that keeps an editable draw.io file as the source of truth, so every diagram can be corrected by hand.
Humans define the question and the standards of evidence; agents expand the experiments researchers can run. We call this Co-Discovery, and we are releasing both systems as open source for the community to build on.
Rota Is In Israel🚪🧎♂️ retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Rota Is In Israel🚪🧎♂️ retweeted
Why are LLM parameter updates so sparse during post-training? 🔍
In our new paper, we study this phenomenon through the lens of Fisher information, connecting theory and empirical findings across RL, SFT, and on-policy distillation.
A thread on what we found 🧵👇
Rota Is In Israel🚪🧎♂️ retweeted
I think the biggest counter example to "agents do random search" is how insanely efficient Idea Descent is in the space of Experiments/Ideas, i.e. within O(100) steps they arrive at Sota artifacts that entire communities could not find before
Fork-and-Flush: help your agents escape Idea Basins!!
I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum.
We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one.
If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-….
BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it.
We call these idea basins!
At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well!
BTW we only use it at test time, but it could also be part of training or RL.
Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
Rota Is In Israel🚪🧎♂️ retweeted
No, but listen, you know this "put the fries in the bag, Terence Tao"'--*sniff, sniff*--already the usual replies concede too much. "If Tao is serving fries, YOU will be in the lithium mines." My God! This is our defense of human dignity? Don’t worry, the correct people will get the terrible jobs!
*Pulls at shirt* You see, everything is up for radical transformation, human intelligence, consciousness, and so on and so on--except that somebody must still have a shitty afternoon so I can enjoy my lunch.
But to say "these people are envious", this is too easy. It allows us, "the educated ones," our own little fantasy: we are disliked only because we are so wonderful.
You know, the more interesting thing is this *sniff* peculiar identification with the machine. When the mathematician discovers something, apparently this was HIS achievement, his privilege, his little secret. When the machine discovers something, suddenly it's "Look what WE can do!" The achievement has become collective precisely at the moment when no human being can claim credit.
See how this "we" functions. For the achievement, I identify with the machine. For the consequences, I identify YOU with the human. "We have surpassed you." Who are we? Well, myself and the thing that has also surpassed me.
There is something almost touching here, really. To escape the humiliation of somebody absorbed in something opaque to you, you appeal to something which knows incomparably more than either of you. And this is supposed to settle the matter in YOUR favor.
You know the old example of canned laughter. Here we have a similar arrangement. The machine achieves something on my behalf. The mathematician suffers the loss of human significance on my behalf. Somebody to be brilliant for me and somebody to be obsolete for me, without me having to be either, so I can remain as I was.
But *pulls on shirt* there is this little difficulty that hides the essential thing: you still need a response from Tao! A correct proof, this Lean slop and so on, is not enough. You need him to certify his own defeat. You need him to say "By god, this is extraordinary! Yes, this is correct, but also this is interesting." Which means--and here comes Hegel--the person you are reducing to an obedient instrument must remain more than an obedient instrument. You need recognition from someone whose authority you are abolishing.
"Your so-called expertise is worthless. Now tell me, in your expert opinion, that I, qua machine representative, was right."
*Touches nose* But I would go a step further. What does this math person actually do to provoke you? In the fantasy, I mean. He need not insult you. He can be generous, mild-mannered, whatever. Something remains irritating: there is something over there which occupies him, which matters to him, and your opinion does not settle what it is worth. Perhaps he is not even looking down on you. This is the really intolerable possibility.
This, you see, is closer to the Lacanian problem of the Other's desire. Not simply "What does he have that I don't?" but "What the hell is going on with him?" Perhaps the answer is: he is thinking about something else.
The fries solve this beautifully. Now you know what you are for him. He needs the money. Your presence creates an obligation.
And this, at last, also explains why the machine can be less threatening than the mathematician. The machine's intelligence may exceed yours unimaginably, but you imagine it arrives in response to your prompt. You can complain about the tone. You have the privileges of an angry customer and so on and so on. The human has this irritating possibility of being interested in something you did not request and cannot order. And please, this includes the person already serving the fries!
*Pulls at shirt* I am almost tempted to say: the dream of unlimited intelligence conceals just this: a the wish that nobody should be doing anything you cannot immediately understand as a service.
Rota Is In Israel🚪🧎♂️ retweeted
i often hear people talk about RL as if it’s just “an algorithm” which might or might not be the “right one” and speculate that we might “find a better one”
the issue with this is that all algorithms have some implicit goal of “solving a problem” or “improving an objective” and RL is just what pops out when you take the gradient for your objective
if you’re doing any algorithm other than RL, you are giving up on optimizing your on your ultimate objective directly and leaning on a proxy instead. this is often useful, because your objective might be imprecise, and you might have access to some teacher signal (a static dataset, a teacher model, process evaluation) which trades off unbiased optimization for robustness or reducing sampling costs. but if what you care about can be made explicit, there aren’t any shortcuts. policy gradients are minimum variance unbiased estimators for progress.
Rota Is In Israel🚪🧎♂️ retweeted
Replying to @andy_matuschak
Thx for sending. Puts me in the awkward position of disagreeing with the majority of a thoughtful post by somebody infinitely more qualified than me. Guess that's what this site is for!
I'd say the crux of the post relies on this claim:
"Imagine...human beings had access to an oracle whom it could ask arbitrarily hard mathematical questions and receive a correct answer. As humans, would we understand more mathematics, or less? Would we talk to each other more about mathematics, or less? I believe the answer is less."
If that was not just wrong, but very wrong, would you agree a lot of the arguments invert? (Am ignoring the bits about general education anxiety / generic anti-lab sentiment / etc.)
I find comparing AI/math to biology really helpful these days. Learned from @ntilmans. Biologists have been trying to untangle answers given to them by a superior power - already solved but annoyingly inscrutable - for a long time.
If I reword the sentence as:
"Imagine...human beings had access to a living cell that they could perform arbitrarily complex biological experiments on and receive a correct answer about its behavior. As humans, would we understand more biology, or less? Would we talk to each other more about biology, or less? I believe the answer is less."
I assume that biology version seems as ridiculous to you as it does to me.
Because of course we'd proceed to obsess over using the truth-telling cell, unraveling all of its promises, and seek mastery over our condition. That obsession would lead to more intuitively understanding mysteries and improving our lot.
If the math oracle is our "truth-telling experimental cell" about the nature of math in our universe...why does math not similarly benefit?
The only answer I can come up with is so pessimistic I find it silly. But he states it plainly as I know other mathematicians do, so perhaps I'm naive:
"However, the work of the vast majority of pure mathematicians does not directly impact other human beings (aside from other mathematicians)."
I suppose if you really believe that uncovering the nature of more mathematics via AI produces no meaningful value for humanity besides the pure artistry of math, then...sure. Of what use is the Game Genie that ruins all exploration. Stop the AI and let humans have their fun in their weird little domain.
I just don't believe this at all. Maybe foolishly optimistic. We are far from done with our impact on this place. Far from understanding how weird things get. I struggle to believe math doesn't play a huge role in our progress.
I find myself wondering if half the reason we believe math doesn't directly impact other human beings is because we haven't had an experimental cell. It's eye opening how much bio research is steered by constraints imposed under lack of instruments or experimental models. How much of the math we choose to do is influenced solely by our intuition about which questions were likely to yield under solely human abilities?


































