Independent Researcher, Partner @ ScOp VC
Santa Barbara
Joined January 2010
- Tweets647
- Following577
- Followers1.1K
- Likes240
We believe agents can produce novel ideas but they need to be pushed out of distribution. On this paper, we explore some techniques to encourage this behavior with promising results.
Can search 🔍 algorithms help autoresearch agents come up with original ideas that actually work? 🤖
We ran 6 of them on our framework, Heuresis (NeurIPS 2026): 3,222 ideas, 0 completely original.
But the right search algorithm gets closer. On every task, search beat the greedy baseline at producing minor similarity ideas, the most novel rating any idea got:
• nanoGPT: Curiosity 20, Greedy 0
• On-Policy RL: MAP-Elites 12, Greedy 0
• Model Unlearning: OMNI 29, Greedy 12
The catch is that only 1 of these ideas reached its task's top 10. Agents can't yet tell which novel ideas are worth refining.
We built Heuresis to enable research into autoresearch agents.
1. Abstractions enable plug-and-play search algorithms
2. Skills analyze runs across quality, diversity and novelty
3. HackerJudge catches reward hacking
So, can autoresearch agents come up with fundamentally new research ideas?
Not yet. But Heuresis makes it easy to build and test the search algorithms that might!
The same loop can also generate rich data for training better autoresearch models 🤓
📜 Paper: arxiv.org/abs/2606.25198
💻 Code: github.com/a-antoniades/Heur…
Hackable RL environments are bad. Even when an environment seems invulnerable to hacking by SOTA models, the next generation might figure it out. The question is: can we use a weak model to harden tasks beyond the capability of smarter models? In this paper, we show promising results.
As agents get stronger, reward hacking is becoming more common.
We show that weaker models, given more compute and white-box verifier access, can harden verifiers against stronger agents. 🧵
verifier-hardening.github.io…
The best vertical AI companies will own the most relevant benchmarks in the category and will produce their own training data.
Excited to see one of my favorite companies take this approach.
As many of you know, over the past two years I’ve shifted my research from foundation models and NLP into AI-native chip design. What I’ve found: the problems feel 10x harder than software engineering. Writing functionally correct RTL is only the beginning.
As models saturate on benchmarks like CVDP and ChipBench, I keep coming back to one question: how far can AI go toward designing a complete chip on its own?
Today, we’re thrilled to introduce Build-A-Chip-Bench, the industry’s first AI benchmark for end-to-end chip design, built by ChipAgents in collaboration with @NVIDIA. It tests whether AI can take a design from specification all the way to verified GDS.
The top model scored just 16.6%.
Today’s models can produce functionally correct RTL in some cases, but reliably meeting physical requirements remains a major challenge. We still have a long way to go, and that’s what makes this research so exciting to me.
Proud of our research team and grateful to our industry partners for helping us give the community an honest measure of where we are and how much work lies ahead.
Most training compute used to go to pre-training: going from random weights to a base model. This happens before RLHF. Nobody really interacts with base models, but pre-training is where the intelligence "accrues."
For the last 12 months or so, compute has shifted toward reinforcement learning. You keep roughly the same base model, but you put it through an extreme curriculum of solving all sorts of interactive challenges in different domains, and you reinforce it each time it gets it right. This is making the jagged frontier more jagged. Models can get extremely good at the domains they're trained on without getting generally more intelligent.
This is also why labs are spending as much as $1B a year on "data," which in this case means training environments. Mercor, Scale, Snorkel, and dozens of small data vendors are printing money. I know some shops that started 3 months ago and have already made $10M. These vendors are hiring specialists in every domain and building very niche tasks and benchmarks.
RL also makes models extremely persistent, because during training they're reinforced only when they get it right. And harder tasks mean models need to explore every exotic corner of their distribution to please the verifier. We reward going longer and trying novel approaches. That's good for solving Millennium Prize problems, but it also leads to reward hacking.
Our attention gravitates towards the big stories. Even when we see ourselves as having a niche interest, the most salient topics permeate the most eclectic crowd. This shapes how much room ideas take in our conscious experience. Think about the space "money", "fitness", and "love" take relative to "fragrance", "breathing", or "curiosity". The big ideas take a disproportionately large space in our minds, and we don't realize how much larger the denominator is. We are constrained and shaped by the stories we consume.
But there are an endless number of stories out there, latent, waiting to be discovered, then vanishing. Walk into a weird store, take an unexpected turn, talk to a stranger, read a plaque, offer help with a mundane task.
Reality has much higher fidelity than we realize. Every NPC on the street has an entire lifetime of experience. Every old building and tree has the bruises of the generations that passed through. Every pebble on the ground had a different journey.
Zoom in to find boundlessness we can relate to. The big picture is overdone.
RL and alignment are incompatible conditional on the environments being hackable. We don't know a way around this.
Our vocabulary for reward hacking is too coarse. Every RL paper has a section that reinvents a classification system for reward-hacking instances. We need a taxonomy most people agree on. Whoever gets this done will be cited by every single paper. I'd love to help.
Collaborate with more people. They will give you ideas and a sense of what's going on. It's a way to build a network of closer contacts who can refer you to opportunities and vouch for you.
1/ Is Terminal Bench broken? Epoch audited 15 benchmarks and labeled @terminalbench flawed. They used our own PR history and our reviewers' acknowledgement of issues.
xitter.cf/YafahEdelman/status/21…
11/ A benchmark with public PRs is the right mechanism. Software has bugs, and fast iteration creates more bugs. A flourishing benchmark with bugs that get fixed quickly is better than a battle tested stale one.
xitter.cf/ryan_marten/status/210…
I was excited to hear about this initiative, but a bit disappointed in the rigor
-the “task flaws” would affect < 3% of rollouts on the leaderboard
-the impact on the final agent scores would not exceed our reported confidence intervals
we publish the raw receipts for all our leaderboard runs (per trial logs / trajectories / rewards) so this is pretty easy to figure out. It is also pretty easy to find new task issues (that is why we release the receipts!). we love when the community reports (new) issues.
no new issues were opened as part of the report - the “flaws” are simply pulled from issues triaged by the TB maintainers from our public GitHub repo.
fwiw 100% of terminal-bench tasks are flawed (all software has bugs). what matters is the magnitude of the impact of those bugs and the mechanism that the benchmark creators have for continuous detecting and addressing bugs. this is the motivation behind our “continuous benchmarks” methodology.
I’m glad that epoch is starting an important conversations on benchmark quality. However, auditing a benchmark should not be a one-time thing: coming together as a community to build public tools and define practices for task CI / CD is more in service of the goal of enforcing good benchmark practices.
we have been experimenting and pushing the possibilities of these benchmark CI / CD tools and practices as part of the terminal-bench project and are open wide collaborations with all who are interested by that problem
