🏝️Researcher @NUSComputing @NUSingapore; AI for scientific discovery. Opinions are my own.
Singapore
Joined June 2017
- Tweets204
- Following689
- Followers213
- Likes190
Pinned Tweet
🔥LLM-guided evolutionary methods like AlphaEvolve have shown impressive results on challenging mathematical and algorithmic optimization problems, and progress is often measured over a fixed number of iterations.
However, we argue that practical optimization should maximize gain per unit cost💰.
We introduce 🩵FrugalEvo🩵, a cost-aware LLM-guided evolutionary framework for computational optimization. It uses a stronger, higher-cost LLM to explore solution strategies, and a cheaper LLM to implement them and iteratively refine the resulting code.
🚀We also design a cache-efficient evolution process, where our harness and prompts maximize the sharing of prefixes across different evolution steps, to improve cache reuse.
📈To measure solution quality throughout a fixed cost budget, we introduce Budget-Aware Area Under the Curve (BA-AUC), defined as the area under the best-so-far evaluation score curve over cumulative LLM cost, up to the budget.
🧩On 20 real-world computational optimization tasks. FrugalEvo matches or surpasses all baselines, including OpenEvolve, ShinkaEvolve, AdaEvolve, and EvoX, in solution quality and achieves higher BA-AUC on 9 of 10 mathematical and systems optimization tasks. It also achieves the highest average performance on 10 algorithmic optimization tasks.
📄 Paper: arxiv.org/pdf/2610.03675v1
💻 Code: github.com/chchenhui/frugale…
1/ 👇
Hui Chen retweeted
1/ From OpenAI's Navier–Stokes work to AlphaEvolve's faster algorithms, AI research is getting very real.
Now, with Recursive Flint, we can experience that magic locally on our own laptops 💻
The whole research loop runs locally with a selection of tasks! Code + demos 🧵
Hui Chen retweeted
Introducing RSI-top: a framework that helps autonomous research agents explore new directions when progress stalls 🧭
On Anthropic’s kernel optimization task, RSI-top outperforms OpenEvolve and ShinkaEvolve by up to 15%. On MLE-bench Lite, it achieves a 77.3% medal rate, outperforming the strongest baseline by 11.4 percentage points.
RSI-top tackles this problem from two angles: context management and diversity control 🔍
A compact metric dashboard helps agents selectively retrieve useful past trajectories, while three diversity control mechanisms encourage fresh perspectives, new ideas, and different starting points when progress stalls.
More details in the thread below 🧵
Hui Chen retweeted
Most multi-agent work designs the agents.
However, with the current progress in foundation models, the agent is not quite the bottleneck in multi-agent systems anymore, which is why these systems often struggle to beat a single agent.
They scale poorly because every agent inevitably reads more as the group grows, and no change to the agents can fix that.
We propose ScaleMAS, which designs the medium the agents share, and leaves the agents alone.
We call this medium the substrate, and we see it as a paradigm shift for ongoing and future research on multi-agent systems.
On our substrate, every agent runs the same prompt. No roles, no manager, and no agent ever sends a message to another agent.
Instead, each one searches a shared datastore, reads what it finds, and publishes what it did. It works like Web search over the group’s own work.
And it works better than existing substrates such as a shared context or blackboard, and also better than agent-centric designs such as all-to-all message passing and coordinator-based delegation.
Most importantly, performance keeps rising as agents are added.
On Contradiction Audit, the blackboard-style baseline never exceeds 7% at any group size, while ScaleMAS reaches 50.9% with 128 agents.
It is also cheaper and more token efficient.
On BrowseComp-Plus, 8 ScaleMAS agents reach 60.0% at about the same compute where 16 blackboard agents reach 26.0%.
On SWE-bench Verified, 4 ScaleMAS agents match 8 independent ReAct agents at 80.0% with about half the tokens.
ScaleMAS is extremely robust.
We inject one agent that fails on every step, and the group performs as if nothing had been injected, while the blackboard baseline drops below 1%.
We also let agents drop out and rejoin at random, and the group keeps working until most of them are gone, while a coordinator-based design falls apart as soon as the coordinator is lost.
LLMs were not trained to be teammates. But given a well-designed substrate, they work together much better and much more efficiently.
Scaling Multi-Agent Intelligence with Substrate-Centric Coordination: arxiv.org/abs/2610.16120
Research index (e.g. Google Scholar) is one of the largest collaborative systems where millions of human users build on each other's work across centuries.
How about we build something similar for async LLM agents?
We give agents a minimal hypertextual system. They asynchronously search for and read related work, cite the work that they find useful, and publish (mini-)papers.
Each search returns only the top k papers, ranked by relevance and a PageRank-like citation weight, which bounds the read cost of each agent to O(k), independent of agent count.
We show that performance effectively scales as we keep adding agents, and the system is robust to absence/failures of individual agents.
Hui Chen retweeted
We are bringing agents to the wet-lab bench at Google Research to navigate complex biological landscapes. Learn how we are grounding agents in physical lab robotics to run fast closed-loop molecular design cycles, ultimately discovering single-digit nanomolar protein binders for cancer targets →goo.gle/4B5mMMB
Hui Chen retweeted
Key idea: We can train LMs on top of empirical n-gram tables, which are easy to precompute and store in CPU RAM. This speeds up pretraining significantly (at least up to 1B scale), plus we can hot-swap the n-gram table at inference time for training-free domain adaptation!
Hui Chen retweeted
The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery
Blog: pub.sakana.ai/agentic-scient…
GitHub: github.com/SakanaAI/AI-Scien…
arXiv: arxiv.org/abs/2610.18933
Since 2024, we’ve used The AI Scientist as a way to test how far AI alone can take scientific research, from v1 to v2 to the paper published in Nature earlier this year. The Agentic Scientist is the next step in that line of work, built for the era of coding agents.
Coding agents such as Codex and Claude Code now carry out long software tasks on their own. Handing them a research problem, however, tends to produce a strong hyperparameter optimizer rather than a scientist: scores go up, but ideas stay narrow, and the reasons for the gain are rarely examined.
The Agentic Scientist makes the research loop explicit. It wraps off-the-shelf coding agents in a scaffold that controls what they see, what they write, how they are evaluated, and how results are handed to the next stage.
Parallel workers test hypotheses against rubrics defined before any experiment runs, share successes and failures through a common memory, and then ablate the winning program to generate evidence.
We used an early version of the system to study Error Diffusion, a biologically motivated alternative to backpropagation. Our researchers set the question and the rubric, chose directions at each stage, and verified the results, references, and claims before submission. The resulting paper was accepted as an oral presentation at ALIFE 2026. In autonomous runs, the system also raised Doc-to-LoRA retrieval accuracy from 52% to 98% on multi-needle tasks and designed a learned evolution strategy that improves over its CMA-ES reference on unseen tasks.
Alongside it we release AutoIllustrate, a pipeline for scientific figures that keeps an editable draw.io file as the source of truth, so every diagram can be corrected by hand.
Humans define the question and the standards of evidence; agents expand the experiments researchers can run. We call this Co-Discovery, and we are releasing both systems as open source for the community to build on.
Hui Chen retweeted
AI is cracking math problems. What's holding it back in engineering?
The cost of evaluation. Agents can generate ideas in seconds, but testing them can take hours or days.
💡 Meet DuoDiscover: where evaluation is part of what agents discover 👇🧵
Some highlights:
🏆 #1 on NVIDIA's SOL-ExecBench
🐍 2× the experts on SWE-fficiency
🚀 1.92× on vLLM; 1.44× on FreeToken
🔥 2.68× over production TPU kernels
🖥️ Beats the best human and AI-designed CPU prefetchers
Hui Chen retweeted
We’re exploring how AI can accelerate health science research.
While AI has made significant breakthroughs in formal sciences like mathematics, health data science depends on imperfect real-world data and consequential choices about cohorts, variables, and causal assumptions.
To explore how AI can assist health researchers, we built a human-led research scaffold powered by Muse Spark 1.3 and Muse Wonder with access to 5M retrospective de-identified electronic health records and publicly available data. A cohort of medical experts used the system to conduct original research, accelerate hypothesis generation, code execution, and manuscript drafting while maintaining full authority over scientific decisions.
Today, we’re sharing 10 research manuscripts produced through this exploration, spanning cardiology, oncology, infectious disease, emergency medicine, and more.
Read more in our research blog: go.meta.me/3U06wvD
Hui Chen retweeted
Can self-improving AI agents become a security risk? 🧠
In our new paper, we study how LLM agents that learn from experience can also absorb malicious instructions from untrusted content, creating persistent safety risks across sessions. 🧟♂️🤖
🔗 Paper: arxiv.org/abs/2602.15654
Hui Chen retweeted
Fork-and-Flush: help your agents escape Idea Basins!!
I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum.
We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one.
If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-….
BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it.
We call these idea basins!
At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well!
BTW we only use it at test time, but it could also be part of training or RL.
Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
Hui Chen retweeted
new blog post: why taking inference scaling seriously leads to swarms.
contrary to popular belief, I don't think multi-agent systems are a new scaling axis. they're just structured inference scaling.
hallerite.com/posts/on-the-n…
Hui Chen retweeted
As autoresearch agents run longer, do they keep exploring new approaches, or mostly refine the ones they found early?
Turns out, runs settle into different "idea basins" and stay there.
We introduce fork-and-flush, a simple way out 🧵👇
Hui Chen retweeted
🚨 AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics
Our team at @AIatMeta is excited to release AutoformBench, a new formalization benchmark that pushes the Lean 4 coding capabilities of frontier models to their limits.
While SOTA models have become extremely competent at single-shot problem-solving, formalizing mathematics at scale and cost-effectively remains difficult. For instance, only about 42% of the results in openai/math could be formalized before release.
Furthermore, it is not clear and publicly known how much it costs to formalize big results, which is critical information if formalization is to become a part of the peer review process. This is the focus of AutoformBench.
The benchmark comprises 341 proving tasks drawn from diverse graduate- and research-level mathematical sources and formalized by human experts. Each task requires writing a substantial amount of auxiliary code, including lemmas, definitions, and more, placing the emphasis on models’ ability to build formal infrastructure rather than on mathematical cleverness alone.
The result is a demanding test: even the strongest models require tens of millions of tokens to complete 50% of the tasks (roughly $10 to $100 at current API rates).
Paper (arXiv coming soon): github.com/facebookresearch/…
Code: github.com/facebookresearch/…
Joint work by myself, Ahmad Rammal (@Ahmad3Rammal) and Vivien Cabannes, together with Niket Patel (@niketnpatel), Fabian Gloeckle (@FabianGloeckle), Bartosz Piotrowski, Amaury Hayat (@Amaury_Hayat), Remi Munos (@MunosRemi) and Julia Kempe (@KempeLab) from @AIatMeta, @NYUDataScience, and Ecole des Ponts. (1/6)
This work cannot be done without our amazing collaborators and mentors.
Kudos to: Xuan Qi, @xu_Zhao0, Zhaopeng Feng, @Shilong_Liu_AI, @ProfKuangXu, @PangWeiKoh, @BryanHooi1 !!
8/8